AI Dictionary of Terms

Chain of Thought (CoT)

A prompting technique that encourages language models to generate intermediate reasoning steps before producing a final answer, dramatically improving performance on complex tasks requiring logical reasoning, mathematics, or multi-step problem-solving.

The Simple Version

Imagine asking someone “What’s 247 × 389?” They could guess, or they could work through it step-by-step: “First, 247 × 300 = 74,100. Then, 247 × 80 = 19,760. Then, 247 × 9 = 2,223. Adding those up: 74,100 + 19,760 + 2,223 = 96,083.”

Chain of thought prompting asks the AI to “show its work” — to think through problems step-by-step rather than jumping straight to an answer. This simple technique dramatically improves accuracy on math, logic, and reasoning tasks.

Detailed Explanation

Introduced by Wei et al. in 2022, chain-of-thought prompting demonstrated that large language models could perform much better on reasoning tasks when encouraged to generate intermediate steps.

How It Works:

  1. Standard Prompting: “What is 247 × 389?” → Model guesses or makes errors
  2. Chain-of-Thought Prompting: “Let’s think step by step. What is 247 × 389?” → Model works through the calculation

Types of Chain-of-Thought:

1. Zero-Shot CoT: Simply add “Let’s think step by step” to the prompt.

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls. 
Each can has 3 tennis balls. How many tennis balls does he have now?
A: Let's think step by step.

2. Few-Shot CoT: Provide examples that include reasoning steps.

Q: There are 15 trees in the grove. Grove workers will plant trees today. 
After they are done, there will be 21 trees. How many trees did they plant?
A: There are 15 trees originally. Then there were 21 trees after some more were planted. 
So there must have been 21 - 15 = 6. The answer is 6.

Q: [Your question here]
A:

3. Self-Consistency: Generate multiple reasoning paths and pick the most common answer.

4. Tree of Thoughts: Explore multiple reasoning branches and backtrack when needed.

Why It Works:

Performance Gains:

Key Characteristics

Business Context

Chain-of-thought prompting is a high-ROI technique for enterprise AI:

When to Use CoT:

When NOT to Use CoT:

Enterprise Applications:

Implementation Considerations:

Real-World Analogy

Showing your work on a math test. Instead of just writing the answer, you write out each step: “First, I’ll factor this equation. Then, I’ll solve for x. Checking my work…” This makes it easier to spot errors, understand your logic, and verify the answer is correct.

Code Example

# Comparing standard vs chain-of-thought prompting
from openai import OpenAI

client = OpenAI()

# Complex reasoning problem
problem = """
A store has 3 types of fruit:
- Apples cost $2 each
- Oranges cost $3 each  
- Bananas cost $1 each

A customer buys 5 apples, 3 oranges, and 7 bananas. 
They pay with a $50 bill. How much change do they get?
"""

# Standard prompting (may make errors)
standard = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": problem}]
)
print("Standard:", standard.choices[0].message.content)

# Chain-of-thought prompting (more accurate)
cot = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": "Solve problems step by step, showing your work."},
        {"role": "user", "content": problem + "\n\nLet's think step by step."}
    ]
)
print("\nChain-of-Thought:", cot.choices[0].message.content)
# The CoT response will show:
# "Step 1: Calculate apple cost: 5 × $2 = $10"
# "Step 2: Calculate orange cost: 3 × $3 = $9"
# "Step 3: Calculate banana cost: 7 × $1 = $7"
# "Step 4: Total cost: $10 + $9 + $7 = $26"
# "Step 5: Change: $50 - $26 = $24"
# "The answer is $24"

Common Misconceptions

Sources & Further Reading