AI Dictionary of Terms

Training

The process of teaching a machine learning model to recognize patterns and make predictions by exposing it to large amounts of data and adjusting its internal parameters (weights) to minimize errors.

The Simple Version

Imagine teaching a child to recognize animals. You show them hundreds of pictures of cats, saying “This is a cat” each time. After seeing enough examples, the child starts to notice patterns: pointy ears, whiskers, certain body shapes. Eventually, they can recognize a cat they’ve never seen before.

Training an AI works the same way. You show the model thousands or millions of examples, and it adjusts its internal “understanding” (mathematical weights) to get better at the task. The more high-quality examples it sees, the better it becomes.

Detailed Explanation

Training is the foundational process that transforms a randomly initialized neural network into a useful AI system. It involves three key phases:

1. Forward Pass:

2. Loss Calculation:

3. Backward Pass (Backpropagation):

Training Loop:

for epoch in range(num_epochs):
    for batch in dataset:
        prediction = model(batch.inputs)
        loss = loss_function(prediction, batch.labels)
        loss.backward()  # Calculate gradients
        optimizer.step()  # Update weights
        optimizer.zero_grad()  # Clear gradients

Types of Training:

Key Hyperparameters:

Key Characteristics

Business Context

Understanding training helps enterprises make informed AI decisions:

Cost Drivers:

Strategic Considerations:

Training vs. Inference:

Real-World Analogy

Learning to drive a car. You start as a novice (random weights). Through practice (training data), you learn to steer, brake, and navigate. Your instructor provides feedback (loss function), and you adjust your technique (weight updates). After thousands of miles (epochs), you become a skilled driver. But you still need to stay alert and adapt to new situations (inference).

Code Example

# Basic training loop using PyTorch
import torch
import torch.nn as nn
import torch.optim as optim

# 1. Define model
model = nn.Sequential(
    nn.Linear(10, 64),
    nn.ReLU(),
    nn.Linear(64, 32),
    nn.ReLU(),
    nn.Linear(32, 1)
)

# 2. Define loss function and optimizer
criterion = nn.MSELoss()  # Mean Squared Error for regression
optimizer = optim.Adam(model.parameters(), lr=0.001)

# 3. Dummy training data
X_train = torch.randn(1000, 10)  # 1000 samples, 10 features
y_train = torch.randn(1000, 1)   # 1000 target values

# 4. Training loop
num_epochs = 100
batch_size = 32

for epoch in range(num_epochs):
    model.train()  # Set model to training mode
    
    # Mini-batch training
    for i in range(0, len(X_train), batch_size):
        batch_X = X_train[i:i+batch_size]
        batch_y = y_train[i:i+batch_size]
        
        # Forward pass
        predictions = model(batch_X)
        loss = criterion(predictions, batch_y)
        
        # Backward pass
        optimizer.zero_grad()  # Clear old gradients
        loss.backward()        # Calculate new gradients
        optimizer.step()       # Update weights
    
    # Print progress
    if (epoch + 1) % 10 == 0:
        print(f"Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item():.4f}")

print("Training complete!")

Common Misconceptions

Sources & Further Reading