Start a fine-tuning job

Here’s a comprehensive example of how to create a fine-tuning job for a Llama 2 model, with specific infrastructure requirements (86 CPUs, 8 GPUs) and training parameters.

To create a fine-tuning job, you'll first create a project, to which you can associate a fine-tuning run. You can also retrieve project information and get a list of all of your projects:

Create a project

import os
from seekrai import SeekrFlow

client = SeekrFlow(api_key=os.environ.get("SEEKR_API_KEY"))

# Create a new project
proj = client.projects.create(name="project-name", description="project-description")

You can also get a list of all projects, or locate a specific one:

Endpoints:

GET v1/flow/fine-tunes List Fine-Tuning Jobs

GET v1/flow/fine-tunes/{fine_tune_id} Retrieve Fine-Tuning Job

# List all projects
client.projects.list()

# Get project info
client.projects.retrieve(proj.id) 

Note: Project ID is an integer and can be found by listing all projects; e.g., project_id=588

Next, specify a TrainingConfig object and an InfrastructureConfig object.

The TrainingConfig defines all parameters that affect the actual code of the training script, such as the base model to be fine-tuned, number of epochs, quantization, etc.

The InfrasructureConfig defines the infrastructure for the fine-tuning job. Gaudi2 is available on SeekrFlow, with other compute options available for on-prem installations and AI appliances.

This example uses 8 Gaudi2 instances, which triggers SeekrFlow to run on multi-card training mode:

Training configuration

Endpoint: POST v1/flow/fine-tunes Create a Fine-Tuning Job

import requests

url = "https://flow.seekr.com/v1/flow/fine-tune"

payload = {
    "infrastructure_config": {
        "n_cpu": 86,
        "n_gpu": 8,
        "memory": 2400
    },
    "training_config": {
        "training_files": ["string"],
        "n_epochs": 10,
        "batch_size": 32,
        "learning_rate": 0.0001,
        "model": "meta-llama/Llama-2-7b-hf",
        "hf_token": "string",
        "experiment_name": "string",
        "max_length": 512,
        "bf16": True,
        "gradient_checkpointing": True
    }
}

headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)
print(response.text)

Fine-tune a base model

Now that you've created your configuration files, you're ready to fine-tune a model!

fine_tune = client.fine_tuning.create(
    training_config=training_config,
    infrastructure_config=infrastructure_config,
    project_id = proj.id,  # NOTE: To associate this fine-tune with a project, we are passing in the ID (an integer) of the project created above. 
)

Tune model hyperparameters

Hyperparameters are the configurable parts of a model's learning process, and tuning them allows you to tweak model performance for optimal results. Hyperparameters play a crucial role in the training process, impacting both performance and training efficiency. Here’s a guide to essential hyperparameters, organized by impact level and the logical order for tuning large text models.

Learning rate

This is the most critical hyperparameter, because it determines whether your model will converge properly. Tune this one first, since it forms the foundation of your optimization process.

Consider impact on convergence: The learning rate controls how much the model’s weights are updated with respect to the loss gradient. A high learning rate can lead to rapid convergence but risks overshooting the optimal solution, while a low learning rate ensures stable convergence but may require more training epochs.

Start with a small value: A common practice is to start with a small learning rate (e.g., 0.001) and adjust based on the training performance.

Batch size

Next in the tuning sequence, batch size directly interacts with learning rate: the effective learning rate is often considered learning rate × batch size, e.g., if you increase batch size from 32 to 64, you might try increasing learning rate by approximately 2× to maintain similar training dynamics. Note: This relationship isn't perfectly linear in practice; at very large batch sizes, more increases in learning rate can lead to instability.

Memory constraints: Larger batch sizes require more memory, but can lead to faster and more stable training due to more accurate gradient estimates.

Training speed: Smaller batch sizes can lead to noisier updates, but may converge faster due to more frequent weight updates.

Experimentation: Start with a moderate batch size (e.g., 32 or 64) and adjust based on memory availability and training speed.

Epochs

Number of epochs should be determined after you've established stable learning rate and batch size settings, because these parameters together will determine how quickly your model converges.

Overfitting: More epochs allow the model to learn more from the data, but also increase the risk of overfitting (where the model learns the task too well, leading to poor generalization on unseen data).

Training time: The number of epochs impacts the total training time. Ensure that the chosen number of epochs balances training time with model performance.

Max length

Max length defines context window capacity and affects the model's ability to understand and generate coherent text. It's a a high-impact architectural decision that affects the model's fundamental capabilities, but is often constrained by hardware limitations.

Sequence length: The maximum length of input sequences the model will handle. Longer sequences can capture more context but require more memory and computation.

Balance length: Choose a length that balances capturing sufficient context with computational efficiency.

Task requirements: Set these based on the typical length of the input data for your task.

The last two mentioned here, bf16 precision and gradient checkpointing, are primarily technical optimizations that affect memory usage and training speed, with less direct impact on model quality:

Bf16 (Bfloat16 precision)

This is typically a technical decision based on hardware capability.

Memory efficiency: Bfloat16 reduces memory usage, allowing for larger models or batch sizes.

Training stability: Maintains training stability while offering computational efficiency.

Hardware support: Ensure your selected hardware supports bf16 precision.

Gradient checkpointing

Memory usage: Gradient checkpointing trades increased computation for reduced memory usage, allowing for training larger models.

Complex models: Particularly useful for training very large models where memory is a constraint.

Training time: May increase training time due to additional computations during backpropagation.

Monitor your fine-tuning run

All job runs are tracked using SeekrFlow's event monitoring and tracking system.

To retrieve the status and progress of a run, use this:

print(client.fine_tuning.retrieve(fine_tune.id).status)

Plot training loss

Note: A training loss chart is automatically generated for you via our UI: Just navigate to Projects and choose your project from the directory.

import matplotlib.pyplot as plt

ft_id = fine_tune.id
events = client.fine_tuning.retrieve(ft_id).events
ft_response_events_sorted = sorted(events, key=lambda x: x.epoch)

epochs = [event.epoch for event in ft_response_events_sorted]
losses = [event.loss for event in ft_response_events_sorted]

plt.figure(figsize=(8, 4))
plt.plot(epochs, losses, marker="o", linestyle="-", color="b")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.title("Training Loss Over Epochs")
plt.grid(True)

max_labels = 10
step = max(1, len(epochs) // max_labels)
plt.xticks(epochs[::step], rotation=45)

plt.tight_layout()
plt.show()

How to interpret a training loss chart

The training loss measures how closely predictions match actual values. A lower value and downward curve forming an elbow shape signal progress; watch out for a flat line or rising lines to indicate learning issues.

Loss: The Y-axis represents training loss, which quantifies the difference between the model's predictions and the actual target values. A lower loss indicates better performance.
Epochs: The upper X-axis shows epochs, where each epoch corresponds to one complete pass through the entire training dataset.
Steps: The lower X-axis represents the training steps, calculated as:
Total Steps = (Total Number of Samples ÷ number of instances * Batch Size) × Number of Epochs

Decreasing loss curve: Indicates that the model is learning and improving its predictions.
Plateauing loss curve: The model may have reached its learning capacity with the current configuration.

• Try adjusting hyperparameters (e.g., learning rate, batch size) and retrain to see if the model improves.

Increasing loss curve: May indicate overfitting or issues with the training process.

• Review data quality to ensure the training data is clean and representative of the problem space, and retrain with a higher-quality dataset.