BroadRouter
Get started

Fine-tuning

Updated

Fine-tuning is the process of continuing to train a pre-trained language model on a smaller, task-specific dataset, such as example conversations, so that its weights change and it performs that task or style more reliably without needing long instructions or examples in every prompt.

Also called: finetuning, supervised fine-tuning, SFT, model fine-tuning

Prompting tells a model what to do each time. Fine-tuning changes the model so it already knows.

How it works

You start from a trained model and run more training on your own examples. In supervised fine-tuning (SFT), each example is a conversation with the answer you want. OpenAI’s guide uses JSONL, “one complete JSON structure on every line”, in the chat format:

{"messages": [{"role": "user", "content": "Classify: 'card charged twice'"}, {"role": "assistant", "content": "billing"}]}

The training job adjusts the weights so those answers become more likely, and returns a new model ID. OpenAI sets the minimum at 10 examples and recommends starting with 50 and evaluating before adding more.

Full fine-tuning updates every weight, which is costly for large models. The LoRA paper by Hu and colleagues calls deploying separate fine-tuned copies of GPT-3, “each with 175B parameters”, prohibitively expensive. Parameter-efficient methods such as LoRA train a small add-on instead. Distillation is a related idea: fine-tune a small model on a larger model’s outputs.

Why it matters

  • Shorter prompts. Behaviour that needed a long few-shot prompt can move into the weights, cutting input tokens per call.
  • Consistency. Good fits are classification, a fixed output style, domain translation and correcting instruction-following failures.
  • Upkeep. A tuned model is pinned to one base model. When the base is retired, you retrain. OpenAI’s guide now says it is winding down its fine-tuning platform, closed to new users, with tuned models served until their base models shut down.
  • Measurement first. Build an eval before training, or you cannot tell whether the tuned model is better.

Fine-tuning is a poor way to add facts that change. Retrieval keeps them current.

On BroadNet Inference

BroadNet’s API reference does not document a fine-tuning API, training jobs or uploading your own weights. It does say that “our Arabic fine-tunes and other models on request” are reached with a private model ID issued with the endpoint. BroadNet lists one such model on request: a fine-tune of Cohere Transcribe Arabic for numbers, dates and names in noisy audio. For the public Arabic transcription model, see the Arabic speech-to-text guide. If you need to train your own, an open-weight model is the usual starting point; the trade-offs are in open-weight vs closed models.

Questions

How many examples do I need to fine-tune an LLM?

OpenAI's guide sets a minimum of 10 and suggests starting with 50 well-made examples, then measuring. If 50 good examples change nothing, rethink the task before adding more.

Fine-tuning or RAG for company knowledge?

Usually retrieval. Fine-tuning teaches behaviour and format well, but facts that change are easier to keep current in documents you retrieve at request time.

Can I fine-tune a model on BroadNet Inference?

BroadNet's docs do not list a fine-tuning API. They mention BroadNet's own Arabic fine-tunes, reached as private endpoints on request.

Read as MarkdownAsk ChatGPTAsk ClaudeAsk Perplexity