Should You Fine-Tune an LLM in 2026? Start With These Three Steps

Should you fine-tune an LLM in 2026? For most bootstrapped products, no, not yet. Write evals, improve the prompt with context and examples, and add retrieval before you train anything. OpenAI's own optimisation guidance puts fine-tuning after evals and prompt engineering, and OpenAI is winding down self-serve fine-tuning, so the decision is now about much more than quality.

TL;DR

• Fine-tuning is one step in an optimisation loop that starts with evals and prompt engineering, not the first move.
• OpenAI's guidance says the prompt engineering process may be all you need for your use case.
• OpenAI has closed new fine-tuning jobs to new organisations and stops existing customers creating new jobs on 6 January 2027.
• Inference on fine-tuned models continues until the underlying base model is deprecated.
• Fine-tune for a narrow, repeated task with clean examples, and use retrieval when the problem is missing or changing knowledge.

Disclosure: ZeroRaised publishes this analysis and is a launch site for bootstrapped products. Facts come from the linked sources as checked on 1 Nov 2026; ZeroRaised has no commercial relationship with the companies named.

Fine-tuning is further training an existing model on your own examples so it learns a task-specific pattern, rather than supplying instructions and examples in the prompt at request time.

Why is fine-tuning the last step, not the first?#

OpenAI's model optimisation guide describes the loop: write evals for a baseline, prompt with context and instructions, fine-tune for some use cases, run evals on representative test data, tweak and repeat (OpenAI). It also suggests writing evals before prompts, in the way behaviour-driven development starts with the behaviour. The order is the point. Without a baseline you cannot tell whether training helped or just changed the output.

The guide is blunt about prompting: it recommends relevant context such as private data or current information, clear instructions and example outputs for few-shot learning, and it says "The prompt engineering process may be all you need in order to get great results for your use case" (OpenAI). For a small team, that sentence should be the default assumption until an eval proves otherwise.

There is also a maintenance reason. LLM output is non-deterministic and behaviour changes between snapshots (OpenAI). A fine-tune is tied to a base model, so every snapshot change is a reason to re-run your evals.

What should you try before fine-tuning?#

A practical order for a bootstrapped team:

1. Write evals. Define what good output looks like and score a baseline. The LLM evaluation guide covers test sets and scoring.
2. Fix the prompt. Add context, clear instructions and a few examples. The system prompt guide covers structure.
3. Add retrieval. If the model lacks private or current information, retrieve it at request time instead of training it into the weights. The RAG explainer covers the pattern.
4. Only then consider fine-tuning for the cases where style, format or a narrow task still falls short.

Retrieval is not a substitute for a good prompt, and a prompt is not a substitute for data. Each step answers a different question, and each one is cheaper to reverse than training.

What is happening to OpenAI's fine-tuning platform?#

OpenAI has been narrowing access since May 2026. On 7 May 2026, creating fine-tuning jobs became unavailable to organisations that had not previously run fine-tuning. On 2 July 2026, it became unavailable to organisations that had not run inference on a fine-tuned model in the past 60 days. On 6 January 2027, active existing customers can no longer create new fine-tuning jobs (OpenAI).

The pricing page states it plainly: "OpenAI is winding down the fine-tuning platform. The platform is no longer accessible to new users, but existing users of the fine-tuning platform will be able to create training jobs for the coming months" (OpenAI).

One detail stops that from being an emergency: inference on fine-tuned models continues until the underlying base model is deprecated (OpenAI). The model optimisation page also notes that evals and fine-tuning workflows are moving into legacy documentation (OpenAI). If you already have a fine-tune in production, you have time, but you should treat it as a migrating dependency, not a permanent one.

When does fine-tuning still make sense?#

It can still make sense when the task is narrow and repeated, you have a large set of clean examples, and the failure is about style, format or a specific classification that prompting has not fixed. A fine-tune can also shorten a long prompt by moving fixed instructions into the weights, which is a cost and latency argument rather than a quality one. That is advice, not a vendor promise.

It makes less sense when the problem is knowledge. If your product needs current prices, policies or customer data, retrieval handles change without retraining. It also makes less sense when you cannot measure the change. Without evals you are buying an unverifiable improvement.

Finally, weigh vendor risk. A fine-tune ties you to a base model and a platform. The open-weight vs closed model comparison and the LLM API guide cover the trade-offs.

What should you do now?#

If you are starting, do the eval and prompt work first, and keep a dataset of good examples as you go. If you already fine-tuned, keep inference running while the base model is supported, preserve your training data and eval suite, and plan a migration path before the platform changes again. If you are evaluating alternatives, compare an open-weight model you can host with the closed provider route, and price the operational work, not just the token rate. The LLM cost guide covers the cost side.

What this means for bootstrapped founders#

Fine-tuning is easy to treat as a shortcut around unclear product thinking. It is not. If you cannot describe what good output looks like in an eval, training will not tell you, and it will not tell your customers either.

The cheaper path is also the more durable one: evals, a better prompt and retrieval are all reversible, and they survive a model change. That matters more for a small team than a marginal quality gain on one benchmark. Keep the training decision for a case where you can measure the gap and the data is already clean.

Where ZeroRaised fits#

ZeroRaised is a launch and discovery site for products built without institutional venture capital or accelerator equity. If you have a product people can try, submit it here; the pricing page explains the current options.

For more practical analysis on shipping AI features without a large budget, join the ZeroRaised newsletter. No promises of traffic or results, just the useful parts.

Sources#

• OpenAI: Deprecations
• OpenAI: API pricing
• OpenAI: Model optimization