# Meta-Harness: End-to-End Optimization of Model Harnesses

Author: Ishwar Jangid
Canonical URL: https://ishwarj.com/papers/meta-harness/
Updated: 2026-10-18
Source: https://github.com/ishwar6/ishwar-books/blob/main/content/papers/meta-harness/index.md


## Why this paper

A language model never works alone. Code around it decides what to remember, what to look up and what to show the model before each call. That code is the **harness**. Change the harness, and the same model can do much better or much worse on the same task.

Harnesses are mostly written by hand: a person reads failures, guesses a fix, and tries again. This paper asks whether an agent can do that job. Its system, **Meta-Harness**, gives a coding agent a folder with every earlier harness, the full record of how each one ran, and its score. The agent reads what it needs and writes a better harness. The paper tests this on three very different tasks and reports gains over hand-written harnesses on each.

## How the six parts are organised

**Before Part 1 (optional): [What a Harness Is](https://ishwarj.com/papers/meta-harness/background-what-a-harness-is/).** The model, the loop around it, and the earlier ways people tried to optimise prompts and programs.

1. **[The Big Idea](https://ishwarj.com/papers/meta-harness/part-1-the-big-idea/)**: the title, Figure 1, the abstract and §1.
2. **[The Method](https://ishwarj.com/papers/meta-harness/part-2-the-method/)**: §2, Table 1, §3, Figure 2 and Algorithm 1. The objective, the filesystem of experience, and the proposer.
3. **[Text Classification](https://ishwarj.com/papers/meta-harness/part-3-text-classification/)**: §4.1, Tables 2 to 5, Figures 3 and 4, and the discovered harnesses of the appendix.
4. **[Math Retrieval and TerminalBench-2](https://ishwarj.com/papers/meta-harness/part-4-math-and-terminalbench/)**: §4.2 and §4.3, Tables 6 and 7, and the discovered harnesses.
5. **[Limits, and What Came After](https://ishwarj.com/papers/meta-harness/part-5-limits-and-what-came-after/)**: §5, the proposer's behaviour in Appendix A, and the later work on self-improving harnesses.
6. **[Build It from Scratch](https://ishwarj.com/papers/meta-harness/part-6-build-it-from-scratch/)**: a small Meta-Harness loop in Python, run on a laptop.

## How to read the boxes

- A **teal "From the paper" box** shows the exact lines of the paper as a highlighted screenshot. Click the screenshot to see it at full size. Under it, **What it says** gives the main words of the paper. **What it means** then explains those lines, and what the authors mean but do not write. When the box shows a different paper, the box gives the name of that paper.
- A **yellow definition box** explains a technical word the first time it occurs. If you know the word, skip the box.
- The small **Paper §3** tag above a heading tells you which section of the paper you read.
- A **green "Key takeaways" box** ends each part with the facts to remember.

Code blocks show real code, and the output under a code block is what that code printed. The scripts are in [`code/papers/meta-harness`](https://github.com/ishwar6/ishwar-books/tree/main/code/papers/meta-harness). They run on a laptop with small open models; the paper used large commercial models, and each part says what that changes.

## What you need to know first

You need to know what a language model is and how a prompt works. Some Python helps for Part 6. The [Transformer series](https://ishwarj.com/papers/transformer/) explains the model inside, but this paper treats the model as a black box.

