RepairAgent

Autonomous LLM-powered bug repair for Java projects — no human intervention needed.

RepairAgent is an autonomous agent that fixes bugs in Java projects using LLMs. It operates in a loop: localize the bug -> analyze the code -> generate a fix -> test it -> iterate — all without human guidance.

On the Defects4J benchmark, RepairAgent correctly fixed 164 bugs, outperforming prior state-of-the-art tools:

Published at ICSE 2025. RepairAgent is the first autonomous agent-based approach to automated program repair.

How It Works

                     RepairAgent Workflow
                     ====================

  +---------------------------------------------------------------+
  |                      LLM-Powered Agent                        |
  |                                                               |
  |   +-----------------+    +------------------+    +----------+ |
  |   |  1. Understand  | -> |   2. Collect Info | -> | 3. Fix   | |
  |   |     the Bug     |    |    to Fix the Bug |    | the Bug  | |
  |   +-----------------+    +------------------+    +----------+ |
  |   | - Extract test  |    | - Search codebase |    | - Write  | |
  |   | - Form          |    | - Extract methods |    |   patch   | |
  |   |   hypothesis    |    | - Find similar    |    | - Run    | |
  |   |                 |    |   patterns        |    |   tests  | |
  |   +-----------------+    +------------------+    +----+-----+ |
  |         ^                                              |      |
  |         |           iterate if tests fail              |      |
  |         +----------------------------------------------+      |
  +---------------------------------------------------------------+

  Input: Buggy Java project + failing test
  Output: Correct patch that passes all tests

The agent has three states, each with specialized commands:

• Understand the bug: reads failing tests, forms hypotheses about root causes
• Collect information: searches the codebase, extracts method signatures and similar code patterns
• Try fixes: generates patches, applies them, runs the test suite, and iterates

Supported Models

RepairAgent supports both OpenAI and Anthropic (Claude) models:

Note: You can pass any OpenAI or Anthropic model name via --model — the table above lists the models with pre-configured cost tracking, but unlisted models work too (cost tracking will be skipped).

gpt-5 family: These models only accept temperature=1.0. RepairAgent handles this automatically — any --temperature value is overridden to 1.0 when using a gpt-5-* model.

Quick Start

The guided CLI handles everything — environment checks, API key setup, model selection, and bug picking:

cd repair_agent
python3 repairagent.py                    # Interactive wizard — walks you through everything

Or run directly without prompts:

python3 repairagent.py run --bugs "Chart 1, Math 5" --model gpt-4o-mini

That's it. The CLI will tell you if anything is missing and help you set it up.

Environment options

You need Java 11, Perl, and Defects4J installed. Pick whichever setup method is easiest for you:

VS Code Dev Container

1. Clone and prepare:

   hljs language-bash

   Copy

   git clone https://github.com/sola-st/RepairAgent.git
   cd RepairAgent/repair_agent
   rm -rf defects4j
   git clone https://github.com/rjust/defects4j.git
   cp -r ../data/buggy-lines defects4j
   cp -r ../data/buggy-methods defects4j
   cd ..
   

2. Open in VS Code, then click "Reopen in Container" (or Command Palette: Dev Containers: Reopen in Container).

3. Run:

   hljs language-bash

   Copy

   cd repair_agent
   python3 repairagent.py
   

Table of Contents

1. Usage
2. Requirements
3. Configuration
4. Analyzing Results
5. Replicating Experiments
6. Our Data
7. Contributing
8. Citation

Usage

Interactive mode

python3 repairagent.py

The wizard guides you through:

1. Environment check — verifies Python, Java, Defects4J, API keys
2. API key setup — configures OpenAI and/or Anthropic keys
3. Model selection — pick from available models
4. Bug selection — enter manually, pick from a project, or load a file
5. Run — executes the agent and shows a results summary

Direct mode (for scripting/CI)

# Single bug
python3 repairagent.py run --bugs "Chart 1" --model gpt-4o-mini

# Multiple bugs
python3 repairagent.py run --bugs "Chart 1, Math 5, Lang 22" --model claude-sonnet-4-20250514

# From a file
python3 repairagent.py run --bugs-file experimental_setups/bugs_list --model gpt-4o-mini

# In Docker
python3 repairagent.py run --docker --bugs "Chart 1" --model gpt-4o-mini

# Custom cycle limit
python3 repairagent.py run --bugs "Chart 1" --model gpt-4o --max-cycles 60

# Custom temperature
python3 repairagent.py run --bugs "Chart 1" --model gpt-4o --temperature 0.5

# Custom hyperparameters file
python3 repairagent.py run --bugs "Chart 1" --model gpt-4o-mini --hyperparams my_hyperparams.json

CLI flags

Setup only

python3 repairagent.py setup                  # Check environment & configure API keys
python3 repairagent.py setup --docker         # Build Docker image
python3 repairagent.py setup --install-deps   # Install all missing dependencies automatically

Shell script (advanced)

For users who prefer the original shell-based workflow:

./run_on_defects4j.sh <bugs_file> <hyperparams_file> [model]

What happens during a run

1. RepairAgent checks out the buggy project version from Defects4J.
2. The agent autonomously analyzes the bug, explores the code, and generates fix candidates.
3. Each candidate is applied and tested against the project's test suite.
4. Logs and results are saved to experimental_setups/experiment_N/ (auto-incremented).

Choosing the LLM model

The --model flag (or third argument to run_on_defects4j.sh) sets all LLM models used by RepairAgent:

• Main agent (fast_llm / smart_llm): drives the agent's reasoning loop
• Static/auxiliary (static_llm): used for mutation generation, fix queries, and auto-completion

For finer control, use environment variables:

export FAST_LLM=gpt-4o-mini       # main agent fast model
export SMART_LLM=gpt-4o           # main agent smart model
export STATIC_LLM=gpt-4o-mini     # auxiliary LLM calls

Configuration

hyperparams.json

Example:

{
  "budget_control": {
    "name": "FULL-TRACK",
    "params": { "#fixes": 4 }
  },
  "repetition_handling": "RESTRICT",
  "commands_limit": 40,
  "external_fix_strategy": 0
}

Analyzing Results

Output structure

Each run creates an experiment folder under experimental_setups/:

experimental_setups/experiment_N/
  logs/                  # Full chat history and command outputs (one file per bug)
  plausible_patches/     # Patches that pass all tests (one JSON file per bug)
  mutations_history/     # Mutant patches generated from prior suggestions
  responses/             # Raw LLM responses at each cycle
  saved_contexts/        # Saved agent contexts
  external_fixes/        # Fixes from external LLM queries (if enabled)

Unified overview

The experiment_overview.py script provides a single consolidated report across all experiments:

cd experimental_setups

# Analyze all experiments
python3 experiment_overview.py

# Analyze a specific range
python3 experiment_overview.py --start 1 --end 10

# JSON output for scripting
python3 experiment_overview.py --json

This produces:

• Grand totals (bugs tested, fixed, plausible patches, queries)
• Per-experiment summary table
• Per-project breakdown
• Per-bug detail with fix status, plausible status, iteration count
• Lists of fixed and plausible-only bugs

Individual analysis scripts

Replicating Experiments

Defects4J

1. Generate execution batches:

   hljs language-bash

   Copy

   python3 get_defects4j_list.py
   

   This creates bug lists under experimental_setups/batches/.

2. Run on each batch:

   hljs language-bash

   Copy

   ./run_on_defects4j.sh experimental_setups/batches/0 hyperparams.json gpt-4o-mini
   

   Replace 0 with the desired batch number. Batches can run in parallel.

3. Analyze results using experiment_overview.py or the individual scripts above.

4. Generate comparison tables (Table III in the paper):

   hljs language-bash

   Copy

   cd experimental_setups
   python3 generate_main_table.py
   

5. Draw Venn diagrams (Figure 6 in the paper):

   hljs language-bash

   Copy

   python3 draw_venn_chatrepair_clean.py
   

GitBug-Java

1. Prepare the GitBug-Java VM (~140 GB disk). See: https://github.com/gitbugactions/gitbug-java
2. Copy RepairAgent into the VM.
3. Run with experimental_setups/gitbuglist as the bugs file.
4. Analyze results using the same scripts.

Our Data

In our experiments, RepairAgent fixed 164 bugs on the Defects4J dataset.

Note: RepairAgent encountered middleware exceptions on 29 bugs, which were not re-run.

Contributing

If you find issues, bugs, or documentation gaps, please open an issue or email the author.

Running the tests

The unit-test suite is fast and needs no Java, Defects4J, or API key — the heavy runtime stack is stubbed (see repair_agent/tests/conftest.py). It runs on every push and pull request via GitHub Actions.

cd repair_agent
python -m venv .venv-test && . .venv-test/bin/activate
pip install -r requirements-dev.txt

# Run the suite
pytest tests -q

# With coverage (gate: the curated module set in .coveragerc must stay >= 70%)
coverage run -m pytest tests -q && coverage report -m --fail-under=70

Citation

If you use RepairAgent in your research, please cite:

@inproceedings{bouzenia2024repairagent,
  title={RepairAgent: An Autonomous, LLM-Based Agent for Program Repair},
  author={Bouzenia, Islem and Pradel, Michael},
  booktitle={Proceedings of the 47th International Conference on Software Engineering (ICSE)},
  year={2025},
  url={https://arxiv.org/abs/2403.17134}
}