ReadPaper Blog
Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution
Code2LoRA addresses the problem that code language models need repository-level knowledge of imports, APIs, and project conventions, while long-context retrieval and per-repository fine-tuning are costly and fragile as code changes. The paper proposes a hypernetwork that turns repository code, or a stream of code diffs, into LoRA adapters for a frozen code model, injecting repository knowledge into parameters with zero inference-time token overhead. On RepoPeftBench, a benchmark of 604 Python repositories, Code2LoRA-Static and Code2LoRA-Evo outperform strong parameter-efficient and context-injection baselines on static and evolving repository tasks.
Source: Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution
Mission Briefing: Why Codebases Are Hard
The paper starts from a practical limitation of code language models: real repositories contain distributed knowledge across files, imports, APIs, tests, and project-specific conventions that cannot be reliably inferred from a single local snippet. Current repository-aware systems often supply this knowledge through long inputs, using retrieval-augmented generation or dependency analysis to bring relevant files into the prompt for every query. The authors argue that this creates repeated inference-time cost, stresses context windows, and depends on retrieval quality when repository context is massive. Per-repository fine-tuning or LoRA adaptation moves knowledge into parameters, but the paper frames that alternative as expensive at repository scale and brittle when commits change the codebase. Code2LoRA is motivated as a way to preserve repository-specific knowledge without repeatedly paying for long prompts or retraining a new adapter after every change.
The Gap: Existing Tools Are Too Static
The related-work discussion positions Code2LoRA between parameter-efficient fine-tuning, hypernetwork-generated adapters, and repository-level code understanding. LoRA, QLoRA, DoRA, weight merging, multi-LoRA routing, LoRACode, and MoLE provide efficient adaptation mechanisms, but the paper emphasizes that they generally treat adapters as static artifacts trained per task, language, or repository. Earlier hypernetwork approaches such as Text2LoRA and Doc2LoRA generate LoRA weights from short task descriptions or single documents, but the authors identify a gap for long, structured, changing software repositories. Repository-level generation methods such as RepoFusion, RepoCoder, RepoFormer, CoCoMIC, R2C2-Coder, and RepoHyper typically route cross-file knowledge through input context or retrieval. Code2LoRA’s central distinction is to generate repository-conditioned adapters directly, so adaptation can generalize to unseen repositories and, in the Evo setting, update as software evolves commit by commit.
Code2LoRA: The Hypernetwork Technique
Code2LoRA is a hypernetwork framework that generates repository-specific LoRA adapters for a frozen code language model, with the paper’s experiments based on Qwen2.5-Coder-1.5B. The architecture has a frozen repository encoder, a trained hypernetwork, and a frozen base LLM that receives the generated adapter for downstream inference. Repository context is compressed using a frozen Qwen3-Embedding-0.6B model: files or diffs are chunked into 4096-token segments with 512-token overlap, embedded, mean-pooled into file vectors, and aggregated into a repository embedding through weighted mean and max pooling. Code2LoRA-Static maps this repository embedding to LoRA matrices for the q, k, v, o, gate, up, and down projection types using a two-layer MLP with GELU, L2 normalization, tanh output heads, and learnable log-scales. The paper trains only the hypernetwork under a standard language-modeling loss, making the generated adapters a parameter-space carrier of repository knowledge rather than an added prompt payload.
Two Forms: Static and Evo
The paper defines two usage scenarios according to when repository knowledge is refreshed. Code2LoRA-Static converts one repository snapshot into a single adapter, which is intended for stable codebases or static comprehension tasks where the snapshot remains a valid source of imports, APIs, and conventions. Code2LoRA-Evo extends the static formulation with a GRU that aggregates sequential code-diff embeddings into a hidden state before adapter generation. This recurrent design maintains an adapter trajectory over the repository lifetime, so each commit can update the repository-specific representation without replacing the original snapshot prior. The authors present this as a hypernetwork formulation for software evolution, where the unit of adaptation is not a global model checkpoint but a repository-specific stream of code changes.
Proof in Battle: RepoPeftBench Results
The empirical evidence comes from RepoPeftBench, a benchmark curated by the authors with 604 Python repositories, including 512 in-distribution repositories and a 92-repository temporal holdout created after the scrape cutoff. The benchmark has a static track with 39,612 training and 11,636 test assertion-completion tasks, and an evolution track with 215,129 training and 86,793 test tasks derived from commit history. Evaluation includes in-repo and cross-repo splits, plus temporal out-of-distribution testing under the commit-derived protocol. On the static track, Code2LoRA-Static achieves 63.8% cross-repo exact match and 66.2% in-repo exact match, matching the per-repository LoRA upper bound while avoiding per-repository training. On the evolution track, Code2LoRA-Evo reaches 60.3% cross-repo exact match, improving by 5.2 percentage points over a single shared LoRA and remaining the strongest method on the temporal OOD holdout according to the excerpt.
