---
title: "Fine-tuning Qwen2.5-1.5B without gradients — Tanuj Dargan"
description: "Can evolutionary strategies, perturbing LoRA weights and keeping what scores well, fine-tune a 1.5B model on math with nothing but a binary reward? And can a cheap gradient-linearised screen pick which perturbations are worth a full evaluation?"
canonical: https://tanujdargan.com/research/es-lora
resume: on request
updated: 2026-11-07
---

# Fine-tuning Qwen2.5-1.5B without gradients

- Author: Tanuj Dargan (https://tanujdargan.com/)
- Where: Independent
- When: May 2026
- Tags: evolution strategies, LoRA, GSM8K
- Web page (with live demo): https://tanujdargan.com/research/es-lora

> Status: the LoRA adapter is published on Hugging Face. Accuracy numbers are not published yet.

## Question

Can evolutionary strategies, perturbing LoRA weights and keeping what scores well, fine-tune a 1.5B model on math with nothing but a binary reward? And can a cheap gradient-linearised screen pick which perturbations are worth a full evaluation?

## Method

- Qwen2.5-1.5B base, LoRA rank 16 on all attention and MLP projections. The reward is exact-match on the boxed answer for GSM8K.
- Each iteration: sample 45 Gaussian perturbations (σ = 1e-3), screen all 45 with a gradient-linearised proxy (−σ·g·ε), evaluate only the top 8 on real reward, z-score those rewards, and step θ ← θ + (α/8)·Σ zₖεₖ with α = 5e-4. Run 15 iterations.
- Ablation: proxy-screened candidates against randomly selected ones on the same problems. If the proxy doesn't beat random, the screening idea is dead.

## Results

- Adapter published on Hugging Face. Accuracy deltas pending confirmation.

## Next

- Report proxy-vs-random with confidence intervals on the 200-problem split.
- Antithetic sampling and rank-based fitness shaping before scaling the population.

## Links

- [Adapter on HF](https://huggingface.co/tanujd/qwen2.5-1.5b-es-lora-math)

---

Profile: https://tanujdargan.com/index.md · Resume: on request (https://tanujdargan.com/#resume or dargantanuj@gmail.com) · All write-ups: https://tanujdargan.com/llms.txt