← All work

2026

LLMs & Agents, NLP, Deep Learning

FinSight-3BFine-Tuned Finance LLM

Fine-tuned Llama-3.2-3B into a finance specialist with QLoRA, trained in about 20 minutes on a 16GB Mac mini with MLX. I then evaluated it rigorously on the FLARE benchmark and served it fully locally.

In numbers

0.617 → 0.748
Financial PhraseBank weighted-F1 (+21%)
0.622 → 0.801
FiQA-SA weighted-F1, matching published GPT-4
0.43%
of parameters trained (13.9M LoRA)

Screenshots

Fig. 1: FinSight-3B vs base Llama-3.2-3B on the FLARE test splits (dotted lines: published FinBERT / GPT-4 / FinMA-7B scores)
Fig. 2: QLoRA training curve on the M4 Mac mini

click any figure to enlarge ↗

What I did

  1. 1.

    Fine-tuned Meta's Llama-3.2-3B-Instruct with QLoRA (rank 16, 13.9M trainable parameters, 0.43% of the model) entirely on a 16GB M4 Mac mini using Apple MLX: about 20 minutes of training, peaking at ~3.4GB of memory.

  2. 2.

    Built the data pipeline from FLARE and FinGPT: normalised labels, balanced classes and de-leaked training data against every frozen test split, removing 16,596 headline rows that would otherwise have inflated the score through memorisation.

  3. 3.

    Evaluated with parser-free log-likelihood rank classification and an identical prompt for base and fine-tuned models: Financial PhraseBank weighted-F1 0.617 → 0.748 (macro-F1 0.582 → 0.752) and FiQA-SA 0.622 → 0.801, on par with published GPT-4.

  4. 4.

    Trained a second LoRA adapter for open-ended finance Q&A and evaluated it with ROUGE-L plus a position-bias-mitigated LLM-as-judge with bootstrap confidence intervals, then debugged it from losing to the base model (win-rate 0.31) to a statistical tie (0.50, 95% CI 0.42–0.58).

  5. 5.

    Fused and quantised the model to 4-bit, measured the quantised model on FLARE too, and served it four ways: an OpenAI-compatible MLX server, a FastAPI service, a Streamlit side-by-side demo and Ollama via GGUF. Private, $0 inference.

Notes

General-purpose LLMs are surprisingly bad at financial language: "costs fell" or "profit rose to EUR 13.1mn" read as neutral to a base model, while an analyst reads them as positive. FinSight-3B closes that gap with parameter-efficient fine-tuning on consumer hardware. It's the training counterpart to my RAG work.

What makes the numbers trustworthy

  • –Fair baseline: the untuned model gets the identical system prompt, query and scoring.
  • –No leakage: any training row whose text appears in a FLARE test split is dropped (16,596 rows).
  • –Parser-free scoring: candidates are ranked by log-probability, so a chatty answer is never mis-scored.
  • –Imbalance surfaced: macro-F1 and confusion matrices, not just accuracy, on a ~60%-neutral dataset.
  • –Quantisation measured, not assumed: the deployed 4-bit model is benchmarked too.

The generative Q&A story

  • –v1 lost to the base model (win-rate 0.31). The eval's own length signal showed the fine-tune had become too terse (64 vs 151 words).
  • –v2 switched to thorough training answers, an 8-bit base, a general-data replay and a repetition penalty, and reached a statistical tie (0.50) with matched lengths and a slight ROUGE-L edge (0.140 → 0.149); an independent re-run reproduced it.

Honest limits: a 3B model sits below FinBERT and FinMA-7B in absolute terms, the headlines task didn't improve on weighted-F1, and FiQA-SA's test set is small (235 rows). The point is the measurable lift from on-device domain fine-tuning.

Built with

Python · MLX · QLoRA · Llama-3.2-3B · Hugging Face · FastAPI · Streamlit · Ollama · GGUF

Next project

ROGII Geosteering →