Fine-tuning 8b AI model on Ada/SPARK

[bladeacer]

So for context I recently got my hands on a laptop with 8 GB VRAM running an NVIDIA graphics card.

I experimented a bit with local models that fit my hardware constraints, though it is safe to say that local models at this range are frankly quite terrible at Ada. I have tried qwen3:8b, gemma4:e4b and the like. They are ok at the common languages like Python but I suppose their training dataset is lacking on Ada.

I tried this on ollama + pi coding agent on WSL, which is probably one of the lighter harness set ups one can use.

Even with AdaCore’s provisioned skills the model is prone to hallucinations and not using Alire properly.

Thus I plan to start working on a finetune for qwen3:8b for it to better understand Ada projects. If you are interested, do drop your repository links if you wish for your repository to be used for training during the fine-tuning process. GitHub technically uses all public GitHub repositories for their AI model training but we are a bit more ethical than that.

Please check that your repository has generally permissive licensing or you explicitly make an exception for use in model training. I do not want AGPLv3 spilling over when the finetuned model should be as permissive as possible for the benefit for everyone (I am thinking Apache 2.0)

Once the finetune is done I plan to have it released on HuggingFace as open weights (not open source, as the underlying Qwen model like almost every other AI model is open weights). That being said, I am to be as transparent with fine-tuning dataset weights and all as possible.

I also do not just want to train it on Ada 2022, as Ada 2012/SPARK 2014 is still widely used out there. The end goal is to have a finetuned model suitable for working on Ada that runs on consumer hardware.

P.S. corrected model name to qwen3:8b instead of qwen3.8, I cannot run qwen3.8 (at least it’s 9b variant iirc) that well with enough context window size to be useful in 8 GB VRAM.

[bladeacer]

Hmm it seems there has been similar fine-tuning done earlier on, I will take inspiration and learn from it

Steelman — a 14B model fine-tuned for Ada 2022 code generation (runs locally)

Hey all — I wanted to share a project I’ve been working on that I think this community might find useful. Steelman is a QLoRA fine-tune of Qwen2.5-Coder-14B-Instruct, trained specifically on compiler-verified Ada 2022 code. It runs locally (Ollama, llama.cpp, etc.) and doesn’t need a cloud API. I built it because every frontier model I tried was genuinely bad at Ada. Claude, GPT, Gemini — they all produce code that looks plausible but won’t compile. So I started generating Ada pairs, verifying…

[VMo]

Hi !

This is an interesting project. Last year I experimented with DeepSeek and upgraded my laptop to 64 Gb / 2Tb for this (before memory got so expensive). Compared to what Opus, Fable and the like offer, it was disappointing. If you want some Ada 83 training do not hesitate to access my Ada 83 compiler sources if helpful.

If you have an established project environment tell where it is and if you need some help.

[bladeacer]

Thanks , Ada 83 would be great to have in the training dataset. As for project environment wise I will start setting it up at a later time. Obtaining quality training data iirc is the most difficult and time consuming part of model fine-tuning/training. I could use synthetic data and other techniques later one but a decently sized corpus of quality source code by the community that is clear/specific to implementation and use case would help immensely.

My educated guess here is that if I can make an 8B model write Ada well larger parameter models will run even better, though we are still some time away from there.

From what I heard models like DeepSeek are pretty good but the local model scene is moving rather quickly even just within this year. I believe there are better models you can now run in 64 GB VRAM (if that is what you have).