2026-07-31T14:00:00Z→2026-07-31T15:00:00Z, more info here:
As to topics, I am very open. I finally had some time to benchmark open models against the commercial frontier models. Massive differences, happy to discuss, though I don’t understand the details sufficiently to truly publish them.
I’d also like to talk about crates. There are a lot of them; the Ada User Awards are enticing people to make more. What else is missing from the user experience or the list of capabilities?
Lastly, I’d love to talk about participation at Embedded World, curious to see who would like to submit a paper.
And of course, any other topics that people would like to bring up.
I’ll probably not be able to attend due to exam preparations but I have an interesting question that I’d like to ask you.
Do you think AI assisted tools should be used in applications that require DO-178C or ISO 26262 certifications? And if so, which stages should or should not use AI?
The context behind the question is a discussion I had with a friend. We were talking about the future of software development and how AI assisted development might be used in safety critical systems. He essentially said: “I would not fly a plane or drive a car that has been vibe coded. If the requirements, implementation and unit tests are all hallucinated we might have a safe system but that system could be bogus!”
But since both of us are not really familiar with both standards, we don’t know how real of a risk AI usage really is for such systems. Therefore, I’d like to ask you, someone who has years of experiences in both fields, what your opinion is on the matter.
With kind regards,
Ada Orbit
PS: A new “main” directory/index for especially qualitative Ada crates would be awesome! And I’m gonna continue working on my Ada User Awards Project once my exam phase is over.
With the proliferation of crates we might run into naming conflicts. Other package managers solve this by using e.g. reverse domain notation as a prefix. This might be worth considering for Alire 3.
Is the recording going to be published? I am quite interested in the topics discussed and sadly I could not attend in the end
Also, I am quite interested in your AI testing as in the past week we saw the release of several new open models or even new versions (new DeepSeek v4!) and the tone has changed a lot about AI economics because of the new open models… Interesting times ahead! It would be quite interesting to test the new models not only on how good they are but also in terms of time and costs
Absolutely! I had a couple of days of vacation, so a bit behind, but the video has been sent to our marketing team for upload and I’ll get a summary out tomorrow.
Apologies for the delay in posting this, but here is the recording from last weeks Ada SPARK Office Hours session, as a summary:
The meeting focused on integrating AI into high-integrity systems and benchmarking language models for software development efficiency.
AI in High-Integrity Systems
Industries are increasingly adopting AI in high-integrity systems by utilizing formal methods to ensure safety. Automating high-integrity software development could significantly reduce costs and enable broader accessibility.
Benchmarking AI Model Performance
Current evaluations of various models reveal that agentic loops effectively generate valid code but entail significant cost and duration concerns. Development of benchmarking frameworks remains ongoing to ensure consistent and accurate data analysis.
Infrastructure and Tooling Strategies
Running large models locally requires substantial hardware, making cloud endpoints a more practical alternative. Efforts are shifting toward publicizing automated code review tools and supporting industry engagement via conference presentations.
It was a quiet one, likely due to vacation and exams.
Some information about open models in the presentation, more to follow in the Office Hours session on August 14th.
The open models are really surprising me, in agentic mode, they are able to solve about the same amount of challenges as the commercial models. I think we should add some more challenges to the set. The top open models are not much cheaper compared to the commercial ones.
To answer @AdaOrbit 's question: Yes, people building high-integrity software are looking into how they can use AI to deliver safe and secure software faster. It is important to realize that safe and secure software is safe and secure due to the process that is being followed with rigorous review and testing. The question is how does AI fit into that while making sure that human engineers own the engineering choices and are own accountability.
This is a topic near and dear to the heart of people at AdaCore and we will have more to demonstrate on this topic soon.
I just watched the video. I am really looking forward to the final results, including the “new” models! I would recommend that you also add DeepSeek Flash (latest version) and maybe other smaller models (Qwen-3.8 28B) as those could be very interesting to your customers. The reason is because I have read that one could use a very good, slow/expensive/hard-to-run-locally models to create a detailed plan/design/features/structure/prototypes/function-signatures and then you delegate the specific to very simple/cheap/direct models that are easy to run locally such as DeepSeek Flash. Also, pretty much every day we are getting new open models that are >50B parameters everyday, which are more and more capable (like Inkling or Laguna, Hy3, MiniMax M3…).
Also, in your slides I would recommend you to list the model+reasoning level. For example, I found GLM 5.2 to be very verbose and do a lot of roundtrips if the reasoning is too high while providing little benefits sometimes.
Also, nice news about the JSON/SPARK code! Looking forward for the blog post.
Thank you for hosting these meetups/Office Hours :D! Best,
Fer
P.S: I have not forgotten about improving the skills with more examples, code, tools, etc
Basically, it is a small model that can be run locally and it is specialized in agentic coding. They show that paired with “smart” AIs you can achieve fairly complex tasks at basically half the price and much faster. It could be interesting to add this to your benchmark for agentic mode.
Yes, I have been including Nemotron. It does not do as well as the top open models (17/24), but it is a LOT cheaper.
However, one of the big learnings is that the smaller the model is, the more guidance you need to give it. For example, with Nemotron, to get any decent output, I had to combine compile, prove and format into a single command and tell the LLM to use that command. Letting it run by itself, it would frequently forget stuff.
