Seeking Ada-based SPEC CPU benchmark candidates

General
1

Hi.

I’m Van Smith, chair of the SPEC CPU committee. We create computer benchmarks which are widely used in the computing industry, academia, media, and governmental organizations. Our benchmarks mainly target servers and workstations, but we are even considering Android compatibility with our next generation product, CPUv8.

The Biden Administration recently published statements declaring an urgent need for a shift to memory safe programming languages to curb future cybersecurity threats. The NSA later followed up with a list of memory safe programming languages which, of course, includes Ada, perhaps the most secure program language in existence today. See:

I am looking for widely used, compute intensive, command line based, open source, memory safe Ada applications that would make suitable CPUv8 benchmark candidates. It is advantageous if application performance scales with thread resources. If you are willing to create the benchmark, there are monetary awards. See:

https://www.spec.org/cpuv8/

Inclusion in the next generation of SPEC CPU could possibly benefit the Ada community. Over the last several decades, few if any server, workstation, desktop, or laptop microprocessors have been developed without targeting SPEC CPU benchmarks for performance optimization and stability testing. Compiler developers also routinely use SPEC CPU benchmarks as primary optimization targets. Inclusion of Ada benchmarks in SPEC CPUv8 might elevate public awareness of the elegance and power of the programming language at this potentially timely transition to memory-safe coding.

Keep in mind that the CPUv8 step process is challenging and only a minor fraction of benchmark submissions will make it into the final release. Additionally, CPUv8 is entering late stages of development, which will make it even more difficult for newly introduced benchmarks to be accepted.

My post here is on my own initiative and it is not a formal action of the Committee. However, I believe that CPUv8 needs representative memory safe benchmark candidates. Due to its mature cross-platform support and Ada’s near-legendary status for safety and security, it might be possible and worthwhile for the SPEC CPU committee to spend time integrating Ada benchmarks into CPUv8.

If you are interested or simply have suggestions, please let me know. Again, this post is my personal initiative. However, it might be impossible for SPEC CPU to support Ada at this late stage of CPUv8 development, or the Committee might immediately reject the idea, so keep that in mind.

2 Likes
2

GHDL maybe?

2 Likes
3

Hi Van,

Thank you for contacting the forums. I have not taken a good look at the tasks and goals, but I would like to add a few comments to your post.

  1. Ada can be quite performant, being close if not beating C to it. However, most Ada code does not tend to be overly optimised. You can see the issue in the Language Benchmark Game, where Ada is left behind by C/C++, Rust and others… This is not because of inefficiencies in the language, but rather a lack of effort/time/interest from the Ada community, we are a much smaller one sadly…

  2. If raw-sheer performance is what you look for, Ada has to be compiled without checks and one needs to be careful with some constructs. The Ada compiler will generate correct code. If undefined-behaviour can lead to better performance at the expense of potentially generating unexpected results, Ada tends to avoid that behaviour.

    a. Though the general consensus is to enable checks unless a performance hit is indeed measurable.

  3. Ada can use parallel tools/standards to improve its performance, such as OpenMP. Here is one such example: parasail/lwt at main · parasail-lang/parasail · GitHub

  4. Ada’s tasks are threads (in GNAT), so, theoretically, any basic Ada program that uses tasks could very easily scale, though most applications do not see a need to use them as they are simple and quick enough, plus tasks are heavily penalised in embedded runtimes (Ravenscar or Jorvik) and SPARK…

    a. Also green-threads are better for some tasks (like waiting for I/O).

  5. You can take a look at the GitHub - ohenley/awesome-ada: A curated list of awesome resources related to the Ada and SPARK programming language repository, where a long list of Ada projects are present, maybe one does pick your interest.

    a. Here are a few that I found that could potentially showcase Ada’s speed:
    i. GitHub - pyjarrett/septum: Context-based code search tool
    ii. GitHub - mosteo/mandelbrot_ascii: Mandelbrot renderer in "ASCII" (unicode actually, but text nonetheless)
    iii. paraffin download | SourceForge.net
    iv. GitHub - jhumphry/Reqrep_Task_Pools: This is an Ada 2012 package that provides a task pool system for jobs which each take the form of a single request that receives a single response.
    v. Fuzzy machine learning framework download | SourceForge.net
    vi. https://unzip-ada.sourceforge.io/

I will take a closer look at the objectives. I see there is a potential economical incentive, so maybe someone steps up…

Thank you for contacting the community.

2 Likes