Measuring AI behavior

NeoMundi Research builds and qualifies a runtime measurement instrument for generative AI systems. Our object of study is the produced execution — the response a system actually returned, under conditions we recorded. Not the internals of a model, and not the truth of its content.

The goal is not to rank models. It is to qualify a measurement instrument through open, inspectable and repeatable experiments.

This Hugging Face profile is the scientific entry point: corpora, experimental material and reproducible verifications. Technical documentation lives on GitHub, which remains the reference.

We are not a certification body, a model provider, or a leaderboard. We publish no composite score and no model ranking.

Measurement → Station → Evidence

Three distinct functions, one chain.

The Python SDK and MCP server are access interfaces to the instrument, not additional stages.

Reproduce a verification

The Evidence verifier runs on a self-contained synthetic package committed to the repository. No network, no credentials, no cost. Python 3 standard library only.

git clone --depth 1 https://github.com/neomundi-io/neomundi-evidence-public.git
cd neomundi-evidence-public/examples/dossier-v1

# 1 — archive integrity
python ../../verify.py NEOMUNDI_EVIDENCE_V1_DEMO.zip --corpus corpus

# 2 — detached Ed25519 attestation
python ../../verify_attestation.py NEOMUNDI_EVIDENCE_V1_DEMO.zip.jws \
    --archive NEOMUNDI_EVIDENCE_V1_DEMO.zip \
    --trust-store trust-store.json --allow-test-keys

Expected output:

# 1
BILAN : 19 PASS, 0 FAIL, 3 NOT TESTED

# 2
BILAN : 10 PASS, 0 FAIL, 5 NOT TESTED
Niveau atteint : cryptographically_attested

Drop --allow-test-keys and step 2 reports 9 PASS, 1 FAIL: the demonstration key is refused by default, so it grants no attestation. The verifier reports in French.

The NOT TESTED lines are deliberate. They name what no local check can establish — export completeness, ledger chaining, archive origin, the signer's legal capacity, independent timestamping, and the truth of the content. A signature establishes commitment and custody, not fact.

Everything in that directory is synthetic: the archive was built for review and the signing key was ephemeral and destroyed. Full walkthrough: auditor's guide.

What a measurement contains

An unavailable signal is reported as unavailable. It is never replaced by zero, and interpretation stays with the consuming system.

Reference corpora

Frozen as 20261105T1200_GEL_CORPUS_GATE0 (5 November 2026), measurement_version 3.3.1.

Operators measured in MED: anthropic, cohere, deepseek, google, infomaniak, moonshot, openai. Two of the seven returned requested_only — the operator did not state which model actually served the request, so only the requested model is established.

Archive hashes, per-provider ledger head hashes and the verification procedure are declared in reference corpora.

Not yet hosted here. GROUNDED-S1 is packaged as a Hub dataset — 151 ledger records, projected to remove three fields that pointed back at a private source conversion — and held back pending a licence decision. MED's content is not redistributable at all. Publishing a corpus depends on redistribution rights and a licence, decided per corpus. We do not publish placeholder datasets, and no dataset is publicly available from this profile today.

Repositories

Source is public and inspectable on GitHub.

Contracts: metric contract · measurement interoperability · interpretation contract

Reuse terms are not granted yet. None of these repositories carries a LICENSE file. The code is published to be read, run and audited; redistribution and derivative use are not licensed. The licence decision is explicitly pending — see the SDK's licence notice. Ask before reusing: contact@neomundi.io.

Research

Research question. How can runtime signals help observe, compare and govern generative systems across real-world contexts?

Current work: runtime stability, factual validity, semantic risk, behavioral drift, cost and efficiency, professional use cases, red-team cases, and instrument qualification.

AI Observatory · metrology validation · non-circularity protocols · use cases and pilots

We collaborate on bounded, documented pilots. contact@neomundi.io

Limitations

These are properties of the instrument, not disclaimers appended to it.

• A measurement is not a verdict on content. It describes properties of a produced text. It does not establish that an answer is correct, medically sound, or legally compliant.
• No ranking. No composite score, no ordering of models or providers.
• End-to-end qualification is partial. The frozen Q-2 chain followed one authorized real execution through Measurement → Station → Evidence, with an Ed25519 signature verified independently against the published trust reference. The technical chain passed; the overall verdict remains PARTIAL on specific scientific and provenance guarantees. Q-2 is a preserved private reference, not a downloadable sample.
• Provenance has a ceiling. requested_only means the served model was never attested by the operator.
• Campaign identity is instrument-verifiable, not third-party reproducible. The recomputation recipe is unpublished. An external evaluation on 6 November 2026, holding the frozen data, concluded not established for one campaign. Treat campaign hashes as verifiable with the instrument and not without it.
• Tamper detection is scoped. 10/10 on the named falsification suite; other suites differ.
• Interface maturity differs. REST, SSE, SDK and MCP are not at the same qualification level — see integration status.
• Programme support is not endorsement. NeoMundi participates in the OVHcloud Startup Program and NVIDIA Inception, with Infomaniak as infrastructure partner. None of these implies certification or endorsement of results.

Links

github.com/neomundi-io — technical reference · neomundi.org — research · neomundi.io — platform · station.neomundi.org — live Station · data protection · contact@neomundi.io