Quickstart

Everything below runs on synthetic data generated in-process. You do not need a dataset, credentials, a GPU, or a network connection.

Install

Python 3.9 or newer. The only required runtime dependency is numpy.

git clone https://github.com/RussellYSW/OpenMed.git
cd OpenMed
pip install -e ".[dev]"

Confirm the install:

python -m pytest -q

cryptography is optional and not installed by ".[dev]", so the default is HMAC. With it installed, the ledger signs blocks with Ed25519. trustfed.ledger.HAVE_CRYPTOGRAPHY tells you which you got. The HMAC fallback gives integrity under a shared secret — it is not a signature, gives no non-repudiation, and a verifier can forge blocks. The default key is a constant in the source and is for demos only.

1. The defense demo

This is the shortest path to seeing what the project is for. Several sites train one Parkinson's rapid-decline classifier; some of them are malicious.

python examples/parkinson_decline/run_demo.py

It walks through, in order:

  • A clean baseline — what the model scores when every site is honest.
  • The same run under poisoning with fedavg, which has no Byzantine tolerance. It degrades under the scaling attack, and collapses outright under sign-flip and adaptive attacks.
  • The same attack against coordinate_median, trimmed_mean, and multi_krum, which hold near the clean baseline.
  • An adaptive attacker that knows which defense is running — the row that separates defenses which actually work from defenses that only survive naive attacks.
  • The attestation gate on, so code-tampered sites are rejected before aggregation.
  • A quality report on the resulting model bundle, as a reviewer would read it.

The demo's quality report ends in FAIL on purpose. The synthetic model has a subgroup performance gap and an incomplete model card, and the gate is supposed to say so. A gate that passes everything is not a gate.

2. The model commons demo

The trust plane end to end — publish, certify, derive, verify, credit.

python examples/model_commons/run_demo.py

It demonstrates:

  1. Publish a model bundle to the registry. The publish event is written to the ledger.
  2. Certify it — and watch the authority refuse the submitting site's own signature, then accept once reviewers from distinct institutions sign.
  3. Derive a fine-tuned model from the certified base. The registry links it to its parent automatically.
  4. Verify lineage — one call walks the derived model's full ancestry and reports whether every step is attested.
  5. Credit — contributors accrue standing, and the reciprocity policy refuses an evaluation request from a site that has never served as an evaluator, with a machine-readable reason.
  6. Tamper evidence — the demo edits a block on disk and re-verifies. Verification fails and names the reason (payload_mutated).

3. The benchmark grid

Compare every aggregator against the default attack set on identical scenarios:

python -m trustfed.benchmark

The default grid is 49 scenarios: 7 aggregators × 4 attacks × 2 Byzantine counts. --attacks selects the others (gaussian, scaling, model_replacement, feature_corruption). The run is deterministic from its seed and finishes quickly. It prints a results table plus the documented tolerance bound for each rule, so a poor cell can be read correctly: if a scenario exceeds a rule's proven tolerance, a bad score there is expected behaviour, not a defect. Reports serialise to JSON, and compare_reports() diffs two runs.

If you are contributing a new defense, this is the bar: add it to AGGREGATORS, run the grid, and put the numbers in your pull request. Everyone's results come from the same harness.

Using the API

A minimal federated run with a robust aggregator and the attestation gate on:

from trustfed.data import make_federated_parkinson
from trustfed.federated import Client, Server, APPROVED_CODE_IDENTITY
from trustfed.aggregation import multi_krum
from trustfed.attestation import MockSoftwareAttestor, measure_code

sites, X_test, y_test = make_federated_parkinson(n_sites=6, seed=0)

attestor = MockSoftwareAttestor(
    root_key=b"demo-root-key",
    approved_measurements=[measure_code(APPROVED_CODE_IDENTITY)],
)

clients = [Client(s.site_id, s.X, s.y, attestor=attestor) for s in sites]

server = Server(
    n_features=X_test.shape[1],
    aggregator=multi_krum,
    attestor=attestor,
    n_byzantine=1,
)

for r in server.fit(clients, X_test, y_test, rounds=10):
    print(f"round {r.round:>2}  auc={r.auc:.3f}  "
          f"accepted={r.n_accepted}  rejected={r.n_rejected}")

Certifying a model — note that the authority refuses the submitting institution's own signature:

from trustfed.ledger import InMemoryLedger
from trustfed.certification import (
    CertificationAuthority, ReviewerKeyring, Reviewer,
    ThresholdPolicy, ConflictOfInterestError,
)

keyring = ReviewerKeyring(seed=b"demo-reviewer-keys")
keyring.register(Reviewer("rev_a1", "INST_A", role="ml"))
keyring.register(Reviewer("rev_b1", "INST_B", role="clinical"))
keyring.register(Reviewer("rev_c1", "INST_C", role="clinical"))

authority = CertificationAuthority(
    keyring,
    ledger=InMemoryLedger(),
    policy=ThresholdPolicy(k=2, min_institutions=2),
)

authority.submit("bundle-123", owner_institution="INST_A", submitted_by="rev_a1")

try:
    authority.review("bundle-123", "rev_a1", "approve", statement="ship it")
except ConflictOfInterestError as exc:
    print("refused:", exc)
    # refused: [self_certification] rev_a1 belongs to the submitting
    #          institution 'INST_A'

authority.review("bundle-123", "rev_b1", "approve", statement="protocol checks out")
authority.review("bundle-123", "rev_c1", "approve", statement="subgroups reviewed")

case = authority.certify("bundle-123")
print(case.state)                    # CertificationState.CERTIFIED
print(authority.verify_log().ok)     # True

See architecture for the full component map and the interfaces to implement when extending any of this.

Next steps

Troubleshooting

SymptomCause and fix
ModuleNotFoundError: trustfed Install in editable mode from the repository root: pip install -e ".[dev]"
Ledger tests skipped cryptography is not installed. Expected — the Ed25519 tests skip rather than fail. Install it to run them.
Quality report says FAIL Expected on the demo bundle. The synthetic model really does have a subgroup gap.
A benchmark cell looks bad Check the tolerance table printed underneath. The scenario may exceed that rule's documented bound.
Something else Open an issue. Quickstart friction is treated as a defect.