Quickstart
Everything below runs on synthetic data generated in-process. You do not need a dataset, credentials, a GPU, or a network connection.
Install
Python 3.9 or newer. The only required runtime dependency is numpy.
git clone https://github.com/RussellYSW/OpenMed.git
cd OpenMed
pip install -e ".[dev]"
Confirm the install:
python -m pytest -q
cryptography is optional and not installed by
".[dev]", so the default is HMAC. With it installed, the
ledger signs blocks with Ed25519.
trustfed.ledger.HAVE_CRYPTOGRAPHY tells you which you got. The HMAC
fallback gives integrity under a shared secret — it is not a signature, gives no
non-repudiation, and a verifier can forge blocks. The default key is a constant in
the source and is for demos only.
1. The defense demo
This is the shortest path to seeing what the project is for. Several sites train one Parkinson's rapid-decline classifier; some of them are malicious.
python examples/parkinson_decline/run_demo.py
It walks through, in order:
- A clean baseline — what the model scores when every site is honest.
- The same run under poisoning with
fedavg, which has no Byzantine tolerance. It degrades under the scaling attack, and collapses outright under sign-flip and adaptive attacks. - The same attack against
coordinate_median,trimmed_mean, andmulti_krum, which hold near the clean baseline. - An adaptive attacker that knows which defense is running — the row that separates defenses which actually work from defenses that only survive naive attacks.
- The attestation gate on, so code-tampered sites are rejected before aggregation.
- A quality report on the resulting model bundle, as a reviewer would read it.
The demo's quality report ends in FAIL on purpose. The synthetic model
has a subgroup performance gap and an incomplete model card, and the gate is
supposed to say so. A gate that passes everything is not a gate.
2. The model commons demo
The trust plane end to end — publish, certify, derive, verify, credit.
python examples/model_commons/run_demo.py
It demonstrates:
- Publish a model bundle to the registry. The publish event is written to the ledger.
- Certify it — and watch the authority refuse the submitting site's own signature, then accept once reviewers from distinct institutions sign.
- Derive a fine-tuned model from the certified base. The registry links it to its parent automatically.
- Verify lineage — one call walks the derived model's full ancestry and reports whether every step is attested.
- Credit — contributors accrue standing, and the reciprocity policy refuses an evaluation request from a site that has never served as an evaluator, with a machine-readable reason.
- Tamper evidence — the demo edits a block on disk and re-verifies. Verification fails and names the reason (
payload_mutated).
3. The benchmark grid
Compare every aggregator against the default attack set on identical scenarios:
python -m trustfed.benchmark
The default grid is 49 scenarios: 7 aggregators × 4 attacks × 2 Byzantine
counts. --attacks selects the others (gaussian,
scaling, model_replacement, feature_corruption).
The run is deterministic from its seed and finishes quickly. It prints a results table
plus the documented tolerance bound for each rule, so a poor cell can be read
correctly: if a scenario exceeds a rule's proven tolerance, a bad score there is
expected behaviour, not a defect. Reports serialise to JSON, and
compare_reports() diffs two runs.
If you are contributing a new defense, this is the bar: add it to
AGGREGATORS, run the grid, and put the numbers in your pull request.
Everyone's results come from the same harness.
Using the API
A minimal federated run with a robust aggregator and the attestation gate on:
from trustfed.data import make_federated_parkinson
from trustfed.federated import Client, Server, APPROVED_CODE_IDENTITY
from trustfed.aggregation import multi_krum
from trustfed.attestation import MockSoftwareAttestor, measure_code
sites, X_test, y_test = make_federated_parkinson(n_sites=6, seed=0)
attestor = MockSoftwareAttestor(
root_key=b"demo-root-key",
approved_measurements=[measure_code(APPROVED_CODE_IDENTITY)],
)
clients = [Client(s.site_id, s.X, s.y, attestor=attestor) for s in sites]
server = Server(
n_features=X_test.shape[1],
aggregator=multi_krum,
attestor=attestor,
n_byzantine=1,
)
for r in server.fit(clients, X_test, y_test, rounds=10):
print(f"round {r.round:>2} auc={r.auc:.3f} "
f"accepted={r.n_accepted} rejected={r.n_rejected}")
Certifying a model — note that the authority refuses the submitting institution's own signature:
from trustfed.ledger import InMemoryLedger
from trustfed.certification import (
CertificationAuthority, ReviewerKeyring, Reviewer,
ThresholdPolicy, ConflictOfInterestError,
)
keyring = ReviewerKeyring(seed=b"demo-reviewer-keys")
keyring.register(Reviewer("rev_a1", "INST_A", role="ml"))
keyring.register(Reviewer("rev_b1", "INST_B", role="clinical"))
keyring.register(Reviewer("rev_c1", "INST_C", role="clinical"))
authority = CertificationAuthority(
keyring,
ledger=InMemoryLedger(),
policy=ThresholdPolicy(k=2, min_institutions=2),
)
authority.submit("bundle-123", owner_institution="INST_A", submitted_by="rev_a1")
try:
authority.review("bundle-123", "rev_a1", "approve", statement="ship it")
except ConflictOfInterestError as exc:
print("refused:", exc)
# refused: [self_certification] rev_a1 belongs to the submitting
# institution 'INST_A'
authority.review("bundle-123", "rev_b1", "approve", statement="protocol checks out")
authority.review("bundle-123", "rev_c1", "approve", statement="subgroups reviewed")
case = authority.certify("bundle-123")
print(case.state) # CertificationState.CERTIFIED
print(authority.verify_log().ok) # True
See architecture for the full component map and the interfaces to implement when extending any of this.
Next steps
- Read the architecture to see how the pieces connect.
- Read the governance draft — especially if you disagree with it.
- Find something to work on in where we need help.
Troubleshooting
| Symptom | Cause and fix |
|---|---|
ModuleNotFoundError: trustfed |
Install in editable mode from the repository root: pip install -e ".[dev]" |
| Ledger tests skipped | cryptography is not installed. Expected — the Ed25519 tests skip rather than fail. Install it to run them. |
Quality report says FAIL |
Expected on the demo bundle. The synthetic model really does have a subgroup gap. |
| A benchmark cell looks bad | Check the tolerance table printed underneath. The scenario may exceed that rule's documented bound. |
| Something else | Open an issue. Quickstart friction is treated as a defect. |