Verifier
The advantage is not that grading happens in a sandbox; plenty of things run in sandboxes. It is that the provider cannot influence the grader, and that this is demonstrable rather than asserted.
Pinned environment
Contracts pin this image by digest. A tag would let the graded environment drift after funding.
How a submission is graded
Steps 2 and 5 are deliberately redundant. The guard already rejects grader-path edits, so the purge should never find anything, but a defense that depends on one check being correct fails when that check is wrong.
-
1
Materialize the pinned base treegit archive emits tree contents only, so .git never exists to leak a reference solution.
-
2
Guard every touched pathA protected- or grader-path violation is a hard reject and the pinned commands never run.
-
3
Apply the provider diffAllowed source paths only, as explicit file changes, never a shell patch of attacker-controlled input.
-
4
Quarantine provider test hookssrc/conftest.py sits inside an allowed path and pytest would still execute it. Allowed to write is not allowed to grade.
-
5
Purge grader paths, inject the buyer's bundleThe graded bytes are the buyer's, whatever the provider submitted.
-
6
Run only the pinned commandsargv vectors with no shell, in a rebuilt environment with no inherited secrets.
-
7
Install the runtime grader guardAn audit hook loaded outside the workspace stops provider code reading the graded tests. Blocking edits was not enough: code that reads them can answer from them without implementing anything.
-
8
Hash the tree and bind the resulttree_hash and submission_sha go into the receipt, so payment is for that exact artifact.
Each bit of the exit code is one destination. The change from 21 to 17 is the whole measurement: the same probe, run twice, against two configurations.
Network posture: measured
A probe executed inside a real Cloud Run Job, encoding each result as a bit of its exit code. Cloud Run surfaced neither stdout nor stderr, so the exit code was the only channel; a loopback control bit was included so a broken probe reports itself instead of masquerading as a passing guarantee.
| Destination | Default Cloud Run | Sealed VPC |
|---|---|---|
| loopback (control) | reachable | reachable |
| 1.1.1.1:443 | reachable | blocked |
| 142.250.72.46:443 | blocked | blocked |
| 93.184.216.34:80 | blocked | blocked |
| DNS resolution | reachable | reachable |
The first configuration reached the public internet. Cloud Run grants egress by default. An earlier version of this system asserted default-deny while the deployed job could reach Cloudflare, and because the manifest writes that field into a signed receipt, it would have signed a false statement. The claim is now deny-tcp-egress; dns-resolution-available. DNS still resolves and is disclosed as a residual signalling channel rather than rounded up.
Anti-gaming defenses
Each attack below is executed against a real repository with a real pytest process and asserted to fail. Mocking the runner would prove nothing; the grade has to actually be computed.
Scope: a run proves whether the submission satisfied the buyer's pinned contract. It does not assess code quality, security, or mergeworthiness.