Security and data handling

Built so the grade can be trusted, and the data cannot leak

What protects the answer keys, the grader and the tables you upload.

  • Answer keys never leave the factoryEach world's sealed specification (planted causes, reference answer) lives only in the compute service; the grade endpoint returns a reward, not the answer. Packs carry the files an agent sees, DATASHEET.md (aggregates) and labels/worlds.csv: per-world kind, difficulty, control scores and, for workflow worlds, the required decision. The labels are for your harness and must be kept away from agents.
  • The verifier runs apart from the agentHarbor tasks declare a separate verifier sandbox that receives only the agent's report or action log, with network access to the grader's host alone. An agent that tampers with its container never reaches the verifier or its key.
  • Grade-only, attempt-limited keysThe key inside a verifier can grade and see the reward, nothing else. An attempt limit per world turns further grades into replays, so a held-out key cannot be used as an oracle.
  • Rate limits and honest failuresGrades are rate-limited per key. When the grader cannot answer, the verifier retries and then records an errored trial, so an outage never enters a training batch as a score.
  • Your table stays yoursUploads go straight to the compute service over a signed, single-use link. Identifiers, free text and dates are set aside at upload; protected attributes are never planted as causes.
  • Deterministic and reproducibleA world regenerates bit for bit from its sealed seed, and grading is counted in compute steps rather than seconds, so the same report always scores the same.
  • Shared links show aggregatesA read-only evidence link shows a batch's datasheet and agent results. It shows only the agents the owner chooses, carries no world files, answer keys, key ids or per-world rewards, never shows an agent graded on fewer than 5 worlds or a per-kind or workflow rate over fewer than 5 worlds or pairs, withholds the cause and scenario distributions of fewer than 10 worlds, expires, and can be revoked.

A SOC 2 report and single sign-on are not yet available. One deployment serves one customer, with one owner account; ask for a dedicated deployment in your own cloud during a pilot.