A contributor hits submit and gets a permanent address back within seconds. That address is how they will later prove the contribution was theirs, claim ownership of it, and — if the data earns — draw the share that ownership entitles them to. Right now the file it points to has not reached Arweave. The address is already written into an on-chain transaction, where it cannot be changed.
That address resolves to a receipt: who made the contribution, what they made, when, and on what grounds it can be trusted. The receipt is a metadata document and it lives off chain. On chain there is only the address pointing at it and a hash, which proves the receipt has not been altered since.
XnY anchors data provenance, assembly relationships and revenue splits on chain¹, but the full metadata document never enters the contract. Only two things are stored there:
bytes32 metadataHash; // keccak256(raw metadata JSON bytes)
string metadataUri; // "arweave://<dataItemId>"arweave://<id> is an address bound to no particular gateway. On read it maps to the HTTP URL of some Arweave gateway, for example https://arweave.net/<id>. Anyone can later fetch the document by it, recompute the hash and compare against the chain, without having to trust any single Arweave gateway.
All five kinds of off-chain metadata are anchored by that same pair of fields, and measure 0.5–1.2 KB compactly serialised²:
| kind | What it holds |
|---|---|
frontier-metadata | The definition of a data domain: name, default policy, submission rules |
campaign-metadata | One collection drive: reward model, budget, eligibility, revenue split |
task-metadata | One concrete task: type, execution, acceptance rules, dependencies |
cf-metadata | One contribution: contributor DID, parent task, payload location, byte count |
dataset-metadata | One dataset version: CF list, contributor Merkle root, cipher suite |
All five share one envelope.
The URI has to be written into the calldata of the transaction that creates the on-chain object. Once that transaction is submitted it cannot be added to, and it cannot be swapped. Get it wrong and there is no second chance: that on-chain record will point forever at an address nothing can be fetched from, the contribution’s provenance cannot be verified, and the contributor cannot prove their claim on the revenue it earns.
Arweave, meanwhile, is asynchronous. The DataItem goes to Turbo — the upload service that accepts signed DataItems, bundles them and submits them to Arweave — and then waits minutes for network confirmation. And an ANS-104 DataItem’s ID is computed from its signature³, with RSA-PSS signing carrying a random salt⁴, so signing the same content a second time produces a different ID.
(Below, gateway service means XnY’s own service; Arweave gateway means the Arweave network’s public read entry point. They are not the same thing.)
What the gateway service does inside the request is therefore only to create an identity: construct and sign the DataItem locally, compute the ID, write the complete signed bytes into PostgreSQL and a separate BackupStore (object storage), and return the URI. Upload and confirmation are handed to the background.
The URI can be settled that early because identity and delivery are pulled apart. What follows takes each in turn: first how hard that identity is — what a 201 proves, and why the ID is decided by the signature; then how delivery is guaranteed — the queue and the failure paths; and finally why this document does not go into the contract.
What a 201 proves
| State | Already true | Not yet true |
|---|---|---|
accepted | Identity created, URI fixed, signed DataItem written to PostgreSQL and to a separate BackupStore | The API did not wait for Turbo to accept it |
uploaded | Turbo has accepted it; it has entered bundling | No Arweave network confirmation observed yet |
confirmed | The DataItem, or the bundle containing it, has been observed in an Arweave block — treated as landed under the current confirmation policy | A public Arweave gateway may still take hours to serve it |
The state at 201 is accepted. It proves a resource has been created with a stable identity whose continued delivery is the gateway service’s responsibility. It does not prove the content is permanently stored, and what we tell users has to follow that split: saying “permanently saved” at uploaded is promising, on Turbo’s behalf, something it has not finished doing. Callers check the current state with GET /v1/arweave/items/{dataItemId}.
After confirmed, what was written cannot be taken back. Every free-text field in the schema carries the same warning in its comment²:
Permanently written to Arweave — do not put personal data in it.
The signature decides the ID, so the signature has to be kept
dataItemId is the SHA-256 of the signature value, base64url-encoded³. The input is that small run of signature bytes, not the whole DataItem. Signing happens locally in the gateway service, so the ID is computed locally too, without touching the Arweave network at any point. That is why a 201 can return the final URI on the spot.
The cost is that the gateway service only gets to sign once. RSA-PSS salt is random, so re-signing changes the ID — and the chain may already have anchored the old one. What is persisted before the 201 returns is therefore the complete signed DataItem, not just the metadata JSON. Lose it, and signing again will not get the same ID back.
Persistence is a double write: BackupStore first, then the PostgreSQL row, and only if both succeed does the request return 201; a failure at either step returns 500. When a worker takes the bytes it verifies that they still derive the same ID, and if the PostgreSQL copy is missing or corrupt it is restored from the BackupStore. The copies are kept even after confirmed, because Turbo’s unbundling and indexing lag by hours, and during that window a public Arweave gateway cannot serve the content.
The keccak256 input and the DataItem’s payload are the same raw JSON bytes as received; the gateway service does not re-serialise². Any change in field order, whitespace or Unicode escaping produces a different hash.
A caller resending the same request will not produce a second identity either. The POST carries an Idempotency-Key header, the server deduplicates per caller and key, and a hit returns the first dataItemId and URI.
The queue lives in PostgreSQL
Once the record commits, that row starts moving through a state machine:
accepted → pushing (claimed by a worker) → uploaded → confirmedThe job is not in an in-memory channel: after a process restart the state is still in the table and a worker comes up and continues the upload. After committing the transaction and before writing the HTTP response, the API sends an empty signal on a channel telling workers the database may hold new work. Workers do not wait for the response to finish. The signal carries no content, so ten wake-ups are no different from one — which is why the channel’s buffer capacity is 1. Polling is kept as a fallback sweep, responsible for finding jobs missed when a wake is lost.
Multiple workers each claim one with FOR UPDATE SKIP LOCKED: a row another worker already holds is skipped rather than queued on the lock⁵. That only stops two workers taking the same row at once. The system as a whole is at-least-once, repeats are allowed, and every resend is the same signed DataItem.
Failure paths
What the paths below have in common is that delivery was interrupted but the identity was not lost. Each one answers the same question: who is now responsible for sending this signed DataItem.
failed is the only state that does not drain on its own.After a crash the row is left in pushing, and once it times out it is reclaimed and returned to the queue. The reclaimer and the worker share one timestamp guard: the timestamp taken at claim time appears in the WHERE of every subsequent UPDATE, so a reclaimed row no longer matches and a stale worker cannot write anything.
Turbo accepting an item but confirmation never arriving is handled by an independent confirmer that runs after uploaded: it checks the Arweave index for evidence of inclusion first, falls back to Turbo’s settled status, and gives anything indexed-but-not-yet-mined a grace period. Only once the deadline passes with no verifiable evidence of inclusion does it enter a capped retransmit, and when that count is spent the row turns to failed.
Among 4xx responses only 402 and 429 stay on the retry path. A 402 means the wallet’s free allowance is exhausted and buying Turbo Credits moves it onto the paid path; a 429 is rate limiting. Those two may succeed next time. Other 4xx will not change on an identical retry, so they turn to failed directly, without consuming retry budget.
failed is the only state in the machine that does not drain on its own. Recoverable causes — a spent budget, a receipt that failed to persist — go back to accepted with an admin tool for another delivery; abandoned ones are closed as retired.
The free allowance only ever decreases, so a 402 is a matter of time. The gateway service polls the wallet’s remaining allowance separately and exports it as a monitoring metric, so it is seen coming.
Why not write it straight into the contract
A kilobyte on Base is roughly 640,000 gas⁶ — about one US cent at the 2026-08-28 gas price (0.006 gwei) and ETH price ($2,497). Cost is not what rules this option out.
Event logs and calldata are far cheaper than storage, but they are historical data. Ordinary nodes may prune historical blocks and receipts (Ethereum is advancing along EIP-4444⁷⁸), so reading them later means relying on nodes that still keep full history. Only storage can be read as current state, and it bills linearly by the byte. The downstream readers of this metadata are HuggingFace, training frameworks and agents⁹, which fetch files over HTTP rather than calling a contract and ABI-decoding the result.
Storage would also freeze the fields. The five metadata kinds each add fields and bump versions independently under their own major.minor², whereas a contract’s storage layout is append-only once live, and changing it means a full UUPS upgrade¹⁰.
The on-chain pair is fixed-length — a bytes32 plus a 53-byte URI, about 80,000 gas, regardless of how large the document is. Anchoring cost being constant is what lets several anchors be batched into one transaction. Variable-length content in a contract cannot do that.
Arweave’s side¹¹ happens to match those constraints: metadata is a kilobyte-scale file, and Turbo is currently free for DataItems that size¹²; Arweave is write-once and long-lived¹³, and can be fetched over HTTP through a gateway¹⁴. Free upload is Turbo’s current policy, not part of the storage model.
The URI can be returned early because the system first saves the identity it cannot deterministically rebuild. The upload itself did not get any faster. Everything after — the queue, the retries, the confirmation — is just carrying that identity to Arweave.
References
- XnY Docs — Architecture. Protocol overview: Contribution → Assetification → Usage; decentralised storage as source of truth. docs.xny.ai/docs/concepts/core-systems/architecture
- XnY Docs — Metadata documents. The five metadata schemas; the exact bytes uploaded to metadataUri are the bytes hashed into metadataHash, with no canonicalisation step in between. docs.xny.ai/docs/developer/schemas
- ANS-104: Bundled Data v2.0. §2 DataItem signature and id: the id is the SHA-256 digest of the signature. github.com/ArweaveTeam/arweave-standards
- RFC 8017, PKCS #1 v2.2. §8.1 / §9.1.1 — RSASSA-PSS encoding carries a random salt. rfc-editor.org/rfc/rfc8017
- PostgreSQL Documentation — SELECT. The Locking Clause: FOR UPDATE … SKIP LOCKED. postgresql.org/docs/current/sql-select.html
- evm.codes — SSTORE (0x55). First write to a zero-valued slot costs 20,000 gas. evm.codes/#55
- EIP-4444: Bound Historical Data in Execution Clients. Pruning of historical blocks and receipts. eips.ethereum.org/EIPS/eip-4444
- Ethereum Foundation Blog — Partial History Expiry. Practical progress on mainnet history pruning (2025-07-08). blog.ethereum.org/2025/07/08/partial-history-exp
- XnY Docs — Key Concepts. Storage / Compute / Serving and their integration with mainstream AI tooling. docs.xny.ai/docs/concepts/core-systems/key-concepts
- OpenZeppelin Contracts — UUPSUpgradeable. The UUPS proxy upgrade pattern (EIP-1822). docs.openzeppelin.com/contracts/5.x/api/proxy
- XnY Docs — Storage, Compute & Serving. Content-addressed storage as source of truth; cloud caches never replace the decentralized record. docs.xny.ai/docs/concepts/core-systems/storage-compute-serving
- ar.io Docs — Upload to Arweave (Turbo). Free uploads under 105 KiB. docs.ar.io/build/upload
- Arweave Yellow Paper. The pay-once, long-term storage endowment model. arweave.org/yellow-paper.pdf
- ar.io Docs — Gateways. Permanence comes from the Arweave network; gateways provide HTTP access, indexing and caching. docs.ar.io/learn/gateways
Links accessed 2026-09-02. Translated from the Chinese original.