Meaning
- Corpus description
- Sources and asserted licenses
- Conversion identity
- Counts and shard hashes
- Index navigation
Grow the public commons
If the source matters,
put it on
the record.
03 / CONTRIBUTE
WALDO turns acquired source material into large canonical, audited shards and a compact Git review record. The bytes stay out of Git. The meaning and accountability stay in public review.
See how to contribute ↓How to contribute
Bring knowledge.
Put it on the record.
A contribution starts with a corpus and ends with an accountable pull request. Large collections are welcome when their origin and asserted terms can be recorded and reviewed. Consumers remain free to select the licenses and sources appropriate for their own use.
-
01
Obtain or build a corpus.
Acquire eligible material from an upstream source, or assemble material you or your organization can contribute. A company might gather its public documentation, product information, and service portfolio so future AI models can learn from an authoritative source.
YOU BRINGRaw source material with a clear origin and asserted terms. -
02
Choose files or a recipe.
If the material is already local, point WALDO at its directory. If it must be fetched or assembled, use a reviewed ingest recipe that declares the metadata and runs the acquisition steps in a controlled temporary workspace. The destination is a logical path inside the writable waldo-index checkout configured on your machine. Public contributions also need a lookaside destination whose published objects reviewers can read; coordinate access with the maintainers before a large ingest.
waldo config set index /path/to/waldo-indexwaldo config set lookaside <public-lookaside-url>waldo index ingest --helpwaldo index ingest <raw-source-directory> <path-in-waldo-index> [flags]waldo index ingest <recipe.yaml> <path-in-waldo-index> [flags] -
03
Let WALDO ingest it.
WALDO probes the input, converts it into canonical Parquet shards, audits the records and totals, uploads the content-addressed objects to the lookaside store configured on your machine, and verifies the published bytes.
WALDO WRITESCanonical Parquet in lookaside storage—no large data objects in Git. -
04
Inspect the index record.
WALDO writes a contribution overlay into its staging area and prints that location. Apply the reported overlay to your index checkout, then inspect the corpus description, sources, asserted licenses, conversion identity, counts, hashes, and public object links before committing anything.
GIT RECORDSThe reviewable meaning, inventory, evidence, and object references. -
05
Sign, commit, and open a pull request.
Review the generated diff, commit it with DCO sign-off, push your branch, and open a pull request. Maintainers and the community can then inspect the claims, run the checks, question the evidence, and improve the contribution.
cp -R <reported-contribution-directory>/. /path/to/waldo-index/git diff --checkgit add .git commit -s -m "Add <corpus name>"
Review and accountability
Review the corpus.
Strengthen the provenance.
DCO ACCOUNTABILITYEvery contribution must carry an accountable sign-off.
The DCO attaches a contributor’s certification to the public Git history. It does not prove that every claim is correct; it makes the contributor and attestation visible so both can be reviewed and challenged.
-
01
Understand
Inspect what was submitted and confirm that its detected content and schema match the proposed corpus.
-
02
Standardize
Convert the material into tokenizer-neutral canonical Parquet with a recorded conversion identity.
-
03
Verify
Reopen every shard and validate its schema, records, hashes, and exact aggregate totals.
-
04
Confirm
Prove that each content-addressed object is available from lookaside storage and matches the published record.
-
05
Challenge
Examine the source and license assertions, conversion identity, counts, object inventory, DCO sign-off, and public diff.
A strong corpus is built through accountable review.
Automation can prove structure, identity, integrity, totals, and availability. Named contributors and human reviewers still have to evaluate usefulness, examine source evidence, challenge rights assertions, and improve the provenance record.
The public record
Bytes over here.
Meaning over here.
This separation keeps large objects efficient without hiding the facts reviewers need to understand what entered the commons.
Canonical bytes
- Self-contained Parquet
- SHA-256 object identity
- No loose media
- No hidden catalogs
- No metadata sidecars
Accountability at scale
Sign the contribution.
Stand behind the claim.
The Developer Certificate of Origin (DCO) is a lightweight accountability mechanism proven across open-source software. A sign-off connects each submitted change to a contributor who certifies that they created it—or otherwise have the right to submit it under the project’s license—and understands that the contribution is public.
Identify the contributor.
git commit -s adds a Signed-off-by trailer using the
contributor’s Git identity.
Make responsibility explicit.
The sign-off records the contributor’s certification instead of leaving provenance as an anonymous upload or an implied promise.
Keep the record public.
The commit, sign-off, review, automated checks, corrections, and history remain together in the shared index.
Accountability is evidence—not magic.
A DCO sign-off does not independently prove that an asserted corpus license is correct or that every underlying document can legally be redistributed. It makes the contribution attributable, explicit, reviewable, and challengeable.
Make the source better
Contribute data.
Review claims.
Build the tool.
WALDO is open infrastructure, not a data vending machine. The commons gets stronger through attributable contribution and skeptical review.