The claims.
The mechanics.
The limits.

OpenWALDO is the first open-source AI project and community: a community-maintained training-data commons, an integrated toolchain, public governance, verifiable provenance, and a workflow for composing, training, validating, exporting, and extending models.

The WALDO command-line tool ingests, verifies, selects, exports, trains, and packages models while preserving provenance. OpenWALDO is not a model vendor or a single frozen dataset; it is a living source foundation maintained in public.

Open source built the software infrastructure the modern world depends on: operating systems, the web, cloud platforms, containers, programming languages, scientific computing, and much of today’s AI tooling. It works because people can inspect, use, improve, and govern a shared foundation together. OpenWALDO brings that proven development model to the full AI stack.

Transparency. Open source replaces unsupported assurances with evidence people can inspect: source material, license assertions, object identities, training plans, and recorded results. It does not guarantee that a model is safe or correct, but it makes meaningful independent scrutiny possible.

Community. Public inputs and open tooling let researchers, builders, creators, and users contribute improvements, find mistakes, challenge claims, and preserve work beyond the priorities of any one lab or vendor.

Scale. Shared infrastructure turns isolated efforts into a reusable foundation. Communities can divide the work of collecting, reviewing, hosting, verifying, and improving data instead of every organization rebuilding the same closed pipeline alone. Understand what makes AI open source.

Open datasets, open-weight models, code releases, and research projects are enormously valuable. OpenWALDO focuses on the ongoing work after publication: its community can continuously propose sources, inspect evidence, review license assertions, correct records, version the corpus, and improve the tools.

The defining difference is that the pieces work together and the community can participate across them. Training data remains a living project; tools and workflows stay open; model lineage remains attached; governance happens in public; and people can build, validate, reproduce, and extend the results. Every improvement can benefit everyone using the shared foundation.

Open weights are enormously valuable: they let people run, study, adapt, and build upon models. OpenWALDO helps those releases go further by preserving the training material, license assertions, object identity, configuration, and run history needed to inspect how a model came to exist.

OpenWALDO gives open-weight releases an inspectable source foundation that a community can keep improving. Understand the role of open weights.

An OpenWALDO corpus BOM is an immutable, machine-readable record of one resolved data selection. It pins the index context, requested paths and license policy, manifest and source identities, resolved shards, exact counts, and hashes.

That record crosses boundaries. Corpus BOMs enter exports and training runs; origin, run, model, and release BOMs connect starting weights, execution, and published artifacts into one inspectable lineage. See how an AI BOM establishes traceability.

A model compose is a strict, portable YAML or JSON definition of a model build. It can declare the architecture or starting model, corpus mixture and filters, ordered training stages, objectives, budgets, evaluation plan, and interaction contract without hard-coding machine-specific execution details.

OpenWALDO created the compose system to make model building easier to understand, reproduce, review, share, and extend. The same compose can be forecast, trained, versioned, and used as the foundation for another model. Understand model composes.

No. Git governs the small, reviewable meaning: corpus descriptions, sources, asserted licenses, conversion identity, counts, and object references. Large canonical Parquet objects live in federated lookaside storage and are addressed by SHA-256.

This keeps review practical without hiding the bytes behind a private catalog.

Metadata verification checks index structure and whether referenced objects are reachable at their declared sizes. Object verification downloads selected objects and proves their content hashes. A full audit also validates canonical records, identities, text hashes, duplicates, and aggregate totals.

These checks establish only what they examine. A hash proves identity and integrity; it does not prove license, quality, safety, or fitness for a model.

The index records attributable license assertions and their scope so users can inspect and filter declared terms before selecting data. Assertions can be reviewed, challenged, and corrected through the public record.

OpenWALDO does not convert an assertion into a legal opinion. Commercial or other use depends on the selected material, its applicable terms, and the user’s own legal analysis.

WALDO currently supports manifest-backed UTF-8 text, Markdown, mbox, JSON, JSONL, Parquet, XML, PDF, and EPUB inputs. Structured formats use explicit mappings, while PDF text layers and EPUB reading order are handled by built-in, versioned adapters.

PDF OCR is not implicit. Multimodal inputs, LaTeX, and other dependency-aware document trees remain planned so unsupported material fails clearly instead of being interpreted by an unrecorded external converter.

Every new ingestion applies a mandatory, versioned privacy-redaction policy before content hashing, deduplication, assessment, and Parquet encoding. It replaces email addresses, IP addresses, phone numbers, and high-confidence credentials with typed placeholders, while retaining names and public attribution. Recognized RFC 822 routing headers are removed.

This is deterministic risk reduction, not a claim of anonymity or regulatory compliance. Indirect identifiers and detector misses can remain, and older shards are not retroactively described as redacted without being rebuilt.

Yes. Corpus selections can be exported as native Parquet or canonical JSONL with an offline-verifiable BOM, then consumed by another training system. WALDO can also train from a blank architecture, a model compose, a managed model, or a supported revision-pinned open-weight origin.

Completed models can be exported as native WALDO, Hugging Face, MLX, GGUF, or Ollama packages while retaining their BOMs. The managed model path intentionally supports a narrower set of architectures and tokenizers today; export remains the general bridge to other stacks.

WALDO supports MLX training on Apple Silicon and PyTorch training on Linux CPU, NVIDIA CUDA, or AMD ROCm systems. TorchTitan supports distributed training on a single multi-GPU Linux host or across multiple hosts when its runtime is installed and compatible.

waldo status reports local readiness, and waldo model forecast checks the selected model on the current host before substantial compute is allocated. WALDO fails explicitly when a usable backend is unavailable; it never silently substitutes simulated training.

Not yet. OpenWALDO currently provides the community corpus, model composes, training and evaluation tooling, provenance, and export paths needed to build models from scratch or extend supported open weights. Reference composes are test and experiment inputs, not model-quality guarantees.

Community-built foundation-model lineages and native sparse-MoE artifacts are planned. Their development will happen through the same open, inspectable process as the rest of the project.

WALDO can pin the exact corpus selection, starting-point identity, architecture, training plan, backend identity, environment observations, and output hashes. That makes the inputs and recorded process reproducible and differences diagnosable.

Bit-for-bit numerical reproduction is a separate claim and can depend on hardware, libraries, kernels, and backend determinism. WALDO records evidence; it does not promise equivalence the execution environment cannot provide.

Yes. WALDO supports local indexes, a retained local cache, and file-based lookaside storage. An organization can use the same content-addressed formats and verification workflow with internal material without publishing it to the public index.

Fully offline operation requires the selected index, objects, tools, and any model dependencies to be available locally.

Begin with a manifest-backed raw directory produced by an acquisition tool, or assemble a manifest from supported local inputs. WALDO verifies the manifest and raw tree, probes the data, applies mandatory privacy redaction and content assessment, deduplicates records, writes canonical Parquet, audits and publishes the objects, then updates the configured writable index.

The result is a compact metadata change that can be reviewed in Git while the large content-addressed objects remain in lookaside storage. You review the change, sign the DCO, commit it, and open a pull request. WALDO never silently commits, pushes, or creates the pull request for you.

The DCO is a contributor attestation used across open-source projects. A Signed-off-by trailer records that the contributor certifies they created the submitted contribution—or otherwise have the right to submit it under the project’s license—and understands that the contribution and sign-off are public.

For WALDO, that puts an accountable identity behind the index change and its source and license assertions. It does not independently prove that those assertions are legally correct. Read how WALDO applies the DCO.

Source and license statements are public, attributable claims—not untouchable conclusions. Questions and corrections should be raised against the index so evidence can be reviewed and the record can change visibly.

Object removal is deliberately explicit because an object may still be referenced by another index, historical revision, or exported BOM. WALDO does not guess that published data is unreferenced.

Today WALDO provides an end-to-end path from manifest-backed ingestion to an auditable corpus, model compose, trained model, and release. That includes built-in text and document adapters, mandatory privacy redaction, content assessment, verification and audit, multiple BOMs, structured conversation and assistant-response training, evaluation, checkpointing, and optional signing.

Training works through MLX on Apple Silicon and PyTorch or TorchTitan on supported Linux systems, including multi-host distributed training. Models can be exported in native WALDO, Hugging Face, MLX, GGUF, and Ollama formats. The core path works today, while interfaces outside the documented compatibility contract may evolve as contributors test and improve them.

Planned work includes multimodal and dependency-aware document ingestion, broader architecture and tokenizer support, community foundation-model and sparse-MoE lineages, preference-training objectives, PyTorch generation, Hugging Face publication, richer regulatory templates, and lookaside mirroring. See the maintained implementation roadmap.