Project DoLittle | Dataset release update
An Auditable Orca Token Dataset
A source-aware path from exact 30-second hydrophone context to a license-separated research token corpus, with raw audio, quality grades, processing choices, and exclusions kept traceable.
We have completed the first evaluator-graded Orca tokenization release for Project DoLittle. The result is not a replacement for source audio and it is not a claim about Orca abundance or behaviour. It is a reproducible research derivative: every tokenized context can be traced to a source, a 30-second time interval, a noise grade, a license partition, and a processing record.
This release follows our earlier Orca corpus update, which documents source-aware context acquisition, quality triage, and the conservative cleanup evaluation that preceded tokenization.
What the Dataset Contains
Each successful context is an exact 30-second segment selected from a frozen manifest. The release records the source identifier, source license, parent-recording order, start and end seconds, evaluator grade, evaluator probabilities, preprocessing route, DAC configuration, logical artifact location, and SHA-256 checksum. Raw recordings remain canonical and are not included in the token packages.
| Evaluator grade | Tokenized | Documented hold |
|---|---|---|
| Quiet | 536 | 0 |
| Soft | 3,204 | 0 |
| Moderate | 17,960 | 4 |
Two contexts that initially produced silent DNS48 outputs were successfully recovered on a later bounded run. Their earlier failures remain in the append-only processing record. Four moderate contexts remain documented holds rather than silently disappearing from the dataset account.
Pipeline and Scope
The preparation derivative exists for this specific research representation. It changes the acoustic signal, is not raw-audio replacement, and is not automatically compatible with detectors trained on the original bandwidth. The 9-codebook configuration was chosen to remain compatible with the Project DoLittle Humpback codec representation.
Interpretation boundary: a lower-noise derivative can improve listening and provide a consistent encoding input, but spectral reduction is not proof that every removed component was noise or that every retained component was biological. We retain the raw context, evaluator record, and preprocessing metadata so those questions remain reviewable.
Sources and Release Partitions
The corpus combines DORI-ONC, DORI-Orcasound, Orcasound podcast material, and DCLDE partitions including DFO-CRP, DFO-WDLP, Orcasound, Scripps, SIMRES, SMRU, UAF, and VFPA. We do not flatten their terms into a single permissive license.
| Upload-ready package | Contexts | Terms |
|---|---|---|
orca-dac9-cc-by-4 | 11,221 | CC BY 4.0 with source attribution |
orca-dac9-cc-by-sa-4 | 8,560 | CC BY-SA 4.0; share-alike conditions apply |
orca-dac9-cc-by-nc-sa-4 | 1,421 | CC BY-NC-SA 4.0; non-commercial and share-alike conditions apply |
| Withheld local-only register | 502 | SanctSound, OOI or unverified terms; plus processing holds. Not for upload. |
The public Hugging Face releases are MMC Orca DAC9 CC-BY-4.0, MMC Orca DAC9 CC-BY-SA-4.0, and MMC Orca DAC9 CC-BY-NC-SA-4.0. Together they publish 21,202 contexts and 492.88 million token IDs. Every repository has a dataset card, explicit license notice, per-context manifest, checksum file, Git LFS attributes, and an uncompressed WebDataset TAR shard. The packages contain only token arrays and release metadata: no raw recording or denoised WAV file is bundled.
Utilities on GitHub
The processing utilities are available on GitHub under the MIT License. The repository includes a start-to-finish reproduction guide, sanitized frozen selection and final-audit metadata, source inventory, checksum summary, and the scripts for quality scoring, manifest-driven DNS48 preparation, spectral A/B/C comparison, DAC9 tokenization, restart-safe finalization, checksum validation, and Hugging Face package assembly. Model weights, archive credentials, raw audio, and derived audio are intentionally excluded.
- Quality and selection: scripts that preserve evaluator grades, probability fields, and source/license partitioning.
- Preparation and debugging: DNS48 batch processing with a free-space floor, silent-output holds, explicit normalization, and spectral diagnostics.
- Tokenization and release: 9-codebook DAC encoding, append-only operational manifests, normalized final audits, checksums, and license-separated WebDataset release preparation.
Reproducibility note: the repository is sufficient to reproduce the processing and verify the release once source recordings and model artifacts have been independently obtained under their own terms. It does not redistribute DNS48 or DAC weights, does not grant audio rights, and the original DAC checkpoint artifact hash was not retained; a reproducer must record the hash of the compatible checkpoint they independently obtain before comparing outputs.
What This Enables
Separating source licenses and keeping raw provenance next to each tokenized context allows future work to evaluate models without losing the history of how a datum entered the corpus. The next research question is not simply how many tokens we have. It is whether downstream models preserve acoustic structure across sources and noise grades, and whether any result can be traced back to the recording conditions and license terms that produced it.