OpenSVBench Scenario-driven SV leaderboard
Reference Document

Minimal Rolling Dataset Example

Rolling Safety

Upload protocol files, not raw audio.

Contributions should describe how to obtain the data, prepare manifests, and construct trials. Maintainers review them separately from the current core ranking.

Do not upload large raw audio. Include download or placement instructions. Use collection="rolling" for accepted catalog patches.

Minimal Rolling Dataset Example

This directory shows the smallest useful shape of a rolling dataset contribution. It does not add anything to the active OpenSVBench catalog. It only demonstrates the files and protocol style expected from a rolling dataset package.

The snippets below are a documentation example, not a real benchmark dataset. They show the expected prepared files and trial format directly. The included script is only a convenience tool if you want to regenerate the same example structure locally.

Example Package Shape

A minimal rolling contribution should make the final prepared shape easy to understand:

example_rolling/
├── prepared/
│   ├── wav.scp
│   ├── wav_all.scp
│   ├── utt2spk
│   ├── spk2utt
│   ├── trials_example_rolling_basic.txt
│   ├── stats_example_rolling_source.json
│   └── stats_example_rolling_basic.json
└── raw_data/
    └── wav/

Example wav.scp:

example_spk01_s1 /path/to/example_rolling/raw_data/wav/example_spk01_s1.wav
example_spk01_s2 /path/to/example_rolling/raw_data/wav/example_spk01_s2.wav
example_spk02_s1 /path/to/example_rolling/raw_data/wav/example_spk02_s1.wav

Example utt2spk:

example_spk01_s1 spk01
example_spk01_s2 spk01
example_spk02_s1 spk02

Example trials_example_rolling_basic.txt:

example_spk01_s1 example_spk01_s2 target example_cross_session
example_spk01_s1 example_spk02_s1 nontarget example_same_session
example_spk01_s2 example_spk02_s2 nontarget example_same_session

Example stats_example_rolling_basic.json:

{
  "trial_set_id": "example_rolling_basic",
  "trials": 9,
  "target": 3,
  "nontarget": 6,
  "condition_counts": {
    "example_cross_session": 3,
    "example_same_session": 6
  }
}

Optional Local Regeneration

python3 examples/rolling_dataset_minimal/prepare_example_rolling.py \
  --data /tmp/opensvbench_rolling_example

The command writes synthetic sine-wave wav files and a manifest/trial shape like the one shown above:

/tmp/opensvbench_rolling_example/example_rolling/
├── prepared/
│   ├── wav.scp
│   ├── wav_all.scp
│   ├── utt2spk
│   ├── spk2utt
│   ├── trials_example_rolling_basic.txt
│   ├── stats_example_rolling_source.json
│   └── stats_example_rolling_basic.json
└── raw_data/
    └── wav/

Trial Protocol

  • target: same example speaker, different recording.
  • nontarget: different example speaker, same session when possible.
  • condition: example_cross_session for target pairs and example_same_session for nontarget pairs.

The generated trial file uses the standard OpenSVBench format:

utt1 utt2 label condition

Catalog Patch

catalog_patch_example.py shows how an accepted contribution could add DatasetSpec(collection="rolling") and ScenarioSpec(collection="rolling"). Do not add the documentation example to the main catalog.