Skip to content

overview: Generate a Dataset Overview Report

The overview command generates a single, self-contained HTML report summarizing a Pascal VOC dataset -- concept distribution, bounding-box geometry, image resolutions, and a sample gallery with boxes drawn. When a manifest.json (from generate) is found alongside the dataset, the report also covers dive distribution, image/video source breakdown, acquisition timeline, and co-occurring imaged moments. This is useful for quickly understanding what's in a dataset, spotting data-quality issues, and reviewing its span before training or QA/QC.

The report is a single .html file -- inline CSS, inline SVG charts, inline base64 thumbnails -- so it opens standalone in any browser, no server or external assets required.

Usage

m3-download overview DATASET_DIR [OPTIONS]

Required Parameters

  • DATASET_DIR: Dataset directory, expecting JPEGImages/ and Annotations/ subfolders

Options

  • -o, --output: (Optional) Path to write the HTML report to (default: DATASET_DIR/overview.html)
  • --manifest-path: (Optional) Path to a generate run's manifest.json (default: DATASET_DIR/manifest.json, used only if present)
  • --json: (Optional) Also write the computed stats as JSON to this path
  • --samples: (Optional) Number of annotated sample images in the gallery, 0 disables it (default: 16)
  • --top-concepts: (Optional) Number of concepts/dives to chart before truncating the long tail (default: 20)
  • --degenerate-box-min-side: (Optional) Boxes with a side (in px) below this are flagged as degenerate (default: 2.0)
  • --extreme-aspect-ratio: (Optional) Boxes with an aspect ratio beyond this (either direction) are flagged (default: 10.0)
  • --seed: (Optional) Random seed for reproducible sample-gallery selection

How It Works

  1. Annotation XML files are matched to their corresponding images by filename stem (warning about any orphan on either side -- an image with no annotation, or an annotation with no image).
  2. Every bounding box is parsed to compute concept counts, box geometry (width/height/area distributions, degenerate and extreme-aspect-ratio flags), and image resolution stats (including declared-vs-actual dimension mismatches) -- all from lightweight XML/header reads, never by decoding every image.
  3. A small, stratified sample of images (spread across concepts so rare ones aren't crowded out) is rendered with its boxes+labels drawn and embedded as thumbnails -- this is the only step that opens actual image bytes, bounded by --samples.
  4. If a manifest.json is found, its per-image records are joined against what's actually on disk (by filename stem, not trusted wholesale) to compute dive distribution, image/video source breakdown, an acquisition timeline, and co-occurring-imaged-moment counts -- with any manifest/disk discrepancies (QA/QC deletions, manually added images) surfaced explicitly.
  5. Everything is composed into one self-contained overview.html.

Long tails

Concept and dive charts truncate to the top --top-concepts by count (a full table is still shown for concepts). The acquisition timeline automatically switches from monthly to yearly buckets if the dataset spans more than three years, so wide time spans stay readable instead of stretching into hundreds of rows.

Examples

Generate a report with default settings:

m3-download overview dataset/

Generate a report and a machine-readable stats JSON alongside it:

m3-download overview dataset/ --json dataset/overview.json

Generate a report with a larger sample gallery and reproducible sampling:

m3-download overview dataset/ --samples 32 --seed 42

Point at a manifest that isn't co-located with the dataset directory:

m3-download overview dataset/ --manifest-path /path/to/manifest.json