`remap`: Remap Concept/Class Labels¶

The remap command group performs bulk renaming of concept (class) labels in Pascal VOC or YOLO annotation files according to a provided mapping file. remap suggest can generate that mapping file for you, using the real taxonomy to merge rare classes.

m3-download remap [voc|yolo|suggest] [OPTIONS]

`remap voc`¶

Bulk renaming of concept (class) names in Pascal VOC annotation XML files. This is useful for harmonizing taxonomic labels, merging similar classes, or simplifying the class hierarchy.

Usage¶

m3-download remap voc MAP_FILE INPUT_DIR [--output-dir OUTPUT_DIR]

Parameters¶

MAP_FILE: File (CSV or JSON) containing the concept remapping definitions
INPUT_DIR: Directory containing Pascal VOC XML files to process
--output-dir: (Optional) Output directory for remapped annotations; if omitted, original files are overwritten
--pretty-print / --no-pretty-print: (Optional) Pretty-print the output XML files (default: --pretty-print)

Data Loss Risk

If --output-dir is not specified, original annotation files will be overwritten without confirmation. Always use --output-dir when testing a new remapping.

Mapping File Formats¶

The MAP_FILE can be provided in two formats:

CSV Format¶

Simple two-column format with original concept names in the first column and target concept names in the second column:

LRJ complex,Benthocodon
Benthocodon pedunculata,Benthocodon
Peniagone sp. A,Peniagone
Peniagone sp. 2,Peniagone
Peniagone sp. 1,Peniagone
Peniagone vitrea,Peniagone
Peniagone vitrea- sp. 1 complex,Peniagone
Peniagone papillata,Peniagone
Scotoplanes sp. A,Scotoplanes
Scotoplanes clarki,Scotoplanes
Scotoplanes globosa,Scotoplanes

JSON Format¶

A JSON object with original concept names as keys and target concept names as values:

{
  "LRJ complex": "Benthocodon",
  "Benthocodon pedunculata": "Benthocodon",
  "Peniagone sp. A": "Peniagone",
  "Peniagone sp. 2": "Peniagone",
  "Peniagone sp. 1": "Peniagone",
  "Peniagone vitrea": "Peniagone",
  "Peniagone vitrea- sp. 1 complex": "Peniagone",
  "Peniagone papillata": "Peniagone",
  "Scotoplanes sp. A": "Scotoplanes",
  "Scotoplanes clarki": "Scotoplanes",
  "Scotoplanes globosa": "Scotoplanes"
}

How It Works¶

The command detects the mapping file format based on file extension
For each XML file in the input directory, it:
- Parses the XML structure
- Checks each <object> element for concept names that need remapping
- Replaces the <name> element content if a match is found
- Writes the modified XML either in-place or to the output directory

Processing Details

Only concept names that match entries in the mapping file are modified
The command reports how many annotation files were modified

Examples¶

Using a CSV mapping file with an output directory:

m3-download remap voc remapping.csv Benthocodon/ --output-dir Benthocodon_remapped/

Using a JSON mapping file and overwriting existing files:

m3-download remap voc taxonomy_map.json annotations/

`remap yolo`¶

Bulk remapping of class IDs in YOLO annotation text files. This is useful for harmonizing label indices, merging similar classes, or removing unwanted classes from your dataset.

Usage¶

m3-download remap yolo MAP_FILE YOLO_DIR [--output-dir OUTPUT_DIR]

Parameters¶

MAP_FILE: File (CSV or JSON) containing the class ID remapping definitions
YOLO_DIR: Directory containing YOLO text annotation files to process
--output-dir: (Optional) Output directory for remapped annotations; if omitted, original files are overwritten

Data Loss Risk

If --output-dir is not specified, original annotation files will be overwritten without confirmation. Always use --output-dir when testing a new remapping.

Mapping File Formats¶

The MAP_FILE can be provided in two formats:

CSV Format¶

Simple two-column format with original class IDs in the first column and target class IDs in the second column:

0,1
2,1
3,2
4,-1

JSON Format¶

A JSON object with original class IDs as keys and target class IDs as values:

{
  "0": 1,
  "2": 1,
  "3": 2,
  "4": -1
}

Special Values

If a class ID is mapped to a negative value (e.g., -1), annotations with that class will be removed entirely.

How It Works¶

The command detects the mapping file format based on file extension
For each text file in the input directory, it:
- Processes each line (each represents one bounding box)
- Checks if the class ID (first value in each line) needs remapping
- Replaces the class ID if a match is found, or removes the line if mapped to a negative value
- Writes the modified annotations either in-place or to the output directory

Processing Details

Only class IDs that match entries in the mapping file are modified
Lines with class IDs mapped to negative values are removed completely
The command reports how many annotation files were modified

Examples¶

Using a CSV mapping file with an output directory:

m3-download remap yolo class_map.csv labels/ --output-dir labels_remapped/

Using a JSON mapping file and overwriting existing files:

m3-download remap yolo class_mapping.json dataset/labels/

Merging multiple classes into one class ID:

m3-download remap yolo merge_classes.csv yolo_annotations/ --output-dir merged_annotations/

Where merge_classes.csv might contain:

1,0
2,0
3,0

This would merge classes 1, 2, and 3 into class 0.

`remap suggest`¶

Suggests a remap mapping for a long-tailed dataset: merges rare concepts up to the coarsest ancestor actually needed to clear a minimum per-class annotation count, using the real VARS taxonomy fetched from a KB server -- not string matching. This produces a mapping file in the exact same format remap voc consumes, so the intended workflow is: review the suggestion, then apply it.

m3-download remap suggest DATASET_DIR --min-count N

Parameters¶

DATASET_DIR: Dataset directory (expects JPEGImages/ and Annotations/ subfolders; only Annotations/ is actually read -- images aren't needed to count concepts)
--min-count: (Required) Minimum per-class annotation count to aim for
-o, --output: (Optional) Path to write the suggested mapping to, .csv or .json (default: DATASET_DIR/remap_suggestion.json)
--count-by: (Optional) localizations (per-box count, matching how remap voc operates) or images (default: localizations)
--target-ranks: (Optional) Comma-separated preferred rank names to merge biological concepts up to (default: genus,family,order,class,phylum,kingdom)
--exclude-targets: (Optional) Comma-separated ancestor names that are never a valid merge target, regardless of rank or depth (default: object,physical object,marine organism)
--max-hierarchy-depth: (Optional) Cap on raw hierarchy hops for concepts with no ranked ancestors at all -- e.g. equipment or substrate concepts (default: 6)
--kb-url: (Optional) Base URL of the VARS KB server (default: https://dsg.mbari.org/kb/v1, the same public endpoint taxonomy uses -- no authentication required)
--workers: (Optional) Parallel KB lookup workers (default: 4)
--report: (Optional) Path to write a detailed per-concept JSON report to

How It Works¶

Every concept's per-class count is tallied from the VOC annotation XMLs.
Concepts already at or above --min-count are left alone. Every concept below it has its ancestor chain fetched from the KB server (in parallel, deduplicated).
For a concept with ranked ancestors (a species, say), the walk-up path is the chain positions whose rank is in --target-ranks, in order -- so a rare species merges straight to genus, then family, skipping over odd intermediate ranks (e.g. "subfamily") in one hop. For a concept with no ranked ancestors at all (equipment, substrate, etc. -- registered in the same KB hierarchy, just without taxonomic ranks), the walk-up path is every raw ancestor hop instead, capped by --max-hierarchy-depth. Either way, --exclude-targets names are never a valid stop, so a concept can never collapse into a meaningless KB-root bucket.
Concepts are grouped by their current name and promoted one step at a time, only while their group's total is still below --min-count -- so a group stops climbing the instant it (and anything else that's merged into the same name, including an already-present coarser concept like a dataset that has both Sebastes and Sebastes diploproa annotated directly) clears the threshold. This finds the minimal sufficient rank, not the coarsest possible one.
A concept that exhausts its walk-up path while still below --min-count is left at its coarsest reached name and reported as merged_insufficient -- it is never forced further.
The resulting {old: new} mapping (only concepts whose name actually changed) is written to --output, directly loadable by remap voc.

Suggests, doesn't apply

This command never modifies annotation files. Review the mapping (and --report, if written) before running m3-download remap voc with it.

Per-concept report statuses

--report records one of four statuses per concept: unchanged (already at/above --min-count), merged (successfully merged to clear the threshold), merged_insufficient (merged as far as the taxonomy/depth cap allows, but still short), or lookup_failed (the KB server didn't recognize the concept -- left unmapped).

Examples¶

Suggest a remap requiring at least 50 localizations per class:

m3-download remap suggest Sebastidae_dataset/ --min-count 50

Suggest a remap and review the full reasoning before applying it:

m3-download remap suggest Sebastidae_dataset/ --min-count 50 -o suggestion.json --report suggestion_report.json
m3-download remap voc suggestion.json Sebastidae_dataset/Annotations --output-dir Sebastidae_remapped/

Restrict merges to genus/family only, leaving anything that can't reach one of those ranks unmerged:

m3-download remap suggest dataset/ --min-count 30 --target-ranks genus,family

remap: Remap Concept/Class Labels¶

remap voc¶

Usage¶

Parameters¶

Mapping File Formats¶

CSV Format¶

JSON Format¶

How It Works¶

Examples¶

remap yolo¶

Usage¶

Parameters¶

Mapping File Formats¶

CSV Format¶

JSON Format¶

How It Works¶

Examples¶

remap suggest¶

Parameters¶

How It Works¶

Examples¶

`remap`: Remap Concept/Class Labels¶

`remap voc`¶

`remap yolo`¶

`remap suggest`¶