Skip to content

Dataset split deletes outputs before validating missing images #266

Description

@YDLuo-1

Environment

  • OS: Windows
  • PPOCRLabel: current main (42362c8)
  • Script: gen_ocr_train_val_test.py

Problem

If Label.txt or rec_gt.txt contains an image path that no longer exists, dataset splitting raises FileNotFoundError at the first missing image. Before reading and validating all source records, however, genDetRecTrainVal recursively deletes every existing det/{train,val,test} and rec/{train,val,test} directory and removes all six output label files. It then writes a partially generated replacement dataset until the copy fails.

This turns a stale annotation reference into a destructive, partially completed split and reports only the first missing path.

Reproduction

  1. Create a PPOCRLabel dataset with valid Label.txt, rec_gt.txt, images, and crop_img files.
  2. Leave one record such as images/missing.png in a label file but remove the referenced image.
  3. Run:
python gen_ocr_train_val_test.py --trainValTestRatio 8:2:0 --datasetRootPath images --detRootPath images/det --recRootPath images/rec --detLabelFileName Label.txt --recLabelFileName rec_gt.txt --recImageDirName crop_img

Actual behavior

The previous split is deleted, some new files are copied, and then the script exits with an error similar to:

FileNotFoundError: [Errno 2] No such file or directory: '...\\images\\missing.png'

The output directories and label files are left incomplete.

Expected behavior

Before deleting or writing any existing output, the script should parse both label files and validate every referenced source image. If any are missing, it should:

  • exit with a non-zero status
  • report all missing paths (not only the first one)
  • leave the previous split untouched

An explicit optional mode to skip missing entries and write a summary could also be useful, but silently skipping them should not be the default because it can hide dataset corruption.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions