Skip to content

mllm_eval/ directory referenced in README is missing from the repository #2

Description

@KetoneOlefine

The evaluation/README.md and REPRODUCE_BAGEL_RESULTS.md both document an AC (Appearance Consistency) scoring pipeline that depends on a mllm_eval/ directory:

  1. Run MLLM-based AC scoring (optional but recommended)
    Launch the evaluation helper in mllm_eval/ to produce AC results with a serving LLM:
cd mllm_eval
bash eval_infer.sh <model_path> default <port>

However, the mllm_eval/ directory (and its scripts build_lmdb.py, mllm_eval.py, eval_infer.sh) is not present in the released repository.

The only AC-related code available is eval/aggregate.py, which can read a pre-existing AC predictions JSON, but cannot produce one.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions