[BUG] 187 multilingual/QA tasks can't load on datasets>=4 (script-based datasets); ifeval-fr/gpqa-fr/bac-fr repos are gone #1406

Describe the bug

lighteval requires datasets>=4.0.0, and datasets 4 dropped support for loading scripts. Six task modules still point at script-based datasets, so every task in them fails before any evaluation runs:

RuntimeError: Dataset scripts are no longer supported, but found mlqa.py

That is 187 task configs in total. Separately, ifeval-fr, gpqa-fr and bac-fr in multilingual/tasks/french.py use fr-gouv-coordination-ia/{IFEval-fr,gpqa-fr,bac-fr}. That organization has no public datasets any more, so those three tasks fail with RepositoryNotFoundError.

siqa (allenai/social_i_qa) and mathqa (allenai/math_qa) also ship loading scripts, but they still load on datasets 5.0.1, so they aren't affected.

To Reproduce

import datasets
datasets.load_dataset("facebook/mlqa", "mlqa.de.de", split="test")
# RuntimeError: Dataset scripts are no longer supported, but found mlqa.py

I checked every config in TASKS_TABLE of the modules above (189 configs) by loading its hf_repo, hf_subset and first evaluation split. The checked versions are lighteval main at b634749 and datasets 5.0.1.

Expected behavior

The tasks load.

Hub's auto-converted parquet exists for all six repos on the refs/convert/parquet branch, with one directory per config. Setting only hf_revision="refs/convert/parquet" isn't enough on datasets 5: load_dataset("facebook/mlqa", "mlqa.de.de", revision="refs/convert/parquet") fails with BuilderConfig 'mlqa.de.de' not found. Loading the per-config files does work:

datasets.load_dataset(
    "parquet",
    data_files={"test": "hf://datasets/facebook/mlqa@refs%2Fconvert%2Fparquet/mlqa.de.de/test/*.parquet"},
    split="test",
)  # 4517 rows; the same approach gives 4000 rows for AmazonScience/mintaka "de"

So these tasks could either use hf_data_files pointing at those files, or be re-uploaded as parquet mirrors under the lighteval org (as was done for lighteval/siqa). I'm happy to open a PR for whichever approach you prefer. I don't know where the French datasets went.

Version info

lighteval main (b634749), datasets 5.0.1, huggingface_hub 1.33, Python 3.12, macOS.

🤖 Generated with Claude Code