Issues

• EvalScope integration: one-command evaluation support for SEED-Bench-2-Plus

  AILab-CVC/SEED-Bench#35 · Filter by author Yunnglin opened on Jul 24, 2026

• Question about the first frame of the 8 sampled frames about SEED-Bench

  AILab-CVC/SEED-Bench#34 · Filter by author wl421 opened on Apr 23, 2026

• huggingface Leaderboard not working

  AILab-CVC/SEED-Bench#31 · Filter by author LindiaC opened on Nov 4, 2024

• Data source for "Transit Maps" model: not listed in this repository? also not listed on Huggingface?

  AILab-CVC/SEED-Bench#29 · Filter by author miku2000 opened on Jun 27, 2024
  1

• Wrong Question Types in SEED-Bench1

  AILab-CVC/SEED-Bench#28 · Filter by author littlepenguin89106 opened on May 17, 2024
  3

• Question about evaluation input format

  AILab-CVC/SEED-Bench#27 · Filter by author yellow-binary-tree opened on May 11, 2024
  1

• any plan to release the original images for SEED-Bench-2-Plus?

  AILab-CVC/SEED-Bench#25 · Filter by author hyunjun-eun opened on May 3, 2024
  1

• Question on multi-image input

  AILab-CVC/SEED-Bench#24 · Filter by author auhowielau opened on Mar 18, 2024
  2

• [bugs] LLaVA-Evaluation : RuntimeError: Expected all tensors to be on the same device

  AILab-CVC/SEED-Bench#21 · Filter by author JJJYmmm opened on Jan 31, 2024
  2

• Muti-GPUs Evaluation

  AILab-CVC/SEED-Bench#20 · Filter by author JJJYmmm opened on Jan 31, 2024

• Question on how task27 generates images

  AILab-CVC/SEED-Bench#19 · Filter by author JunZhan2000 opened on Jan 16, 2024
  2

• a lot of data with more questions than pictures in SEED-Bench-2 level L2, is this reasonable?

  AILab-CVC/SEED-Bench#15 · Filter by author nemonameless opened on Dec 12, 2023
  5

• In-Context Example Selection Process

  AILab-CVC/SEED-Bench#14 · Filter by author mustafaadogan opened on Dec 2, 2023
  1

• VLMs vs LLMs evaluation

  AILab-CVC/SEED-Bench#12 · Filter by author idan-tankel opened on Oct 22, 2023
  1

• Easy way to probe result examples?

  AILab-CVC/SEED-Bench#11 · Filter by author chancharikmitra opened on Sep 22, 2023
  1

• Request for the interface of minigpt4 and llava

  AILab-CVC/SEED-Bench#10 · Filter by author Richar-Du opened on Sep 22, 2023
  2

• Reproduce the Qwen-VL SOTAs results

  AILab-CVC/SEED-Bench#9 · Filter by author jinze1994 opened on Sep 4, 2023
  2

• Support for evaluation of other VLM models like MiniGPT-4, mPLUG-Owl, Llava, and VPGTrans

  AILab-CVC/SEED-Bench#8 · Filter by author WesleyHsieh0806 opened on Aug 30, 2023
  2

• This benchmark could lead to wrong conclusion.

  AILab-CVC/SEED-Bench#7 · Filter by author dannyhung1128 opened on Aug 23, 2023
  3

• Add InstructBlip Flan-T5-xl and InstructBlip Flan-T5-xxl

  AILab-CVC/SEED-Bench#5 · Filter by author brianjking opened on Aug 7, 2023
  1

• Update for mPLUG-Owl

  AILab-CVC/SEED-Bench#3 · Filter by author MAGAer13 opened on Aug 2, 2023
  1

• Update for Otter-Image-MPT7B and Otter-Video

  AILab-CVC/SEED-Bench#1 · Filter by author Luodian opened on Aug 1, 2023
  1