Skip to content

Issues: MMMU-Benchmark/MMMU

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
1 Open 28 Closed
1 Open 28 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Enquiry about the usage about your dataset
#33 by tunantu was closed Sep 6, 2024
1
ls
#29 by yuanze-lin was closed Aug 6, 2024
.tsv file
#28 by beichenzbc was closed Jul 15, 2024
validation_Materials_25 answer seems wrong?
#27 by Zarjagen was closed May 29, 2024
2
How to convert images and prompt to the HF parquet?
#25 by Gumpest was closed May 30, 2024
2
GPT4o
#24 by dirtycomputer was closed May 13, 2024
3
Can you release more result files from the validation leaderboard?
#23 by kyleliang919 was closed May 7, 2024
1
There's an error in one of the Correct Examples for genetics
#22 by eabase was closed May 8, 2024
3
Fail to connect to the homepage: https://mmmu-benchmark.github.io/
#21 by shannany0606 was closed Mar 28, 2024
2
No (supported) data files found in /MMMU/MMMU
#20 by Xiaolong-RRL was closed Mar 23, 2024
3
gpt4v refuse to answer/ insist on "I'm sorry, but I'm unable to view images" these kind of things
#19 by SweetGUOguo was closed Mar 9, 2024
2
RuntimeError: The size of tensor a (162) must match the size of tensor b (7) at non-singleton dimension 1
#18 by nrikoh was closed May 9, 2024
17
PNG files does not convert to RGB
#17 by y-vectorfield was closed Feb 15, 2024
4
process_single_sample function's question
#16 by bruceisme was closed Jan 31, 2024
2
Request for answer_dict.json for test and dev
#15 by boxin-wbx was closed Jan 21, 2024
1
Link for the open source methods in Leaderboard
#14 by XiongweiWu was closed May 7, 2024
1
Image and JSON dataset.
#13 by sxj1215 was closed Jan 4, 2024
6
How was "prompt engineering" performed?
#12 by mckinziebrandon was closed Dec 18, 2023
2
Mismatch of the data label in Eval code
#11 by XiongweiWu was closed Dec 15, 2023
1
Representing LLaVa-1.5-13b
#10 by teasgen was closed Dec 8, 2023
7
Model Evaluatation
#9 by Rubics-Xuan was closed Dec 10, 2023
11
Question about "Text as Input"
#8 by fxmeng was closed Dec 6, 2023
2
Why is every answer in Structural Engineering just "?"
#7 by mckinziebrandon was closed Dec 3, 2023
1
How are the image types defined and labeled?
#6 by CCYChongyanChen was closed Dec 6, 2023
2
Prompts
#5 by teasgen was closed Dec 4, 2023
3
ProTip! Type g i on any issue or pull request to go back to the issue listing page.