OpenHands/evaluation/benchmarks/mint/config_variables.py
OpenHands 678436da30
Fix issue #5222: [Refactor]: Refactor the evaluation directory (#5223)
Co-authored-by: Engel Nyst <enyst@users.noreply.github.com>
2024-11-25 08:35:52 -05:00

10 lines
433 B
Python

TASK_INFO_MAP = {
# === Reasoning ===
'gsm8k': {'class': 'ReasoningTask', 'type': 'reasoning'},
'math': {'class': 'ReasoningTask', 'type': 'reasoning'},
'mmlu': {'class': 'MultipleChoiceTask', 'type': 'reasoning'},
'theoremqa': {'class': 'TheoremqaTask', 'type': 'reasoning'},
'mbpp': {'class': 'MBPPTask', 'type': 'code_generation'},
'humaneval': {'class': 'HumanEvalTask', 'type': 'code_generation'},
}