New Math Benchmark Dataset Enhances LLMs for Portuguese

The article announces the introduction of Math-PT, a dataset developed for evaluating large language models (LLMs) in mathematical reasoning specifically in European and Brazilian Portuguese. Comprising 1,729 problems sourced from national competitions and exams, this dataset aims to address the linguistic gap in existing benchmarks predominantly available in English. By evaluating state-of-the-art LLMs against Math-PT, researchers found that while high-performing models succeeded in multiple-choice formats, their effectiveness diminished with more complex question formats.

The release of the Math-PT dataset underlines a significant step towards inclusivity in AI research, particularly in the realm of language diversity. By providing a resource tailored to Portuguese speakers, it enhances the capability of LLMs without increasing reliance on foreign resources, contributing to a more equitable technological landscape. This initiative supports ongoing efforts to diversify AI training datasets and methodologies, potentially leading to more robust and adaptable models in mathematical reasoning tasks.