https://github.com/amazon-science/mxeval - v1.2

Canonical solutions release

We have released canonical solutions for certain popular langauges (v1.2). The detailed numbers of the solutions for each langauge are listed below.

Dataset | # solutions | # problems -- | -- | -- MBCPP | 773 | 848 MBCSP | 725 | 968 MBJP | 874 | 966 MBJSP | 938 | 966 MBKP | 796 | 966 MBPHP | 950 | 966 MBPP | 960 | 974 MBRBP | 784 | 966 MBTSP | 967 | 968

@wshiqi-aws 👏 🎉 thank you for the PR!

- Python
Published by sk-g over 2 years ago

https://github.com/amazon-science/mxeval - v1.1

v1.1

Bugfix:

Fix unit test cases for 47 problems’ test assertions for C#/TypeScript/Go, which represents ~5% of all problems: * Root cause of the issue is a possibility for the input parameters to the canonical solutions get mutated as a side-affect which cause the captured input to mismatch. * We fix this issue by saving another copy of the function input before passing it for execution. * Update README with new results for MBGP, MBTSP, MBCSP with the fix.

- Python
Published by sk-g about 3 years ago

https://github.com/amazon-science/mxeval - First release

- Python
Published by benathi over 3 years ago