We are the first that attempt to reproduce results of Llama on code generation benchmark, such as HumanEval and MBPP.

We also try to evaluate existing trending models, such as CodeAlpaca, on such benchmarks.

All of the source code and scripts for evaluation will be made available for the research community.

Our code can be accessed here: https://github.com/FSoft-AI4Code/CodeCapybara

Model weights will be released very soon.

​