Replication of evals on HumanEval and MBPP

by ekurtic - opened Oct 14, 2023

Oct 14, 2023

Hi,
Could you please share the commands you used to run evals on HumanEval and MBPP tasks so I can try to reproduce the reported numbers?
I am not sure if I am hitting the right params like temp, top_p, etc.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment