Commit 8eed623
authored
Update MultiPL-E to v3 prompts
I've put together a new MultiPL-E release, v3 and this updates the harness to point to the new prompts.
There are fixes to old prompts as well as support for new languages, both described on the MultiPL-E dataset page. The nature of fixes is that some PLs have their scores go up a little. I've tested with with StarCoder2-15B (using VLLM) and will continue testing other models.1 parent 334efb7 commit 8eed623
1 file changed
+1
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
83 | | - | |
| 83 | + | |
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
| |||
0 commit comments