Skip to content
Navigation Menu
Sign in
Appearance settings
Platform
AI CODE CREATION
GitHub Copilot
Write better code with AI
GitHub Copilot app
Direct agents from issue to merge
MCP Registry
Integrate external tools
DEVELOPER WORKFLOWS
Actions
Automate any workflow
Codespaces
Instant dev environments
Issues
Plan and track work
Code Review
Manage code changes
Code Quality
Enforce quality at merge
APPLICATION SECURITY
GitHub Advanced Security
Find and fix vulnerabilities
Code security
Secure your code as you build
Secret protection
Stop leaks before they start
EXPLORE
Why GitHub
Documentation
Blog
Changelog
Marketplace
View all features
Solutions
BY COMPANY SIZE
Enterprises
Small and medium teams
Startups
Nonprofits
BY USE CASE
App Modernization
DevSecOps
DevOps
CI/CD
View all use cases
BY INDUSTRY
Healthcare
Financial services
Manufacturing
Government
View all industries
View all solutions
Resources
EXPLORE BY TOPIC
AI
Software Development
DevOps
Security
View all topics
EXPLORE BY TYPE
Customer stories
Events & webinars
Ebooks & reports
Business insights
GitHub Skills
SUPPORT & SERVICES
Documentation
Customer support
Community forum
Trust center
Partners
View all resources
Open Source
COMMUNITY
GitHub Sponsors
Fund open source developers
PROGRAMS
Security Lab
Maintainer Community
Accelerator
GitHub Stars
Archive Program
REPOSITORIES
Topics
Trending
Collections
Enterprise
ENTERPRISE SOLUTIONS
Enterprise platform
AI-powered developer platform
AVAILABLE ADD-ONS
GitHub Advanced Security
Enterprise-grade security features
Copilot for Business
Enterprise-grade AI features
Premium Support
Enterprise-grade 24/7 support
Pricing
Search
/
Sign in
Sign up
Appearance settings
You signed in with another tab or window.
Reload
to refresh your session.
You signed out in another tab or window.
Reload
to refresh your session.
You switched accounts on another tab or window.
Reload
to refresh your session.
Dismiss alert
{{ message }}
Uh oh!
There was an error while loading.
Please reload this page
.
bigcode-project
/
bigcode-evaluation-harness
Public
Notifications
You must be signed in to change notification settings
Fork
260
Star
1.1k
Code
Issues
61
Pull requests
35
Actions
Projects
Security and quality
0
Insights
Additional navigation options
Code
Issues
Pull requests
Actions
Projects
Security and quality
Insights
Commits
Branch selector
main
User selector
All users
Datepicker
All time
Commit history
Commits on Jul 22, 2025
Update multiple.py
Show description for 8fc5bae
arjunguha
committed
8fc5bae
View commit details
Copy full SHA for 8fc5bae
Browse repository at this point
Commits on Jul 1, 2025
fix: Multiple-E dataset fix go_test.go path for test execution
sagtanih
authored and
arjunguha
committed
f17770e
View commit details
Copy full SHA for f17770e
Browse repository at this point
Commits on Oct 31, 2024
Merge pull request #274 from younesbelkada/main
Show description for 6116c6a
loubnabnl
authored
6116c6a
View commit details
Copy full SHA for 6116c6a
Browse repository at this point
Commits on Oct 20, 2024
Merge pull request #269 from zkcpku/main
Show description for f246ab2
loubnabnl
authored
f246ab2
View commit details
Copy full SHA for f246ab2
Browse repository at this point
Commits on Sep 26, 2024
Use MultiPL-E 3.1 dataset
arjunguha
committed
7c02b2f
View commit details
Copy full SHA for 7c02b2f
Browse repository at this point
MultiPL-E now supports Dart, OCaml, Elixir, Haskell, Clojure
arjunguha
committed
28c0eb2
View commit details
Copy full SHA for 28c0eb2
Browse repository at this point
Commits on Sep 23, 2024
Update requirements.txt
younesbelkada
authored
516d364
View commit details
Copy full SHA for 516d364
Browse repository at this point
Commits on Sep 5, 2024
fix typo bug in mbppplus
zkcpku
committed
598b03e
View commit details
Copy full SHA for 598b03e
Browse repository at this point
Commits on Sep 3, 2024
Merge pull request #265 from shehrozek-cerebras/update_pyext_pkg_dep
Show description for f0b81a9
loubnabnl
authored
f0b81a9
View commit details
Copy full SHA for f0b81a9
Browse repository at this point
Commits on Aug 29, 2024
downgrade pyext dep
shehrozek-cerebras
committed
fa7f511
View commit details
Copy full SHA for fa7f511
Browse repository at this point
Commits on Jul 14, 2024
Merge pull request #255 from bigcode-project/arjunguha-patch-1
Show description for 0f3e95f
loubnabnl
authored
0f3e95f
View commit details
Copy full SHA for 0f3e95f
Browse repository at this point
Merge pull request #254 from bigcode-project/MultiPL-Ev3
Show description for a83b1ca
loubnabnl
authored
a83b1ca
View commit details
Copy full SHA for a83b1ca
Browse repository at this point
Commits on Jul 12, 2024
Update Dockerfile-multiple
Show description for 06108e4
arjunguha
authored
06108e4
View commit details
Copy full SHA for 06108e4
Browse repository at this point
Update MultiPL-E to v3 prompts
Show description for 8eed623
arjunguha
authored
8eed623
View commit details
Copy full SHA for 8eed623
Browse repository at this point
Commits on Jun 24, 2024
Merge pull request #244 from meher-m/transformers_fix
Show description for 334efb7
loubnabnl
authored
334efb7
View commit details
Copy full SHA for 334efb7
Browse repository at this point
Commits on Jun 21, 2024
Update warning message
Show description for cc9033c
meher-m
and
loubnabnl
authored
cc9033c
View commit details
Copy full SHA for cc9033c
Browse repository at this point
Merge pull request #247 from anil-gurbuz/leaderboard-documentation-fix
Show description for 4659ecd
loubnabnl
authored
4659ecd
View commit details
Copy full SHA for 4659ecd
Browse repository at this point
Commits on Jun 18, 2024
Added --save_generations argument
anil-gurbuz
committed
0641b1b
View commit details
Copy full SHA for 0641b1b
Browse repository at this point
Commits on Jun 11, 2024
Adding warning when we catch this error to notify user of behavior
meher-m
committed
c165bb8
View commit details
Copy full SHA for c165bb8
Browse repository at this point
Upgrading to catch only the specific error caused by new version of transformers package
meher-m
committed
44b5e41
View commit details
Copy full SHA for 44b5e41
Browse repository at this point
Commits on Jun 7, 2024
Adding support for transformers>=4.40.2 to avoid crash with mbpp
meher-m
committed
3dd0076
View commit details
Copy full SHA for 3dd0076
Browse repository at this point
Commits on May 29, 2024
Merge pull request #238 from Elfsong/mercury
Show description for f0f2b52
loubnabnl
authored
f0f2b52
View commit details
Copy full SHA for f0f2b52
Browse repository at this point
Commits on May 28, 2024
Refine code format
Elfsong
committed
b73221c
View commit details
Copy full SHA for b73221c
Browse repository at this point
Update usage doc
Elfsong
committed
a90b5d6
View commit details
Copy full SHA for a90b5d6
Browse repository at this point
Commits on May 26, 2024
Merge branch 'main' into mercury
Elfsong
authored
67c4409
View commit details
Copy full SHA for 67c4409
Browse repository at this point
Update readme
Elfsong
committed
49e6f24
View commit details
Copy full SHA for 49e6f24
Browse repository at this point
Format
Elfsong
committed
4b05ad8
View commit details
Copy full SHA for 4b05ad8
Browse repository at this point
Fix a diff
Elfsong
committed
31d4b47
View commit details
Copy full SHA for 31d4b47
Browse repository at this point
Update
Elfsong
committed
2d5991a
View commit details
Copy full SHA for 2d5991a
Browse repository at this point
remove diffs
Elfsong
committed
ba34e71
View commit details
Copy full SHA for ba34e71
Browse repository at this point
Remove print
Elfsong
committed
5baa2c0
View commit details
Copy full SHA for 5baa2c0
Browse repository at this point
recoveing original code
Elfsong
committed
b581e05
View commit details
Copy full SHA for b581e05
Browse repository at this point
Prepare for merging
Elfsong
committed
a1a930c
View commit details
Copy full SHA for a1a930c
Browse repository at this point
Merge branch 'mercury' of https://github.com/Elfsong/bigcode-evaluation-harness into mercury
Elfsong
committed
d9385f3
View commit details
Copy full SHA for d9385f3
Browse repository at this point
Update
Elfsong
committed
0447089
View commit details
Copy full SHA for 0447089
Browse repository at this point
Previous
Next
You can’t perform that action at this time.
