Docstring consistency #168
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 30 commits
December 11, 2024 06:46
added 15 commits
June 12, 2025 18:31
nushib
reviewed
Jul 9, 2025
| try: | ||
| # Check if string is a tuple representation and has more than one element | ||
| if s[0] == '(' and s[-1] == ')' and len(sl) > 1: | ||
| # Evaluate each element using latex2sympy and round the result to 2 decimal places |
nushib
reviewed
Jul 10, 2025
| if len(string.split("\\approx")) == 2: | ||
| string = string.split("\\approx")[-1] | ||
|
|
||
| # Fix sqrt values not wrapped in curly braces. Note: The function _fix_sqrt is not provided. |
| except: | ||
| ans = s | ||
|
|
||
| # If there's a closing bracket without an opening bracket before it, consider everything before it. |
There was a problem hiding this comment.
why is the comment here removed?
| def transform(self, df: pd.DataFrame) -> pd.DataFrame: | ||
| """ | ||
| Some models (e.g. GPT3.5) sometimes does not follow instructions and do not add the "Output:\n" marker in its format | ||
| add_output_marker adds a "Output:\n" string to the model output prior to parsing, if it detects a list start, i.e. "1." |
| if last_open == -1: | ||
| return None | ||
| sliced = model_output[last_open:] | ||
| # Grab everything between <final_answer> and </final_answer> |
| if title == "Justification": | ||
|
|
||
| if title == "Justification".lower(): |
There was a problem hiding this comment.
seems like an ok change but flagging anyways
| Returns: | ||
| int: The token usage for the row. | ||
| int or float: The extracted token count if present, otherwise NaN. |
| @@ -1,359 +1,509 @@ | |||
| # This file was authored by BenchAgents authors and is being reused under the MIT license. | |||
There was a problem hiding this comment.
this is a bit hard to review because the whole code was removed and added again
| results_strict = test_instruction_following_strict( | ||
| row["prompt"], row["response"], row["instruction_id_list"], row["kwargs"] | ||
| ) | ||
| results_loose = {} |
There was a problem hiding this comment.
minor edit, previously not here
| """ | ||
| # Pattern for numbers with commas | ||
| pattern_commas = r"-?\b\d{1,3}(?:,\d{3})+\b" | ||
| # Pattern for scientific notation |
There was a problem hiding this comment.
removed comment, several of these in this function
| if answer == pred_i: | ||
| correct = True | ||
| break | ||
| else: # gold_i is a string |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
1- Adds docstrings where missing, and improves consistency of existing ones (Google style)
2- Introduces a Eureka pipeline config for adding docstrings to a set of python files.