v1.4.0 #43
Merged
v1.4.0 #43
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* chore: add codeowners file * chore: add python poetry action and docs workflow * chore: update pre-commit file * chore: update docs * chore: update logo * chore: add cicd pipeline for automated deployment * chore: update poetry version * chore: fix action versioning * chore: add gitattributes to ignore line count in jupyter notebooks * chore: add and update docstrings * chore: fix end of files * chore: update action versions * Update README.md --------- Co-authored-by: mo374z <schlager.mo@t-online.de>
* chore: fix workflow execution * chore: fix version check in CICD pipeline
* update gitignore * initial implementation of opro * formatting of prompt template * added opro test run * opro refinements * fixed sampling error * add docs to opro * fix pre commit issues# * fix pre commit issues# * fixed end of line
* fixed pre commit config and removed end of file line breaks in tempaltes * added /
* added prompt_creation.py * change version
* Remove deepinfra file * change langchain-community version
* renamed get_tasks to get_task and change functionality accordingly. moved templates and data_sets * init * move templates to templates.py * Add nested asyncio to make it useable in notebooks * Update README.md * changed getting_started.ipynb and created helper functions * added sampling of initial population * fixed config * fixed callbacks * adjust runs * fix run evaluation api token * fix naming convention in opro, remove on epoch end for logger callback, fixed to allow for numeric values in class names * Update promptolution/llms/api_llm.py Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com> * fixed comments * Update pyproject.toml * resolve comments --------- Co-authored-by: mo374z <schlager.mo@t-online.de> Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com> Co-authored-by: Moritz Schlager <87517800+mo374z@users.noreply.github.com>
* implemented random selector * added random search selector * increased version count * fix typos * Update promptolution/predictors/base_predictor.py Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com> * Update promptolution/tasks/classification_tasks.py Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com> * resolve comments * resolve comments --------- Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com>
* Update release-notes.md * Fix release note links
This reverts commit e23dd74.
* Delete Experiment files * Removed config necessities * improved opro meta-prompts * added read from data frame feature * changed required python version to 3.9
* delete poetry.lock and upgrade transformers dependency * Update release-notes.md
* make vllm class deterministic * fixes in prompt creation
* fixes in the "MarkerBasedClassificator" * generalize the Classificator * add verbosity and callback handling in EvoPromptGA * add timestamp to the callback
* v1.3.1 (#37) #### Added features * new features for the VLLM Wrapper (accept seeding to ensure reproducibility) * fixes in the "MarkerBasedClassificator" * fixes in prompt creation and task description handling * generalize the Classificator * add verbosity and callback handling in EvoPromptGA * add timestamp to the callback * removed datasets from repo * changed task creation (now by default with a dataset) * add generation prompt to vllm input * allow for parquet as fileoutput callback * added sys_prompts * change usage of csv callbacks * add system prompt to token counts * fix merge issues * drag system prompts from api to task * added release notes --------- Co-authored-by: Moritz Schlager <87517800+mo374z@users.noreply.github.com>
* v1.3.1 (#37) #### Added features * new features for the VLLM Wrapper (accept seeding to ensure reproducibility) * fixes in the "MarkerBasedClassificator" * fixes in prompt creation and task description handling * generalize the Classificator * add verbosity and callback handling in EvoPromptGA * add timestamp to the callback * removed datasets from repo * changed task creation (now by default with a dataset) * opro reimplementation according to the paper * fine opro implementation * opro test scripts alignment * implement opro review * small fix in score handling * adjust hyperparameters * add early stopping at convergence to opro * Update promptolution/optimizers/opro.py Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com> --------- Co-authored-by: Moritz Schlager <87517800+mo374z@users.noreply.github.com> Co-authored-by: mo374z <schlager.mo@t-online.de> Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com>
* v1.3.2 (#40) #### Added features * Allow for configuration and evaluation of system prompts in all LLM-Classes * CSV Callback is now FileOutputCallback and able to write Parquet files * Fixed LLM-Call templates in VLLM * refined OPRO-implementation to be closer to the paper * implement api calls * removed default for system messages * roll back renaming --------- Co-authored-by: mo374z <schlager.mo@t-online.de>
* add token count, flexible batch size and kwargs to vllm class * add testing script for implementation * fix batch size calculation * small changes * add revision test * add argument to parser * max model len to int * remove script * Change version and Release notes * changed callback behaviour and impelemented token count callback * init * small corrections * changes to prevent merge conflicts * small changes * first tests * add api_test * a lot has changed for gangsters * fix * added * removed * fix * Update promptolution/predictors/__init__.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * add testing to ci pipeline * Update action.yml * fix test dependencies in pipeline * Add further test dependencies * Refactor dependency groups * Add getting started notebook to documentation * Add workflow call trigger to docs workflow * Add CI and Docs status badges * Add temporary file for docs testing * Remove temporary file for docs testing * Update notebooks/getting_started.ipynb Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com> * Update notebooks/getting_started.ipynb Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com> * comments * end2end * added * new * more * change * update dependencies * added literals to get optimizer * added release notes --------- Co-authored-by: mo374z <schlager.mo@t-online.de> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Timo Heiß <ti-heiss@t-online.de> Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR releases version v1.4.0 of Promptolution and introduces significant reworks across LLM modules, configuration management, and documentation. Key changes include the introduction of ExperimentConfig, refactoring of API- and local-based LLM implementations, and updates to documentation and CI workflows.
Reviewed Changes
Copilot reviewed 59 out of 59 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| promptolution/llms/vllm.py | Updated import mechanism, added config support, and cleanup adjustments in vLLM initialization. |
| promptolution/llms/local_llm.py | Updated import and constructor parameters to include ExperimentConfig. |
| promptolution/llms/base_llm.py | Adjusted abstract method signatures and constructor to support config-based initialization. |
| promptolution/llms/api_llm.py | Reworked async invocation logic and API client integration using OpenAI standards. |
| promptolution/init.py and others | Refactored factory function and config handling across the package. |
| docs/, notebooks/, .github/workflows/* | Updated documentation, release notes, and CI/CD workflows. |
timo282
approved these changes
Apr 22, 2025
mo374z
approved these changes
Apr 22, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.