Refactor/interfaces (#42)

* add token count, flexible batch size and kwargs to vllm class

* add testing script for implementation

* fix batch size calculation

* small changes

* add revision test

* add argument to parser

* max model len to int

* remove script

* Change version and Release notes

* changed callback behaviour and impelemented token count callback

* init

* small corrections

* changes to prevent merge conflicts

* small changes

* first tests

* add api_test

* a lot has changed for gangsters

* fix

* added

* removed

* fix

* Update promptolution/predictors/__init__.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add testing to ci pipeline

* Update action.yml

* fix test dependencies in pipeline

* Add further test dependencies

* Refactor dependency groups

* Add getting started notebook to documentation

* Add workflow call trigger to docs workflow

* Add CI and Docs status badges

* Add temporary file for docs testing

* Remove temporary file for docs testing

* Update notebooks/getting_started.ipynb

Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com>

* Update notebooks/getting_started.ipynb

Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com>

* comments

* end2end

* added

* new

* more

* change

* update dependencies

* added literals to get optimizer

* added release notes

---------

Co-authored-by: mo374z <schlager.mo@t-online.de>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Timo Heiß <ti-heiss@t-online.de>
Co-authored-by: Timo Heiß <87521684+timo282@users.noreply.github.com>

Author: 4 people

.coverage

@@ -34,5 +34,6 @@ runs:
         path: .venv
         key: ${{ runner.os }}-poetry-${{ inputs.poetry-version }}-${{ hashFiles('**/poetry.lock') }}
     - name: Install dependencies
-      run: poetry install --no-interaction --no-root --only ${{ inputs.groups }}
+      run:
+        poetry install --no-interaction --no-root --only ${{ inputs.groups }}
       shell: bash

.github/actions/python-poetry/action.yml

@@ -21,11 +21,11 @@ jobs:
       - name: Set up Python and Poetry
         uses: ./.github/actions/python-poetry
         with:
-          groups: dev
+          groups: main,dev,test
       - name: Run pre-commit
         uses: pre-commit/action@v3.0.1
-      # - name: Run tests
-      #   run: poetry run python -m pytest
+      - name: Run tests
+        run: poetry run python -m pytest
 
   build:
     runs-on: ubuntu-latest

.github/workflows/ci.yml

@@ -4,6 +4,7 @@ on:
   push:
     branches:
       - main
+  workflow_call:
 
 permissions:
   contents: write
@@ -22,6 +23,10 @@ jobs:
         with:
           groups: "docs"
 
+      - name: Generate notebook examples
+        run: |
+          poetry run jupyter nbconvert --to markdown --allow-errors --output-dir docs/examples notebooks/*.ipynb
+
       - name: Deploy docs
         run: |
           poetry run mkdocs gh-deploy --force

.github/workflows/docs.yml

@@ -9,3 +9,5 @@ dist/
 outputs/
 results/
 poetry.lock
+CLAUDE.md
+**/CLAUDE.local.md

.gitignore

@@ -1,4 +1,8 @@
 ![promptolution](https://github.com/user-attachments/assets/84c050bd-61a1-4f2e-bc4e-874d9b4a69af)
+
+[![CI](https://github.com/finitearth/promptolution/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/finitearth/promptolution/actions/workflows/ci.yml)
+[![Docs](https://github.com/finitearth/promptolution/actions/workflows/docs.yml/badge.svg?branch=main)](https://github.com/finitearth/promptolution/actions/workflows/docs.yml)
+
 # Promptolution
 Promptolution is a library that provides a modular and extensible framework for implementing prompt tuning experiments. It offers a user-friendly interface to assemble the core components for various prompt optimization tasks.
 

README.md

@@ -1,6 +1,6 @@
 # Config
 
-::: promptolution.config.Config
+::: promptolution.config.ExperimentConfig
     options:
       show_root_heading: true
       show_root_full_path: false

docs/api/config.md

@@ -1,5 +1,17 @@
 # Release Notes
 
+## Release v1.4.0
+### What's changed
+#### Added features
+* Reworked APILLM to allow for calls to any API that follows the OpenAI API format
+* Added graceful failing in optimization runs, allowing to obtain results after an error
+* Reworked configs to ExperimentConfig, allowing to parse any attributes
+
+### Further Changes:
+* Reworked getting started notebook
+* Added tests for the entire package, covering roughly 80% of the codebase
+* Reworked dependency and import structure to allow the usage of a subset of the package
+
 ## Release v1.3.2
 ### What's changed
 #### Added features

docs/examples/getting_started.md

@@ -53,6 +53,8 @@ nav:
     - Tasks: api/tasks.md
     - Callbacks: api/callbacks.md
     - Config: api/config.md
+  - Tutorials:
+    - Getting Started: examples/getting_started.md
 
 markdown_extensions:
   - pymdownx.highlight: