Skip to content

Conversation

@bauersimon
Copy link
Member

Early merger is #95.

@bauersimon bauersimon requested a review from zimmski May 7, 2024 10:22
@bauersimon bauersimon self-assigned this May 7, 2024
@bauersimon bauersimon force-pushed the ollama branch 2 times, most recently from b3da82a to 51ad917 Compare May 7, 2024 16:35
@bauersimon
Copy link
Member Author

TODO: tests pass when targeted manually but running make test fails nearly all the tests with the reason that the local Ollama server is unreachable. I guess this is because we always use the default port and the tests run in parallel, but running them with the -p 1 flag does not work either.

@bauersimon
Copy link
Member Author

bauersimon commented May 8, 2024

# === TERMINAL A ===
# Checkout and install Ollama as part of tools.
git fresh-checkout ollama
make install-all

# === TERMINAL B
# Ollama server instance which manages all models.
~/.eval-dev-quality/bin/ollama serve &

# === TERMINAL A ===
# Pull some models (check https://ollama.com/library for more but watch out, many are several GBs large).
~/.eval-dev-quality/bin/ollama pull qwen:0.5b
~/.eval-dev-quality/bin/ollama pull tinyllama:latest

# === TERMINAL B ===
# Kill the server, or don't cause our eval sould either start a new one or reuse the existing one.

# === TERMINAL A ===
# Run the eval with only the models we pulled.
eval-dev-quality evaluate --model ollama/qwen:0.5b --model ollama/tinyllama:latest

@Munsio @ahumenberger

@Munsio
Copy link
Contributor

Munsio commented May 8, 2024

I've run the bash script and it gave me the following output:
https://gist.github.com/Munsio/8dcae3ff9d58c0ccf25401287ea42619

@bauersimon bauersimon force-pushed the ollama branch 3 times, most recently from 89e9eff to 1311539 Compare May 8, 2024 10:32
@bauersimon bauersimon marked this pull request as ready for review May 8, 2024 10:36
zimmski pushed a commit that referenced this pull request May 11, 2024
zimmski pushed a commit that referenced this pull request May 11, 2024
@zimmski zimmski force-pushed the ollama branch 2 times, most recently from f603a82 to e256bda Compare May 11, 2024 19:43
@zimmski zimmski added the enhancement New feature or request label May 11, 2024
@zimmski zimmski added this to the v0.5.0 milestone May 11, 2024
@zimmski zimmski linked an issue May 11, 2024 that may be closed by this pull request
10 tasks
@zimmski zimmski force-pushed the ollama branch 13 times, most recently from f44693f to cc46ef3 Compare May 15, 2024 12:20
@zimmski zimmski merged commit 6f0e539 into main May 15, 2024
@zimmski zimmski deleted the ollama branch May 15, 2024 12:54
@zimmski zimmski mentioned this pull request May 16, 2024
45 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integrate Ollama

4 participants