Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] agent example (w/ sandboxable Tools!) & improved OAI compatibility layer (in Python) #6389

Closed
wants to merge 73 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
0d47c43
gguf: add GGUFReader.read_field(field) method + read template example
ochafik Apr 27, 2024
0d1d46e
grammars: add troubleshooting section to readme
ochafik Apr 8, 2024
63d1324
server.py: hacky code
ochafik Mar 25, 2024
ffc7436
agents: scripts to run scripts as sandboxed fastapi servers
ochafik Mar 26, 2024
d5d9993
server.py: default tools work!
ochafik Mar 26, 2024
8afd4de
server.py: make tools work w/ mixtral-8x7b-instruct
ochafik Mar 27, 2024
aa9605c
server.py: kinda api-compliant output, disabled grammar
ochafik Mar 27, 2024
a406293
server.py: reenable grammar, accommodate mistral's escaped underscores
ochafik Mar 27, 2024
63a384d
server.py: raise n_predict
ochafik Mar 28, 2024
5f3de16
server.py: pass all request options, comments in ts sigs, render tool…
ochafik Mar 28, 2024
59b4114
server.py: refactor chat handlers
ochafik Mar 29, 2024
253b68d
server.py: crude reactor
ochafik Mar 29, 2024
e874565
agent: split code from openai example
ochafik Mar 29, 2024
b63f91a
Update agent.py
ochafik Mar 29, 2024
c340e8c
Update example_weather_tools.py
ochafik Mar 29, 2024
ce2fb01
agent: add --allow_parallel_calls
ochafik Mar 29, 2024
ea34bd3
agent/openai:nits
ochafik Mar 29, 2024
80c7930
openai: fix message merging for mixtral (parallel calls)
ochafik Mar 29, 2024
9ab493f
Update prompting.py
ochafik Mar 29, 2024
e0c8af4
agent: --style
ochafik Mar 29, 2024
b4e292e
Create requirements.txt
ochafik Mar 29, 2024
d1d8602
agent: disable parallel by default
ochafik Mar 29, 2024
eb9a552
agent: nits
ochafik Mar 29, 2024
3da30ed
agent: fix functionary tool_calls templating
ochafik Mar 29, 2024
ff6563a
Delete test.sh
ochafik Mar 29, 2024
dd11bb6
agent: format still broken
ochafik Mar 29, 2024
22b980f
agent: update readme
ochafik Mar 29, 2024
61f35e0
agent: prepare to test various templates
ochafik Mar 29, 2024
d8a53ea
openai: test features of templates at runtime, to make sure no bits o…
ochafik Mar 30, 2024
ad2f4c1
Update test_chat_handlers.py
ochafik Mar 30, 2024
3c3eff5
openai: quiet + update prompt output
ochafik Mar 30, 2024
6935503
openai: refactor chat handler vs. template
ochafik Mar 30, 2024
d9f30f8
Update test_chat_handlers.md
ochafik Mar 30, 2024
da2067a
openai: only special-format assistant in thoughtful mode
ochafik Mar 30, 2024
09de4eb
openai: actually use thoughtful examples in tests
ochafik Mar 30, 2024
19811a4
openai: tests didn't catch output format
ochafik Mar 30, 2024
22fe86d
openai tools: TS signatures work well too at a fraction of the eval cost
ochafik Mar 30, 2024
6e52a9c
Update test_chat_handlers.md
ochafik Apr 8, 2024
701a66d
agent: fix response_format
ochafik Apr 9, 2024
b447a74
agent: revert to json schemas (ts not ready for refs)
ochafik Apr 9, 2024
85820f4
agent: fix sandbox dockerfile
ochafik Apr 9, 2024
6880f1d
agent: support basic openapi tools (incl. from fastify sandbox)
ochafik Apr 9, 2024
0532680
agent: nits
ochafik Apr 9, 2024
a634e03
agent: cache_prompt=True
ochafik Apr 10, 2024
9fe269e
openai: nit
ochafik Apr 10, 2024
a61ebeb
agent: hint at math import in python tool
ochafik Apr 10, 2024
24e34f1
agent: nit
ochafik Apr 10, 2024
1475b1e
agent: fix killing of subprocesses
ochafik Apr 10, 2024
6c00378
agent: nits
ochafik Apr 10, 2024
082d54d
agent: rename fake weather tools
ochafik Apr 10, 2024
f9afb04
agent: python tool: test serializability of variables
ochafik Apr 10, 2024
a98f483
agent: python tool: return errors
ochafik Apr 10, 2024
ea0c31b
agent: ensure DATA_DIR exists
ochafik Apr 10, 2024
89dcc06
agent: mypy type fixes
ochafik Apr 10, 2024
0120f7c
agent: fix wait --std-tools
ochafik Apr 10, 2024
09c2565
grammars: early exit when no next_candidates to reject
ochafik Apr 21, 2024
00c709e
grammars: cache decoded tokens
ochafik Apr 21, 2024
8d503ef
grammars: faster llama_grammar_copy
ochafik Apr 21, 2024
b4a00ce
Merge branch 'gguf-read' into agent-example
ochafik Apr 27, 2024
7675ac6
Merge remote-tracking branch 'origin/master' into agent-example
ochafik Apr 30, 2024
312e20b
openai: update after merge
ochafik Apr 30, 2024
ca1a640
server: tool call grammar-constraints
ochafik May 2, 2024
2b2127c
agent: url params
ochafik May 2, 2024
e41b6ce
server: update tool calling, introduce system prompt for json schema
ochafik May 2, 2024
a1d64cf
openai: function call arguments must be returned stringified!
ochafik May 18, 2024
3f5a25f
Merge remote-tracking branch 'origin/master' into agent-example
ochafik May 18, 2024
5ea637e
openai: fix merge
ochafik May 21, 2024
6dadcd2
Merge remote-tracking branch 'origin/master' into agent-example
ochafik May 21, 2024
c8458fa
openai: make content optional for tool call grammar gen
ochafik May 22, 2024
a39e6e0
openai: pretty indent json response
ochafik May 22, 2024
793f4ff
agent: support OpenAI: --endpoint https://api.openai.com --auth "Bear…
ochafik May 22, 2024
a1c4aac
server: ultra basic tools, tool_choice, tool_calls support
ochafik May 22, 2024
298c098
Merge remote-tracking branch 'origin/master' into agent-example
ochafik Jun 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
agent: fix sandbox dockerfile
  • Loading branch information
ochafik committed Apr 27, 2024
commit 85820f4401f030165b28c2a7d1af9fd2652a5070
8 changes: 4 additions & 4 deletions examples/agent/run_sandboxed_tools.sh
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,14 @@ echo "
RUN mkdir /src /data

# Copy resources in increasing likelihood of change, to keep as much as possible cached
COPY fastify-requirements.txt /root
COPY fastify-requirements.txt /root/
RUN pip install -r /root/fastify-requirements.txt
COPY script-requirements.txt /root
COPY script-requirements.txt /root/
RUN pip install -r /root/script-requirements.txt
COPY fastify.py utils.py /root
COPY fastify.py utils.py /root/examples/agent/

WORKDIR /data
ENTRYPOINT PYTHONPATH=/src python /root/fastify.py --port=$PORT '/src/$( basename "$script" )'
ENTRYPOINT PYTHONPATH=/src:/root python -m examples.agent.fastify --port=$PORT '/src/$( basename "$script" )'
" | docker build "$BUILD_DIR" -f - -t "$LLAMA_IMAGE_NAME"

echo "#"
Expand Down