Skip to content

LLM attribute calculation #287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 65 commits into from
Feb 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
cc0c65b
build: init LLM attribute calculation
andhreljaKern Jan 13, 2025
9ad0f9e
chore: submodules/model
andhreljaKern Jan 13, 2025
8f2b04c
perf: separate exec_env from sample LLM ac
andhreljaKern Jan 14, 2025
08d40da
perf: add additional_config column to attribute table
andhreljaKern Jan 14, 2025
abd574c
chore: update submodules/model
andhreljaKern Jan 14, 2025
4c0d7ed
feat: attribute calculation DB updates
andhreljaKern Jan 14, 2025
80f3c57
fix: revert custom llm functions
andhreljaKern Jan 14, 2025
8d99b4a
adds llm template function for llm attribute calc
DerKernigeFeuerpfeil Jan 14, 2025
af87bcf
adds llm code wrap to the attribute calculation of exec env
DerKernigeFeuerpfeil Jan 14, 2025
ef51bfd
chore: update submodules
andhreljaKern Jan 16, 2025
0e2e579
style: recalculate alembic migration
andhreljaKern Jan 16, 2025
24e6ee9
feat: LLM_RESPONSE attribute calculation
andhreljaKern Jan 16, 2025
37b09b6
style: syntax updates
andhreljaKern Jan 16, 2025
1b6616d
chore: development cleanup
andhreljaKern Jan 17, 2025
00fa1af
fix: PR review
andhreljaKern Jan 17, 2025
16849fe
perf: avoid variable overrides by users
andhreljaKern Jan 17, 2025
a8f92d1
adds better LLM config error message
DerKernigeFeuerpfeil Jan 20, 2025
5cc2adc
adds llm connection check before running LLM attribution calc
DerKernigeFeuerpfeil Jan 20, 2025
7e35dc0
adds user prompt validation + code improvements
DerKernigeFeuerpfeil Jan 20, 2025
1dc42b6
adds JSON mode to openai API code
DerKernigeFeuerpfeil Jan 20, 2025
b5ee7e5
Adds add config to id endpoint for attributes
JWittmeyer Jan 21, 2025
461a128
style: fix flake8 warnings
andhreljaKern Jan 21, 2025
27cd681
fix: error message definition
andhreljaKern Jan 21, 2025
51bf48a
perf: llm_response_tmpl LLM kwargs update
andhreljaKern Jan 21, 2025
6ec97c1
perf: add `run-llm-playground` route
andhreljaKern Jan 21, 2025
492d435
fix: variable formatting in tmpl + llm_config for playground
andhreljaKern Jan 23, 2025
d0e0316
fix: set defaults for endpoint and apiVersion llm_config
andhreljaKern Jan 23, 2025
435e13c
In call changes
JWittmeyer Jan 23, 2025
b164ca0
fix: minor fixes
andhreljaKern Jan 23, 2025
7e97bb9
fix: singular string in f-string expression
andhreljaKern Jan 23, 2025
885b27f
perf: skip setting progress for llm playground
andhreljaKern Jan 23, 2025
ef3ad35
fix: playground logs + raising exceptions
andhreljaKern Jan 23, 2025
9a05079
error message fix
JWittmeyer Jan 23, 2025
1872fd3
Adds str cast only for none str values in result dict
JWittmeyer Jan 23, 2025
7dc4aa2
Removes logs from none playgroudn ac return
JWittmeyer Jan 23, 2025
410f2b5
style: omit error_message variable definitions
andhreljaKern Jan 23, 2025
3a47d16
style: minor fixes
andhreljaKern Jan 23, 2025
ecfb0eb
fix: apiBase and apiVersion as non-default keys
andhreljaKern Jan 23, 2025
bc09d2f
add single quote stop sequence to ensure json dumps works
JWittmeyer Jan 23, 2025
e70cfd2
Adds correct defalult values
JWittmeyer Jan 24, 2025
e077268
multi line sys prompt
JWittmeyer Jan 27, 2025
0450485
feat: add asynchronous chat completion and configurable number of wor…
DerKernigeFeuerpfeil Jan 28, 2025
8e6c979
feat: add retry mechanism for LLM API calls with configurable parameters
DerKernigeFeuerpfeil Jan 28, 2025
df170c2
Change to recusive approach for str conversion & async in playground …
JWittmeyer Jan 28, 2025
a113fcf
Adds retry execeded error message
JWittmeyer Jan 28, 2025
2054f3c
Return result on error case as error result
JWittmeyer Jan 28, 2025
8752ba6
Adds LLM response to text like datatypes
JWittmeyer Feb 3, 2025
9f370fb
perf: llm ac caching
andhreljaKern Feb 3, 2025
c78f3ef
chore: pending llm cache endpoint
andhreljaKern Feb 3, 2025
2a50929
Merge branch 'llm-ac' of https://github.com/code-kern-ai/refinery-gat…
andhreljaKern Feb 3, 2025
4405894
feat: enhance get_llm_response with caching mechanism
DerKernigeFeuerpfeil Feb 3, 2025
078d934
perf: llm-ac-cache endpoint
andhreljaKern Feb 4, 2025
4dccdce
style: llm-ac-cache return payload
andhreljaKern Feb 4, 2025
83c79a7
refactor: rename get_llm_config to get_llm_config_a2vybg for clarity
DerKernigeFeuerpfeil Feb 4, 2025
4d81d96
style: variable rename
andhreljaKern Feb 4, 2025
37072dd
perf: add s3 obj deletion on attr deletion
andhreljaKern Feb 4, 2025
c9f5f46
perf: simpler conditional in controller/attribute/util.py
andhreljaKern Feb 6, 2025
a68511d
style: PR fixes
andhreljaKern Feb 6, 2025
a68d210
fix: apiKey deletion from frontend - attributes
andhreljaKern Feb 6, 2025
94e3385
perf: remove global ref to static var
andhreljaKern Feb 7, 2025
29da545
chore: pr review
andhreljaKern Feb 13, 2025
2c4aa46
chore: pr review
andhreljaKern Feb 13, 2025
1a76c31
Merge wiht dev
JWittmeyer Feb 13, 2025
e66b380
Fix alembic
JWittmeyer Feb 13, 2025
0820dac
Submodule merge
JWittmeyer Feb 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions alembic/versions/10c48793371d_adds_attribute_additional_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
"""adds attribute `additional_config` and llm_config

Revision ID: 10c48793371d
Revises: 0c8eb3ff1c71
Create Date: 2025-01-15 17:08:30.137845

"""
from alembic import op
import sqlalchemy as sa


# revision identifiers, used by Alembic.
revision = '10c48793371d'
down_revision = '0c8eb3ff1c71'
branch_labels = None
depends_on = None


def upgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.add_column('attribute', sa.Column('additional_config', sa.JSON(), nullable=True, comment='used when data_type == LLM_RESPONSE'))
# ### end Alembic commands ###


def downgrade():
# ### commands auto generated by Alembic - please adjust! ###
op.drop_column('attribute', 'additional_config')
# ### end Alembic commands ###
Loading