Skip to content

Commit

Permalink
Release 4.0.0 (#382)
Browse files Browse the repository at this point in the history
* Update package

* Bugfixes

* Add migration file

* adding ability to add to dataset from traces tab

* fixing z-index bug

* fixing llm parsing, disable button for non llm traces

* cleaning up api providers page

* more bug fixes, auditing models

* adding api key auth to create project api key (#270)

* Improvements to prompt playground (#272)

* fixing static text area, fixing overlap

* creating custo ui for playground taces

* fetching traces on dialog open

* fixes

* fixes

* fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* fix

* fix

* support for o1-preview and o1-mini (#275)

* fixing live prompt bug (#277)

* DSPy enhancements (#280)

* DSPy experiments

* LiteLLM support

* eval charts

* experiment metrics

* empty state

* minor fix

* fix

* clearing invite form fields on send (#279)

* Track project creation count metrics (#271)

* adding posthog

* bug fixes, adding new env vars

* adding posthog api key to dockerfile

* updating read me

* removing sensitive data

* removing more data

* adding sign up count by team

* minor

* update

* fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* minor

* minor (#282)

* add vercel ai pricing (#283)

* fix (#284)

* Fixes (#289)

* Add `Gemini` Cost table (#291)

* add gemini cost tables

* remove protobuf helper

* Fixing token counts if `output_tokens` is `0` + adding embedding cost tables (#292)

* make tracing endpoint `route.ts` oTel compatible backend. (#298)

* make tracing endpoint `route.ts` oTel compatible backend.

* handle null events

* fallback for status code

* remove console statement

* Upload csv feature (#265)

* all changes for upload csv feature

* upddated package.json.lock

* updated package.lock.json

* update package lock

* fixes

* validation

---------

Co-authored-by: Oluwadabira Akinwumi <dabira.akinwumi@gmail.com>
Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update package lock (#299)

* all changes for upload csv feature

* upddated package.json.lock

* updated package.lock.json

* update package lock

* fixes

* validation

* validation

---------

Co-authored-by: Oluwadabira Akinwumi <dabira.akinwumi@gmail.com>

* add proto files inside `next.config.mjs` and migrate protofiles to root level (#302)

* Support xAI (#306)

* support xai

* support xai

* Upmerge from main (#316)

* Release 1.5.11 (#172)

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating descriptions to optional

* prio python

* cleanup and fixes

* more bug fixes

* more fixes

* remove console log

* updating trace_service functions

* add migration

* add format function, updating from day to hour

* adding dropwdown menu

* updating query key

* updating query keys v2

* clean up

* fix bug

* Minor bugfix (#102)

* Pagination bug

* Bug fix

* Minor bugfix

* fixes

* Bug fixes to Evaluations  (#111)

* Pagination bug

* Bug fix

* Bug fixes to evals v2

* Minor fix

* upgrading tiktoken, adding omni to pricing (#110)

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update package

* Dataset bugfix (#113)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor linking fix (#114)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor fix

* api bug fix (#116)

* fixing dropdown color (#117)

* Support for Image inputs (#119)

* Pagination bug

* Bug fix

* Support image inputs

* Bug fix Spans with status_code = "ERROR" (#120)

* Pagination bug

* Bug fix

* Support image inputs

* Bug fixes

* Dylan/s3en 2265 save prompt and model settings directly from playground (#99)

* bug fixes, adding save prompt button in playground

* adding save prompt dialog

* adding create registry dialog

* working registry selection

* Adding new prompt to exist registry

* storing

* button padding fix

* refactor, adding combobox

* increasing dropdwon width

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* adding image display

* adding property filtering (#123)

* optimizing cost function

* optimizing per cost per hour function

* removing console statements

* updating queryKey for datasets

* init advanced filter dialog

* adding operator combobox

* updating ui

* adding attribute type

* bug fixes

* cleanup

* remove console

* updating queryKeys

* adding missing table checks

* adding more checks

* chore: update clickhouse docker for user management

* style: remove blank space

* updating save prompt button

* Redesign of Manual Evaluations (#134)

* Pagination bug

* Bug fix

* Schema changes for evals

* Fix APIs for evals

* Minor bugfixes

* Bugfix

* Evaluation changes

* Manual eval simplification

* Minor fix

* Manual evaluations UX changes

* Chart title fix

* Disable experiments

* fix

* Dylan/s3en 2318 add the ability to import a conversation from the traces to (#133)

* adding import conversation button

* init import conversation dialog

* adding trace rows

* fixing scroll bug

* adding import prompt to playground

* fixes

* Minor

* Bugfixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* storing

* remove import for images

* importing attribute stuff

* fixing operations display

* cleaning up advanced filters

* cleanup

* docs: update docker instructions

* removing console, fixing button

* add stars history (#139)

* Dylan/s3en 2401 ability to filter by user for metrics and traces (#138)

* adding user id combobox, api

* fixing sql query

* removing user filter on deselect{

* moving user combobox into filter dialog

* adding filtering by user for token charts

* bug fixes, more userId route, function updates

* latency filtering

* updating to alphanumeric

* removing console, comment

* updating default filter operation to and

* adding clear all button, cleanup

* Dylan/s3en 2350 aggregate metrics by models v2 (#141)

* adding user id combobox, api

* fixing sql query

* removing user filter on deselect{

* moving user combobox into filter dialog

* adding filtering by user for token charts

* bug fixes, more userId route, function updates

* latency filtering

* updating to alphanumeric

* removing console, comment

* updating default filter operation to and

* adding clear all button, cleanup

* init commit model combobox

* adding api, updating function, adding to metrics page

* adding model to charts, apis, service

* remove consoles

* Automated Evaluations - Support for Inspect AI (#142)

* Pagination bug

* Bug fix

* Experiments view

* re-order

* Comparison view

* Experiment report

* Experiments - hook up APIs

* Experiments pagination

* deduplicate paginated data

* Fix loading and error states - experiments

* Rename experiments and evaluations

* Add loading spinner

* Link evaluation

* Link to evals

* Fixes

* Minor UI fixes

* Bugfixes

* DSPy support (#144)

* Pagination bug

* Bug fix

* Bugfixes

* DSPy support

* Update README

* Prompt registry and Playground integration (#146)

* Pagination bug

* Bug fix

* Bugfixes

* Playground enhancements

* Fix

* Playground improvment

* Bug fix

* Minor bug fixes (#148)

* Pagination bug

* Bug fix

* Bugfixes

* Bugfixes

* feat: adding example code for starters

* Bugfixes (#152)

* Pagination bug

* Bug fix

* Bugfixes

* crewai support

* fix ollama bug

* updating card link to traces

* adding copy to clipboard for prompt id

* Dylan/s3en 2415 expose project creation api for hivenetwork (#156)

* adding api key auth for project, displaying team id

* migrating, adding api key gen to teams page

* removing comment

* removing apiKeyHash from project return

* Minorfixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Dylan/s3en 2393 update UI tracing tab to show the prompt version being used v2 (#154)

* Adding prompt id to traces table

* adding prompt combobox, api route, service function

* adding prompt version to traces tab

* Fix

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* fix for generating api key

* dynamically loading attributes and vendors

* updating langtrace attributes sdk

* Fix database migration drift (#166)

* Pagination bug

* Bug fix

* Bugfixes

* Fix migration drift

* Dylan/s3en 2507 introduce a settings page for project (#163)

* proejct settings init

* adding settings page

* adding delete card

* Minor fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Dylan/s3en 2527 surface project id on project settings page (#165)

* proejct settings init

* adding settings page

* adding delete card

* surfacing project id on settings page

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* decreasing popup size, translating right

* scrolling fix

* fixing scrolling in attributes list

* Support for Azure AD Login for Self Hosted Setups (#170)

* add azure SSO

* add azure SSO

* Update README

* Azure AD variables

---------

Co-authored-by: Obinna Okafor <obinna.okafor01@gmail.com>
Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Add migration file

* Add migration file for ext_expires_in (#171)

* Pagination bug

* Bug fix

* Update README.md (#76)

Update screenshot

* Update README.md (#77)

Add reference architecture

* Release  (#80)

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

---------

Co-authored-by: Karthik Kalyanaraman <105607645+karthikscale3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>

* Release 1.3.0 (#85)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>

* Release 1.3.1 (#87)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>

* Release 1.3.2 (#92)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Release 1.3.2 (#95)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Release 1.3.4 (#100)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Release 1.4.0 (#108)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating descriptions to optional

* prio python

* cleanup and fixes

* more bug fixes

* more fixes

* remove console log

* updating trace_service functions

* add migration

* add format function, updating from day to hour

* adding dropwdown menu

* updating query key

* updating query keys v2

* clean up

* fix bug

* Minor bugfix (#102)

* Pagination bug

* Bug fix

* Minor bugfix

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Bump next from 14.1.0 to 14.1.1 (#93)

Bumps [next](https://github.com/vercel/next.js) from 14.1.0 to 14.1.1.
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v14.1.0...v14.1.1)

---
updated-dependencies:
- dependency-name: next
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update package

* Release 1.4.2 (#112)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating descriptions to optional

* prio python

* cleanup and fixes

* more bug fixes

* more fixes

* remove console log

* updating trace_service functions

* add migration

* add format function, updating from day to hour

* adding dropwdown menu

* updating query key

* updating query keys v2

* clean up

* fix bug

* Minor bugfix (#102)

* Pagination bug

* Bug fix

* Minor bugfix

* fixes

* Bug fixes to Evaluations  (#111)

* Pagination bug

* Bug fix

* Bug fixes to evals v2

* Minor fix

* upgrading tiktoken, adding omni to pricing (#110)

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update package

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Release 1.4.3 (#115)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating descriptions to optional

* prio python

* cleanup and fixes

* more bug fixes

* more fixes

* remove console log

* updating trace_service functions

* add migration

* add format function, updating from day to hour

* adding dropwdown menu

* updating query key

* updating query keys v2

* clean up

* fix bug

* Minor bugfix (#102)

* Pagination bug

* Bug fix

* Minor bugfix

* fixes

* Bug fixes to Evaluations  (#111)

* Pagination bug

* Bug fix

* Bug fixes to evals v2

* Minor fix

* upgrading tiktoken, adding omni to pricing (#110)

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update package

* Dataset bugfix (#113)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor linking fix (#114)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor fix

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Release 1.4.4 (#118)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating descriptions to optional

* prio python

* cleanup and fixes

* more bug fixes

* more fixes

* remove console log

* updating trace_service functions

* add migration

* add format function, updating from day to hour

* adding dropwdown menu

* updating query key

* updating query keys v2

* clean up

* fix bug

* Minor bugfix (#102)

* Pagination bug

* Bug fix

* Minor bugfix

* fixes

* Bug fixes to Evaluations  (#111)

* Pagination bug

* Bug fix

* Bug fixes to evals v2

* Minor fix

* upgrading tiktoken, adding omni to pricing (#110)

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update package

* Dataset bugfix (#113)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor linking fix (#114)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor fix

* api bug fix (#116)

* fixing dropdown color (#117)

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Release 1.4.5 (#121)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating descriptions to optional

* prio python

* cleanup and fixes

* more bug fixes

* more fixes

* remove console log

* updating trace_service functions

* add migration

* add format function, updating from day to hour

* adding dropwdown menu

* updating query key

* updating query keys v2

* clean up

* fix bug

* Minor bugfix (#102)

* Pagination bug

* Bug fix

* Minor bugfix

* fixes

* Bug fixes to Evaluations  (#111)

* Pagination bug

* Bug fix

* Bug fixes to evals v2

* Minor fix

* upgrading tiktoken, adding omni to pricing (#110)

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update package

* Dataset bugfix (#113)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor linking fix (#114)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor fix

* api bug fix (#116)

* fixing dropdown color (#117)

* Support for Image inputs (#119)

* Pagination bug

* Bug fix

* Support image inputs

* Bug fix Spans with status_code = "ERROR" (#120)

* Pagination bug

* Bug fix

* Support image inputs

* Bug fixes

* Dylan/s3en 2265 save prompt and model settings directly from playground (#99)

* bug fixes, adding save prompt button in playground

* adding save prompt dialog

* adding create registry dialog

* working registry selection

* Adding new prompt to exist registry

* storing

* button padding fix

* refactor, adding combobox

* increasing dropdwon width

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Update README.md (#124)

* adding table exist checks

* Release 1.5.0 (#136)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating descriptions to optional

* prio python

* cleanup and fixes

* more bug fixes

* more fixes

* remove console log

* updating trace_service functions

* add migration

* add format function, updating from day to hour

* adding dropwdown menu

* updating query key

* updating query keys v2

* clean up

* fix bug

* Minor bugfix (#102)

* Pagination bug

* Bug fix

* Minor bugfix

* fixes

* Bug fixes to Evaluations  (#111)

* Pagination bug

* Bug fix

* Bug fixes to evals v2

* Minor fix

* upgrading tiktoken, adding omni to pricing (#110)

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update package

* Dataset bugfix (#113)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor linking fix (#114)

* Pagination bug

* Bug fix

* Dataset bugfix

* Minor fix

* api bug fix (#116)

* fixing dropdown color (#117)

* Support for Image inputs (#119)

* Pagination bug

* Bug fix

* Support image inputs

* Bug fix Spans with status_code = "ERROR" (#120)

* Pagination bug

* Bug fix

* Support image inputs

* Bug fixes

* Dylan/s3en 2265 save prompt and model settings directly from playground (#99)

* bug fixes, adding save prompt button in playground

* adding save prompt dialog

* adding create registry dialog

* working registry selection

* Adding new prompt to exist registry

* storing

* button padding fix

* refactor, adding combobox

* increasing dropdwon width

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* adding image display

* adding property filtering (#123)

* optimizing cost function

* optimizing per cost per hour function

* removing console statements

* updating queryKey for datasets

* updating queryKeys

* adding missing table checks

* adding more checks

* chore: update clickhouse docker for user management

* style: remove blank space

* updating save prompt button

* Redesign of Manual Evaluations (#134)

* Pagination bug

* Bug fix

* Schema changes for evals

* Fix APIs for evals

* Minor bugfixes

* Bugfix

* Evaluation changes

* Manual eval simplification

* Minor fix

* Manual evaluations UX changes

* Chart title fix

* Disable experiments

* fix

* Dylan/s3en 2318 add the ability to import a conversation from the traces to (#133)

* adding import conversation button

* init import conversation dialog

* adding trace rows

* fixing scroll bug

* adding import prompt to playground

* fixes

* Minor

* Bugfixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

---------

Co-authored-by: Darshit Suratwala <darshit@scale3labs.com>
Co-authored-by: darshit-s3 <119623510+darshit-s3@users.noreply.github.com>
Co-authored-by: dylan <dylan@scale3labs.com>
Co-authored-by: dylanzuber-scale3 <116033320+dylanzuber-scale3@users.noreply.github.com>
Co-authored-by: Rohit Kadhe <rohit@scale3labs.com>
Co-authored-by: Rohit Kadhe <113367036+rohit-kadhe@users.noreply.github.com>
Co-authored-by: MayuriS24 <159064413+MayuriS24@users.noreply.github.com>

* Release 1.5.1 (#143)

* Pagination bug

* Bug fix

* chore: add docker cmd

* Compatibility fixes for SDK version 2.0.0 (#69)

* Pagination bug

* Bug fix

* Fix for schema changes

* Render tool calling

* Support for Langgraph, Qdrant & Groq  (#73)

* Pagination bug

* Bug fix

* Add langgraph support

* QDrant support

* Add Groq support

* update README

* update README

* feat: optimise docker image for self host setup

* adding api access to traces endpoint

* clean up

* refactor

* feat: add clickhouse db create on app start (#79)

* docs: add railway deploy, fix sdk badges (#81)

* untrack .env

* Revert "untrack .env"

This reverts commit 4551d7eb33a22b4ba94be8eb1e20d620c6e0a567.

* Playground and Prompt Management (#83)

* Pagination bug

* Bug fix

* Playground - basic implementation

* Playground - streaming and nonstreaming

* Move playground inside project

* API key flow

* Api key

* Playground refactor

* Add chat hookup

* anthropic streaming support

* Bug fixes to openai playground

* Anthropic bugfixes

* Anthropic bugfix

* Cohere first iteration

* Cohere role fixes

* Cohere api fix

* Parallel running

* Playground cost calculation non streaming

* playground - streaming token calculation

* latency and cost

* Support for Groq

* Add model name

* Prompt management views

* Remove current promptset flow

* Prompt management API hooks

* Prompt registry final

* Playground bugfixes

* Bug fix playground

* Rearrange project nav

* Fix playground

* Fix prompts

* Bugfixes

* Minor fix

* Prompt versioning bugfix

* Bugfix

* fix: clickhouse table find queries (#82)

* Fix to surface multiple LLM requests inside LLM View (#84)

* Pagination bug

* Bug fix

* Fix for surfacing multiple LLM requests in LLMView

* Minor bugfixes (#86)

* Pagination bug

* Bug fix

* Bugfixes

* api to fetch promptset with prompt filters

* bug fixes

* fix invalid redirect

* fix invalid status code

* Project Switcher (#90)

* Pagination bug

* Bug fix

* Project switcher

* Feat: dataset download (#60)

* API: download dataset

* API: Download dataset

* updated download-dataset api

* Updated: download_dataset api

* Updated download dataset API

* Updated Download API: changed Response to Next Response, add a condition to ensure max page size is 500

* updated the download-dataset API: fixed the format and removed redundant lines of code

* Updated download_daatset API: file name and removed 'id' param

* Added the Download dataset button.

* Merged developemnt into my branch

* Updated button size

* Fixes

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* Update prompt registry with instructions to fetch prompts (#91)

* Pagination bug

* Bug fix

* Update prompt registry

* Minor bugfix (#94)

* Pagination bug

* Bug fix

* Minor bugfix

* chore: update github repo badges

* optimizing token count function

* Add GPT4-O Pricing and Playground (#98)

* Pagination bug

* Bug fix

* Add GPT4-O support

* Add GPT4-O support

* Update cost

* Dylan/s3en 2234 add perplexity support to playground (#89)

* adding perplexity to playground types

* adding ui stuff:'

* adding perplexity chat api

* fixing perplexity model dropdown

---------

Co-authored-by: Karthik Kalyanaraman <karthik@scale3labs.com>

* api changes

* add api access to get api and fix all bugs

* bug fix

* bug fix

* updating d…
  • Loading branch information
24 people authored Jan 8, 2025
1 parent bb3f123 commit 227af2f
Show file tree
Hide file tree
Showing 70 changed files with 3,626 additions and 3,089 deletions.
494 changes: 0 additions & 494 deletions app/(protected)/project/[project_id]/annotations/annotations.tsx

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -121,41 +121,41 @@ export default function Evaluation() {
const columns: ColumnDef<RunSample>[] = [
{
accessorKey: "input",
size: 500,
size: 400,
enableResizing: true,
header: "Input",
cell: ({ row }) => {
const input = row.getValue("input") as string;
return <p className="text-sm font-semibold">{input}</p>;
return <p className="text-sm">{input}</p>;
},
},
{
accessorKey: "target",
size: 500,
size: 400,
enableResizing: true,
header: "Target",
cell: ({ row }) => {
const target = row.getValue("target") as string;
return <p className="text-sm font-semibold">{target}</p>;
return <p className="text-sm">{target}</p>;
},
},
{
accessorKey: "output",
size: 500,
size: 400,
enableResizing: true,
header: "Output",
cell: ({ row }) => {
const output = row.getValue("output") as string;
return <p className="text-sm font-semibold">{output}</p>;
return <p className="text-sm">{output}</p>;
},
},
{
accessorKey: "model",
size: 200,
size: 150,
enableResizing: true,
header: "Model",
cell: ({ row }) => {
return <p className="text-sm font-semibold">{run?.model}</p>;
return <p className="text-sm">{run?.model}</p>;
},
},
{
Expand All @@ -174,14 +174,10 @@ export default function Evaluation() {
{scores.map((score, idx) => {
// remove underscore from scorer
let scorer = score?.scorer?.replace(/_/g, " ") || "";
// capitalize the entire scorer
if (scorer) {
scorer = scorer.toUpperCase();
}
return (
<Badge key={idx} className="flex gap-2">
<p className="text-xs">{scorer}:</p>
<p className="text-xl font-bold">{score.value}</p>
<p className="text-xs">{score.value}</p>
</Badge>
);
})}
Expand Down Expand Up @@ -242,7 +238,11 @@ export default function Evaluation() {

return (
<div className="w-full flex flex-col gap-4">
<div className="px-12 py-12 flex justify-between bg-muted">
<Button variant="outline" onClick={() => router.back()} className="w-fit">
<ChevronLeft className="text-muted-foreground" size={20} />
Back
</Button>
<div className="flex justify-between">
<div className="flex flex-col gap-2">
<div className="flex items-center gap-2">
<h1 className="text-sm font-semibold">Run ID</h1>
Expand Down Expand Up @@ -271,43 +271,25 @@ export default function Evaluation() {
</Badge>
)}
</div>
<Link href={EVALUATIONS_DOCS_URL} target="_blank">
<Button
variant={run && run?.samples?.length > 0 ? "outline" : "default"}
>
New Evaluation
<FlaskConical className="ml-1 h-4 w-4" />
<ArrowTopRightIcon className="ml-1 h-4 w-4" />
</Button>
</Link>
{run?.scores?.map((score, i) => (
<div className="flex flex-col" key={i}>
<p className="text-md font-semibold">{score.scorer}</p>
{score?.metrics.map((metric, j) => (
<div className="flex gap-2" key={j}>
<p className="text-sm font-semibold capitalize">
{metric.name}:
</p>
<p className="text-sm font-semibold">
{parseFloat(metric.value).toFixed(2)}
</p>
</div>
))}
</div>
))}
</div>
<div className="flex flex-col gap-6 w-full px-12">
<div className="flex gap-2">
{run?.scores?.map((score, i) => (
<div
className="flex flex-col gap-0 p-2 border rounded-md border-muted bg-muted shadow-md"
key={i}
>
<p className="text-md font-semibold mb-3">{score.scorer}</p>
{score?.metrics.map((metric, j) => (
<div className="flex gap-2" key={j}>
<p className="text-sm font-semibold capitalize">
{metric.name}:
</p>
<p className="text-sm font-semibold">
{parseFloat(metric.value).toFixed(2)}
</p>
</div>
))}
</div>
))}
</div>
<div className="flex flex-col gap-6 w-full">
<div className="flex justify-between items-center">
<div className="flex items-center gap-2">
<Button variant="outline" onClick={() => router.back()}>
<ChevronLeft className="text-muted-foreground" size={20} />
Back
</Button>
<DropdownMenu open={openDropdown} onOpenChange={setOpenDropdown}>
<DropdownMenuTrigger asChild>
<Button variant="outline" className="ml-auto">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,6 @@ export default function Compare() {
});
const [columnVisibility, setColumnVisibility] = useState<VisibilityState>({});
const [openDropdown, setOpenDropdown] = useState(false);
const [openSheet, setOpenSheet] = useState(false);
const [rowIndex, setRowIndex] = useState<number>(-1);

const { isLoading: experimentsLoading, error: experimentsError } = useQuery({
queryKey: ["fetch-runs-query", projectId, ...runIds],
Expand Down Expand Up @@ -113,27 +111,27 @@ export default function Compare() {
const columns: ColumnDef<RunView>[] = [
{
accessorKey: "input",
size: 500,
size: 400,
enableResizing: true,
header: "Input",
cell: ({ row }) => {
const input = row.original.samples[0].input;
return <p className="text-sm font-semibold">{input}</p>;
return <p className="text-sm">{input}</p>;
},
},
{
accessorKey: "target",
size: 500,
size: 400,
enableResizing: true,
header: "Target",
cell: ({ row }) => {
const target = row.original.samples[0].target;
return <p className="text-sm font-semibold">{target}</p>;
return <p className="text-sm">{target}</p>;
},
},
...runs.map((run, i) => ({
accessorKey: `output-${i}`,
size: 500,
size: 400,
enableResizing: true,
header: `Output - (${run.model})`,
cell: ({ row }: any) => {
Expand Down Expand Up @@ -201,16 +199,16 @@ export default function Compare() {

return (
<div className="w-full flex flex-col gap-4">
<div className="px-12 py-12 flex flex-col gap-2 bg-muted">
<Button variant="outline" className="w-fit" onClick={() => router.back()}>
<ChevronLeft className="text-muted-foreground" size={20} />
Back
</Button>
<div className="flex flex-col gap-2">
<h1 className="text-md font-semibold">Comparing Runs</h1>
<p className="text-sm w-1/2">{runIds.join(", ")}</p>
</div>
<div className="flex flex-col gap-12 w-full px-12">
<div className="flex flex-col gap-6 w-full">
<div className="flex items-center gap-2">
<Button variant="outline" onClick={() => router.back()}>
<ChevronLeft className="text-muted-foreground" size={20} />
Back
</Button>
<DropdownMenu open={openDropdown} onOpenChange={setOpenDropdown}>
<DropdownMenuTrigger asChild>
<Button variant="outline" className="ml-auto">
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"use client";

import { PathBreadCrumbs } from "@/components/dataset/path-breadcrumbs";
import ModelEvalMetricsChart from "@/components/evaluations/model-eval-metrics-chart";
import ModelScorerMetricsChart from "@/components/evaluations/model-scorer-metrics-chart";
import { ChartTypesDropDown } from "@/components/shared/chart-types";
Expand Down Expand Up @@ -406,41 +405,9 @@ export default function Evaluations() {
}, [table.getState().columnSizingInfo, table.getState().columnSizing]);

return (
<div className="w-full flex flex-col gap-4">
<div className="px-12 py-12 flex justify-between bg-muted">
<h1 className="text-3xl font-semibold">Evaluations</h1>
<div className="flex gap-2">
<Button
variant={currentData.length > 0 ? "default" : "outline"}
disabled={comparisonRunIds.length < 2}
onClick={() => {
// append comparisonRunIds to query params. & only from the second run id
const query = comparisonRunIds
.map((runId, i) => (i === 0 ? "" : "&") + "run_id=" + runId)
.join("");
router.push(
`/project/${projectId}/datasets/dataset/${datasetId}/evaluations/compare?${query}`
);
}}
>
Compare
</Button>
<Link href={EVALUATIONS_DOCS_URL} target="_blank">
<Button variant={currentData.length > 0 ? "outline" : "default"}>
New Evaluation
<FlaskConical className="ml-1 h-4 w-4" />
<ArrowTopRightIcon className="ml-1 h-4 w-4" />
</Button>
</Link>
</div>
</div>
<div>
{!fetchExperiments.isLoading && (
<div className="flex flex-col gap-12 w-full px-12">
<div className="flex items-center gap-2">
{datasetId && (
<PathBreadCrumbs projectId={projectId} datasetId={datasetId} />
)}
</div>
<div className="flex flex-col gap-3">
<div className="flex items-center justify-between">
<div className="flex gap-2">
Expand All @@ -456,6 +423,23 @@ export default function Evaluations() {
>
<RefreshCwIcon className="w-4 h-4" />
</Button>
<Button
variant={currentData.length > 0 ? "default" : "outline"}
disabled={comparisonRunIds.length < 2}
onClick={() => {
// append comparisonRunIds to query params. & only from the second run id
const query = comparisonRunIds
.map(
(runId, i) => (i === 0 ? "" : "&") + "run_id=" + runId
)
.join("");
router.push(
`/project/${projectId}/datasets/dataset/${datasetId}/evaluations/compare?${query}`
);
}}
>
Compare
</Button>
<Button
disabled={
fetchEvalMetrics.isLoading ||
Expand Down Expand Up @@ -529,29 +513,27 @@ export default function Evaluations() {
</div>
</div>
{viewMetrics && (
<div className="flex gap-4 flex-col w-full border rounded-md p-2">
<div className="flex gap-4 flex-col w-full">
<div className="flex gap-4 items-center overflow-x-scroll">
<ChartTypesDropDown
chartType={chartType}
setChartType={setChartType}
/>
</div>
{!fetchEvalMetrics.isLoading && (
<div className="flex flex-row gap-4 pb-6 overflow-x-scroll">
<div className="flex flex-row gap-4 flex-wrap">
{!fetchEvalMetrics.isLoading && (
<ModelScorerMetricsChart
chartType={chartType}
data={fetchEvalMetrics?.data?.scorer_metrics || {}}
/>
</div>
)}
{!fetchEvalMetrics.isLoading && (
<div className="flex flex-row gap-4 pb-12">
)}
{!fetchEvalMetrics.isLoading && (
<ModelEvalMetricsChart
chartType={chartType}
data={fetchEvalMetrics?.data?.eval_metrics || {}}
/>
</div>
)}
)}
</div>
</div>
)}
</div>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,5 @@ export default async function Page() {
redirect("/login");
}

return (
<>
<Experiments />
</>
);
return <Experiments />;
}
Loading

0 comments on commit 227af2f

Please sign in to comment.