tag:github.com,2008:https://github.com/OpenGenerativeAI/llm-colosseum/releases 2024-11-20T11:10:45Z tag:github.com,2008:Repository/776301107/v0.0.3 2024-11-20T11:12:52Z <p>This releases add Multimodal models support!</p> <p>We ran the benchmark again with the newest models from Mistral, OpenAI, Anthropic, and Together.</p> <h2>What's Changed</h2> <ul> <li>Feat: Add Vision Model to the Game by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Jeerhz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Jeerhz">@Jeerhz</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2659377455" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/69" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/69/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/69">#69</a></li> <li>Feat: Compute Elo Rankings by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Jeerhz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Jeerhz">@Jeerhz</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2671527642" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/70" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/70/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/70">#70</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/Jeerhz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/Jeerhz">@Jeerhz</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2659377455" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/69" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/69/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/69">#69</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/OpenGenerativeAI/llm-colosseum/compare/v0.0.2...v0.0.3"><tt>v0.0.2...v0.0.3</tt></a></p> oulianov tag:github.com,2008:Repository/776301107/v0.0.2 2024-10-09T14:08:22Z <h2>What's Changed</h2> <ul> <li>Adds human controls (Linux only) and simplify code by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/oulianov/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/oulianov">@oulianov</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2206078941" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/25" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/25/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/25">#25</a></li> <li>Added support for anthropic and others by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/SamPink/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/SamPink">@SamPink</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2229609730" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/44" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/44/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/44">#44</a></li> <li>ollama bug fix by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/SamPink/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/SamPink">@SamPink</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2265242572" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/51" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/51/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/51">#51</a></li> <li>fix typo by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zhimin-z/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zhimin-z">@zhimin-z</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2387218061" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/55" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/55/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/55">#55</a></li> <li>feat: Add Bedrock support ⛰️ by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/JGalego/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/JGalego">@JGalego</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2442771393" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/59" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/59/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/59">#59</a></li> <li>Ensuring bugfix for pydantic is in place by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/nickschuetz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/nickschuetz">@nickschuetz</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2451668829" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/61" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/61/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/61">#61</a></li> <li>chore: update dashboard.ipynb by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/eltociear/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/eltociear">@eltociear</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2434045087" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/58" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/58/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/58">#58</a></li> <li>Adding Dockerfile and docker-compose.yml by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/harshkasat/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/harshkasat">@harshkasat</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2483982883" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/64" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/64/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/64">#64</a></li> <li>Rename ollama.py to local.py by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/oulianov/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/oulianov">@oulianov</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2488797352" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/65" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/65/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/65">#65</a></li> <li>feat: Add Cerebras API via llama-index by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/vithursant/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/vithursant">@vithursant</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2503663599" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/66" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/66/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/66">#66</a></li> <li>Simplify requirements.txt by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/oulianov/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/oulianov">@oulianov</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2510058554" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/67" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/67/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/67">#67</a></li> <li>Adds back mistral support by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/oulianov/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/oulianov">@oulianov</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2510110669" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/68" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/68/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/68">#68</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/SamPink/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/SamPink">@SamPink</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2229609730" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/44" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/44/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/44">#44</a></li> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zhimin-z/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zhimin-z">@zhimin-z</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2387218061" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/55" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/55/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/55">#55</a></li> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/JGalego/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/JGalego">@JGalego</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2442771393" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/59" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/59/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/59">#59</a></li> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/nickschuetz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/nickschuetz">@nickschuetz</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2451668829" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/61" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/61/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/61">#61</a></li> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/eltociear/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/eltociear">@eltociear</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2434045087" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/58" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/58/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/58">#58</a></li> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/harshkasat/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/harshkasat">@harshkasat</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2483982883" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/64" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/64/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/64">#64</a></li> <li><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/vithursant/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/vithursant">@vithursant</a> made their first contribution in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="2503663599" data-permission-text="Title is private" data-url="https://github.com/OpenGenerativeAI/llm-colosseum/issues/66" data-hovercard-type="pull_request" data-hovercard-url="/OpenGenerativeAI/llm-colosseum/pull/66/hovercard" href="https://github.com/OpenGenerativeAI/llm-colosseum/pull/66">#66</a></li> </ul> <p><strong>Full Changelog</strong>: <a class="commit-link" href="https://github.com/OpenGenerativeAI/llm-colosseum/compare/v0.0.1...v0.0.2"><tt>v0.0.1...v0.0.2</tt></a></p> oulianov tag:github.com,2008:Repository/776301107/v0.0.1 2024-03-24T17:37:49Z <details open="" class="details-reset border rounded-2"> <summary class="px-3 py-2"> <svg aria-hidden="true" height="16" viewBox="0 0 16 16" version="1.1" width="16" data-view-component="true" class="octicon octicon-device-camera-video"> <path d="M16 3.75v8.5a.75.75 0 0 1-1.136.643L11 10.575v.675A1.75 1.75 0 0 1 9.25 13h-7.5A1.75 1.75 0 0 1 0 11.25v-6.5C0 3.784.784 3 1.75 3h7.5c.966 0 1.75.784 1.75 1.75v.675l3.864-2.318A.75.75 0 0 1 16 3.75Zm-6.5 1a.25.25 0 0 0-.25-.25h-7.5a.25.25 0 0 0-.25.25v6.5c0 .138.112.25.25.25h7.5a.25.25 0 0 0 .25-.25v-6.5ZM11 8.825l3.5 2.1v-5.85l-3.5 2.1Z"></path> </svg> <span class="m-1">6v6720.mp4</span> <span class="dropdown-caret"></span> </summary> <video src="https://private-user-images.githubusercontent.com/19614572/316328567-6c54a7af-bc07-4de5-a66f-24bf754d182a.mp4?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjIxODI1MzUsIm5iZiI6MTc2MjE4MjIzNSwicGF0aCI6Ii8xOTYxNDU3Mi8zMTYzMjg1NjctNmM1NGE3YWYtYmMwNy00ZGU1LWE2NmYtMjRiZjc1NGQxODJhLm1wND9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTExMDMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUxMTAzVDE1MDM1NVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTgxZWU3MWUyOGM5NTQwY2Y4MzVmMDRiZWRjNTYwZDBlODc0YWIzODUzN2YzMTAyODM5NmU4YjMyYzA2OGI1MjMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.IosVEw1perMFv9Dqk-cHiaq0PbmK7iU7xtGFNYo_rno" data-canonical-src="https://private-user-images.githubusercontent.com/19614572/316328567-6c54a7af-bc07-4de5-a66f-24bf754d182a.mp4?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NjIxODI1MzUsIm5iZiI6MTc2MjE4MjIzNSwicGF0aCI6Ii8xOTYxNDU3Mi8zMTYzMjg1NjctNmM1NGE3YWYtYmMwNy00ZGU1LWE2NmYtMjRiZjc1NGQxODJhLm1wND9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTExMDMlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUxMTAzVDE1MDM1NVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTgxZWU3MWUyOGM5NTQwY2Y4MzVmMDRiZWRjNTYwZDBlODc0YWIzODUzN2YzMTAyODM5NmU4YjMyYzA2OGI1MjMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.IosVEw1perMFv9Dqk-cHiaq0PbmK7iU7xtGFNYo_rno" controls="controls" muted="muted" class="d-block rounded-bottom-2 border-top width-fit" style="max-height:640px; min-height: 200px"> </video> </details> StanGirard