Skip to content

Support Open WebUI as a Unified LLM Provider #37

@JBGruber

Description

@JBGruber

We only need to make a few changes to support Open WebUI. I already made the bare minimum changes in https://github.com/JBGruber/rollama/tree/open-webui. The authentication can already be attached as described in #7.

The main advatages are:

  • user authentication is possible, which allows for a central Ollama server, e.g., in a research lab. Everyone who has been granted access can then query models through their own account
  • we can compare the models available through Ollama to proprietary ones

This is the current syntax, which could use some improvement, but works:

options(rollama_headers = list(Authorization = "Bearer sk-6b82a5443e7145d4a779922592701ee3"))
query(q = "Why is the sky blue?", 
            model = "gpt-4o", 
            server = "http://localhost:3000", 
            endpoint = "/api/chat/completions")
#>                      
#>  ── Answer from gpt-4o-2024-08-06 ───────────────────────────────────────────────────────────────
#>  The sky appears blue due to a phenomenon called Rayleigh scattering. When sunlight enters Earth's
#>  atmosphere, it is made up of many different colors, each with different wavelengths. Blue light has a
#>  shorter wavelength and is scattered in all directions by the gases and particles in the atmosphere more
#>  than other colors with longer wavelengths, like red or yellow.
#>  
#>  As sunlight passes through the atmosphere, the blue light is scattered more widely across the sky and is
#>  more likely to be seen from any given point. This widespread scattering of blue light makes the sky
#>  appear predominantly blue to our eyes during the day. Near sunrise and sunset, the sky can take on red
#>  or orange hues because the sun is lower on the horizon, and its light passes through more of the
#>  atmosphere, scattering shorter wavelengths out of the direct line of sight and allowing longer
#>  wavelengths like red and orange to dominate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions