Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow custom URL for the used GPT API #15

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

DanielBoettner
Copy link

This add a boolean and a string setting to the module.

The boolean would determine if a custom URL is to be used.
If so the string from that custom URL is used instead of the OpenAI api.

This makes the module usable with e.g. LM Studio.

Allow to pick a custom url for a GPT API.
This allows to use self hosted Open AI compatible APIs
@vizovitin
Copy link
Owner

Thank you very much for your interest and pull request! I assume this is very much alike #14.

Some questions, if you don't mind:

  1. I assume the respective documentation for LM Studio is here.
  2. Which version of Foundry VTT did you test this on?
  3. Could you share your own specific use case for the feature? Not just "use with LM Studio", but along the lines of "I want to run ... kind of LLM locally and expose it in ... way to Foundry and use with the ... game system, because I want ... or I find that this is better for ...". I want to understand your motivation and possible additional points for improvement for the module.

@vizovitin vizovitin added the enhancement New feature or request label Apr 2, 2024
@DanielBoettner
Copy link
Author

  1. Correct. This isn't restriceted to LMStudio but basically makes the module compatible with every OpenAI compatible API. So LM Studio, https://github.com/oobabooga/text-generation-webui or SaaS.
  2. I did test in the current v12 (.318 I think) build.

I locally actually update some other modules (https://github.com/TheComamba/UnKenny, https://github.com/adougherty/ai-tools, https://github.com/Halftonex/npc-generator-gpt )

In short you'll,

  • already find some pre trained/refined models on huggingface
  • will soon find smaller models with appropriate context windows so it will become attractive for more people to run GAI on local machines
  • So special models with pre trained behavior freeing up the context window while still delivering specific infos)

For some time I contributed to the npc sheet 5e modules and I wrote the lootpopulator module (merged into the npc sheet at some point).

All of the above mentioned modules could benefit from the following features IMHO.

  • Setting for the service (this PR basically)
  • Settings for Model (GPT3, ...4, Mistral, Olama; Gemma, ...)
  • Settings for Streaming mode (in other words if used for chat or even generation, this could be done async/non blocking in the background. Not sure if the chat could also be "streamed".
  • Edit/enhance/add the "commands" for the models.
    • based on the model the initial prompt can be very important but also take up space in the context window

So my current goal is to prepare the lootsheet for v12.
Fixing some bugs, preparing for the application framework and theming in dnd 3(foundry system).

Combining above mentioned modules and getting them ready for v12. Basically brings fully fledged NPC with a full background, items, relations, properties, ... that can trade, barter and chat with the players on the press of a button or the drop of a token.

And all this without beeing depended on someone else :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants