Default Index

[gfsysa]

Carrying this over from feature request issue 23...

Bug? -- If the default llama-index is not base, but base still exists and you begin a chat with chat-with-file enableed, the model will not find your indexed files... Have to switch to chat with files mode, select the database and switch back to Chat.

could you please describe in more detail, step by step with example setup? Unfortunately, I can't reproduce this problem.

More context:
I have two indexes, base : Base, and custom : Custom

On start-up, under debug Indexes:
Current idx: base
Indexes (list):

[
    {
        "id": "base",
        "name": "Base"
    },
    {
        "id": "custom",
        "name": "Custom"
    }
]

Plugin settings:

• Chat with files (Llama-index, inline): custom
• • Ask Llama-index first (enabled)
• Command: Files I/O (Index to use when indexing files): base (Note this setting was a challenge to find or changed maybe??)
• Context History: (I thought I was able to designate where to save context history, more specifcally the Auto-Index conversation history, which I had set to base, but maybe I'm confusing it with the Files I/O setting, but I cannot find that setting any longer).

Effectively, on startup 'base' is the index being referenced; however, I have 'Ask Llama-index first' enabled, but I don't think it's being referenced. I can change the current index by changing the Mode to 'Chat with files', selecting my Custom index (this setting is not visible in 'Chat' mode), and then switching back to 'Chat'

I suspect we should be able to set the default index on startup. The other use case I'm facing is that I'm trying to keep my curated data separate, so my data is in Custom, and the chat history and context I think should be in the base index.

[gfsysa]

I also just noticed this related item in Debug: Database (SQLite) ...

My Vector Store is set to RedisVectorStore localhost 6379, ie:
Current storage: RedisVectorStore
DB items (ctx), DB items (file), both are referencing SimpleVectorStore

I switched the default store to SimpleVectorStore and cleared the base index, which wiped the ctx, but the files are still indexed in the SimpleVectorStore.

Additionally, I am noting that all prompts and interactions are being stored in SQLite when looking at the DB Viewer. I assume this in not knowledge for the model, but I'm also not sure how to manage this database... I assume it's the logger, but only some of my history seems valuable to me, and I suppose I can purge, but can we selectively purge/delete and is this data in use?

[szczyglis-dev]

About the first comment - this works a bit differently:

• The index selected in Chat with files mode is only applicable to that mode (not the plugin).
• The index selected in the Chat with files plugin only applies to the plugin in all other modes, like Chat.

^ these two things are separate configurations.

The index to which conversations/context are indexed (in auto mode) is to be set in Settings -> Llama-index -> Update -> ID of index for auto-indexing.

The index chosen in the Files I/O plugin is the index used only during file indexing with commands executed using this plugin.
The Context history plugin does not use an index, it searches in real-time through the database (sqlite), not in the index.
Including context from the index (where the database is indexed) is done only with the Chat with files plugin (or Chat with files mode).

About DB viewer:

Refresh the database view (via refresh button) to update the records after index clearance; there is no auto-refresh in the DB viewer.

The DB Viewer is for debug/development purposes; you can delete records manually from the database, but if you delete a record with context (ctx_ tables) from the DB, it will delete the context items from the app. If you delete the mapping between vector store and real files / DB records IDs (from idx_ tables), you will lose your mapping between the index and the indexed data. You may experiment with deleting records manually if you want, but remember you are deleting data from the app by doing this and you may lose some references.