Pass frontend context to `qa/chat_streamed`

[jankrepl]

Description

The AI Assistant is a sidebar that is visible across all pages of Explore (in the future it will be also Build + Experiment). However, right now the LLM itself is not aware of what page the user is on. However, one can get that information quite easily in React (and then send to our backend)

import { usePathname, useSearchParams } from 'next/navigation'
 
export function Whatever() {
  const pathname = usePathname()
  const searchParams = useSearchParams()


  const url = `${pathname}?${searchParams}`
  // send request to /qa/chat_streamed and include the `url` in the request body

This ticket is about taking that url: str and turning it into a couple of sentences in the system prompt. Most likely the following two pieces of information should do it

• explanation of the frontend page structure (this has overlaps with Fetch metadata about platform (description, abbreviations, ...) #320 )
• what page we are on + query params

UPDATE
We decided to turn it into a tool. The workflow would look something like this

1. The frontend sends frontend_url: str with each request to qa/chat_streamed
2. We insert it to context variables
3. Define a new tool (~ContextAnalyzer) that has an empty input schema and takes in a single context variable frontend_url
4. Iplement arun while using as many if elif as you want
5. Return tool output that looks something like this

{
   "page_description": "blablabla",
}

6. Add a rule (system prompt) that will force the LLM to evoke the tool whenever the user's question is related to context (edited)

Why?

The way the frontend is designed makes you believe the AI Assistant knows where you are. So for example if you ask it "What do I see on this page?" it should give you something reasonable.

Comments

• Upstream ticket: https://github.com/openbraininstitute/prod-explore-functionality/issues/246
• This is mostly for the future - not an official requirement but IMO can make the user experience way better
• IMO we should not start this before the frontend is fully migrated to entitycore. For example the brain region query params in explore still use nexus identifiers.
• Conceptually similar to Navigate tool (clickable IDs) #346 since it would require the LLM to understand the page structure + query params of the official frontend
• The url (path + query params) is not the only state the frontend has but we can always add more things in the future
• We should probably have an env var that can disable this entire feature (useless in our frontend)
• I know this is reminiscent of the question suggestion user journey but note that here we don't work with any exotic types and we only care about the page we are currently on.
• Similar idea exists in Cursor - it always knows what file you have open and what line you are on

Bonus

• If we feel adventurous we can start applying different rules in (Assemble system prompt dynamically using reusable rules #387 ) on different pages. This would require us to put some metadata in the rules frontmatter that would help us filter whether that rule is applicable for a given page. It is equivalent to what cursor does with the glob parameter (e.g. *.py - the rule only applies if you are working on a Python file)
• ^ the same logic can be done for tool selection (different pages could have different whitelisted tools) - sounds like an overkill

[jankrepl]

We decided to go for a tool approach and somehow make sure the frontend_url makes it into context variables. This tool will have no other inputs. We can call the tool ContextAnalyzer (or something along those lines) inspired by Amazon Q.

[evakennyobi]

This is in code review @BoBer78 ? thanks

[BoBer78]

The first version should be done for tomorrow, then we can iterate.
In the long run it would be better to have some mechanism in the front end that automatically extracts the main information from the current webpage.