Is there any cache feature for function like owlv2_sam2_video_tracking?

[claude89757]

it seems like that when i test this fuction，it need to call api to create code?

[dillonalaird]

Hey @claude89757, I'm not sure I understand your question. The APIs you need to call to create code are just the agent LLMs, so this is Claude and Gemini. Once the code is written, the code will utilize tools which call our APIs. For example, we are hosting OwlV2 and SAM2 for you so can call our endpoints to use them.

[claude89757]

Hey @claude89757, I'm not sure I understand your question. The APIs you need to call to create code are just the agent LLMs, so this is Claude and Gemini. Once the code is written, the code will utilize tools which call our APIs. For example, we are hosting OwlV2 and SAM2 for you so can call our endpoints to use them.

Thans for your reply!

What I mean is: Is there a way for the user to run the CV-related code implemented by the vision agent via LLM locally, rather than having it hosted in the cloud?

Also, does running owlv2_sam2_video_tracking generate a new code every time, even when testing the same task repeatedly?

[dillonalaird]

I see, I think this information might help:

• The code generated by the LLMs is run locally (if you use this github repo vs running it from the website va.landing.ai). It runs via a local Jupyter notebook, the code can be found here https://github.com/landing-ai/vision-agent/blob/main/vision_agent/utils/execute.py#L391
• To run the LLMs themselves locally you can modify the config file found here https://github.com/landing-ai/vision-agent/blob/main/vision_agent/configs/config.py#L10 to use OllamaLMM found here https://github.com/landing-ai/vision-agent/blob/main/vision_agent/lmm/lmm.py#L239 the config would look something like this:

class Config(BaseModel):
    # for vision_agent_v2
    agent: Type[LMM] = Field(default=OllamaLMM)
    agent_kwargs: dict = Field(
        default_factory=lambda: {
            "model_name": "gemma3",
            "temperature": 0.0,
            "image_size": 768,
        }
    )

So the model name is the model name used by Ollama https://ollama.com/library/gemma3 A couple things to note:

• The agents need to be multimodal, so you need an agent that can consume an image. You can test it on other regular LLMs like DeepSeek but I'm not sure how well it will work

• Most open source LLMs, particularly the smaller ones (<32B) I've found cannot run the agentic workflow and have too many issues, for example they can't output code in the correct format. This is mainly why I haven't added a config for them, you will likely see many edge cases it crashes on so be ware

• When you run the function owlv2_sam2_video_tracking it does not generate new code, it only calls owlv2 and sam2 tools. When you call VisionAgentV2 it may the function owlv2_sam2_video_tracking repeatedly as it tests it out and tries to figure out if it works for the task, just like you would call it multiple times as you figure out how to use it.

Hope that helps!