Skip to content

Benchmark not clearing VRAM #25

Open
@LeonDBL

Description

@LeonDBL

System: Pop OS 22.04

ROCk module is loaded
Runtime Version: 1.13
Runtime Ext Version: 1.4
Name: gfx1031
Uuid: GPU-XX
Marketing Name: AMD Radeon RX 6750 XT
Vendor Name: AMD

GPU: MSI 6750XT 12GB

Issue:

Running the benchmark, it doesn't detect the GPU, however it does correctly select suitable models to load and run (8GB yml).

After the first model (mistral:7b) has loaded, run and tested, it then loads llama3.1:8b, however, it doesn't clear/flush the VRAM first, and both models end up loaded, the benchmark hangs and the GPU load returns to zero (VRAM is still loaded).

Metadata

Metadata

Assignees

No one assigned

    Labels

    InvestigationInvestigate user's questions

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions