Skip to content

Evaluation frame counts inconsistent #32

Open
@my-other-github-account

Description

Hello,

I see in the paper that default MLLM configs were largely used, but frame counts were increased where applicable.

Certain models such as LongVA appear to support video contexts up to 1000 frames, but only 128 are used in the benchmark. If models can handle the extra frame context, it seems like it could potentially help their performance.

What determines the frame counts?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions