Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Possible) Incorrect assignment of image mean/std values in clip_model_load() #99

Open
tino926 opened this issue Sep 6, 2024 · 0 comments

Comments

@tino926
Copy link

tino926 commented Sep 6, 2024

The current implementation of clip_model_load() seems incorrectly assigns mean and standard deviation values to the image_mean and image_std fields.

The existing code fetches data for all three color channels (presumably RGB) but fails to iterate through the retrieved arrays. This results in all channels being assigned the first value of the respective image_mean and image_std arrays.

The original code is as this:

    int idx_mean = get_key_idx(ctx, KEY_IMAGE_MEAN);
    int idx_std = get_key_idx(ctx, KEY_IMAGE_STD);
    for (int i = 0; i < 3; ++i) {
        new_clip->image_mean[i] = *((float *)gguf_get_arr_data(ctx, idx_mean));
        new_clip->image_std[i] = *((float *)gguf_get_arr_data(ctx, idx_std));
    }

To rectify this, the code should access the correct index within the retrieved arrays for each color channel, as shown below:

    int idx_mean = get_key_idx(ctx, KEY_IMAGE_MEAN);
    int idx_std = get_key_idx(ctx, KEY_IMAGE_STD);
    for (int i = 0; i < 3; ++i) {
        new_clip->image_mean[i] = *(((float *)gguf_get_arr_data(ctx, idx_mean)) + i);
        new_clip->image_std[i] = *(((float *)gguf_get_arr_data(ctx, idx_std)) + i);
    }
bluetarpmedia added a commit to bluetarpmedia/clip.cpp that referenced this issue Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant