Update representation.py

[dakotah-jones]

Add explicit "color_face" argument.

Tickets

#1426

What has been done

With this PR I've added color_face="bgr" to detection.extract_faces call.

Previously: When using a detector_backend the final image sent to the model for embedding was BGR" format.
Now: When using a detector_backend the final image sent to the model for embedding is "RGB" format.

How to test

make lint && make test

[serengil]

I think we should change the skip case's behaviour, not default branch. All thresholds are tuned for default behaviour.

[dakotah-jones]

I think that would depend on the expected input of embedding = model.forward(img). (here)

Is the expected input to the model supposed to be RGB or BGR format?
Please clarify this for me.

If img is expected to be in BGR format, then I agree with you and I'll update the PR.
If img is expected to be in RGB format, then I disagree. The default case is incorrectly sending in img as BGR.

[dakotah-jones]

The comments seem to suggest that img should be in RGB format when the model is called. (here)

[serengil]

Then, that comment should be fixed, too.

That line was updated in this PR. I missed it.

#1402

[dakotah-jones]

For clarification sake.
Is the expected input to the model supposed to be RGB or BGR format?

It seems as though you are inferring that img sent for embedding should be in BGR format.

I ask because this is the opposite of my understanding.
I'm using this to generate FaceNet image embeddings. It's my understanding that FaceNet was trained on 160x160 images in RGB format.
It then seems strange that we would be calculating the embeddings on images in BGR format

[dakotah-jones]

To be clear, I'm talking about the final state of img, not the initial state of the image loaded in.

[serengil]

BGR

[dakotah-jones]

Thank you for the clarification.

The changes have been reverted.
The skip case has been updated.
The comment you mentioned from #1402 has been reverted.

[serengil]

Did you run the tests after your change? Seems tests are broken.

[dakotah-jones]

Sorry for the delay.
One of the local .pkl files was causing my tests to pass locally, but then it would fail on push.

__find_bulk_embeddings was sending RGB images to represent when represent was expecting BGR images as input.

[dakotah-jones]

BGR

Looking into the available FacialRecognition models.
It looks like some of the models used under the hood expect input as RGB and some expect input as BGR.

At the moment represent is always sending BGR format images to model.forward.

Let me know if this looks correct.

Expects BGR:

• VGG-Face
• OpenFace
• SFace
• GhostFaceNet

Expects RGB:

• FaceNet
• FaceNet512
• DeepFace
• DeepId
• Dlib
• ArcFace

[serengil]

We are sending BGR to all models via model.forward

We are not doing anything with RGB.

[dakotah-jones]

I understand the image being sent to model.forward is always BGR format.
That is a good thing. Having consistent input for FacialRecognition class implementation makes it easy to understand and use.

I'm not suggesting changing the input for model.forward.

I'm talking about the case where the underlying model has been trained on RGB images and we are sending in BGR images.

For instance, the DlibClient already has pre-processing steps to convert the image from BGR to RGB.
This is a case where you are using RGB.

I was just wondering if there were other cases, like DlibClient, where that type of pre-processing would be beneficial.

You may have a specific reason why most of these models take in BGR images as input. I would like to understand the reasoning behind that decision, for my own personal curiosity.