Added UDF for emotion detection #356

Anirudh58 · 2022-09-12T18:42:29Z

No description provided.

gaurav274 · 2022-09-12T18:46:40Z

eva/udfs/emotion_detector.py

+        self.model.load_state_dict(model_state["net"])
+
+        # move to GPU and set to evaluation mode
+        self.to_device("0")


We shouldn't need this right?

Hmm without this, I get an error saying input tensors are CUDA but the model is not. I am inheriting AbstractClassifierUDF and implementing the classify method.

def to_device(self, device: str): should have moved the model to GPU. Can you verify if it is called?

Yes I tried that. It does not automatically move. to_device() is not being called.

gaurav274 · 2022-09-12T18:47:23Z

eva/udfs/emotion_detector.py

+        # load model
+        self.model = VGG("VGG19")
+        model_state = torch.load(
+            os.path.join(EVA_DEFAULT_DIR, "data", "models", "emotion_detector.t7")


Can we upload this to google drive and pull it from there? It won't work on collab

Oh yes sure, I can do that. But what should the path here be then?

2 points:

Where is the t7 model file? It is not part of the PR

@gaurav274 we need a better system of storing model files

jarulraj · 2022-09-12T19:38:08Z

@Anirudh58 I shared a link to the Google drive folder. We should also add some test cases.

Anirudh58 · 2022-09-13T00:49:38Z

@Anirudh58 I shared a link to the Google drive folder. We should also add some test cases.

Right I'll upload the model to the drive. And yes I'll add a few testcases for this UDF

gaurav274 · 2022-09-13T15:08:45Z

eva/udfs/emotion_detector.py

+        self.model.load_state_dict(model_state["net"])
+
+        # move to GPU and set to evaluation mode
+        self.to_device("0")


def to_device(self, device: str): should have moved the model to GPU. Can you verify if it is called?

gaurav274 · 2022-09-13T15:13:20Z

eva/udfs/emotion_detector.py

+            # predict
+            ncrops, c, h, w = np.shape(inputs)
+            inputs = inputs.view(-1, c, h, w)
+            inputs = inputs.cuda()


@xzdandy @LordDarkula How will this call work? How to figure which GPU to move the data on?

LordDarkula · 2022-09-13T20:55:39Z

eva/udfs/emotion_detector.py

+        # load model
+        self.model = VGG("VGG19")
+        model_state = torch.load(
+            os.path.join(EVA_DEFAULT_DIR, "data", "models", "emotion_detector.t7")


2 points:

Where is the t7 model file? It is not part of the PR

@gaurav274 we need a better system of storing model files

LordDarkula · 2022-09-13T20:57:29Z

eva/udfs/emotion_detector.py

+
+
+# helper class for VGG
+class VGG(nn.Module):


Under our current system you should be using PytorchAbstractClassifierUDF which inherists from AbstractClassifierUDF and nn.Module already.

This should be fine as I guess because it is a helper class

LordDarkula · 2022-09-13T21:18:55Z

eva/udfs/emotion_detector.py

+from eva.udfs.abstract_udf import AbstractClassifierUDF
+from eva.udfs.gpu_compatible import GPUCompatible
+
+# VGG configuration


What kind of configuration is this? Is this the structure of the neural network? A small comment would be nice

Yes, it specifies the architecture. But yes, let me clean it up a little.

gaurav274 · 2022-09-14T05:16:30Z

eva/udfs/emotion_detector.py

+        return nn.Sequential(*layers)
+
+
+class EmotionDetector(AbstractClassifierUDF, GPUCompatible):


We can maybe inherit this from PytorchAbstractClassifierUDF, and then to_device should be taken care of automatically.

I referenced the approach from FaceDetector because it had a lot of pre-processing/trasnforms. But yes, I will try inheriting from PytorchAbstractClassifierUDF

gaurav274 · 2022-09-15T22:08:06Z

@Anirudh58 @LordDarkula We need to close this PR asap. It is important for the release. What are the missing questions that we need to address?

jarulraj · 2022-09-17T23:36:40Z

We are planning to use this approach for downloading models using Dropbox links --
https://github.com/Emilien-mipt/fer-pytorch/blob/main/fer_pytorch/pre_trained_models.py#L12

@gaurav274 @LordDarkula

gaurav274 · 2022-09-18T03:09:01Z

@Anirudh58 Are we planning to make the changes Joy mentioned in this PR?
We can also pass the model_dir in the torch.utils.model_zoo.load_url function. How about we keep all these models in the ~/.eva/udfs/models/? @LordDarkula @jarulraj

jarulraj · 2022-09-20T04:43:21Z

@Anirudh58 Are we planning to make the changes Joy mentioned in this PR? We can also pass the model_dir in the torch.utils.model_zoo.load_url function. How about we keep all these models in the ~/.eva/udfs/models/? @LordDarkula @jarulraj

Sure, that should work.

Anirudh58 requested review from gaurav274 and jarulraj September 12, 2022 18:42

Anirudh58 force-pushed the movie_analysis branch from d60bed7 to bb34fd9 Compare September 12, 2022 18:48

gaurav274 requested changes Sep 12, 2022

View reviewed changes

gaurav274 reviewed Sep 13, 2022

View reviewed changes

Anirudh58 force-pushed the movie_analysis branch 2 times, most recently from 70c59a9 to 5ddfc0b Compare September 13, 2022 18:29

LordDarkula requested changes Sep 13, 2022

View reviewed changes

gaurav274 reviewed Sep 14, 2022

View reviewed changes

gaurav274 force-pushed the movie_analysis branch from 5ddfc0b to 6e87ebe Compare September 16, 2022 10:32

Anirudh58 force-pushed the movie_analysis branch from 6e87ebe to 5df970b Compare September 17, 2022 22:05

Anirudh58 force-pushed the movie_analysis branch from 965c07d to 1a4066b Compare September 18, 2022 02:10

Anirudh58 added 6 commits September 20, 2022 19:35

Added UDF for emotion detection

4c2f7cb

Added test for emotion detector

7da337b

Inheriting from PytorchAbstractClassifierUDF

db233ed

temporarily ignoring udf from coveralls

8437ee9

Added code to remotely pull models

f56220f

fix conflicts in coveragerc

2e7f755

Anirudh58 force-pushed the movie_analysis branch from 1a4066b to 2e7f755 Compare September 21, 2022 02:05

gaurav274 approved these changes Sep 22, 2022

View reviewed changes

gaurav274 merged commit ebc071d into georgia-tech-db:master Sep 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added UDF for emotion detection #356

Added UDF for emotion detection #356

Anirudh58 commented Sep 12, 2022

gaurav274 Sep 12, 2022

Anirudh58 Sep 13, 2022

gaurav274 Sep 13, 2022

Anirudh58 Sep 13, 2022

gaurav274 Sep 12, 2022

Anirudh58 Sep 13, 2022

LordDarkula Sep 13, 2022

jarulraj commented Sep 12, 2022

Anirudh58 commented Sep 13, 2022

gaurav274 Sep 13, 2022

gaurav274 Sep 13, 2022

LordDarkula Sep 13, 2022

LordDarkula Sep 13, 2022

gaurav274 Sep 14, 2022 •

edited

Loading

LordDarkula Sep 13, 2022

Anirudh58 Sep 16, 2022

gaurav274 Sep 14, 2022

Anirudh58 Sep 16, 2022

gaurav274 commented Sep 15, 2022 •

edited

Loading

jarulraj commented Sep 17, 2022 •

edited

Loading

gaurav274 commented Sep 18, 2022 •

edited

Loading

jarulraj commented Sep 20, 2022

		return nn.Sequential(*layers)


		class EmotionDetector(AbstractClassifierUDF, GPUCompatible):

Added UDF for emotion detection #356

Added UDF for emotion detection #356

Conversation

Anirudh58 commented Sep 12, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jarulraj commented Sep 12, 2022

Anirudh58 commented Sep 13, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gaurav274 Sep 14, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gaurav274 commented Sep 15, 2022 • edited Loading

jarulraj commented Sep 17, 2022 • edited Loading

gaurav274 commented Sep 18, 2022 • edited Loading

jarulraj commented Sep 20, 2022

gaurav274 Sep 14, 2022 •

edited

Loading

gaurav274 commented Sep 15, 2022 •

edited

Loading

jarulraj commented Sep 17, 2022 •

edited

Loading

gaurav274 commented Sep 18, 2022 •

edited

Loading