Chrome Extension Mostra i Computer Vision Tags di Facebook

Da Aprile 2016, Facebook aggiunge automaticamente i tag alt alle immagini che carichi, che descrivono con keywords il contenuto delle immagini:

<img csrc="https://facebook.com/some-url.jpg"
alt="Image may contain: golf, grass, outdoor and nature"
width="316" height="237">

They are labeling your images using a Deep ConvNet built by Facebook's FAIR team.

On one hand, this is really great. It improves accessibility for blind users who depend on screen readers which are only capable of processing text.

But I think a lot of internet users don't realize the amount of information that is now routinely extracted from photographs. Facebook (and Google, Apple, Amazon, etc) can easily tell from your photographs if you have a pet dog, if you collect cameras, if you play golf, if you have children, or if you are just really into sunglasses. There's nothing stopping them from using this information to show you relevant ads just based on photos of you - even if another user uploaded the photo and didn't even directly tag you!

This is a very simple Chrome Extension that I hacked together in a few minutes to make it easy to see the tags that Facebook is automatically applying to you and your friends' photos. Once you install it, all the photos you see on your Facebook timeline will automatically be overlaid with their tags, like this:

The goal is simply to make everyone aware of the kind of information that is routinely extracted from your own images today. This isn't some technology that is 'coming soon', but this is something in wide-spread production use now by every major internet company. You can even learn how to implement this kind of system yourself using widely available open-source software.

Here's some examples of the kinds of things that Facebook detects in your photos:

Objects in your photos

It's possible to tell what kinds of objects are in your photos and identify your interests based on that.

Activities you are doing

Whether you are eating, running, or playing golf, it's possible for an algorithm to tell what you are doing in your photographs.

When and where a photo was taken

You can tell if a photo was taken inside or outside, what the time of day was, what physical landmarks (mountains, trees, water, etc) were around, and so on.

Events you are attending

It's even possible to tell which kind of sporting event you are attending.

These are just a few examples. See what kinds of things Facebook has found in your own photos!

Installing this Chrome Extension

Option 1: From the Chrome Web Store

1. Click here and choose "+ Add to Chrome"

Option 2: From Source

1. Clone this repo so you have a copy in a folder locally.
2. Open chrome://extensions in the location or go to Tools > Extensions
3. Enable Developer mode by checking the checkbox in the upper-right corner.
4. Click on the button labelled Load unpacked extension....
5. Select the directory where you cloned this repo to.
6. Visit Facebook!

Installing in Firefox

From the Add-ons website:

1. Click here and choose "+ Add to Firefox"

Caveats

1. This only works for English-speaking Facebook users. I made no attempt to make this work for any other language. Sorry! But you could easily fork this and make it work for a different language. There's not a lot to this.

• Álvaro Sarasúa made a Spanish version of this extension here.
• Nicolas Raube made a German version of this extension here.

Press

• New York Magazine select/all - See What Facebook Thinks Is in Your Photos