Expanded motivation for Fourier transforms of color reps

hw_colors.ipynb

@@ -14,7 +14,7 @@
    "outputs": [],
    "source": [
     "__author__ = \"Christopher Potts\"\n",
-    "__version__ = \"CS224u, Stanford, Spring 2021\""
+    "__version__ = \"CS224u, Stanford, Summer 2021\""
    ]
   },
   {
@@ -398,7 +398,36 @@
     "```\n",
     "\n",
     "  and collect these `f_jkl` values in a list of 27 values. Additionally, in Python, [`2j` produces a value with `real` and `imag` attributes](https://docs.python.org/3.7/library/cmath.html). Each element `f_jkl` should have these components. If you concatenate the `real` and `imag` parts of all the `f_jkl`, you will have a 54-dimensional representation, as in the paper. Remember to start with an HSV representation, and with $h$ in $[0, 360]$, $s$ in $[0, 200]$, and $v$ in $[0, 200]$ (or else do the scaling differently). Note that the values in our corpus are in HLS format, [which are easily converted to HSV](https://en.wikipedia.org/wiki/HSL_and_HSV#HSV_to_HSL).\n",
+    "  \n",
+    "* It's natural to ask why this Fourier transform is useful in the current context. This is a challenging question, and I don't have a complete answer, but here is an intuitive observation: if you consider the raw color representations to be embeddings, then you can see very quickly that our standard geometric notions are totally out of line with our intuitions about the colors themselves. For example, here is a plot where we simply vary the hue dimension while keeping the other dimensions constant:\n",
     "\n",
+    "  <img src=\"fig/colors-hue-hls.png\" alt=\"A series of very different colors with cosine distances from orange ranging from 0 to 0.19\" />\n",
+    "\n",
+    "  I've printed the cosine distances from the leftmost color above each patch. They all look pretty similar. Now, you might say, well at least the distances are sort of proportional to how different the colors are from the first. However, that argument seems to crumble when we do the same experiment but now varying the saturation dimension:\n",
+    "\n",
+    "  <img src=\"fig/colors-saturation-hls.png\" alt=\"A series of very similar purple-ish colors with cosine distances from gray-purple ranging from 0 to 0.19\" />\n",
+    "\n",
+    "  These colors are all quite simular intuitively. Notice, though, that the cosine distances are identical to my first plot. Of course! Cosine distances doesn't care about the nature of these dimensions! The underlying color space is a cylinder, not a regular Euclidean 3d space!\n",
+    "  \n",
+    "  The Fourier transformation that we apply is remapping the colors into approximately the cylindrical space that we want. It is at least capturing some the circular/radial relationships that are inherent in the space. Thus, here are plots corresponding to the above, but now where the colors have been transformed for the cosine comparisons. \n",
+    "  \n",
+    "  First, the hue variation:\n",
+    "  \n",
+    "  <img src=\"fig/colors-hue-fourier.png\" alt=\"A series of very different colors with cosine distances from orange that are generally large (near 1.0)\" />\n",
+    "\n",
+    "  And then saturation:\n",
+    "  \n",
+    "  <img src=\"fig/colors-saturation-fourier.png\" alt=\"A series of very similar purple-ish colors with cosine distances from gray purple that seem aligned with visual color similarity\" />\n",
+    "  \n",
+    "  These distances seem much better aligned with intuitions to me, and I think that's quite general. Thus, even if our networks can in principle learn this remapping, it's very helpful to at least start them closer to where we want them to be.\n",
+    "  \n",
+    "  If you want to go one layer deeper, then the [Zhang and Lu 2002](https://www.sciencedirect.com/science/article/pii/S092359650200084X) paper that Monroe et al. 2016 cite is pretty intuitive. It's for the 2d case, but that actually makes the ideas somewhere more accessible, since they can easily plot the original and remapped feature spaces."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
     "The following test seeks to ensure only that the output of your `represent_color_context` will be compatible with the models we are creating:"
    ]
   },