mandelbotpaper.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Mandelbot: a novel method of analyzing music for its perception-based compability with Mandelbrot \"zoom\" sequences\n",
    "Eliyahu Suskind, Priscila Morales\n",
    "\n",
    "Run the cell below each time the notebook is started or restarted to ensure that if you change any code in the library, this notebook will use the latest version of the library code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%run src/mandelbot_alpha.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Introduction:**\n",
    "\n",
    "A Mandelbrot set is a specific fractal set created by points on the complex plane. A point on the complex plane is considered a part of the Mandelbrot set if it does not diverge to infinity when iterated from Z=0 in the following function: \n",
    "\n",
    "$$f_{C}(z)= z^{2}+C$$\n",
    "\n",
    "For instance, the point (-1, 0i) is considered part of the Mandelbrot sequence as it alternates between the points (-1, 0i) and (0, 0i) in an infinite \"orbit\". Points that are part of the Mandelbrot set can also diverge on a non-infinite point.<sup>[1]</sup> \n",
    "\n",
    "Visual depictions of the Mandelbrot sequence are created by coloring all points that are part of the Mandelbrot set in black and all points that aren't a myriad of different possible colors. The color of each non-Mandelbrot point is determined by how quickly the point reaches a defined \"escape value\" (a large non-infinite point outside the mandelbrot set representative of infinity).<sup>[2]</sup>  \n",
    "\n",
    "In recent years, videos of Mandelbrot zoom sequences, or a video depicting the camera zooming into a specific point on the Mandelbrot sequence, have grown in popularity among individuals outside the STEM community. The most popular video of this type on popular video-streaming platform [YouTube](https://www.youtube.com/) has 2.5 million views as of March 2022 and was featured on the TV show \"The Big Bang Theory\". Many of the comments from the viewers reference music, aesthetics, and psychedelics, with only a handful referencing relevant mathematics.<sup>[3]</sup>\n",
    "\n",
    "Thus, we seek to analyze trends in the characteristics of songs average individuals associate with Mandelbrot zoom sequences and use this data to develop a framework for identifying whether a song is a good perceptual fit for a mandelbrot zoom sequence. The software we developed for this purpose shall henceforth be referred to as \"the Mandelbot\".\n",
    "\n",
    "### **Methods:**\n",
    "\n",
    "First, in order to get data on songs people associate with Mandelbrot sequences, we carried out a random sampling campaign. For those whom we ended up sampling, we began by showing them a controlled Mandelbrot zoom sequence and asking them what generally came to their mind. Then, we narrowed the question down by asking them if any specific songs or musical artists came to mind. If the interviewee named an artist, we asked what specific songs by said artist would pair well with the zoom sequence in their opinion. After the interviewee named a song, we would then play the named song and ask the interviewee whether or not it paired well. If the interviewee answered yes, then we added it to a database playlist hosted by song streaming service [Spotify](https://www.spotify.com/us/). We then repeated this process with the interviewee for an hour. As of March, 31st 2022, we reached a sample size of n=25 people.\n",
    "\n",
    "In order to scrape the data collected from the playlist database and convert it into forms that are suitable for data analytics, we coded the first half of the Mandelbot: Mandelbot Alpha. Mandelbot Alpha uses the lightweight python-based Spotify API [Spotipy](https://pypi.org/project/spotipy/) to scrape the data from the database into one large, unorganized dictionary-containing list assigned to the variable name ```all_track_data```. From this list, we then extracted 12 seperate quantitative song characteristics<sup>[4]</sup> into their own respective lists denoted by the prefix ```track_``` (for instance, a list containing the key of all the songs would be assigned the variable name of ```track_key``` and so forth). We additionally extracted the qualitative characteristic of song title into the list ```all_track_titles``` for ease of indexing. Finally, we compiled all the relevant lists containing song data into a larger list-containing list beginning with our quantitative indexing list (```all_track_titles```) followed by our quantitative lists and assigned it the variable name of ```all_track_chars```. This was in order to allow Mandelbot Alpha to easily be able to convert all of our data into a comma separated value (.CSV) file for ease of further data analysis. See the code cell below for a demonstration of Mandelbot Alpha pulling and organizing track data into ```all_track_chars```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(all_track_chars)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We then utilized an ingrained library of [Plyplot](https://matplotlib.org/3.5.1/tutorials/introductory/pyplot.html) eight functions within Mandelbot Alpha to find trends within the data. These eight functions consisted of histograms for track characteristics not categorized by Spotify and comparative scatterplots for track characteristics of similar categories as dictated by Spotify (these being: mood, property, and context).<sup>[5]</sup> The only exception to this latter rule is one scatterplot for song loudness - which only has one characteristic displayed as there are an uneven number of categorized characteristics.\n",
    "\n",
    "### **Error Analysis:**\n",
    "\n",
    "During sampling, we noticed that many of our participants selected songs with large sentimental value to them. For instance, one participant chose a song they associated with their graduation from a rehab clinic. Many participants also expressed feeling nostalgic while viewing the Mandelbrot zoom sequence, which could affect song choice subconsciously in a similar way. Additionally, due to the novelty of Mandelbrot zoom sequences, some individuals initially picked songs with characteristics they later determined were unsuitable for Mandelbrot zoom sequence pairing after they acclimated to the zoom sequence.\n",
    "\n",
    "### **Results:**\n",
    "\n",
    "After reviewing all eight graphs, we noticed three distinct trends within the track data. The first trend we noticed was in comparing track speechiness with track instrumentalness. Both are measures of the amount of vocal content within a specific track on a scale of 0 to 1. Run the cell below to generate the relevant scatterplot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_svi()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As one can see, despite a fairly even distribution of instrumentalness throughout its entire range, the majority of the tracks have extremely low speechiness. The vast majority of songs in our database have a speechiness score lower than 0.06. As our database will continue to expand as we sample past the publishing of this paper, we have provided a code cell below to calculate the percentage of songs which have a speechiness lower than 0.06."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "i = 0\n",
    "for item in track_speechiness:\n",
    "    if item < 0.06:\n",
    "        i += 1\n",
    "result = int((i / len(track_speechiness)) * 100)\n",
    "print(f\"The current percentage of songs with a speechiness lower than 0.06 is approx {result}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This seems to point to individuals perceiving songs with less vocal interruptions better with a Mandelbrot zoom sequence. The second trend we noticed was in comparing track liveness with track acousticness. Liveness is a measure from 0 to 1 that guesses the probability that the track was performed in front of a live audience; the closer to 1 the value becomes, the more qualities of a live-performed track it has. Acousticness is a measure of how acoustic a track is and is also measured from 0 to 1. Both of these characteristics are categorized by Spotify as context characteristics. Run the cell below to generate the relevant scatterplot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_contextg()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As one can see, the tracks in our data tend to have both low acousticness and liveness, with this trend being significantly more visible with track liveness. Tracks individuals associate with Mandelbrot zoom sequences typically have acousticness scores lower than 0.3 and liveness scores lower than 0.2. As before, run the code cell below to calculate the percentage of tracks in our database with an acousticness score lower than 0.3 and tracks with a liveness score lower than 0.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "i = 0\n",
    "j = 0\n",
    "for item in track_acousticness:\n",
    "    if item < 0.3:\n",
    "        i += 1\n",
    "for item in track_liveness:\n",
    "    if item < 0.2:\n",
    "        j += 1\n",
    "result1 = int((i / len(track_acousticness)) * 100)\n",
    "result2 = int((j / len(track_liveness)) * 100)\n",
    "print(f\"The current percentage of songs with an acousticness lower than 0.3 is approx {result1}% and the percentage with an liveness lower than 0.2 is approx {result2}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Much like with the last trend, this seems to point to individuals perceiving songs with less interruptions associated with live performances better with a Mandelbrot zoom sequence. Additionally, individuals seem to associate more electronic songs with Mandelbrot zoom sequences, as opposed to more acoustic-based songs. The last trend we noticed was in analyzing track loudness. Loudness is a measure of track volume in decibels as averaged across the entire song. Loudness values typically range from 0 dB to -60 dB on Spotify. This is because Spotify adjusts tracks to -14dB in line with International Telecommunication Union standards.<sup>[6]</sup> Run the cell below to generate the relevant scatterplot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_louds()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As one can see, the tracks in our data tend to lie above the adjusted track average - as displayed by the red dotted line.  As before, run the code cell below to calculate the percentage of tracks in our database louder than the adjusted average."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "i = 0\n",
    "for item in track_loudness:\n",
    "    if item > -14:\n",
    "        i += 1\n",
    "result = int((i / len(track_speechiness)) * 100)\n",
    "print(f\"The current percentage of songs louder than the adjusted average is approx {result}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This seems to point to individuals associating louder songs with Mandelbrot zoom sequences more frequently than quieter songs. In addition to these three trends, it is important to acknowledge that the non-categorical track characteristics typically falls in line with overall song averages. This includes song length forming a normal distribution around 4 minutes, the majority of songs being in a time signature of 4, and there being an even distribution of song keys within normal ranges. Equally important is the fact that track tempo and energy have a wide variety: being equally spread out between a tempo of 75 and 180 beats per minute and an energy of 0.2 to 1. Run the code cell below to numerically and graphically demonstrate this latter observation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "create_tve()\n",
    "i = 0\n",
    "j = 0\n",
    "for item in track_tempo:\n",
    "    if 75 <= item <= 180:\n",
    "            i += 1\n",
    "for item in track_energy:\n",
    "    if item >= 0.2:\n",
    "            j += 1\n",
    "result1 = int((i / len(track_tempo)) * 100)\n",
    "result2 = int((j / len(track_energy)) * 100)\n",
    "print(f\"The current percentage of songs with a tempo between 75 and 180 bpm is approx {result1}% and the percentage with an energy  score between 0.2 and 1 is approx {result2}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **Conclusions:**\n",
    "\n",
    "From the three significant trends that are seemingly apparent in our graphs, a few clear conclusions can be drawn. First, song consistency is important for perceptual compatibility with Mandelbrot zoom sequences. While pacing matters less as shown by the wide variance in tempo, interruptions from liveness, speechiness, or an unusual time signature ruins perceptual compatibility for most songs. Second, individuals perceive more electronically-based songs better than they perceive more acoustic-based songs. This connection is further supported by the interviews we held during our sampling efforts with one participant saying:\n",
    "\n",
    "> (The Mandelbrot zoom sequence)... reminds me of, like, funky robots. Like Jet Set, ya know?<sup>[7]</sup>\n",
    "\n",
    "Third, individuals perceive louder songs better with Mandelbrot zoom sequences. Out of all the significant trends noticed, this one is the least clear as to why this occurs. Accordingly, it is wise to conduct further research into this trend to uncover the root causes behind it.\n",
    "\n",
    "Overall, it is clear that certain song characteristics change how well the average individual perceives it paired with a Mandelbrot zoom sequence. This falls in line with our current understanding of how the human brain reconciles sight and sound, and governs how our senses work to process information. Consequently, we have created a second half of the Mandelbot+, the Mandelbot Beta, which seeks to create optimized playlists based off of these trends. The Mandelbot Beta uses the k-means machine learning algorithm to partition the larger database into *k* distinct, non-overlapping clusters. The algorithm assigns each observation to exactly one of the *k* clusters. Currently, the algorithm creates 2 clusters, defined as energetic and calm moods. As our sampling campaign continues beyond this project and the database grows in magnitude, the k-means algorithm will continue to shift when more clusters become visible. Another next step might be to explore multiple regression models for predicting whether a song is a good perceptual fit with Mandelbrot zoom sequences. Further analysis might include potential areas in music moods, color chromatic variations in specific mandelbrot iterations, adding the backgrounds of artists/producers, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **References:**\n",
    "\n",
    "[1]: The Mandelbrot set, theme and variations. Tan, Lei. Cambridge University Press, 2000. ISBN 978-0-521-77476-5.\n",
    "\n",
    "[2]: García, Francisco; Ángel Fernández; Javier Barrallo; Luis Martín. \"Coloring Dynamical Systems in the Complex Plane\" (PDF).\n",
    "\n",
    "[3]: Maths Town. “Eye of the Universe - Mandelbrot Fractal Zoom.” YouTube.com, YouTube, 28 Aug. 2017, https://www.youtube.com/watch?v=pCpLWbHVNhk. \n",
    "\n",
    "[4]: These are: danceability, energy, key, loudness, speechiness, acousticness, instrumentalness, liveness, valence, tempo, time signature, and track duration in minutes.\n",
    "\n",
    "[5]: Spotify. “Features: Spotify for Developers.” Features | Spotify for Developers, Spotify, https://developer.spotify.com/discover/. \n",
    "\n",
    "[6]: Spotify. “Help - Loudness Normalization – Spotify for Artists.” Fans Make It Possible – Spotify for Artists, Spotify, https://artists.spotify.com/en/help/article/loudness-normalization. \n",
    "\n",
    "[7]: This is in reference to the 2000 video game \"Jet Set Radio\", in which the player vandalizes a futuristic Tokyo with graffiti. Its soundtrack is prominent throughout gameplay and composed of heavily electronic songs<sup>[8]</sup> - some of which became part of our sample.\n",
    "\n",
    "[8]: Anthony Caufield; Nichola Caufield. Jet Set Radio: The Rude Awakening (motion picture). Sega. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}