002. recommender system basic with Python - 1 content based filtering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 블로그 설명\n",
    "\n",
    "**해당 자료에 대한 설명은 아래 블로그에도 올려두었습니다.**\n",
    "- https://lsjsj92.tistory.com/565\n",
    "\n",
    "----\n",
    "\n",
    "이전 자료에서 다루었던 추천 시스템을 실습으로 살펴봅니다.\n",
    "\n",
    "해당 자료는 아래 리스트에서 참고했습니다.\n",
    "\n",
    "- https://www.kaggle.com/rounakbanik/movie-recommender-systems\n",
    "- https://www.kaggle.com/ibtesama/getting-started-with-a-movie-recommendation-system\n",
    "\n",
    "\n",
    "데이터는 kaggle의 **The movies Dataset (https://www.kaggle.com/rounakbanik/the-movies-dataset)** 을 사용했습니다.\n",
    "\n",
    "\n",
    "가장 먼저 데이터 전처리와 콘텐츠 기반(content based filtering)으로 시작합니다.\n",
    "\n",
    "# 데이터 전처리"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from ast import literal_eval\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.metrics.pairwise import cosine_similarity"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터를 불러옵니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.read_csv('./movie_data/tmdb_5000_movies.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>budget</th>\n",
       "      <th>genres</th>\n",
       "      <th>homepage</th>\n",
       "      <th>id</th>\n",
       "      <th>keywords</th>\n",
       "      <th>original_language</th>\n",
       "      <th>original_title</th>\n",
       "      <th>overview</th>\n",
       "      <th>popularity</th>\n",
       "      <th>production_companies</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>release_date</th>\n",
       "      <th>revenue</th>\n",
       "      <th>runtime</th>\n",
       "      <th>spoken_languages</th>\n",
       "      <th>status</th>\n",
       "      <th>tagline</th>\n",
       "      <th>title</th>\n",
       "      <th>vote_average</th>\n",
       "      <th>vote_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>237000000</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...</td>\n",
       "      <td>http://www.avatarmovie.com/</td>\n",
       "      <td>19995</td>\n",
       "      <td>[{\"id\": 1463, \"name\": \"culture clash\"}, {\"id\":...</td>\n",
       "      <td>en</td>\n",
       "      <td>Avatar</td>\n",
       "      <td>In the 22nd century, a paraplegic Marine is di...</td>\n",
       "      <td>150.437577</td>\n",
       "      <td>[{\"name\": \"Ingenious Film Partners\", \"id\": 289...</td>\n",
       "      <td>[{\"iso_3166_1\": \"US\", \"name\": \"United States o...</td>\n",
       "      <td>2009-12-10</td>\n",
       "      <td>2787965087</td>\n",
       "      <td>162.0</td>\n",
       "      <td>[{\"iso_639_1\": \"en\", \"name\": \"English\"}, {\"iso...</td>\n",
       "      <td>Released</td>\n",
       "      <td>Enter the World of Pandora.</td>\n",
       "      <td>Avatar</td>\n",
       "      <td>7.2</td>\n",
       "      <td>11800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>300000000</td>\n",
       "      <td>[{\"id\": 12, \"name\": \"Adventure\"}, {\"id\": 14, \"...</td>\n",
       "      <td>http://disney.go.com/disneypictures/pirates/</td>\n",
       "      <td>285</td>\n",
       "      <td>[{\"id\": 270, \"name\": \"ocean\"}, {\"id\": 726, \"na...</td>\n",
       "      <td>en</td>\n",
       "      <td>Pirates of the Caribbean: At World's End</td>\n",
       "      <td>Captain Barbossa, long believed to be dead, ha...</td>\n",
       "      <td>139.082615</td>\n",
       "      <td>[{\"name\": \"Walt Disney Pictures\", \"id\": 2}, {\"...</td>\n",
       "      <td>[{\"iso_3166_1\": \"US\", \"name\": \"United States o...</td>\n",
       "      <td>2007-05-19</td>\n",
       "      <td>961000000</td>\n",
       "      <td>169.0</td>\n",
       "      <td>[{\"iso_639_1\": \"en\", \"name\": \"English\"}]</td>\n",
       "      <td>Released</td>\n",
       "      <td>At the end of the world, the adventure begins.</td>\n",
       "      <td>Pirates of the Caribbean: At World's End</td>\n",
       "      <td>6.9</td>\n",
       "      <td>4500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      budget                                             genres  \\\n",
       "0  237000000  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...   \n",
       "1  300000000  [{\"id\": 12, \"name\": \"Adventure\"}, {\"id\": 14, \"...   \n",
       "\n",
       "                                       homepage     id  \\\n",
       "0                   http://www.avatarmovie.com/  19995   \n",
       "1  http://disney.go.com/disneypictures/pirates/    285   \n",
       "\n",
       "                                            keywords original_language  \\\n",
       "0  [{\"id\": 1463, \"name\": \"culture clash\"}, {\"id\":...                en   \n",
       "1  [{\"id\": 270, \"name\": \"ocean\"}, {\"id\": 726, \"na...                en   \n",
       "\n",
       "                             original_title  \\\n",
       "0                                    Avatar   \n",
       "1  Pirates of the Caribbean: At World's End   \n",
       "\n",
       "                                            overview  popularity  \\\n",
       "0  In the 22nd century, a paraplegic Marine is di...  150.437577   \n",
       "1  Captain Barbossa, long believed to be dead, ha...  139.082615   \n",
       "\n",
       "                                production_companies  \\\n",
       "0  [{\"name\": \"Ingenious Film Partners\", \"id\": 289...   \n",
       "1  [{\"name\": \"Walt Disney Pictures\", \"id\": 2}, {\"...   \n",
       "\n",
       "                                production_countries release_date     revenue  \\\n",
       "0  [{\"iso_3166_1\": \"US\", \"name\": \"United States o...   2009-12-10  2787965087   \n",
       "1  [{\"iso_3166_1\": \"US\", \"name\": \"United States o...   2007-05-19   961000000   \n",
       "\n",
       "   runtime                                   spoken_languages    status  \\\n",
       "0    162.0  [{\"iso_639_1\": \"en\", \"name\": \"English\"}, {\"iso...  Released   \n",
       "1    169.0           [{\"iso_639_1\": \"en\", \"name\": \"English\"}]  Released   \n",
       "\n",
       "                                          tagline  \\\n",
       "0                     Enter the World of Pandora.   \n",
       "1  At the end of the world, the adventure begins.   \n",
       "\n",
       "                                      title  vote_average  vote_count  \n",
       "0                                    Avatar           7.2       11800  \n",
       "1  Pirates of the Caribbean: At World's End           6.9        4500  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(4803, 20)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터에 대한 설명은 참조 링크에 들어가면 자세히 쓰여져 있습니다.  \n",
    "여기서 제가 일반적으로 많이 사용하는 컬럼은 아래와 같습니다.  \n",
    "\n",
    "- genres : 영화 장르\n",
    "- keywords : 영화의 키워드\n",
    "- original_language : 영화 언어 \n",
    "- title : 제목\n",
    "- vote_average : 평점 평균\n",
    "- vote_count :  평점 카운트\n",
    "- popularity : 인기도\n",
    "- overview : 개요 설명\n",
    "\n",
    "등등 같은 컬럼을 사용할 예정입니다. 다른 컬럼은 일단 여기서 그렇게 중요하지 않게 사용합니다.  \n",
    "사실, release_data와 같은 컬럼도 중요할 수 있습니다. 최신 영화를 추천할 수도 있으니까요. 하지만 여기서는 사용하지 않겠습니다.  \n",
    "\n",
    "\n",
    "가장 먼저 **전처리**를 조금 해주어야 합니다.  \n",
    "먼저 우리가 사용할 데이터부터 뽑아보죠.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = data[['id','genres', 'vote_average', 'vote_count','popularity','title',  'keywords', 'overview']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "그리고 vote_average값을 변경해주어야 합니다.   \n",
    "현재 vote_average는 조금 **불공정**하게 되어 있습니다.\n",
    "\n",
    "왜냐하면, vote 수가 적은데(예를 들어 3개) 3개 전부 5점이라고 하면 vote가 5점으로 되어 있기 때문입니다.  \n",
    "하지만, vote 수가 많을수록 vote_average가 떨어질 수 밖에 없습니다. 많은 사람들이 평가를 하니까요.  \n",
    "\n",
    "그래서 이런 불공정을 처리하기 위해 imdb에서 처리한 방법이 있습니다.  \n",
    "해당 이슈는 url : https://www.quora.com/How-does-IMDbs-rating-system-work 에서 확인할 수 있습니다.\n",
    "\n",
    "그에 대한 답은 아래와 같습니다.\n",
    "\n",
    "![1](https://user-images.githubusercontent.com/24634054/71774470-d1470c80-2fb2-11ea-8a1e-aa018dd6d25a.JPG)\n",
    "\n",
    "- r : 개별 영화 평점\n",
    "- v : 개별 영화에 평점을 투표한 횟수\n",
    "- m : 250위 안에 들어야 하는 최소 투표 (정하기 나름인듯. 난 500이라고 하면 500으로 해도 되고.)\n",
    "- c : 전체 영화에 대한 평균 평점\n",
    "\n",
    "여기서 m은 **500위로 가정하고 진행하겠습니다.** \n",
    "\n",
    "먼저 m부터 찾아보죠. 500위 정도로 들어오게 하려면 vote_count가 상위 몇 %이어야 할까요?  \n",
    "이는 quantile을 이용해서 구할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1683.8999999999987"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp_m = data['vote_count'].quantile(0.89)\n",
    "tmp_m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(529, 8)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp_data = data.copy().loc[data['vote_count'] >= tmp_m]\n",
    "tmp_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "상위 90%로 했을 때 481개가 들어옵니다.   \n",
    "89%로 하면 529개가 들어오게 됩니다. 저는 90%로 가정하고 진행하도록 하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "del tmp_data\n",
    "\n",
    "m = data['vote_count'].quantile(0.9)\n",
    "data = data.loc[data['vote_count'] >= m]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>genres</th>\n",
       "      <th>vote_average</th>\n",
       "      <th>vote_count</th>\n",
       "      <th>popularity</th>\n",
       "      <th>title</th>\n",
       "      <th>keywords</th>\n",
       "      <th>overview</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19995</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...</td>\n",
       "      <td>7.2</td>\n",
       "      <td>11800</td>\n",
       "      <td>150.437577</td>\n",
       "      <td>Avatar</td>\n",
       "      <td>[{\"id\": 1463, \"name\": \"culture clash\"}, {\"id\":...</td>\n",
       "      <td>In the 22nd century, a paraplegic Marine is di...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>285</td>\n",
       "      <td>[{\"id\": 12, \"name\": \"Adventure\"}, {\"id\": 14, \"...</td>\n",
       "      <td>6.9</td>\n",
       "      <td>4500</td>\n",
       "      <td>139.082615</td>\n",
       "      <td>Pirates of the Caribbean: At World's End</td>\n",
       "      <td>[{\"id\": 270, \"name\": \"ocean\"}, {\"id\": 726, \"na...</td>\n",
       "      <td>Captain Barbossa, long believed to be dead, ha...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>206647</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...</td>\n",
       "      <td>6.3</td>\n",
       "      <td>4466</td>\n",
       "      <td>107.376788</td>\n",
       "      <td>Spectre</td>\n",
       "      <td>[{\"id\": 470, \"name\": \"spy\"}, {\"id\": 818, \"name...</td>\n",
       "      <td>A cryptic message from Bond’s past sends him o...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>49026</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 80, \"nam...</td>\n",
       "      <td>7.6</td>\n",
       "      <td>9106</td>\n",
       "      <td>112.312950</td>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>[{\"id\": 849, \"name\": \"dc comics\"}, {\"id\": 853,...</td>\n",
       "      <td>Following the death of District Attorney Harve...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>49529</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2124</td>\n",
       "      <td>43.926995</td>\n",
       "      <td>John Carter</td>\n",
       "      <td>[{\"id\": 818, \"name\": \"based on novel\"}, {\"id\":...</td>\n",
       "      <td>John Carter is a war-weary, former military ca...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       id                                             genres  vote_average  \\\n",
       "0   19995  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...           7.2   \n",
       "1     285  [{\"id\": 12, \"name\": \"Adventure\"}, {\"id\": 14, \"...           6.9   \n",
       "2  206647  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...           6.3   \n",
       "3   49026  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 80, \"nam...           7.6   \n",
       "4   49529  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...           6.1   \n",
       "\n",
       "   vote_count  popularity                                     title  \\\n",
       "0       11800  150.437577                                    Avatar   \n",
       "1        4500  139.082615  Pirates of the Caribbean: At World's End   \n",
       "2        4466  107.376788                                   Spectre   \n",
       "3        9106  112.312950                     The Dark Knight Rises   \n",
       "4        2124   43.926995                               John Carter   \n",
       "\n",
       "                                            keywords  \\\n",
       "0  [{\"id\": 1463, \"name\": \"culture clash\"}, {\"id\":...   \n",
       "1  [{\"id\": 270, \"name\": \"ocean\"}, {\"id\": 726, \"na...   \n",
       "2  [{\"id\": 470, \"name\": \"spy\"}, {\"id\": 818, \"name...   \n",
       "3  [{\"id\": 849, \"name\": \"dc comics\"}, {\"id\": 853,...   \n",
       "4  [{\"id\": 818, \"name\": \"based on novel\"}, {\"id\":...   \n",
       "\n",
       "                                            overview  \n",
       "0  In the 22nd century, a paraplegic Marine is di...  \n",
       "1  Captain Barbossa, long believed to be dead, ha...  \n",
       "2  A cryptic message from Bond’s past sends him o...  \n",
       "3  Following the death of District Attorney Harve...  \n",
       "4  John Carter is a war-weary, former military ca...  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "C = data['vote_average'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "6.962993762993763\n",
      "1838.4000000000015\n"
     ]
    }
   ],
   "source": [
    "print(C)\n",
    "print(m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def weighted_rating(x, m=m, C=C):\n",
    "    v = x['vote_count']\n",
    "    R = x['vote_average']\n",
    "    \n",
    "    return ( v / (v+m) * R ) + (m / (m + v) * C)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['score'] = data.apply(weighted_rating, axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>genres</th>\n",
       "      <th>vote_average</th>\n",
       "      <th>vote_count</th>\n",
       "      <th>popularity</th>\n",
       "      <th>title</th>\n",
       "      <th>keywords</th>\n",
       "      <th>overview</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19995</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...</td>\n",
       "      <td>7.2</td>\n",
       "      <td>11800</td>\n",
       "      <td>150.437577</td>\n",
       "      <td>Avatar</td>\n",
       "      <td>[{\"id\": 1463, \"name\": \"culture clash\"}, {\"id\":...</td>\n",
       "      <td>In the 22nd century, a paraplegic Marine is di...</td>\n",
       "      <td>7.168053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>285</td>\n",
       "      <td>[{\"id\": 12, \"name\": \"Adventure\"}, {\"id\": 14, \"...</td>\n",
       "      <td>6.9</td>\n",
       "      <td>4500</td>\n",
       "      <td>139.082615</td>\n",
       "      <td>Pirates of the Caribbean: At World's End</td>\n",
       "      <td>[{\"id\": 270, \"name\": \"ocean\"}, {\"id\": 726, \"na...</td>\n",
       "      <td>Captain Barbossa, long believed to be dead, ha...</td>\n",
       "      <td>6.918271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>206647</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...</td>\n",
       "      <td>6.3</td>\n",
       "      <td>4466</td>\n",
       "      <td>107.376788</td>\n",
       "      <td>Spectre</td>\n",
       "      <td>[{\"id\": 470, \"name\": \"spy\"}, {\"id\": 818, \"name...</td>\n",
       "      <td>A cryptic message from Bond’s past sends him o...</td>\n",
       "      <td>6.493333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>49026</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 80, \"nam...</td>\n",
       "      <td>7.6</td>\n",
       "      <td>9106</td>\n",
       "      <td>112.312950</td>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>[{\"id\": 849, \"name\": \"dc comics\"}, {\"id\": 853,...</td>\n",
       "      <td>Following the death of District Attorney Harve...</td>\n",
       "      <td>7.492998</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>49529</td>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...</td>\n",
       "      <td>6.1</td>\n",
       "      <td>2124</td>\n",
       "      <td>43.926995</td>\n",
       "      <td>John Carter</td>\n",
       "      <td>[{\"id\": 818, \"name\": \"based on novel\"}, {\"id\":...</td>\n",
       "      <td>John Carter is a war-weary, former military ca...</td>\n",
       "      <td>6.500396</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       id                                             genres  vote_average  \\\n",
       "0   19995  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...           7.2   \n",
       "1     285  [{\"id\": 12, \"name\": \"Adventure\"}, {\"id\": 14, \"...           6.9   \n",
       "2  206647  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...           6.3   \n",
       "3   49026  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 80, \"nam...           7.6   \n",
       "4   49529  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...           6.1   \n",
       "\n",
       "   vote_count  popularity                                     title  \\\n",
       "0       11800  150.437577                                    Avatar   \n",
       "1        4500  139.082615  Pirates of the Caribbean: At World's End   \n",
       "2        4466  107.376788                                   Spectre   \n",
       "3        9106  112.312950                     The Dark Knight Rises   \n",
       "4        2124   43.926995                               John Carter   \n",
       "\n",
       "                                            keywords  \\\n",
       "0  [{\"id\": 1463, \"name\": \"culture clash\"}, {\"id\":...   \n",
       "1  [{\"id\": 270, \"name\": \"ocean\"}, {\"id\": 726, \"na...   \n",
       "2  [{\"id\": 470, \"name\": \"spy\"}, {\"id\": 818, \"name...   \n",
       "3  [{\"id\": 849, \"name\": \"dc comics\"}, {\"id\": 853,...   \n",
       "4  [{\"id\": 818, \"name\": \"based on novel\"}, {\"id\":...   \n",
       "\n",
       "                                            overview     score  \n",
       "0  In the 22nd century, a paraplegic Marine is di...  7.168053  \n",
       "1  Captain Barbossa, long believed to be dead, ha...  6.918271  \n",
       "2  A cryptic message from Bond’s past sends him o...  6.493333  \n",
       "3  Following the death of District Attorney Harve...  7.492998  \n",
       "4  John Carter is a war-weary, former military ca...  6.500396  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(481, 9)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이렇게 weighted_score가 완성되었습니다.\n",
    "\n",
    "또한, 지금 장르와 키워드를 보시면 조금 독특한 구조의 데이터를 가지고 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>genres</th>\n",
       "      <th>keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...</td>\n",
       "      <td>[{\"id\": 1463, \"name\": \"culture clash\"}, {\"id\":...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[{\"id\": 12, \"name\": \"Adventure\"}, {\"id\": 14, \"...</td>\n",
       "      <td>[{\"id\": 270, \"name\": \"ocean\"}, {\"id\": 726, \"na...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              genres  \\\n",
       "0  [{\"id\": 28, \"name\": \"Action\"}, {\"id\": 12, \"nam...   \n",
       "1  [{\"id\": 12, \"name\": \"Adventure\"}, {\"id\": 14, \"...   \n",
       "\n",
       "                                            keywords  \n",
       "0  [{\"id\": 1463, \"name\": \"culture clash\"}, {\"id\":...  \n",
       "1  [{\"id\": 270, \"name\": \"ocean\"}, {\"id\": 726, \"na...  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['genres', 'keywords']].head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "list 내부에 dictionary가 있는 구조로 되어있습니다.  \n",
    "이렇게 표현한 이유는 하나의 영화가 하나의 장르에만 속하지 않고, 하나의 키워드만 있지 않기 때문입니다.  \n",
    "그리고 문제가 지금 내부에는 **문자열**로 들어가 있는 것입니다.\n",
    "\n",
    "이를 해결하기 위해서 ast 패키지를 사용해야합니다. ast내부에 literal_eval을 사용해보죠  \n",
    "\n",
    "그러면 list와 dictionary 형태로 바뀌게 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['genres'] = data['genres'].apply(literal_eval)\n",
    "data['keywords'] = data['keywords'].apply(literal_eval)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>genres</th>\n",
       "      <th>keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[{'id': 28, 'name': 'Action'}, {'id': 12, 'nam...</td>\n",
       "      <td>[{'id': 1463, 'name': 'culture clash'}, {'id':...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[{'id': 12, 'name': 'Adventure'}, {'id': 14, '...</td>\n",
       "      <td>[{'id': 270, 'name': 'ocean'}, {'id': 726, 'na...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              genres  \\\n",
       "0  [{'id': 28, 'name': 'Action'}, {'id': 12, 'nam...   \n",
       "1  [{'id': 12, 'name': 'Adventure'}, {'id': 14, '...   \n",
       "\n",
       "                                            keywords  \n",
       "0  [{'id': 1463, 'name': 'culture clash'}, {'id':...  \n",
       "1  [{'id': 270, 'name': 'ocean'}, {'id': 726, 'na...  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['genres', 'keywords']].head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그럼 이제 장르와 키워드를 id를 제거한 후 name만 뽑아내면 끝나겠죠?  \n",
    "우리에게 저 id 값을 필요없으니까요. 장르가 무엇이고 키워드가 무엇인지만 알면됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "data['genres'] = data['genres'].apply(lambda x : [d['name'] for d in x]).apply(lambda x : \" \".join(x))\n",
    "data['keywords'] = data['keywords'].apply(lambda x : [d['name'] for d in x]).apply(lambda x : \" \".join(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>genres</th>\n",
       "      <th>vote_average</th>\n",
       "      <th>vote_count</th>\n",
       "      <th>popularity</th>\n",
       "      <th>title</th>\n",
       "      <th>keywords</th>\n",
       "      <th>overview</th>\n",
       "      <th>score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19995</td>\n",
       "      <td>Action Adventure Fantasy Science Fiction</td>\n",
       "      <td>7.2</td>\n",
       "      <td>11800</td>\n",
       "      <td>150.437577</td>\n",
       "      <td>Avatar</td>\n",
       "      <td>culture clash future space war space colony so...</td>\n",
       "      <td>In the 22nd century, a paraplegic Marine is di...</td>\n",
       "      <td>7.168053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>285</td>\n",
       "      <td>Adventure Fantasy Action</td>\n",
       "      <td>6.9</td>\n",
       "      <td>4500</td>\n",
       "      <td>139.082615</td>\n",
       "      <td>Pirates of the Caribbean: At World's End</td>\n",
       "      <td>ocean drug abuse exotic island east india trad...</td>\n",
       "      <td>Captain Barbossa, long believed to be dead, ha...</td>\n",
       "      <td>6.918271</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                                    genres  vote_average  vote_count  \\\n",
       "0  19995  Action Adventure Fantasy Science Fiction           7.2       11800   \n",
       "1    285                  Adventure Fantasy Action           6.9        4500   \n",
       "\n",
       "   popularity                                     title  \\\n",
       "0  150.437577                                    Avatar   \n",
       "1  139.082615  Pirates of the Caribbean: At World's End   \n",
       "\n",
       "                                            keywords  \\\n",
       "0  culture clash future space war space colony so...   \n",
       "1  ocean drug abuse exotic island east india trad...   \n",
       "\n",
       "                                            overview     score  \n",
       "0  In the 22nd century, a paraplegic Marine is di...  7.168053  \n",
       "1  Captain Barbossa, long believed to be dead, ha...  6.918271  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "다음장에서 사용할 데이터이므로 미리 저장을 합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.to_csv('./movie_data/pre_tmdb_5000_movies.csv', index = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "자! 이렇게 어느정도 전처리가 마무리 되었습니다.\n",
    "\n",
    "이제 본격적으로 진행해보죠.\n",
    "\n",
    "# 콘텐츠 기반 필터링 추천(Content based filtering)\n",
    "\n",
    "콘텐츠 기반으로 추천을 하고자 합니다. 콘텐츠 기반 필터링을 이용해서 추천을 진행하는 것은 비슷한 콘텐츠를 사용자에게 추천하는 것을 말합니다.  \n",
    "\n",
    "여기서 비슷한 콘텐츠는 무엇일까요? 대표적으로 '장르'가 될 수 있습니다.   \n",
    "따라서 content based filtering 추천에서는 이 '장르'를 이용해서 추천을 하겠습니다.  \n",
    "\n",
    "현재 장르는 문자열로 되어 있습니다. 이 문자열을 숫자로 바꾸어 벡터화 시켜야겠죠? 이것부터 진행하죠\n",
    "\n",
    "## 해당 작업은 2021.08.01에 수정되었습니다.\n",
    "\n",
    "- 데이터 처리를 잘못한 것을 발견해서 급하게 수정하였습니다.\n",
    "- 위에서 전처리한 데이터는 참고만 부탁드립니다!\n",
    "- 위에서 처리한 방법은 나중에 movielens 데이터 전처리 할 때 필요한 요소일 수 있으니 삭제하지 않고 유지하겠습니다.\n",
    "\n",
    "\n",
    "데이터는 아래와 같은 것을 사용합니다.\n",
    "- movie_metadata.csv\n",
    "    - 제목과 장르 등의 영화 메타 데이터\n",
    "- keywords.csv\n",
    "    - 영화 id에 따라 keyword 값"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(32269, 4)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/leesoojin/opt/anaconda3/envs/soojin/lib/python3.8/site-packages/IPython/core/interactiveshell.py:3145: DtypeWarning: Columns (10) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  has_raised = await self.run_ast_nodes(code_ast.body, cell_name,\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>original_language</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>862</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 16, 'name': 'Animation'}, {'id': 35, '...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8844</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 12, 'name': 'Adventure'}, {'id': 14, '...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15602</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 10749, 'name': 'Romance'}, {'id': 35, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>31357</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 35, 'name': 'Comedy'}, {'id': 18, 'nam...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11862</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 35, 'name': 'Comedy'}]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                        title original_language  \\\n",
       "0    862                    Toy Story                en   \n",
       "1   8844                      Jumanji                en   \n",
       "2  15602             Grumpier Old Men                en   \n",
       "3  31357            Waiting to Exhale                en   \n",
       "4  11862  Father of the Bride Part II                en   \n",
       "\n",
       "                                              genres  \n",
       "0  [{'id': 16, 'name': 'Animation'}, {'id': 35, '...  \n",
       "1  [{'id': 12, 'name': 'Adventure'}, {'id': 14, '...  \n",
       "2  [{'id': 10749, 'name': 'Romance'}, {'id': 35, ...  \n",
       "3  [{'id': 35, 'name': 'Comedy'}, {'id': 18, 'nam...  \n",
       "4                     [{'id': 35, 'name': 'Comedy'}]  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_data = pd.read_csv('./data/movie_dataset/movies_metadata.csv')\n",
    "movie_data =  movie_data.loc[movie_data['original_language'] == 'en', :]\n",
    "movie_data = movie_data[['id', 'title', 'original_language', 'genres']]\n",
    "\n",
    "print(movie_data.shape)\n",
    "movie_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(46419, 2)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>862</td>\n",
       "      <td>[{'id': 931, 'name': 'jealousy'}, {'id': 4290,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8844</td>\n",
       "      <td>[{'id': 10090, 'name': 'board game'}, {'id': 1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15602</td>\n",
       "      <td>[{'id': 1495, 'name': 'fishing'}, {'id': 12392...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>31357</td>\n",
       "      <td>[{'id': 818, 'name': 'based on novel'}, {'id':...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11862</td>\n",
       "      <td>[{'id': 1009, 'name': 'baby'}, {'id': 1599, 'n...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                                           keywords\n",
       "0    862  [{'id': 931, 'name': 'jealousy'}, {'id': 4290,...\n",
       "1   8844  [{'id': 10090, 'name': 'board game'}, {'id': 1...\n",
       "2  15602  [{'id': 1495, 'name': 'fishing'}, {'id': 12392...\n",
       "3  31357  [{'id': 818, 'name': 'based on novel'}, {'id':...\n",
       "4  11862  [{'id': 1009, 'name': 'baby'}, {'id': 1599, 'n..."
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_keyword = pd.read_csv('./data/movie_dataset/keywords.csv')\n",
    "print(movie_keyword.shape)\n",
    "movie_keyword.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터 2개를 가져와서 id를 맞춰 merge 해줍니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(32852, 5)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>original_language</th>\n",
       "      <th>genres</th>\n",
       "      <th>keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>862</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 16, 'name': 'Animation'}, {'id': 35, '...</td>\n",
       "      <td>[{'id': 931, 'name': 'jealousy'}, {'id': 4290,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8844</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 12, 'name': 'Adventure'}, {'id': 14, '...</td>\n",
       "      <td>[{'id': 10090, 'name': 'board game'}, {'id': 1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15602</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 10749, 'name': 'Romance'}, {'id': 35, ...</td>\n",
       "      <td>[{'id': 1495, 'name': 'fishing'}, {'id': 12392...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>31357</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 35, 'name': 'Comedy'}, {'id': 18, 'nam...</td>\n",
       "      <td>[{'id': 818, 'name': 'based on novel'}, {'id':...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11862</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>en</td>\n",
       "      <td>[{'id': 35, 'name': 'Comedy'}]</td>\n",
       "      <td>[{'id': 1009, 'name': 'baby'}, {'id': 1599, 'n...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                        title original_language  \\\n",
       "0    862                    Toy Story                en   \n",
       "1   8844                      Jumanji                en   \n",
       "2  15602             Grumpier Old Men                en   \n",
       "3  31357            Waiting to Exhale                en   \n",
       "4  11862  Father of the Bride Part II                en   \n",
       "\n",
       "                                              genres  \\\n",
       "0  [{'id': 16, 'name': 'Animation'}, {'id': 35, '...   \n",
       "1  [{'id': 12, 'name': 'Adventure'}, {'id': 14, '...   \n",
       "2  [{'id': 10749, 'name': 'Romance'}, {'id': 35, ...   \n",
       "3  [{'id': 35, 'name': 'Comedy'}, {'id': 18, 'nam...   \n",
       "4                     [{'id': 35, 'name': 'Comedy'}]   \n",
       "\n",
       "                                            keywords  \n",
       "0  [{'id': 931, 'name': 'jealousy'}, {'id': 4290,...  \n",
       "1  [{'id': 10090, 'name': 'board game'}, {'id': 1...  \n",
       "2  [{'id': 1495, 'name': 'fishing'}, {'id': 12392...  \n",
       "3  [{'id': 818, 'name': 'based on novel'}, {'id':...  \n",
       "4  [{'id': 1009, 'name': 'baby'}, {'id': 1599, 'n...  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_data.id = movie_data.id.astype(int)\n",
    "movie_keyword.id = movie_keyword.id.astype(int)\n",
    "movie_data = pd.merge(movie_data, movie_keyword, on='id')\n",
    "print(movie_data.shape)\n",
    "movie_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 전처리\n",
    "\n",
    "앞서 진행했던 전처리 방법 그대로 적용해서 전처리를 진행하겠습니다.  \n",
    "위에서 설명이 쓰여져 있지만, 데이터 내부엔 list(dict()) 형태로 구성되어 있습니다. 또한 이 안에는 **문자열** 형태로 들어가있구요.  \n",
    "이를 처리하기 위해서 ast의 literal_eval 함수를 사용합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "movie_data['genres'] = movie_data['genres'].apply(literal_eval)\n",
    "movie_data['genres'] = movie_data['genres'].apply(lambda x : [d['name'] for d in x]).apply(lambda x : \" \".join(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "movie_data['keywords'] = movie_data['keywords'].apply(literal_eval)\n",
    "movie_data['keywords'] = movie_data['keywords'].apply(lambda x : [d['name'] for d in x]).apply(lambda x : \" \".join(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>original_language</th>\n",
       "      <th>genres</th>\n",
       "      <th>keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>862</td>\n",
       "      <td>Toy Story</td>\n",
       "      <td>en</td>\n",
       "      <td>Animation Comedy Family</td>\n",
       "      <td>jealousy toy boy friendship friends rivalry bo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8844</td>\n",
       "      <td>Jumanji</td>\n",
       "      <td>en</td>\n",
       "      <td>Adventure Fantasy Family</td>\n",
       "      <td>board game disappearance based on children's b...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15602</td>\n",
       "      <td>Grumpier Old Men</td>\n",
       "      <td>en</td>\n",
       "      <td>Romance Comedy</td>\n",
       "      <td>fishing best friend duringcreditsstinger old men</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>31357</td>\n",
       "      <td>Waiting to Exhale</td>\n",
       "      <td>en</td>\n",
       "      <td>Comedy Drama Romance</td>\n",
       "      <td>based on novel interracial relationship single...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>11862</td>\n",
       "      <td>Father of the Bride Part II</td>\n",
       "      <td>en</td>\n",
       "      <td>Comedy</td>\n",
       "      <td>baby midlife crisis confidence aging daughter ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id                        title original_language  \\\n",
       "0    862                    Toy Story                en   \n",
       "1   8844                      Jumanji                en   \n",
       "2  15602             Grumpier Old Men                en   \n",
       "3  31357            Waiting to Exhale                en   \n",
       "4  11862  Father of the Bride Part II                en   \n",
       "\n",
       "                     genres                                           keywords  \n",
       "0   Animation Comedy Family  jealousy toy boy friendship friends rivalry bo...  \n",
       "1  Adventure Fantasy Family  board game disappearance based on children's b...  \n",
       "2            Romance Comedy   fishing best friend duringcreditsstinger old men  \n",
       "3      Comedy Drama Romance  based on novel interracial relationship single...  \n",
       "4                    Comedy  baby midlife crisis confidence aging daughter ...  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TF-IDF 벡터화\n",
    "\n",
    "전처리한 데이터를 TF-IDF 방법을 이용해 벡터로 만들어줍니다.  \n",
    "저는 장르와 keyword를 하나로 합친 후 tfidf vector로 만들어주었습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "tfidf_vector = TfidfVectorizer()\n",
    "#tfidf_vector = TfidfVectorizer(ngram_range=(1,2))\n",
    "tfidf_matrix = tfidf_vector.fit_transform(movie_data['genres'] + \" \" + movie_data['keywords']).toarray()\n",
    "#tfidf_matrix = tfidf_vector.fit_transform(movie_data['genres']).toarray()\n",
    "tfidf_matrix_feature = tfidf_vector.get_feature_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(32852, 11437)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_matrix.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(32852, 11437)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>077</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>13</th>\n",
       "      <th>1500s</th>\n",
       "      <th>15th</th>\n",
       "      <th>16th</th>\n",
       "      <th>17th</th>\n",
       "      <th>1812</th>\n",
       "      <th>18th</th>\n",
       "      <th>...</th>\n",
       "      <th>βάφτηκε</th>\n",
       "      <th>γη</th>\n",
       "      <th>κόκκινο</th>\n",
       "      <th>το</th>\n",
       "      <th>χώμα</th>\n",
       "      <th>миньоны</th>\n",
       "      <th>卧底肥妈</th>\n",
       "      <th>绝地奶霸</th>\n",
       "      <th>自然界大事件</th>\n",
       "      <th>超级妈妈</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Toy Story</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jumanji</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grumpier Old Men</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Waiting to Exhale</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Father of the Bride Part II</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 11437 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                             077   10   11   13  1500s  15th  16th  17th  \\\n",
       "title                                                                      \n",
       "Toy Story                    0.0  0.0  0.0  0.0    0.0   0.0   0.0   0.0   \n",
       "Jumanji                      0.0  0.0  0.0  0.0    0.0   0.0   0.0   0.0   \n",
       "Grumpier Old Men             0.0  0.0  0.0  0.0    0.0   0.0   0.0   0.0   \n",
       "Waiting to Exhale            0.0  0.0  0.0  0.0    0.0   0.0   0.0   0.0   \n",
       "Father of the Bride Part II  0.0  0.0  0.0  0.0    0.0   0.0   0.0   0.0   \n",
       "\n",
       "                             1812  18th  ...  βάφτηκε   γη  κόκκινο   το  \\\n",
       "title                                    ...                               \n",
       "Toy Story                     0.0   0.0  ...      0.0  0.0      0.0  0.0   \n",
       "Jumanji                       0.0   0.0  ...      0.0  0.0      0.0  0.0   \n",
       "Grumpier Old Men              0.0   0.0  ...      0.0  0.0      0.0  0.0   \n",
       "Waiting to Exhale             0.0   0.0  ...      0.0  0.0      0.0  0.0   \n",
       "Father of the Bride Part II   0.0   0.0  ...      0.0  0.0      0.0  0.0   \n",
       "\n",
       "                             χώμα  миньоны  卧底肥妈  绝地奶霸  自然界大事件  超级妈妈  \n",
       "title                                                                 \n",
       "Toy Story                     0.0      0.0   0.0   0.0     0.0   0.0  \n",
       "Jumanji                       0.0      0.0   0.0   0.0     0.0   0.0  \n",
       "Grumpier Old Men              0.0      0.0   0.0   0.0     0.0   0.0  \n",
       "Waiting to Exhale             0.0      0.0   0.0   0.0     0.0   0.0  \n",
       "Father of the Bride Part II   0.0      0.0   0.0   0.0     0.0   0.0  \n",
       "\n",
       "[5 rows x 11437 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_matrix = pd.DataFrame(tfidf_matrix, columns=tfidf_matrix_feature, index = movie_data.title)\n",
    "print(tfidf_matrix.shape)\n",
    "tfidf_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 유사도 구하기\n",
    "\n",
    "이렇게 만들어진 tf-idf vector를 코사인 유사도를 활용해서 유사도 값을 구해줍니다.  \n",
    "이렇게 하면 영화 개수(n)만큼 n x n의 matirx 형태가 나오게 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 5min, sys: 11 s, total: 5min 11s\n",
      "Wall time: 1min 5s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "cosine_sim = cosine_similarity(tfidf_matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(32852, 32852)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cosine_sim.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(32852, 32852)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>title</th>\n",
       "      <th>Toy Story</th>\n",
       "      <th>Jumanji</th>\n",
       "      <th>Grumpier Old Men</th>\n",
       "      <th>Waiting to Exhale</th>\n",
       "      <th>Father of the Bride Part II</th>\n",
       "      <th>Heat</th>\n",
       "      <th>Sabrina</th>\n",
       "      <th>Tom and Huck</th>\n",
       "      <th>Sudden Death</th>\n",
       "      <th>GoldenEye</th>\n",
       "      <th>...</th>\n",
       "      <th>Deep Hearts</th>\n",
       "      <th>The Morning After</th>\n",
       "      <th>House of Horrors</th>\n",
       "      <th>Shadow of the Blair Witch</th>\n",
       "      <th>The Burkittsville 7</th>\n",
       "      <th>Caged Heat 3000</th>\n",
       "      <th>Robin Hood</th>\n",
       "      <th>Betrayal</th>\n",
       "      <th>Satan Triumphant</th>\n",
       "      <th>Queerama</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Toy Story</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.041569</td>\n",
       "      <td>0.008708</td>\n",
       "      <td>0.006937</td>\n",
       "      <td>0.005595</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.006456</td>\n",
       "      <td>0.059202</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.051110</td>\n",
       "      <td>0.028298</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Jumanji</th>\n",
       "      <td>0.041569</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.065065</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.165721</td>\n",
       "      <td>0.028302</td>\n",
       "      <td>0.011462</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.039299</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Grumpier Old Men</th>\n",
       "      <td>0.008708</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.035846</td>\n",
       "      <td>0.010906</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.033363</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.099628</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.106819</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Waiting to Exhale</th>\n",
       "      <td>0.006937</td>\n",
       "      <td>0.065065</td>\n",
       "      <td>0.035846</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.093741</td>\n",
       "      <td>0.003806</td>\n",
       "      <td>0.063686</td>\n",
       "      <td>0.027484</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.135728</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.121701</td>\n",
       "      <td>0.037622</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Father of the Bride Part II</th>\n",
       "      <td>0.005595</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.010906</td>\n",
       "      <td>0.093741</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.038016</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.064015</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 32852 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "title                        Toy Story   Jumanji  Grumpier Old Men  \\\n",
       "title                                                                \n",
       "Toy Story                     1.000000  0.041569          0.008708   \n",
       "Jumanji                       0.041569  1.000000          0.000000   \n",
       "Grumpier Old Men              0.008708  0.000000          1.000000   \n",
       "Waiting to Exhale             0.006937  0.065065          0.035846   \n",
       "Father of the Bride Part II   0.005595  0.000000          0.010906   \n",
       "\n",
       "title                        Waiting to Exhale  Father of the Bride Part II  \\\n",
       "title                                                                         \n",
       "Toy Story                             0.006937                     0.005595   \n",
       "Jumanji                               0.065065                     0.000000   \n",
       "Grumpier Old Men                      0.035846                     0.010906   \n",
       "Waiting to Exhale                     1.000000                     0.093741   \n",
       "Father of the Bride Part II           0.093741                     1.000000   \n",
       "\n",
       "title                            Heat   Sabrina  Tom and Huck  Sudden Death  \\\n",
       "title                                                                         \n",
       "Toy Story                    0.000000  0.006456      0.059202      0.000000   \n",
       "Jumanji                      0.000000  0.000000      0.165721      0.028302   \n",
       "Grumpier Old Men             0.000000  0.033363      0.000000      0.000000   \n",
       "Waiting to Exhale            0.003806  0.063686      0.027484      0.000000   \n",
       "Father of the Bride Part II  0.000000  0.038016      0.000000      0.000000   \n",
       "\n",
       "title                        GoldenEye  ...  Deep Hearts  The Morning After  \\\n",
       "title                                   ...                                   \n",
       "Toy Story                     0.000000  ...          0.0           0.051110   \n",
       "Jumanji                       0.011462  ...          0.0           0.000000   \n",
       "Grumpier Old Men              0.000000  ...          0.0           0.099628   \n",
       "Waiting to Exhale             0.000000  ...          0.0           0.135728   \n",
       "Father of the Bride Part II   0.000000  ...          0.0           0.064015   \n",
       "\n",
       "title                        House of Horrors  Shadow of the Blair Witch  \\\n",
       "title                                                                      \n",
       "Toy Story                            0.028298                        0.0   \n",
       "Jumanji                              0.039299                        0.0   \n",
       "Grumpier Old Men                     0.000000                        0.0   \n",
       "Waiting to Exhale                    0.000000                        0.0   \n",
       "Father of the Bride Part II          0.000000                        0.0   \n",
       "\n",
       "title                        The Burkittsville 7  Caged Heat 3000  Robin Hood  \\\n",
       "title                                                                           \n",
       "Toy Story                                    0.0              0.0    0.000000   \n",
       "Jumanji                                      0.0              0.0    0.000000   \n",
       "Grumpier Old Men                             0.0              0.0    0.106819   \n",
       "Waiting to Exhale                            0.0              0.0    0.121701   \n",
       "Father of the Bride Part II                  0.0              0.0    0.000000   \n",
       "\n",
       "title                        Betrayal  Satan Triumphant  Queerama  \n",
       "title                                                              \n",
       "Toy Story                    0.000000               0.0       0.0  \n",
       "Jumanji                      0.000000               0.0       0.0  \n",
       "Grumpier Old Men             0.000000               0.0       0.0  \n",
       "Waiting to Exhale            0.037622               0.0       0.0  \n",
       "Father of the Bride Part II  0.000000               0.0       0.0  \n",
       "\n",
       "[5 rows x 32852 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cosine_sim_df = pd.DataFrame(cosine_sim, index = movie_data.title, columns = movie_data.title)\n",
    "print(cosine_sim_df.shape)\n",
    "cosine_sim_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Content Based Recommend\n",
    "\n",
    "이제 content base recsys 결과를 뽑아내기 위해 함수를 하나 만들어줍니다. 이 함수는 아래와 같은 기능을 담당합니다. \n",
    "\n",
    "- target title( 추천 결과를 조회할 영화 제목 )에 따라서 코사인 유사도를 구한 matrix에서 유사도 데이터를 가져옴\n",
    "- 유사도 데이터 중 가장 유사도 값이 큰 데이터를 가져옴\n",
    "    - 가져올 때 top k개를 가져옴\n",
    "- 해당 추천 값 출력"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def genre_recommendations(target_title, matrix, items, k=10):\n",
    "    recom_idx = matrix.loc[:, target_title].values.reshape(1, -1).argsort()[:, ::-1].flatten()[1:k+1]\n",
    "    recom_title = items.iloc[recom_idx, :].title.values\n",
    "    recom_genre = items.iloc[recom_idx, :].genres.values\n",
    "    target_title_list = np.full(len(range(k)), target_title)\n",
    "    target_genre_list = np.full(len(range(k)), items[items.title == target_title].genres.values)\n",
    "    d = {\n",
    "        'target_title':target_title_list,\n",
    "        'target_genre':target_genre_list,\n",
    "        'recom_title' : recom_title,\n",
    "        'recom_genre' : recom_genre\n",
    "    }\n",
    "    return pd.DataFrame(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>target_title</th>\n",
       "      <th>target_genre</th>\n",
       "      <th>recom_title</th>\n",
       "      <th>recom_genre</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>The Dark Knight</td>\n",
       "      <td>Drama Action Crime Thriller</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>The Burglar</td>\n",
       "      <td>Crime Drama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>Batman Begins</td>\n",
       "      <td>Action Crime Drama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>Batman &amp; Robin</td>\n",
       "      <td>Action Crime Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>Batman</td>\n",
       "      <td>Fantasy Action</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>Raffles</td>\n",
       "      <td>Adventure Comedy Crime Drama History Romance T...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>Hero at Large</td>\n",
       "      <td>Action Comedy Drama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>DC Showcase: Catwoman</td>\n",
       "      <td>Action Adventure Animation Science Fiction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>DC Super Hero Girls: Hero of the Year</td>\n",
       "      <td>Animation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>The Dark Knight Rises</td>\n",
       "      <td>Action Crime Drama Thriller</td>\n",
       "      <td>Batman Returns</td>\n",
       "      <td>Action Fantasy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            target_title                 target_genre  \\\n",
       "0  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "1  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "2  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "3  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "4  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "5  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "6  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "7  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "8  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "9  The Dark Knight Rises  Action Crime Drama Thriller   \n",
       "\n",
       "                             recom_title  \\\n",
       "0                        The Dark Knight   \n",
       "1                            The Burglar   \n",
       "2                          Batman Begins   \n",
       "3                         Batman & Robin   \n",
       "4                                 Batman   \n",
       "5                                Raffles   \n",
       "6                          Hero at Large   \n",
       "7                  DC Showcase: Catwoman   \n",
       "8  DC Super Hero Girls: Hero of the Year   \n",
       "9                         Batman Returns   \n",
       "\n",
       "                                         recom_genre  \n",
       "0                        Drama Action Crime Thriller  \n",
       "1                                        Crime Drama  \n",
       "2                                 Action Crime Drama  \n",
       "3                               Action Crime Fantasy  \n",
       "4                                     Fantasy Action  \n",
       "5  Adventure Comedy Crime Drama History Romance T...  \n",
       "6                                Action Comedy Drama  \n",
       "7         Action Adventure Animation Science Fiction  \n",
       "8                                          Animation  \n",
       "9                                     Action Fantasy  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "genre_recommendations('The Dark Knight Rises', cosine_sim_df, movie_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}