Pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2ee0e2c1",
   "metadata": {},
   "source": [
    "# 2022年5月14日 - 第14个恋恋日献礼\n",
    "### 东方歌词生成器+高性能平行翻译器 (Paralleled Touhou Lyric Generator and Translator, PaToL-GT)\n",
    "#### 以及古明地恋主题曲（哈德曼的妖怪少女）歌词专用生成器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "87fb85b7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "table {float:left}\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%html\n",
    "<style>\n",
    "table {float:left}\n",
    "</style>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6ecc7e78",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 锁住numpy和pytorch的随机数种子（保证复现性，在实用的时候可以不锁）\n",
    "import numpy as np\n",
    "import torch\n",
    "\n",
    "seed=514\n",
    "np.random.seed(seed)\n",
    "torch.manual_seed(seed)\n",
    "torch.cuda.manual_seed(seed)\n",
    "torch.backends.cudnn.deterministic = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bcde59e8",
   "metadata": {},
   "source": [
    "### 生成器和翻译器都是通过微调日文GPT2实现的\n",
    "使用了**thbwiki**上的**7122**首东方曲训练了生成器，恋恋专用的生成器以此为基础使用了其中的**91**首+从**東方同人CDの歌詞＠wiki**额外收集的**68**首（可能有重复）原曲为**哈德曼的妖怪少女**的歌词训练。\\\n",
    "翻译器首先使用了**600k**条中日维基平行语料进行预训练，然后又在抽取到的**93k（thbwiki）+547k（萌娘百科）** 的日->中歌词语料上迁移后继续训练。\n",
    "#### 依赖库：\n",
    "numpy, pytorch, transformers (为了合并词典，必须最新版的**4.18.0**)\\\n",
    "rjieba (中文分词器)\\\n",
    "fugashi, sentencepiece (日文分词器)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3182c431",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入预定义好的生成和翻译模型框架\n",
    "from models import TouhouMusicTranslator, TouhouMusicGenerator\n",
    "\n",
    "# 设置显卡加速（一般生成需要4GB的显存，使用Beam Search需要8GB），如果没有可用显卡也可以使用CPU\n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a69e22b",
   "metadata": {},
   "source": [
    "## 东方歌词翻译器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3bc10ea9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 为翻译器加载预训练好的状态\n",
    "translator = TouhouMusicTranslator(f'./translator_anime_music_medium_state.pth', device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dfd24ae",
   "metadata": {},
   "source": [
    "### 和Google翻译器的性能比较\n",
    "\n",
    "**评价指标: BERTScore** from [ICLR2020](https://openreview.net/forum?id=SkeHuCVFDr)\\\n",
    "**测试集**和**开发集**各由128句东方曲中歌词及其翻译组成。\n",
    "    \n",
    "| 翻译器 |  开发集 | 测试集 |\n",
    "| :---------| :-----: | :------: |\n",
    "| Google |  68.8   |   67.3   |\n",
    "| PaToL  |  **73.8**   |   **71.6**   |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "861ca87a",
   "metadata": {},
   "source": [
    "**翻译效率**\n",
    "    \n",
    "| 翻译模式 |  CPU | GPU |\n",
    "| :---------| :-----: | :------: |\n",
    "| Greedy |  2.539秒/句   |   0.088秒/句   |\n",
    "| Beam Search (size=6)  |  4.116秒/句   |   0.095秒/句   |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3c9c32f",
   "metadata": {},
   "source": [
    "**以下是一些样例**："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9bf20bf3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "日文原文： 閉じた恋の瞳\n",
      "....................................................................................................\n",
      "Google翻译： 闭合的爱眼睛\n",
      "PaToL翻译： 紧闭的恋爱之瞳\n",
      "标准翻译： 紧闭的恋之瞳\n",
      "--------------------------------------------------\n",
      "日文原文： 儚さ故 美しい\n",
      "....................................................................................................\n",
      "Google翻译： 优雅而美丽\n",
      "PaToL翻译： 因虚幻而美丽\n",
      "标准翻译： 因虚幻而美丽\n",
      "--------------------------------------------------\n",
      "日文原文： シアワセなんてものは\n",
      "....................................................................................................\n",
      "Google翻译： siawase\n",
      "PaToL翻译： 幸福什么的\n",
      "标准翻译： 幸福这种东西\n",
      "--------------------------------------------------\n",
      "日文原文： 枯れぬ花を咲かせたい 輝く銀の花束\n",
      "....................................................................................................\n",
      "Google翻译： 一束闪亮的银花束，想和枯萎的花朵开花\n",
      "PaToL翻译： 想让那永不枯萎的花朵绽放闪耀的银花束\n",
      "标准翻译： 多想花朵绽放永不枯萎 光辉闪耀的银色花束\n",
      "--------------------------------------------------\n",
      "日文原文： （荷取）: 希望　溢れる　(魔理沙：明日を行く)\n",
      "....................................................................................................\n",
      "Google翻译： （收集）：充满希望（Marisa：明天去）\n",
      "PaToL翻译： （荷取）:希望满溢着（魔理沙：前往明天）\n",
      "标准翻译： （荷取）:相信希望 (魔理沙：向明日前进)\n",
      "--------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# 导入Google翻译器\n",
    "from googletrans import Translator\n",
    "google_translator = Translator(service_urls=['translate.google.cn'])\n",
    "\n",
    "with open('./translate_cases.txt', encoding='utf-8') as fp:\n",
    "    cases = [eval(line) for line in fp.read().strip().split('\\n')]\n",
    "    \n",
    "for case in cases:\n",
    "    translation = translator.translate(case['ja'])\n",
    "    google_translation = google_translator.translate(case['ja'], dest='zh-CN').text\n",
    "    print(\"日文原文：\", case['ja']); print('.'*100)\n",
    "    print(\"Google翻译：\", google_translation)\n",
    "    print(\"PaToL翻译：\", translation)\n",
    "    print(\"标准翻译：\", case['zh']); print('-'*50)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37c1f3c0",
   "metadata": {},
   "source": [
    "## 东方歌词生成器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "29812f0d",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 加载东方歌词生成器的状态\n",
    "generator = TouhouMusicGenerator(f'./gpt2_touhou_medium_state.pth', translator=translator, device=device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "22ea94cf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "作曲：ZUN    作词：GPT2-Touhou\n",
      "--------------------------------------------------\n",
      "歌词： 閉じた恋の瞳に惑う\n",
      "翻译： 被封闭的恋爱之瞳所迷惑\n",
      "--------------------------------------------------\n",
      "歌词： 声を聞きたくて聳え立つ\n",
      "翻译： 想要听到你的声音才是我的目标\n",
      "--------------------------------------------------\n",
      "歌词： 標識が消えれば 見えにくい暗がり\n",
      "翻译： 看不清路标就会变得模糊的黑暗\n",
      "--------------------------------------------------\n",
      "歌词： まるで木々が芽吹く頃に咲く花のようだから\n",
      "翻译： 就像在树木萌芽之时绽放的花朵一样\n",
      "--------------------------------------------------\n",
      "歌词： 君 どうかひとつ\n",
      "翻译： 请让我一个\n",
      "--------------------------------------------------\n",
      "歌词： 叶えてよ一度でいい\n",
      "翻译： 想实现一次就好\n",
      "--------------------------------------------------\n",
      "歌词： その想い響いて\n",
      "翻译： 回响着这份思念\n",
      "--------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "prefix = \"閉じた恋の瞳\"\n",
    "print(\"作曲：ZUN    作词：GPT2-Touhou\")\n",
    "print(\"-\"*50)\n",
    "lyrics = generator.generate(prefix, temperature=1.0, step_range=(32, 64), log=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b067091",
   "metadata": {},
   "source": [
    "## 东方歌词生成器（恋恋专用版）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a1b00f0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 切换歌词生成器的状态为恋恋专用~\n",
    "generator.load(f'./gpt2_koishi_medium_state.pth')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "15fa5b08",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "作曲：ZUN    作词：GPT2-Koishi\n",
      "--------------------------------------------------\n",
      "歌词： 閉じた恋の瞳で見るあこがれは\n",
      "翻译： 用紧闭的爱之瞳所见的憧憬\n",
      "--------------------------------------------------\n",
      "歌词： 黒い薔薇の檻\n",
      "翻译： 黑色蔷薇的牢笼\n",
      "--------------------------------------------------\n",
      "歌词： 重力の彼方へ\n",
      "翻译： 向着重力的彼方\n",
      "--------------------------------------------------\n",
      "歌词： おとぎごっこ さて\n",
      "翻译： 来玩捉迷藏吧\n",
      "--------------------------------------------------\n",
      "歌词： 今宵の野望は\n",
      "翻译： 今夜的野心\n",
      "--------------------------------------------------\n",
      "歌词： 私はあなた\n",
      "翻译： 我就是你\n",
      "--------------------------------------------------\n",
      "歌词： あなたが引き寄せる 誰かの想いを察して\n",
      "翻译： 你所吸引的是谁的思念\n",
      "--------------------------------------------------\n",
      "歌词： with you inside and outside\n",
      "翻译： 在你的身边\n",
      "--------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "prefix = \"閉じた恋の瞳\"\n",
    "print(\"作曲：ZUN    作词：GPT2-Koishi\")\n",
    "print(\"-\"*50)\n",
    "lyrics = generator.generate(prefix, temperature=1.0, step_range=(32, 64), log=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b88bdfff",
   "metadata": {},
   "source": [
    "本项目为基于东方project二次创作以及动画歌曲开发的辅助工具，项目中所使用的歌词以及翻译内容的版权均属于对应的艺术创作者和百科社区贡献者，故请勿用于商业用途。\\\n",
    "将本工具用于东方project或者动画作品领域时，无需注明来源，若用于其它领域时，希望加以注明。对训练数据的使用需要遵循源版权者的规定。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}