DecodET_Starter_Kit.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DecodET_Starter_Kit",
      "provenance": [],
      "collapsed_sections": [
        "vYNJxWFkMzGf",
        "83d0p2KzNlZC"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vYNJxWFkMzGf",
        "colab_type": "text"
      },
      "source": [
        "# Setup & Imports"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e3HzxQAqjdMr",
        "colab_type": "text"
      },
      "source": [
        "Mount to drive if needed "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z--g0H-1DyDj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import files\n",
        "from IPython.display import Image\n",
        "from google.colab import drive\n",
        "drive.mount('/gdrive')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FrHqy-BbjYlb",
        "colab_type": "text"
      },
      "source": [
        "Install Tape dependencies \n",
        "\n",
        "Github: https://github.com/songlab-cal/tape"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jw6kCO1dMums",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip3 install tape_proteins\n",
        "!pip3 install torch==1.5.1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wO2SQIR1jgou",
        "colab_type": "text"
      },
      "source": [
        "Imports"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fq8aef0WEjxU",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import string\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import typing\n",
        "import copy\n",
        "import json\n",
        "import logging\n",
        "import os\n",
        "from io import open\n",
        "import math\n",
        "\n",
        "import torch\n",
        "from torch.nn.utils.weight_norm import weight_norm\n",
        "from torch import nn\n",
        "import torch.nn.functional as F\n",
        "\n",
        "from tape import ProteinBertModel, TAPETokenizer"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "83d0p2KzNlZC",
        "colab_type": "text"
      },
      "source": [
        "# Download & Load Train Data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jsyHk_SxoJQg",
        "colab_type": "text"
      },
      "source": [
        "The Train Data and Test Data is available here: http://www.cbs.dtu.dk/services/NetSurfP/"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8Tueyz9Sn7Vv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!wget http://www.cbs.dtu.dk/services/NetSurfP-2.0/training_data/Train_HHblits.npz"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mKSkCxmFqr2i",
        "colab_type": "text"
      },
      "source": [
        "Generally we don't unzip .npz files. We use ``` np.load() ``` and load it directly to memory. But, the ```data.npy``` file is too large and crashes the Colab instance when trying to load. So, firsr unzip it below and then memory map the file. \n",
        "\n",
        "Before unzipping, you can also copy the .npz file to Google Drive using ```!cp $filename $path_to_gdrive``` and then load using ```np.load(path_to_gdrive, mmap_mode='r')```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TK2r5Lz6hOf9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!unzip /content/Train_HHblits.npz  ## Zipped file is 420 MB, when unzipped it is 9GB+!"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bgEZyDWjr9NE",
        "colab_type": "text"
      },
      "source": [
        "A memory-mapped array is kept on disk. However, it can be accessed and sliced like any ndarray. Memory mapping is especially useful for accessing small fragments of large files without reading the entire file into memory. Syntax: ```np.load(filepath, mmap_mode='r')```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s1vUOKKtNuZb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_data = np.load('/content/data.npy', mmap_mode='r')\n",
        "train_labels = np.load('/content/pdbids.npy', mmap_mode='r')  ## 9GB file!"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "30If4qjZM_Pu",
        "colab_type": "text"
      },
      "source": [
        "# Breaking down the [train data](http://www.cbs.dtu.dk/services/NetSurfP/)\n",
        "\n",
        "We have shown for one protein. It is your job to do it for the rest."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IWHU1iRZvsOJ",
        "colab_type": "text"
      },
      "source": [
        "The data is represented this way. There are ```10848``` different protein sequences and largest sequence is ```1632``` amino acids long. And, each amino acid has the following data:\n",
        "\n",
        "```\n",
        " [0:20] Amino Acids (sparse encoding)\n",
        " Unknown residues are stored as an all-zero vector\n",
        " [20:50] hmm profile\n",
        " [50] Seq mask (1 = seq, 0 = empty)\n",
        " [51] Disordered mask (0 = disordered, 1 = ordered)\n",
        " [52] Evaluation mask (For CB513 dataset, 1 = eval, 0 = ignore)\n",
        " [53] ASA (isolated)\n",
        " [54] ASA (complexed)\n",
        " [55] RSA (isolated)\n",
        " [56] RSA (complexed)\n",
        " [57:65] Q8 GHIBESTC (Q8 -> Q3: HHHEECCC)\n",
        " [65:67] Phi+Psi\n",
        " [67] ASA_max\n",
        "```\n",
        "Read the paper for better understanding. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JD3N3ymbgLnJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "c8086075-7324-47a2-dc19-95a6fe8ceeaa"
      },
      "source": [
        "train_data.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10848, 1632, 68)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DKaaw7Fv0Ez9",
        "colab_type": "text"
      },
      "source": [
        "Converting the Sparse encoded amino acid sequence and the experimentally determined secondary structure to the fasta format. Basically fasta format is a string of single letter codes of amino acids of a protein. So, if a protein is made of 300 amino acids, the fasta format will contain a string of 300 letters representing the amino acids present in the protein."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fj2GYuPCyLKn",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "code = ['A', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'K', 'L', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'V', 'W', 'Y']\n",
        "ss = ['H','H','H','E','E','C','C','C']"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_h7N6p85JZ3I",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "protein_0 = train_data[0, :, :]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qOzvi1YigfO8",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 74
        },
        "outputId": "1c0136d6-b816-452a-da91-6a99658c8b1d"
      },
      "source": [
        "length = 0\n",
        "fasta = ''\n",
        "for i in range(train_data.shape[1]):\n",
        "  for j in range(20):\n",
        "    if(protein_0[i][j]==1):\n",
        "      fasta = fasta + code[j]\n",
        "      length+=1\n",
        "\n",
        "fasta"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic": {
              "type": "string"
            },
            "text/plain": [
              "'MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPILSRVGDGTQDNLSGAEKAVQVKVKALPDAQFEVVHSLAKWKRQTLGQHDFSAGEGLYTHMKALRPDEDRLSPLHSVYVDQWDWERVMGDGERQFSTLKSTVEAIWAGIKATEAAVSEEFGLAPFLPDQIHFVHSQELLSRYPDLDAKGRERAIAKDLGAVFLVGIGGKLSDGHRHDVRAPDYDDWSTPSELGHAGLNGDILVWNPVLEDAFELSSMGIRVDADTLKHQLALTGDEDRLELEWHQALLRGEMPQTIGGGIGQSRLTMLLLQLPHIGQVQAGVWPAAVRESVPSLL'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SagYLn6fGbVJ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 74
        },
        "outputId": "d614760d-473c-4c89-f37e-ebee96057e46"
      },
      "source": [
        "structure = ''\n",
        "secondary_structure = []\n",
        "for i in range(train_data.shape[1]):\n",
        "  for j in range(57,65):\n",
        "    if(protein_0[i][j]==1):\n",
        "      structure = structure + ss[j-57]\n",
        "\n",
        "structure"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic": {
              "type": "string"
            },
            "text/plain": [
              "'CCCCHHHHHHHHHHHHHHHHHHHHHHHCEEECCCCCEEECCCCCCCCCCCCCCCCEECCCCCCCCCEEECCCCCCHHHHHHHHCCCCCCCEEEEEEEEECCCCCCCCCCCCCEEEEEEEEEECCCCCCCHHHHHHHHHHHHHHHHHHHHHHHHHCCCCCCCCCCCEEEEHHHHHHHCCCCCHHHHHHHHHHHHCEEEEECCCCCCCCCCCCCCCCCCCECCCCECCCCCECCEEEEEEEECCCCEEEEEEEEEEECCHHHHHHHHHHHCCCCHHHCHHHHHHHCCCCCCEEEEEEEHHHHHHHHHCCCCHHHCCCCCCCHHHHHHCCCCC'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LNBsiy8e0Ci9",
        "colab_type": "text"
      },
      "source": [
        "#Input the sequences into the BERT encoder"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5Fo17_URNKki",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model = ProteinBertModel.from_pretrained('bert-base')\n",
        "tokenizer = TAPETokenizer(vocab='iupac')  # iupac is the vocab for TAPE models, use unirep for the UniRep model\n",
        "\n",
        "\n",
        "sequence = fasta\n",
        "token_ids = torch.tensor([tokenizer.encode(sequence)])\n",
        "output = model(token_ids)\n",
        "sequence_output = output[0]\n",
        "pooled_output = output[1]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kChtvN1_2eMn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "24d13a42-a33f-4499-8777-f35cffd92131"
      },
      "source": [
        "sequence_output.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([1, 332, 768])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vXJw-csGkq07",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "d4829bcd-9b33-4e9a-f920-2e002671f70b"
      },
      "source": [
        "pooled_output.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([1, 768])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 50
        }
      ]
    }
  ]
}