roberta-fit-stratkfold-augtrain.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## RoBERTa Base Fine-Tuning with Better Training Strategies\n\n### Steps:\n1. [RoBERTa Pretraining](https://www.kaggle.com/rhtsingh/commonlit-readability-prize-roberta-torch-itpt) on Task Data\n2. [RoBERTa Finetuing]() - This Notebook.\n3. [RoBERTa Inference](https://www.kaggle.com/rhtsingh/commonlit-readability-prize-roberta-torch-infer) with weights available.\n\n\n### Updates\n- For further improvement, consider experimenting with various performance optimization strategies explained in this notebook [Speeding up Transformer w/ Optimization Strategies\n](https://www.kaggle.com/rhtsingh/speeding-up-transformer-w-optimization-strategies). The notebook explains 5 optimization strategies in-depth with code. All these techniques are promising and can improve the model performance both in terms of speed and accuracy.\n   - Dynamic Padding and Uniform Length Batching\n   - Gradient Accumulation\n   - Freeze Embedding\n   - Numeric Precision Reduction\n   - Gradient Checkpointing\n   \n- After many experiments, I think complex scheduling strategies like `linear with warmup` or `cosine with warmup` etc. might not work with pretraining as well as finetuning as the dataset is small. I don't exactly remember where but I read simple scheduling generally works better than complex ones with small corpus. I am not sure about this one but it's worth experimenting with. I have two suggestions to try,\n  - Scheduling after epoch.\n  - No scheduling at all.\n\n- Another thing worth trying is using RAdam with Lookahead instead of AdamW. RAdam does not require the warm-up tuning that depends on data size. I'll add the code here soon.  ","metadata":{}},{"cell_type":"markdown","source":"### Load Data","metadata":{"papermill":{"duration":0.017372,"end_time":"2021-05-11T19:26:26.916863","exception":false,"start_time":"2021-05-11T19:26:26.899491","status":"completed"},"tags":[],"id":"vietnamese-nursery"}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\n\n#train = pd.read_csv('../input/commonlitreadabilityprize/train.csv')\ntrain = pd.read_csv('../input/augmented-train-data/aug_train_data_same_distrib.csv')\ntest = pd.read_csv('../input/commonlitreadabilityprize/test.csv')","metadata":{"id":"PyldEJ_jq4yx","execution":{"iopub.status.busy":"2021-06-03T19:49:54.723501Z","iopub.execute_input":"2021-06-03T19:49:54.726803Z","iopub.status.idle":"2021-06-03T19:49:54.806188Z","shell.execute_reply.started":"2021-06-03T19:49:54.726766Z","shell.execute_reply":"2021-06-03T19:49:54.805326Z"},"trusted":true},"execution_count":12,"outputs":[]},{"cell_type":"code","source":"\"\"\"\nfrom sklearn import model_selection\ndef create_folds(data, num_splits):\n    data[\"kfold\"] = -1\n    kf = model_selection.KFold(n_splits=num_splits, shuffle=True, random_state=2021)\n    for f, (t_, v_) in enumerate(kf.split(X=data)):\n        data.loc[v_, 'kfold'] = f\n    return data\ntrain = create_folds(train, num_splits=5)\n\"\"\";","metadata":{"papermill":{"duration":0.828531,"end_time":"2021-05-11T19:26:27.914917","exception":false,"start_time":"2021-05-11T19:26:27.086386","status":"completed"},"tags":[],"id":"nearby-expert","execution":{"iopub.status.busy":"2021-06-03T19:49:54.810798Z","iopub.execute_input":"2021-06-03T19:49:54.813472Z","iopub.status.idle":"2021-06-03T19:49:54.819460Z","shell.execute_reply.started":"2021-06-03T19:49:54.813432Z","shell.execute_reply":"2021-06-03T19:49:54.818517Z"},"trusted":true},"execution_count":13,"outputs":[]},{"cell_type":"code","source":"from sklearn import model_selection\n\n#stratified k fold \ndef create_folds(data, num_splits):\n    data[\"kfold\"] = -1\n    bins = data[\"bins\"].to_numpy()\n    #kf = model_selection.KFold(n_splits=num_splits, shuffle=True, random_state=2021)\n    kf = model_selection.StratifiedKFold(n_splits=num_splits, shuffle=True, random_state=2021)\n    for f, (t_, v_) in enumerate(kf.split(data, bins)):\n        data.loc[v_, 'kfold'] = f\n    return data\n\ntrain = create_folds(train, num_splits=5)","metadata":{"execution":{"iopub.status.busy":"2021-06-03T19:49:54.824614Z","iopub.execute_input":"2021-06-03T19:49:54.826178Z","iopub.status.idle":"2021-06-03T19:49:54.847063Z","shell.execute_reply.started":"2021-06-03T19:49:54.826143Z","shell.execute_reply":"2021-06-03T19:49:54.846341Z"},"trusted":true},"execution_count":14,"outputs":[]},{"cell_type":"markdown","source":"### Import Dependencies - Modelling","metadata":{"papermill":{"duration":0.017397,"end_time":"2021-05-11T19:26:27.950181","exception":false,"start_time":"2021-05-11T19:26:27.932784","status":"completed"},"tags":[],"id":"legal-float"}},{"cell_type":"code","source":"%matplotlib inline\nfrom glob import glob\nimport os\nimport matplotlib.pyplot as plt\nimport json\nfrom collections import defaultdict\nimport gc\ngc.enable()","metadata":{"papermill":{"duration":0.026039,"end_time":"2021-05-11T19:26:27.994289","exception":false,"start_time":"2021-05-11T19:26:27.96825","status":"completed"},"tags":[],"id":"complete-breakdown","execution":{"iopub.status.busy":"2021-06-03T19:49:54.851248Z","iopub.execute_input":"2021-06-03T19:49:54.854009Z","iopub.status.idle":"2021-06-03T19:49:54.863106Z","shell.execute_reply.started":"2021-06-03T19:49:54.853974Z","shell.execute_reply":"2021-06-03T19:49:54.862080Z"},"trusted":true},"execution_count":15,"outputs":[]},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nfrom torch.optim.optimizer import Optimizer\nimport torch.optim.lr_scheduler as lr_scheduler\nfrom torch.utils.data import (\n    Dataset, DataLoader, \n    SequentialSampler, RandomSampler\n)\nfrom transformers import AutoConfig\nfrom transformers import (\n    get_cosine_schedule_with_warmup, \n    get_cosine_with_hard_restarts_schedule_with_warmup,\n    get_linear_schedule_with_warmup\n)\nfrom transformers import AdamW\nfrom transformers import AutoTokenizer\nfrom transformers import AutoModel\nfrom transformers import MODEL_FOR_SEQUENCE_CLASSIFICATION_MAPPING\nfrom IPython.display import clear_output\nfrom tqdm import tqdm, trange","metadata":{"papermill":{"duration":7.228235,"end_time":"2021-05-11T19:26:35.239965","exception":false,"start_time":"2021-05-11T19:26:28.01173","status":"completed"},"tags":[],"id":"willing-affairs","execution":{"iopub.status.busy":"2021-06-03T19:49:54.868713Z","iopub.execute_input":"2021-06-03T19:49:54.871411Z","iopub.status.idle":"2021-06-03T19:49:54.880263Z","shell.execute_reply.started":"2021-06-03T19:49:54.871376Z","shell.execute_reply":"2021-06-03T19:49:54.879477Z"},"trusted":true},"execution_count":16,"outputs":[]},{"cell_type":"markdown","source":"### Convert Examples to Features","metadata":{"papermill":{"duration":0.020552,"end_time":"2021-05-11T19:26:35.281616","exception":false,"start_time":"2021-05-11T19:26:35.261064","status":"completed"},"tags":[],"id":"palestinian-immunology"}},{"cell_type":"code","source":"def convert_examples_to_features(data, tokenizer, max_len, is_test=False):\n    data = data.replace('\\n', '')\n    tok = tokenizer.encode_plus(\n        data, \n        max_length=max_len, \n        truncation=True,\n        return_attention_mask=True,\n        return_token_type_ids=True\n    )\n    curr_sent = {}\n    padding_length = max_len - len(tok['input_ids'])\n    curr_sent['input_ids'] = tok['input_ids'] + ([tokenizer.pad_token_id] * padding_length)\n    curr_sent['token_type_ids'] = tok['token_type_ids'] + \\\n        ([0] * padding_length)\n    curr_sent['attention_mask'] = tok['attention_mask'] + \\\n        ([0] * padding_length)\n    return curr_sent","metadata":{"papermill":{"duration":0.028976,"end_time":"2021-05-11T19:26:35.330531","exception":false,"start_time":"2021-05-11T19:26:35.301555","status":"completed"},"tags":[],"id":"irish-fossil","execution":{"iopub.status.busy":"2021-06-03T19:49:54.887529Z","iopub.execute_input":"2021-06-03T19:49:54.890034Z","iopub.status.idle":"2021-06-03T19:49:54.898821Z","shell.execute_reply.started":"2021-06-03T19:49:54.890000Z","shell.execute_reply":"2021-06-03T19:49:54.898054Z"},"trusted":true},"execution_count":17,"outputs":[]},{"cell_type":"markdown","source":"### Dataset Retriever","metadata":{"papermill":{"duration":0.019304,"end_time":"2021-05-11T19:26:35.369266","exception":false,"start_time":"2021-05-11T19:26:35.349962","status":"completed"},"tags":[],"id":"subject-entertainment"}},{"cell_type":"code","source":"class DatasetRetriever(Dataset):\n    def __init__(self, data, tokenizer, max_len, is_test=False):\n        self.data = data\n        if 'excerpt' in self.data.columns:\n            self.excerpts = self.data.excerpt.values.tolist()\n        else:\n            self.excerpts = self.data.text.values.tolist()\n        self.targets = self.data.target.values.tolist()\n        self.tokenizer = tokenizer\n        self.is_test = is_test\n        self.max_len = max_len\n    \n    def __len__(self):\n        return len(self.data)\n    \n    def __getitem__(self, item):\n        excerpt, label = self.excerpts[item], self.targets[item]\n        features = convert_examples_to_features(\n            excerpt, self.tokenizer, \n            self.max_len, self.is_test\n        )\n        return {\n            'input_ids':torch.tensor(features['input_ids'], dtype=torch.long),\n            'token_type_ids':torch.tensor(features['token_type_ids'], dtype=torch.long),\n            'attention_mask':torch.tensor(features['attention_mask'], dtype=torch.long),\n            'label':torch.tensor(label, dtype=torch.double),\n        }","metadata":{"papermill":{"duration":0.029976,"end_time":"2021-05-11T19:26:35.418819","exception":false,"start_time":"2021-05-11T19:26:35.388843","status":"completed"},"tags":[],"id":"solar-group","execution":{"iopub.status.busy":"2021-06-03T19:49:54.904650Z","iopub.execute_input":"2021-06-03T19:49:54.908107Z","iopub.status.idle":"2021-06-03T19:49:54.920412Z","shell.execute_reply.started":"2021-06-03T19:49:54.908074Z","shell.execute_reply":"2021-06-03T19:49:54.919557Z"},"trusted":true},"execution_count":18,"outputs":[]},{"cell_type":"markdown","source":"### Model","metadata":{"papermill":{"duration":0.01885,"end_time":"2021-05-11T19:26:35.456835","exception":false,"start_time":"2021-05-11T19:26:35.437985","status":"completed"},"tags":[],"id":"proprietary-genre"}},{"cell_type":"code","source":"class CommonLitModel(nn.Module):\n    def __init__(\n        self, \n        model_name, \n        config,  \n        multisample_dropout=False,\n        output_hidden_states=False\n    ):\n        super(CommonLitModel, self).__init__()\n        self.config = config\n        self.roberta = AutoModel.from_pretrained(\n            model_name, \n            output_hidden_states=output_hidden_states\n        )\n        self.layer_norm = nn.LayerNorm(config.hidden_size)\n        if multisample_dropout:\n            self.dropouts = nn.ModuleList([\n                nn.Dropout(0.5) for _ in range(5)\n            ])\n        else:\n            self.dropouts = nn.ModuleList([nn.Dropout(0.3)])\n        #self.regressor = nn.Linear(config.hidden_size*2, 1)\n        self.regressor = nn.Linear(config.hidden_size, 1)\n        self._init_weights(self.layer_norm)\n        self._init_weights(self.regressor)\n \n    def _init_weights(self, module):\n        if isinstance(module, nn.Linear):\n            module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)\n            if module.bias is not None:\n                module.bias.data.zero_()\n        elif isinstance(module, nn.Embedding):\n            module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)\n            if module.padding_idx is not None:\n                module.weight.data[module.padding_idx].zero_()\n        elif isinstance(module, nn.LayerNorm):\n            module.bias.data.zero_()\n            module.weight.data.fill_(1.0)\n \n    def forward(\n        self, \n        input_ids=None,\n        attention_mask=None,\n        token_type_ids=None,\n        labels=None\n    ):\n        outputs = self.roberta(\n            input_ids,\n            attention_mask=attention_mask,\n            token_type_ids=token_type_ids,\n        )\n        sequence_output = outputs[1]\n        sequence_output = self.layer_norm(sequence_output)\n \n        # max-avg head\n        # average_pool = torch.mean(sequence_output, 1)\n        # max_pool, _ = torch.max(sequence_output, 1)\n        # concat_sequence_output = torch.cat((average_pool, max_pool), 1)\n \n        # multi-sample dropout\n        for i, dropout in enumerate(self.dropouts):\n            if i == 0:\n                logits = self.regressor(dropout(sequence_output))\n            else:\n                logits += self.regressor(dropout(sequence_output))\n        \n        logits /= len(self.dropouts)\n \n        # calculate loss\n        loss = None\n        if labels is not None:\n            # regression task\n            loss_fn = torch.nn.MSELoss()\n            logits = logits.view(-1).to(labels.dtype)\n            loss = torch.sqrt(loss_fn(logits, labels.view(-1)))\n        \n        output = (logits,) + outputs[2:]\n        return ((loss,) + output) if loss is not None else output","metadata":{"papermill":{"duration":0.037145,"end_time":"2021-05-11T19:26:35.512991","exception":false,"start_time":"2021-05-11T19:26:35.475846","status":"completed"},"tags":[],"id":"resident-kazakhstan","execution":{"iopub.status.busy":"2021-06-03T19:49:54.928981Z","iopub.execute_input":"2021-06-03T19:49:54.932315Z","iopub.status.idle":"2021-06-03T19:49:54.953591Z","shell.execute_reply.started":"2021-06-03T19:49:54.932268Z","shell.execute_reply":"2021-06-03T19:49:54.952669Z"},"trusted":true},"execution_count":19,"outputs":[]},{"cell_type":"markdown","source":"### Lamb Optimizer","metadata":{"papermill":{"duration":0.019467,"end_time":"2021-05-11T19:26:35.552336","exception":false,"start_time":"2021-05-11T19:26:35.532869","status":"completed"},"tags":[],"id":"going-tractor"}},{"cell_type":"code","source":"class Lamb(Optimizer):\n    # Reference code: https://github.com/cybertronai/pytorch-lamb\n\n    def __init__(\n        self,\n        params,\n        lr: float = 1e-3,\n        betas = (0.9, 0.999),\n        eps: float = 1e-6,\n        weight_decay: float = 0,\n        clamp_value: float = 10,\n        adam: bool = False,\n        debias: bool = False,\n    ):\n        if lr <= 0.0:\n            raise ValueError('Invalid learning rate: {}'.format(lr))\n        if eps < 0.0:\n            raise ValueError('Invalid epsilon value: {}'.format(eps))\n        if not 0.0 <= betas[0] < 1.0:\n            raise ValueError(\n                'Invalid beta parameter at index 0: {}'.format(betas[0])\n            )\n        if not 0.0 <= betas[1] < 1.0:\n            raise ValueError(\n                'Invalid beta parameter at index 1: {}'.format(betas[1])\n            )\n        if weight_decay < 0:\n            raise ValueError(\n                'Invalid weight_decay value: {}'.format(weight_decay)\n            )\n        if clamp_value < 0.0:\n            raise ValueError('Invalid clamp value: {}'.format(clamp_value))\n\n        defaults = dict(lr=lr, betas=betas, eps=eps, weight_decay=weight_decay)\n        self.clamp_value = clamp_value\n        self.adam = adam\n        self.debias = debias\n\n        super(Lamb, self).__init__(params, defaults)\n\n    def step(self, closure = None):\n        loss = None\n        if closure is not None:\n            loss = closure()\n\n        for group in self.param_groups:\n            for p in group['params']:\n                if p.grad is None:\n                    continue\n                grad = p.grad.data\n                if grad.is_sparse:\n                    msg = (\n                        'Lamb does not support sparse gradients, '\n                        'please consider SparseAdam instead'\n                    )\n                    raise RuntimeError(msg)\n\n                state = self.state[p]\n\n                # State initialization\n                if len(state) == 0:\n                    state['step'] = 0\n                    # Exponential moving average of gradient values\n                    state['exp_avg'] = torch.zeros_like(\n                        p, memory_format=torch.preserve_format\n                    )\n                    # Exponential moving average of squared gradient values\n                    state['exp_avg_sq'] = torch.zeros_like(\n                        p, memory_format=torch.preserve_format\n                    )\n\n                exp_avg, exp_avg_sq = state['exp_avg'], state['exp_avg_sq']\n                beta1, beta2 = group['betas']\n\n                state['step'] += 1\n\n                # Decay the first and second moment running average coefficient\n                # m_t\n                exp_avg.mul_(beta1).add_(grad, alpha=1 - beta1)\n                # v_t\n                exp_avg_sq.mul_(beta2).addcmul_(grad, grad, value=1 - beta2)\n\n                # Paper v3 does not use debiasing.\n                if self.debias:\n                    bias_correction = math.sqrt(1 - beta2 ** state['step'])\n                    bias_correction /= 1 - beta1 ** state['step']\n                else:\n                    bias_correction = 1\n\n                # Apply bias to lr to avoid broadcast.\n                step_size = group['lr'] * bias_correction\n\n                weight_norm = torch.norm(p.data).clamp(0, self.clamp_value)\n\n                adam_step = exp_avg / exp_avg_sq.sqrt().add(group['eps'])\n                if group['weight_decay'] != 0:\n                    adam_step.add_(p.data, alpha=group['weight_decay'])\n\n                adam_norm = torch.norm(adam_step)\n                if weight_norm == 0 or adam_norm == 0:\n                    trust_ratio = 1\n                else:\n                    trust_ratio = weight_norm / adam_norm\n                state['weight_norm'] = weight_norm\n                state['adam_norm'] = adam_norm\n                state['trust_ratio'] = trust_ratio\n                if self.adam:\n                    trust_ratio = 1\n\n                p.data.add_(adam_step, alpha=-step_size * trust_ratio)\n\n        return loss","metadata":{"papermill":{"duration":0.039162,"end_time":"2021-05-11T19:26:35.610963","exception":false,"start_time":"2021-05-11T19:26:35.571801","status":"completed"},"tags":[],"id":"photographic-crack","execution":{"iopub.status.busy":"2021-06-03T19:49:54.959008Z","iopub.execute_input":"2021-06-03T19:49:54.961588Z","iopub.status.idle":"2021-06-03T19:49:54.987910Z","shell.execute_reply.started":"2021-06-03T19:49:54.961553Z","shell.execute_reply":"2021-06-03T19:49:54.986927Z"},"trusted":true},"execution_count":20,"outputs":[]},{"cell_type":"markdown","source":"### Differential Learning Rate and Weight Decay","metadata":{"papermill":{"duration":0.019229,"end_time":"2021-05-11T19:26:35.648664","exception":false,"start_time":"2021-05-11T19:26:35.629435","status":"completed"},"tags":[],"id":"exceptional-asbestos"}},{"cell_type":"code","source":"def get_optimizer_params(model):\n    # differential learning rate and weight decay\n    param_optimizer = list(model.named_parameters())\n    learning_rate = 5e-5\n    no_decay = ['bias', 'gamma', 'beta']\n    group1=['layer.0.','layer.1.','layer.2.','layer.3.']\n    group2=['layer.4.','layer.5.','layer.6.','layer.7.']    \n    group3=['layer.8.','layer.9.','layer.10.','layer.11.']\n    group_all=['layer.0.','layer.1.','layer.2.','layer.3.','layer.4.','layer.5.','layer.6.','layer.7.','layer.8.','layer.9.','layer.10.','layer.11.']\n    optimizer_parameters = [\n        {'params': [p for n, p in model.roberta.named_parameters() if not any(nd in n for nd in no_decay) and not any(nd in n for nd in group_all)],'weight_decay': 0.01},\n        {'params': [p for n, p in model.roberta.named_parameters() if not any(nd in n for nd in no_decay) and any(nd in n for nd in group1)],'weight_decay': 0.01, 'lr': learning_rate/2.6},\n        {'params': [p for n, p in model.roberta.named_parameters() if not any(nd in n for nd in no_decay) and any(nd in n for nd in group2)],'weight_decay': 0.01, 'lr': learning_rate},\n        {'params': [p for n, p in model.roberta.named_parameters() if not any(nd in n for nd in no_decay) and any(nd in n for nd in group3)],'weight_decay': 0.01, 'lr': learning_rate*2.6},\n        {'params': [p for n, p in model.roberta.named_parameters() if any(nd in n for nd in no_decay) and not any(nd in n for nd in group_all)],'weight_decay': 0.0},\n        {'params': [p for n, p in model.roberta.named_parameters() if any(nd in n for nd in no_decay) and any(nd in n for nd in group1)],'weight_decay': 0.0, 'lr': learning_rate/2.6},\n        {'params': [p for n, p in model.roberta.named_parameters() if any(nd in n for nd in no_decay) and any(nd in n for nd in group2)],'weight_decay': 0.0, 'lr': learning_rate},\n        {'params': [p for n, p in model.roberta.named_parameters() if any(nd in n for nd in no_decay) and any(nd in n for nd in group3)],'weight_decay': 0.0, 'lr': learning_rate*2.6},\n        {'params': [p for n, p in model.named_parameters() if \"roberta\" not in n], 'lr':1e-3, \"momentum\" : 0.99},\n    ]\n    return optimizer_parameters","metadata":{"papermill":{"duration":0.034924,"end_time":"2021-05-11T19:26:35.702699","exception":false,"start_time":"2021-05-11T19:26:35.667775","status":"completed"},"tags":[],"id":"worth-distance","execution":{"iopub.status.busy":"2021-06-03T19:49:54.993598Z","iopub.execute_input":"2021-06-03T19:49:54.996309Z","iopub.status.idle":"2021-06-03T19:49:55.018178Z","shell.execute_reply.started":"2021-06-03T19:49:54.996272Z","shell.execute_reply":"2021-06-03T19:49:55.017310Z"},"trusted":true},"execution_count":21,"outputs":[]},{"cell_type":"markdown","source":"### Utilities","metadata":{"papermill":{"duration":0.018135,"end_time":"2021-05-11T19:26:35.739429","exception":false,"start_time":"2021-05-11T19:26:35.721294","status":"completed"},"tags":[],"id":"respiratory-improvement"}},{"cell_type":"code","source":"def make_model(model_name='../input/robertaitpt/', num_labels=1):\n    tokenizer = AutoTokenizer.from_pretrained('roberta-base')\n    config = AutoConfig.from_pretrained(model_name)\n    config.update({'num_labels':num_labels})\n    model = CommonLitModel(model_name, config=config)\n    return model, tokenizer\n\ndef make_optimizer(model, optimizer_name=\"AdamW\"):\n    optimizer_grouped_parameters = get_optimizer_params(model)\n    kwargs = {\n            'lr':5e-5,\n            'weight_decay':0.01,\n            # 'betas': (0.9, 0.98),\n            # 'eps': 1e-06\n    }\n    if optimizer_name == \"LAMB\":\n        optimizer = Lamb(optimizer_grouped_parameters, **kwargs)\n        return optimizer\n    elif optimizer_name == \"Adam\":\n        from torch.optim import Adam\n        optimizer = Adam(optimizer_grouped_parameters, **kwargs)\n        return optimizer\n    elif optimizer_name == \"AdamW\":\n        optimizer = AdamW(optimizer_grouped_parameters, **kwargs)\n        return optimizer\n    else:\n        raise Exception('Unknown optimizer: {}'.format(optimizer_name))\n\ndef make_scheduler(optimizer, decay_name='linear', t_max=None, warmup_steps=None):\n    if decay_name == 'step':\n        scheduler = optim.lr_scheduler.MultiStepLR(\n            optimizer,\n            milestones=[30, 60, 90],\n            gamma=0.1\n        )\n    elif decay_name == 'cosine':\n        scheduler = lrs.CosineAnnealingLR(\n            optimizer,\n            T_max=t_max\n        )\n    elif decay_name == \"cosine_warmup\":\n        scheduler = get_cosine_schedule_with_warmup(\n            optimizer,\n            num_warmup_steps=warmup_steps,\n            num_training_steps=t_max\n        )\n    elif decay_name == \"linear\":\n        scheduler = get_linear_schedule_with_warmup(\n            optimizer, \n            num_warmup_steps=warmup_steps, \n            num_training_steps=t_max\n        )\n    else:\n        raise Exception('Unknown lr scheduler: {}'.format(decay_type))    \n    return scheduler    \n\ndef make_loader(\n    data, \n    tokenizer, \n    max_len,\n    batch_size,\n    fold=0\n):\n    train_set, valid_set = data[data['kfold']!=fold], data[data['kfold']==fold]\n    train_dataset = DatasetRetriever(train_set, tokenizer, max_len)\n    valid_dataset = DatasetRetriever(valid_set, tokenizer, max_len)\n\n    train_sampler = RandomSampler(train_dataset)\n    train_loader = DataLoader(\n        train_dataset, \n        batch_size=batch_size, \n        sampler=train_sampler, \n        pin_memory=True, \n        drop_last=False, \n        num_workers=4\n    )\n\n    valid_sampler = SequentialSampler(valid_dataset)\n    valid_loader = DataLoader(\n        valid_dataset, \n        batch_size=batch_size // 2, \n        sampler=valid_sampler, \n        pin_memory=True, \n        drop_last=False, \n        num_workers=4\n    )\n\n    return train_loader, valid_loader","metadata":{"papermill":{"duration":0.032912,"end_time":"2021-05-11T19:26:35.790678","exception":false,"start_time":"2021-05-11T19:26:35.757766","status":"completed"},"tags":[],"id":"ahead-steering","execution":{"iopub.status.busy":"2021-06-03T19:49:55.023521Z","iopub.execute_input":"2021-06-03T19:49:55.026339Z","iopub.status.idle":"2021-06-03T19:49:55.044824Z","shell.execute_reply.started":"2021-06-03T19:49:55.026302Z","shell.execute_reply":"2021-06-03T19:49:55.044025Z"},"trusted":true},"execution_count":22,"outputs":[]},{"cell_type":"markdown","source":"### Metrics","metadata":{"papermill":{"duration":0.018561,"end_time":"2021-05-11T19:26:35.827595","exception":false,"start_time":"2021-05-11T19:26:35.809034","status":"completed"},"tags":[],"id":"written-resort"}},{"cell_type":"code","source":"class AverageMeter(object):\n    def __init__(self):\n        self.reset()\n\n    def reset(self):\n        self.val = 0\n        self.avg = 0\n        self.sum = 0\n        self.count = 0\n        self.max = 0\n        self.min = 1e5\n\n    def update(self, val, n=1):\n        self.val = val\n        self.sum += val * n\n        self.count += n\n        self.avg = self.sum / self.count\n        if val > self.max:\n            self.max = val\n        if val < self.min:\n            self.min = val","metadata":{"papermill":{"duration":0.027034,"end_time":"2021-05-11T19:26:35.873026","exception":false,"start_time":"2021-05-11T19:26:35.845992","status":"completed"},"tags":[],"id":"changed-monte","execution":{"iopub.status.busy":"2021-06-03T19:49:55.050517Z","iopub.execute_input":"2021-06-03T19:49:55.053450Z","iopub.status.idle":"2021-06-03T19:49:55.063307Z","shell.execute_reply.started":"2021-06-03T19:49:55.053416Z","shell.execute_reply":"2021-06-03T19:49:55.062473Z"},"trusted":true},"execution_count":23,"outputs":[]},{"cell_type":"markdown","source":"### Trainer","metadata":{"papermill":{"duration":0.01885,"end_time":"2021-05-11T19:26:35.910429","exception":false,"start_time":"2021-05-11T19:26:35.891579","status":"completed"},"tags":[],"id":"atmospheric-correspondence"}},{"cell_type":"code","source":"class Trainer:\n    def __init__(self, model, optimizer, scheduler, scalar=None, log_interval=1, evaluate_interval=1):\n        self.model = model\n        self.optimizer = optimizer\n        self.scheduler = scheduler\n        self.scalar = scalar\n        self.log_interval = log_interval\n        self.evaluate_interval = evaluate_interval\n        self.evaluator = Evaluator(self.model, self.scalar)\n\n    def train(self, train_loader, valid_loader, epoch, \n              result_dict, tokenizer, fold):\n        count = 0\n        losses = AverageMeter()\n        self.model.train()\n        \n        for batch_idx, batch_data in enumerate(train_loader):\n            input_ids, attention_mask, token_type_ids, labels = batch_data['input_ids'], \\\n                batch_data['attention_mask'], batch_data['token_type_ids'], batch_data['label']\n            input_ids, attention_mask, token_type_ids, labels = \\\n                input_ids.cuda(), attention_mask.cuda(), token_type_ids.cuda(), labels.cuda()\n            \n            if self.scalar is not None:\n                with torch.cuda.amp.autocast():\n                    outputs = self.model(\n                        input_ids=input_ids,\n                        attention_mask=attention_mask,\n                        token_type_ids=token_type_ids,\n                        labels=labels\n                    )\n            else:\n                outputs = self.model(\n                    input_ids=input_ids,\n                    attention_mask=attention_mask,\n                    token_type_ids=token_type_ids,\n                    labels=labels\n                )\n\n            loss, logits = outputs[:2]\n            count += labels.size(0)\n            losses.update(loss.item(), input_ids.size(0))\n            \n            if self.scalar is not None:\n                self.scalar.scale(loss).backward()\n                self.scalar.step(self.optimizer)\n                self.scalar.update()\n            else:\n                loss.backward()\n                self.optimizer.step()\n\n            self.scheduler.step()\n            self.optimizer.zero_grad()\n\n            if batch_idx % self.log_interval == 0:\n                _s = str(len(str(len(train_loader.sampler))))\n                ret = [\n                    ('epoch: {:0>3} [{: >' + _s + '}/{} ({: >3.0f}%)]').format(epoch, count, len(train_loader.sampler), 100 * count / len(train_loader.sampler)),\n                    'train_loss: {: >4.5f}'.format(losses.avg),\n                ]\n                print(', '.join(ret))\n            \n            if batch_idx % self.evaluate_interval == 0:\n                result_dict = self.evaluator.evaluate(\n                    valid_loader, \n                    epoch, \n                    result_dict, \n                    tokenizer\n                )\n                if result_dict['val_loss'][-1] < result_dict['best_val_loss']:\n                    print(\"{} epoch, best epoch was updated! valid_loss: {: >4.5f}\".format(epoch, result_dict['val_loss'][-1]))\n                    result_dict[\"best_val_loss\"] = result_dict['val_loss'][-1]\n                    torch.save(self.model.state_dict(), f\"model{fold}.bin\")\n\n        result_dict['train_loss'].append(losses.avg)\n        return result_dict","metadata":{"papermill":{"duration":0.033486,"end_time":"2021-05-11T19:26:35.962579","exception":false,"start_time":"2021-05-11T19:26:35.929093","status":"completed"},"tags":[],"id":"chubby-liberty","execution":{"iopub.status.busy":"2021-06-03T19:49:55.069759Z","iopub.execute_input":"2021-06-03T19:49:55.073024Z","iopub.status.idle":"2021-06-03T19:49:55.095096Z","shell.execute_reply.started":"2021-06-03T19:49:55.072986Z","shell.execute_reply":"2021-06-03T19:49:55.094286Z"},"trusted":true},"execution_count":24,"outputs":[]},{"cell_type":"markdown","source":"### Evaluator","metadata":{"papermill":{"duration":0.019241,"end_time":"2021-05-11T19:26:36.000319","exception":false,"start_time":"2021-05-11T19:26:35.981078","status":"completed"},"tags":[],"id":"interested-glass"}},{"cell_type":"code","source":"class Evaluator:\n    def __init__(self, model, scalar=None):\n        self.model = model\n        self.scalar = scalar\n    \n    def worst_result(self):\n        ret = {\n            'loss':float('inf'),\n            'accuracy':0.0\n        }\n        return ret\n\n    def result_to_str(self, result):\n        ret = [\n            'epoch: {epoch:0>3}',\n            'loss: {loss: >4.2e}'\n        ]\n        for metric in self.evaluation_metrics:\n            ret.append('{}: {}'.format(metric.name, metric.fmtstr))\n        return ', '.join(ret).format(**result)\n\n    def save(self, result):\n        with open('result_dict.json', 'w') as f:\n            f.write(json.dumps(result, sort_keys=True, indent=4, ensure_ascii=False))\n    \n    def load(self):\n        result = self.worst_result\n        if os.path.exists('result_dict.json'):\n            with open('result_dict.json', 'r') as f:\n                try:\n                    result = json.loads(f.read())\n                except:\n                    pass\n        return result\n\n    def evaluate(self, data_loader, epoch, result_dict, tokenizer):\n        losses = AverageMeter()\n\n        self.model.eval()\n        total_loss = 0\n        with torch.no_grad():\n            for batch_idx, batch_data in enumerate(data_loader):\n                input_ids, attention_mask, token_type_ids, labels = batch_data['input_ids'], \\\n                    batch_data['attention_mask'], batch_data['token_type_ids'], batch_data['label']\n                input_ids, attention_mask, token_type_ids, labels = input_ids.cuda(), \\\n                    attention_mask.cuda(), token_type_ids.cuda(), labels.cuda()\n                \n                if self.scalar is not None:\n                    with torch.cuda.amp.autocast():\n                        outputs = self.model(\n                            input_ids=input_ids,\n                            attention_mask=attention_mask,\n                            token_type_ids=token_type_ids,\n                            labels=labels\n                        )\n                else:\n                    outputs = self.model(\n                        input_ids=input_ids,\n                        attention_mask=attention_mask,\n                        token_type_ids=token_type_ids,\n                        labels=labels\n                    )\n                \n                loss, logits = outputs[:2]\n                losses.update(loss.item(), input_ids.size(0))\n\n        print('----Validation Results Summary----')\n        print('Epoch: [{}] valid_loss: {: >4.5f}'.format(epoch, losses.avg))\n\n        result_dict['val_loss'].append(losses.avg)        \n        return result_dict","metadata":{"papermill":{"duration":0.034578,"end_time":"2021-05-11T19:26:36.054175","exception":false,"start_time":"2021-05-11T19:26:36.019597","status":"completed"},"tags":[],"id":"demonstrated-delhi","execution":{"iopub.status.busy":"2021-06-03T19:49:55.100219Z","iopub.execute_input":"2021-06-03T19:49:55.103444Z","iopub.status.idle":"2021-06-03T19:49:55.122689Z","shell.execute_reply.started":"2021-06-03T19:49:55.103407Z","shell.execute_reply":"2021-06-03T19:49:55.121817Z"},"trusted":true},"execution_count":25,"outputs":[]},{"cell_type":"markdown","source":"### Config","metadata":{"papermill":{"duration":0.018509,"end_time":"2021-05-11T19:26:36.091323","exception":false,"start_time":"2021-05-11T19:26:36.072814","status":"completed"},"tags":[],"id":"solved-ownership"}},{"cell_type":"code","source":"def config(fold=0):\n    torch.manual_seed(2021)\n    torch.cuda.manual_seed(2021)\n    torch.cuda.manual_seed_all(2021)\n    epochs = 8\n    max_len = 250\n    batch_size = 16\n\n    model, tokenizer = make_model(model_name='../input/robertaitpt/', num_labels=1)\n    train_loader, valid_loader = make_loader(\n        train, tokenizer, max_len=max_len,\n        batch_size=batch_size, fold=fold\n    )\n\n    import math\n    num_update_steps_per_epoch = len(train_loader)\n    max_train_steps = epochs * num_update_steps_per_epoch\n    warmup_proportion = 0\n    if warmup_proportion != 0:\n        warmup_steps = math.ceil((max_train_steps * 2) / 100)\n    else:\n        warmup_steps = 0\n\n    optimizer = make_optimizer(model, \"AdamW\")\n    scheduler = make_scheduler(\n        optimizer, decay_name='cosine_warmup', \n        t_max=max_train_steps, \n        warmup_steps=warmup_steps\n    )    \n\n    if torch.cuda.device_count() >= 1:\n        print('Model pushed to {} GPU(s), type {}.'.format(\n            torch.cuda.device_count(), \n            torch.cuda.get_device_name(0))\n        )\n        model = model.cuda() \n    else:\n        raise ValueError('CPU training is not supported')\n\n    # scaler = torch.cuda.amp.GradScaler()\n    scaler = None\n\n    result_dict = {\n        'epoch':[], \n        'train_loss': [], \n        'val_loss' : [], \n        'best_val_loss': np.inf\n    }\n    return (\n        model, tokenizer, \n        optimizer, scheduler, \n        scaler, train_loader, \n        valid_loader, result_dict, \n        epochs\n    )","metadata":{"_kg_hide-output":true,"papermill":{"duration":0.028282,"end_time":"2021-05-11T19:26:36.138369","exception":false,"start_time":"2021-05-11T19:26:36.110087","status":"completed"},"tags":[],"id":"addressed-function","execution":{"iopub.status.busy":"2021-06-03T19:49:55.128593Z","iopub.execute_input":"2021-06-03T19:49:55.131410Z","iopub.status.idle":"2021-06-03T19:49:55.144973Z","shell.execute_reply.started":"2021-06-03T19:49:55.131376Z","shell.execute_reply":"2021-06-03T19:49:55.144000Z"},"trusted":true},"execution_count":26,"outputs":[]},{"cell_type":"markdown","source":"### Run","metadata":{"papermill":{"duration":0.020208,"end_time":"2021-05-11T19:26:36.177987","exception":false,"start_time":"2021-05-11T19:26:36.157779","status":"completed"},"tags":[],"id":"nonprofit-causing"}},{"cell_type":"code","source":"def run(fold=0):\n    model, tokenizer, optimizer, scheduler, scaler, \\\n        train_loader, valid_loader, result_dict, epochs = config(fold)\n    \n    import time\n    trainer = Trainer(model, optimizer, scheduler, scaler)\n    train_time_list = []\n\n    for epoch in range(epochs):\n        result_dict['epoch'] = epoch\n\n        torch.cuda.synchronize()\n        tic1 = time.time()\n\n        result_dict = trainer.train(train_loader, valid_loader, epoch, \n                                    result_dict, tokenizer, fold)\n\n        torch.cuda.synchronize()\n        tic2 = time.time() \n        train_time_list.append(tic2 - tic1)\n\n    torch.cuda.empty_cache()\n    del model, tokenizer, optimizer, scheduler, \\\n        scaler, train_loader, valid_loader,\n    gc.collect()\n    return result_dict","metadata":{"papermill":{"duration":0.032278,"end_time":"2021-05-11T19:26:36.229422","exception":false,"start_time":"2021-05-11T19:26:36.197144","status":"completed"},"tags":[],"id":"suspended-anniversary","execution":{"iopub.status.busy":"2021-06-03T19:49:55.150991Z","iopub.execute_input":"2021-06-03T19:49:55.154208Z","iopub.status.idle":"2021-06-03T19:49:55.164510Z","shell.execute_reply.started":"2021-06-03T19:49:55.154172Z","shell.execute_reply":"2021-06-03T19:49:55.163594Z"},"trusted":true},"execution_count":27,"outputs":[]},{"cell_type":"code","source":"result_list = []\nfor fold in range(2,3):\n    print('----')\n    print(f'FOLD: {fold}')\n    result_dict = run(fold)\n    result_list.append(result_dict)\n    print('----')","metadata":{"papermill":{"duration":2940.88873,"end_time":"2021-05-11T20:15:37.137957","exception":false,"start_time":"2021-05-11T19:26:36.249227","status":"completed"},"tags":[],"id":"flush-clause","outputId":"0bb37720-4c50-4662-8128-176e7008e03d","execution":{"iopub.status.busy":"2021-06-03T19:49:55.277063Z","iopub.execute_input":"2021-06-03T19:49:55.277411Z","iopub.status.idle":"2021-06-04T03:59:51.950351Z","shell.execute_reply.started":"2021-06-03T19:49:55.277378Z","shell.execute_reply":"2021-06-04T03:59:51.949091Z"},"trusted":true},"execution_count":28,"outputs":[{"name":"stdout","text":"----\nFOLD: 2\n","output_type":"stream"},{"name":"stderr","text":"Some weights of RobertaModel were not initialized from the model checkpoint at ../input/robertaitpt/ and are newly initialized: ['roberta.pooler.dense.weight', 'roberta.pooler.dense.bias']\nYou should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n","output_type":"stream"},{"name":"stdout","text":"Model pushed to 1 GPU(s), type Tesla P100-PCIE-16GB.\nepoch: 000 [  16/3571 (  0%)], train_loss: 1.57088\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.26739\n0 epoch, best epoch was updated! valid_loss: 1.26739\nepoch: 000 [  32/3571 (  1%)], train_loss: 1.40065\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.18344\n0 epoch, best epoch was updated! valid_loss: 1.18344\nepoch: 000 [  48/3571 (  1%)], train_loss: 1.32948\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.96853\n0 epoch, best epoch was updated! valid_loss: 0.96853\nepoch: 000 [  64/3571 (  2%)], train_loss: 1.22893\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.04471\nepoch: 000 [  80/3571 (  2%)], train_loss: 1.16833\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.09794\nepoch: 000 [  96/3571 (  3%)], train_loss: 1.15737\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.00718\nepoch: 000 [ 112/3571 (  3%)], train_loss: 1.14297\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.94318\n0 epoch, best epoch was updated! valid_loss: 0.94318\nepoch: 000 [ 128/3571 (  4%)], train_loss: 1.12404\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.10056\nepoch: 000 [ 144/3571 (  4%)], train_loss: 1.11939\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.21193\nepoch: 000 [ 160/3571 (  4%)], train_loss: 1.14427\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.10538\nepoch: 000 [ 176/3571 (  5%)], train_loss: 1.12386\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.95105\nepoch: 000 [ 192/3571 (  5%)], train_loss: 1.10770\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.94937\nepoch: 000 [ 208/3571 (  6%)], train_loss: 1.08775\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.04484\nepoch: 000 [ 224/3571 (  6%)], train_loss: 1.08580\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.04964\nepoch: 000 [ 240/3571 (  7%)], train_loss: 1.07832\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.01983\nepoch: 000 [ 256/3571 (  7%)], train_loss: 1.07977\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.95330\nepoch: 000 [ 272/3571 (  8%)], train_loss: 1.08713\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.88179\n0 epoch, best epoch was updated! valid_loss: 0.88179\nepoch: 000 [ 288/3571 (  8%)], train_loss: 1.07835\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.89884\nepoch: 000 [ 304/3571 (  9%)], train_loss: 1.06566\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.99138\nepoch: 000 [ 320/3571 (  9%)], train_loss: 1.05992\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.94761\nepoch: 000 [ 336/3571 (  9%)], train_loss: 1.06360\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.84091\n0 epoch, best epoch was updated! valid_loss: 0.84091\nepoch: 000 [ 352/3571 ( 10%)], train_loss: 1.05785\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.86910\nepoch: 000 [ 368/3571 ( 10%)], train_loss: 1.04738\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.00212\nepoch: 000 [ 384/3571 ( 11%)], train_loss: 1.05237\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.01959\nepoch: 000 [ 400/3571 ( 11%)], train_loss: 1.04681\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.87551\nepoch: 000 [ 416/3571 ( 12%)], train_loss: 1.03732\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.79777\n0 epoch, best epoch was updated! valid_loss: 0.79777\nepoch: 000 [ 432/3571 ( 12%)], train_loss: 1.03067\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.81281\nepoch: 000 [ 448/3571 ( 13%)], train_loss: 1.02596\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.90109\nepoch: 000 [ 464/3571 ( 13%)], train_loss: 1.01691\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.97196\nepoch: 000 [ 480/3571 ( 13%)], train_loss: 1.01526\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.87053\nepoch: 000 [ 496/3571 ( 14%)], train_loss: 1.01100\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.74106\n0 epoch, best epoch was updated! valid_loss: 0.74106\nepoch: 000 [ 512/3571 ( 14%)], train_loss: 0.99841\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.72177\n0 epoch, best epoch was updated! valid_loss: 0.72177\nepoch: 000 [ 528/3571 ( 15%)], train_loss: 0.99146\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.76348\nepoch: 000 [ 544/3571 ( 15%)], train_loss: 0.97690\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.74261\nepoch: 000 [ 560/3571 ( 16%)], train_loss: 0.97005\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.69192\n0 epoch, best epoch was updated! valid_loss: 0.69192\nepoch: 000 [ 576/3571 ( 16%)], train_loss: 0.95858\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.79586\nepoch: 000 [ 592/3571 ( 17%)], train_loss: 0.94897\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.85381\nepoch: 000 [ 608/3571 ( 17%)], train_loss: 0.94502\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.80373\nepoch: 000 [ 624/3571 ( 17%)], train_loss: 0.94366\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.72278\nepoch: 000 [ 640/3571 ( 18%)], train_loss: 0.94128\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.72733\nepoch: 000 [ 656/3571 ( 18%)], train_loss: 0.94096\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.71947\nepoch: 000 [ 672/3571 ( 19%)], train_loss: 0.94136\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.70063\nepoch: 000 [ 688/3571 ( 19%)], train_loss: 0.93954\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66960\n0 epoch, best epoch was updated! valid_loss: 0.66960\nepoch: 000 [ 704/3571 ( 20%)], train_loss: 0.93520\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.70228\nepoch: 000 [ 720/3571 ( 20%)], train_loss: 0.92717\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.77547\nepoch: 000 [ 736/3571 ( 21%)], train_loss: 0.92312\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67901\nepoch: 000 [ 752/3571 ( 21%)], train_loss: 0.91833\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.86396\nepoch: 000 [ 768/3571 ( 22%)], train_loss: 0.90988\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.94930\nepoch: 000 [ 784/3571 ( 22%)], train_loss: 0.90667\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.79161\nepoch: 000 [ 800/3571 ( 22%)], train_loss: 0.90282\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68868\nepoch: 000 [ 816/3571 ( 23%)], train_loss: 0.89520\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.72176\nepoch: 000 [ 832/3571 ( 23%)], train_loss: 0.89239\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.83869\nepoch: 000 [ 848/3571 ( 24%)], train_loss: 0.88897\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.85462\nepoch: 000 [ 864/3571 ( 24%)], train_loss: 0.88718\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.72895\nepoch: 000 [ 880/3571 ( 25%)], train_loss: 0.88297\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68666\nepoch: 000 [ 896/3571 ( 25%)], train_loss: 0.88360\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.78165\nepoch: 000 [ 912/3571 ( 26%)], train_loss: 0.87805\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.83222\nepoch: 000 [ 928/3571 ( 26%)], train_loss: 0.87648\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.75649\nepoch: 000 [ 944/3571 ( 26%)], train_loss: 0.87532\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67709\nepoch: 000 [ 960/3571 ( 27%)], train_loss: 0.87440\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.71224\nepoch: 000 [ 976/3571 ( 27%)], train_loss: 0.87138\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.79370\nepoch: 000 [ 992/3571 ( 28%)], train_loss: 0.87049\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.77738\nepoch: 000 [1008/3571 ( 28%)], train_loss: 0.86503\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.71597\nepoch: 000 [1024/3571 ( 29%)], train_loss: 0.86351\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67565\nepoch: 000 [1040/3571 ( 29%)], train_loss: 0.86018\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68502\nepoch: 000 [1056/3571 ( 30%)], train_loss: 0.86154\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.69400\nepoch: 000 [1072/3571 ( 30%)], train_loss: 0.86134\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68577\nepoch: 000 [1088/3571 ( 30%)], train_loss: 0.85636\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.69314\nepoch: 000 [1104/3571 ( 31%)], train_loss: 0.85704\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.70336\nepoch: 000 [1120/3571 ( 31%)], train_loss: 0.85300\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.69099\nepoch: 000 [1136/3571 ( 32%)], train_loss: 0.84962\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.65583\n0 epoch, best epoch was updated! valid_loss: 0.65583\nepoch: 000 [1152/3571 ( 32%)], train_loss: 0.84834\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64337\n0 epoch, best epoch was updated! valid_loss: 0.64337\nepoch: 000 [1168/3571 ( 33%)], train_loss: 0.84551\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.65536\nepoch: 000 [1184/3571 ( 33%)], train_loss: 0.84199\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67489\nepoch: 000 [1200/3571 ( 34%)], train_loss: 0.84127\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.71827\nepoch: 000 [1216/3571 ( 34%)], train_loss: 0.83922\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66989\nepoch: 000 [1232/3571 ( 35%)], train_loss: 0.83603\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64515\nepoch: 000 [1248/3571 ( 35%)], train_loss: 0.83229\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.69329\nepoch: 000 [1264/3571 ( 35%)], train_loss: 0.83088\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.75256\nepoch: 000 [1280/3571 ( 36%)], train_loss: 0.83157\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.75917\nepoch: 000 [1296/3571 ( 36%)], train_loss: 0.83162\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68101\nepoch: 000 [1312/3571 ( 37%)], train_loss: 0.82995\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67748\nepoch: 000 [1328/3571 ( 37%)], train_loss: 0.82827\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.71457\nepoch: 000 [1344/3571 ( 38%)], train_loss: 0.83054\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68827\nepoch: 000 [1360/3571 ( 38%)], train_loss: 0.82908\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68897\nepoch: 000 [1376/3571 ( 39%)], train_loss: 0.82870\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.77637\nepoch: 000 [1392/3571 ( 39%)], train_loss: 0.82912\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68799\nepoch: 000 [1408/3571 ( 39%)], train_loss: 0.82641\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64938\nepoch: 000 [1424/3571 ( 40%)], train_loss: 0.82762\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.65621\nepoch: 000 [1440/3571 ( 40%)], train_loss: 0.82670\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.75493\nepoch: 000 [1456/3571 ( 41%)], train_loss: 0.82415\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.78396\nepoch: 000 [1472/3571 ( 41%)], train_loss: 0.82441\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67361\nepoch: 000 [1488/3571 ( 42%)], train_loss: 0.82332\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63436\n0 epoch, best epoch was updated! valid_loss: 0.63436\nepoch: 000 [1504/3571 ( 42%)], train_loss: 0.82012\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66197\nepoch: 000 [1520/3571 ( 43%)], train_loss: 0.81765\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62825\n0 epoch, best epoch was updated! valid_loss: 0.62825\nepoch: 000 [1536/3571 ( 43%)], train_loss: 0.81440\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63619\nepoch: 000 [1552/3571 ( 43%)], train_loss: 0.81257\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64322\nepoch: 000 [1568/3571 ( 44%)], train_loss: 0.81262\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63155\nepoch: 000 [1584/3571 ( 44%)], train_loss: 0.81057\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.61894\n0 epoch, best epoch was updated! valid_loss: 0.61894\nepoch: 000 [1600/3571 ( 45%)], train_loss: 0.80816\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62874\nepoch: 000 [1616/3571 ( 45%)], train_loss: 0.80583\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63119\nepoch: 000 [1632/3571 ( 46%)], train_loss: 0.80353\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.61789\n0 epoch, best epoch was updated! valid_loss: 0.61789\nepoch: 000 [1648/3571 ( 46%)], train_loss: 0.80181\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62296\nepoch: 000 [1664/3571 ( 47%)], train_loss: 0.79943\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63917\nepoch: 000 [1680/3571 ( 47%)], train_loss: 0.79899\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62946\nepoch: 000 [1696/3571 ( 47%)], train_loss: 0.79778\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66268\nepoch: 000 [1712/3571 ( 48%)], train_loss: 0.79788\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63139\nepoch: 000 [1728/3571 ( 48%)], train_loss: 0.79615\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.65547\nepoch: 000 [1744/3571 ( 49%)], train_loss: 0.79433\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62795\nepoch: 000 [1760/3571 ( 49%)], train_loss: 0.79140\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63461\nepoch: 000 [1776/3571 ( 50%)], train_loss: 0.79073\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.60300\n0 epoch, best epoch was updated! valid_loss: 0.60300\nepoch: 000 [1792/3571 ( 50%)], train_loss: 0.78999\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62286\nepoch: 000 [1808/3571 ( 51%)], train_loss: 0.78769\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63971\nepoch: 000 [1824/3571 ( 51%)], train_loss: 0.78647\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67001\nepoch: 000 [1840/3571 ( 52%)], train_loss: 0.78700\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.65146\nepoch: 000 [1856/3571 ( 52%)], train_loss: 0.78644\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64672\nepoch: 000 [1872/3571 ( 52%)], train_loss: 0.78597\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63617\nepoch: 000 [1888/3571 ( 53%)], train_loss: 0.78531\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.61917\nepoch: 000 [1904/3571 ( 53%)], train_loss: 0.78427\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64269\nepoch: 000 [1920/3571 ( 54%)], train_loss: 0.78340\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63610\nepoch: 000 [1936/3571 ( 54%)], train_loss: 0.78211\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.60966\nepoch: 000 [1952/3571 ( 55%)], train_loss: 0.77979\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.69286\nepoch: 000 [1968/3571 ( 55%)], train_loss: 0.77806\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.79779\nepoch: 000 [1984/3571 ( 56%)], train_loss: 0.77846\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68995\nepoch: 000 [2000/3571 ( 56%)], train_loss: 0.77818\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63543\nepoch: 000 [2016/3571 ( 56%)], train_loss: 0.77723\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64271\nepoch: 000 [2032/3571 ( 57%)], train_loss: 0.77708\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62486\nepoch: 000 [2048/3571 ( 57%)], train_loss: 0.77526\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62724\nepoch: 000 [2064/3571 ( 58%)], train_loss: 0.77421\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62640\nepoch: 000 [2080/3571 ( 58%)], train_loss: 0.77290\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63633\nepoch: 000 [2096/3571 ( 59%)], train_loss: 0.77240\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68288\nepoch: 000 [2112/3571 ( 59%)], train_loss: 0.77219\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67589\nepoch: 000 [2128/3571 ( 60%)], train_loss: 0.76922\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66222\nepoch: 000 [2144/3571 ( 60%)], train_loss: 0.76827\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.61734\nepoch: 000 [2160/3571 ( 60%)], train_loss: 0.76782\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68006\nepoch: 000 [2176/3571 ( 61%)], train_loss: 0.76658\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.92409\nepoch: 000 [2192/3571 ( 61%)], train_loss: 0.76826\n----Validation Results Summary----\nEpoch: [0] valid_loss: 1.02193\nepoch: 000 [2208/3571 ( 62%)], train_loss: 0.76988\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.83761\nepoch: 000 [2224/3571 ( 62%)], train_loss: 0.76953\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.59325\n0 epoch, best epoch was updated! valid_loss: 0.59325\nepoch: 000 [2240/3571 ( 63%)], train_loss: 0.76906\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.70514\nepoch: 000 [2256/3571 ( 63%)], train_loss: 0.76761\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.89211\nepoch: 000 [2272/3571 ( 64%)], train_loss: 0.76943\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.82959\nepoch: 000 [2288/3571 ( 64%)], train_loss: 0.77140\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66605\nepoch: 000 [2304/3571 ( 65%)], train_loss: 0.77235\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64547\nepoch: 000 [2320/3571 ( 65%)], train_loss: 0.77110\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.70552\nepoch: 000 [2336/3571 ( 65%)], train_loss: 0.77189\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68462\nepoch: 000 [2352/3571 ( 66%)], train_loss: 0.77195\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.61420\nepoch: 000 [2368/3571 ( 66%)], train_loss: 0.77099\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64757\nepoch: 000 [2384/3571 ( 67%)], train_loss: 0.77082\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.69909\nepoch: 000 [2400/3571 ( 67%)], train_loss: 0.77238\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63874\nepoch: 000 [2416/3571 ( 68%)], train_loss: 0.77208\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.59288\n0 epoch, best epoch was updated! valid_loss: 0.59288\nepoch: 000 [2432/3571 ( 68%)], train_loss: 0.77031\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.77905\nepoch: 000 [2448/3571 ( 69%)], train_loss: 0.77181\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.84008\nepoch: 000 [2464/3571 ( 69%)], train_loss: 0.77250\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.71848\nepoch: 000 [2480/3571 ( 69%)], train_loss: 0.77219\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.57070\n0 epoch, best epoch was updated! valid_loss: 0.57070\nepoch: 000 [2496/3571 ( 70%)], train_loss: 0.76947\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.65232\nepoch: 000 [2512/3571 ( 70%)], train_loss: 0.76770\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.79545\nepoch: 000 [2528/3571 ( 71%)], train_loss: 0.76714\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.76505\nepoch: 000 [2544/3571 ( 71%)], train_loss: 0.76781\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66632\nepoch: 000 [2560/3571 ( 72%)], train_loss: 0.76623\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64661\nepoch: 000 [2576/3571 ( 72%)], train_loss: 0.76450\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.68771\nepoch: 000 [2592/3571 ( 73%)], train_loss: 0.76292\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.73926\nepoch: 000 [2608/3571 ( 73%)], train_loss: 0.76150\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.77162\nepoch: 000 [2624/3571 ( 73%)], train_loss: 0.75987\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.75840\nepoch: 000 [2640/3571 ( 74%)], train_loss: 0.75946\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.70026\nepoch: 000 [2656/3571 ( 74%)], train_loss: 0.75849\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.70637\nepoch: 000 [2672/3571 ( 75%)], train_loss: 0.75722\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.78216\nepoch: 000 [2688/3571 ( 75%)], train_loss: 0.75802\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.81076\nepoch: 000 [2704/3571 ( 76%)], train_loss: 0.75929\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.72360\nepoch: 000 [2720/3571 ( 76%)], train_loss: 0.75817\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63477\nepoch: 000 [2736/3571 ( 77%)], train_loss: 0.75703\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.61833\nepoch: 000 [2752/3571 ( 77%)], train_loss: 0.75644\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.60661\nepoch: 000 [2768/3571 ( 78%)], train_loss: 0.75569\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.60037\nepoch: 000 [2784/3571 ( 78%)], train_loss: 0.75616\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.59531\nepoch: 000 [2800/3571 ( 78%)], train_loss: 0.75462\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.61648\nepoch: 000 [2816/3571 ( 79%)], train_loss: 0.75350\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62568\nepoch: 000 [2832/3571 ( 79%)], train_loss: 0.75207\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63202\nepoch: 000 [2848/3571 ( 80%)], train_loss: 0.75233\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.59607\nepoch: 000 [2864/3571 ( 80%)], train_loss: 0.75075\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.57127\nepoch: 000 [2880/3571 ( 81%)], train_loss: 0.74953\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64029\nepoch: 000 [2896/3571 ( 81%)], train_loss: 0.74997\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.78886\nepoch: 000 [2912/3571 ( 82%)], train_loss: 0.75027\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.76854\nepoch: 000 [2928/3571 ( 82%)], train_loss: 0.75031\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.60125\nepoch: 000 [2944/3571 ( 82%)], train_loss: 0.74952\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.57302\nepoch: 000 [2960/3571 ( 83%)], train_loss: 0.74937\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.57564\nepoch: 000 [2976/3571 ( 83%)], train_loss: 0.74839\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.59147\nepoch: 000 [2992/3571 ( 84%)], train_loss: 0.74911\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.55372\n0 epoch, best epoch was updated! valid_loss: 0.55372\nepoch: 000 [3008/3571 ( 84%)], train_loss: 0.74783\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.54416\n0 epoch, best epoch was updated! valid_loss: 0.54416\nepoch: 000 [3024/3571 ( 85%)], train_loss: 0.74687\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.54538\nepoch: 000 [3040/3571 ( 85%)], train_loss: 0.74641\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.54300\n0 epoch, best epoch was updated! valid_loss: 0.54300\nepoch: 000 [3056/3571 ( 86%)], train_loss: 0.74598\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.53549\n0 epoch, best epoch was updated! valid_loss: 0.53549\nepoch: 000 [3072/3571 ( 86%)], train_loss: 0.74480\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.55188\nepoch: 000 [3088/3571 ( 86%)], train_loss: 0.74388\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.59053\nepoch: 000 [3104/3571 ( 87%)], train_loss: 0.74282\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.56943\nepoch: 000 [3120/3571 ( 87%)], train_loss: 0.74247\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.53469\n0 epoch, best epoch was updated! valid_loss: 0.53469\nepoch: 000 [3136/3571 ( 88%)], train_loss: 0.74156\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.53831\nepoch: 000 [3152/3571 ( 88%)], train_loss: 0.74019\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.60457\nepoch: 000 [3168/3571 ( 89%)], train_loss: 0.73989\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63496\nepoch: 000 [3184/3571 ( 89%)], train_loss: 0.73980\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.58446\nepoch: 000 [3200/3571 ( 90%)], train_loss: 0.74023\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.53981\nepoch: 000 [3216/3571 ( 90%)], train_loss: 0.73861\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.58966\nepoch: 000 [3232/3571 ( 91%)], train_loss: 0.73845\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.62043\nepoch: 000 [3248/3571 ( 91%)], train_loss: 0.73774\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.55576\nepoch: 000 [3264/3571 ( 91%)], train_loss: 0.73706\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.56637\nepoch: 000 [3280/3571 ( 92%)], train_loss: 0.73682\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.58902\nepoch: 000 [3296/3571 ( 92%)], train_loss: 0.73628\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.55631\nepoch: 000 [3312/3571 ( 93%)], train_loss: 0.73591\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.56377\nepoch: 000 [3328/3571 ( 93%)], train_loss: 0.73496\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66461\nepoch: 000 [3344/3571 ( 94%)], train_loss: 0.73457\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.66331\nepoch: 000 [3360/3571 ( 94%)], train_loss: 0.73471\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.58136\nepoch: 000 [3376/3571 ( 95%)], train_loss: 0.73488\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.56887\nepoch: 000 [3392/3571 ( 95%)], train_loss: 0.73406\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64218\nepoch: 000 [3408/3571 ( 95%)], train_loss: 0.73433\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.67280\nepoch: 000 [3424/3571 ( 96%)], train_loss: 0.73412\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64583\nepoch: 000 [3440/3571 ( 96%)], train_loss: 0.73405\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.59255\nepoch: 000 [3456/3571 ( 97%)], train_loss: 0.73354\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.60706\nepoch: 000 [3472/3571 ( 97%)], train_loss: 0.73207\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.64977\nepoch: 000 [3488/3571 ( 98%)], train_loss: 0.73211\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.63676\nepoch: 000 [3504/3571 ( 98%)], train_loss: 0.73200\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.56215\nepoch: 000 [3520/3571 ( 99%)], train_loss: 0.73180\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.52945\n0 epoch, best epoch was updated! valid_loss: 0.52945\nepoch: 000 [3536/3571 ( 99%)], train_loss: 0.73135\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.55593\nepoch: 000 [3552/3571 ( 99%)], train_loss: 0.73071\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.53347\nepoch: 000 [3568/3571 (100%)], train_loss: 0.73032\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.53181\nepoch: 000 [3571/3571 (100%)], train_loss: 0.73003\n----Validation Results Summary----\nEpoch: [0] valid_loss: 0.52804\n0 epoch, best epoch was updated! valid_loss: 0.52804\nepoch: 001 [  16/3571 (  0%)], train_loss: 0.72898\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54926\nepoch: 001 [  32/3571 (  1%)], train_loss: 0.53264\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.58351\nepoch: 001 [  48/3571 (  1%)], train_loss: 0.52343\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54844\nepoch: 001 [  64/3571 (  2%)], train_loss: 0.53816\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52479\n1 epoch, best epoch was updated! valid_loss: 0.52479\nepoch: 001 [  80/3571 (  2%)], train_loss: 0.50743\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54380\nepoch: 001 [  96/3571 (  3%)], train_loss: 0.50727\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54151\nepoch: 001 [ 112/3571 (  3%)], train_loss: 0.48984\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54344\nepoch: 001 [ 128/3571 (  4%)], train_loss: 0.48464\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52723\nepoch: 001 [ 144/3571 (  4%)], train_loss: 0.49202\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57877\nepoch: 001 [ 160/3571 (  4%)], train_loss: 0.48418\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.69329\nepoch: 001 [ 176/3571 (  5%)], train_loss: 0.49693\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.67666\nepoch: 001 [ 192/3571 (  5%)], train_loss: 0.50043\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57871\nepoch: 001 [ 208/3571 (  6%)], train_loss: 0.51331\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54140\nepoch: 001 [ 224/3571 (  6%)], train_loss: 0.51076\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61337\nepoch: 001 [ 240/3571 (  7%)], train_loss: 0.51607\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60628\nepoch: 001 [ 256/3571 (  7%)], train_loss: 0.50784\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56529\nepoch: 001 [ 272/3571 (  8%)], train_loss: 0.50613\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.59877\nepoch: 001 [ 288/3571 (  8%)], train_loss: 0.51668\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.69376\nepoch: 001 [ 304/3571 (  9%)], train_loss: 0.53745\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.75004\nepoch: 001 [ 320/3571 (  9%)], train_loss: 0.54550\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.67399\nepoch: 001 [ 336/3571 (  9%)], train_loss: 0.53608\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55157\nepoch: 001 [ 352/3571 ( 10%)], train_loss: 0.53701\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.59821\nepoch: 001 [ 368/3571 ( 10%)], train_loss: 0.54026\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.70956\nepoch: 001 [ 384/3571 ( 11%)], train_loss: 0.54349\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.65352\nepoch: 001 [ 400/3571 ( 11%)], train_loss: 0.54652\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54148\nepoch: 001 [ 416/3571 ( 12%)], train_loss: 0.54978\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57292\nepoch: 001 [ 432/3571 ( 12%)], train_loss: 0.55380\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61876\nepoch: 001 [ 448/3571 ( 13%)], train_loss: 0.55184\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55677\nepoch: 001 [ 464/3571 ( 13%)], train_loss: 0.55026\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51576\n1 epoch, best epoch was updated! valid_loss: 0.51576\nepoch: 001 [ 480/3571 ( 13%)], train_loss: 0.54285\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56922\nepoch: 001 [ 496/3571 ( 14%)], train_loss: 0.53411\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61997\nepoch: 001 [ 512/3571 ( 14%)], train_loss: 0.53675\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57901\nepoch: 001 [ 528/3571 ( 15%)], train_loss: 0.53680\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54714\nepoch: 001 [ 544/3571 ( 15%)], train_loss: 0.53651\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53714\nepoch: 001 [ 560/3571 ( 16%)], train_loss: 0.53313\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57633\nepoch: 001 [ 576/3571 ( 16%)], train_loss: 0.53421\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60857\nepoch: 001 [ 592/3571 ( 17%)], train_loss: 0.53456\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57196\nepoch: 001 [ 608/3571 ( 17%)], train_loss: 0.53698\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52425\nepoch: 001 [ 624/3571 ( 17%)], train_loss: 0.53364\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54437\nepoch: 001 [ 640/3571 ( 18%)], train_loss: 0.53494\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55099\nepoch: 001 [ 656/3571 ( 18%)], train_loss: 0.52890\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55462\nepoch: 001 [ 672/3571 ( 19%)], train_loss: 0.52712\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53254\nepoch: 001 [ 688/3571 ( 19%)], train_loss: 0.52464\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55778\nepoch: 001 [ 704/3571 ( 20%)], train_loss: 0.52467\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60872\nepoch: 001 [ 720/3571 ( 20%)], train_loss: 0.52623\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.58200\nepoch: 001 [ 736/3571 ( 21%)], train_loss: 0.52879\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51819\nepoch: 001 [ 752/3571 ( 21%)], train_loss: 0.52918\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52058\nepoch: 001 [ 768/3571 ( 22%)], train_loss: 0.52814\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53470\nepoch: 001 [ 784/3571 ( 22%)], train_loss: 0.52408\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55555\nepoch: 001 [ 800/3571 ( 22%)], train_loss: 0.52268\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53443\nepoch: 001 [ 816/3571 ( 23%)], train_loss: 0.52090\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53950\nepoch: 001 [ 832/3571 ( 23%)], train_loss: 0.51672\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.62000\nepoch: 001 [ 848/3571 ( 24%)], train_loss: 0.52112\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.63277\nepoch: 001 [ 864/3571 ( 24%)], train_loss: 0.52028\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57620\nepoch: 001 [ 880/3571 ( 25%)], train_loss: 0.51673\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52953\nepoch: 001 [ 896/3571 ( 25%)], train_loss: 0.51685\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55964\nepoch: 001 [ 912/3571 ( 26%)], train_loss: 0.51577\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57327\nepoch: 001 [ 928/3571 ( 26%)], train_loss: 0.51583\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53034\nepoch: 001 [ 944/3571 ( 26%)], train_loss: 0.51544\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52089\nepoch: 001 [ 960/3571 ( 27%)], train_loss: 0.51202\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57187\nepoch: 001 [ 976/3571 ( 27%)], train_loss: 0.50956\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55046\nepoch: 001 [ 992/3571 ( 28%)], train_loss: 0.50937\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52979\nepoch: 001 [1008/3571 ( 28%)], train_loss: 0.50715\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.62915\nepoch: 001 [1024/3571 ( 29%)], train_loss: 0.51157\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.64243\nepoch: 001 [1040/3571 ( 29%)], train_loss: 0.51224\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56326\nepoch: 001 [1056/3571 ( 30%)], train_loss: 0.51030\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52716\nepoch: 001 [1072/3571 ( 30%)], train_loss: 0.51149\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54021\nepoch: 001 [1088/3571 ( 30%)], train_loss: 0.51017\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54228\nepoch: 001 [1104/3571 ( 31%)], train_loss: 0.51188\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52184\nepoch: 001 [1120/3571 ( 31%)], train_loss: 0.51057\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51644\nepoch: 001 [1136/3571 ( 32%)], train_loss: 0.51055\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54720\nepoch: 001 [1152/3571 ( 32%)], train_loss: 0.51154\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60667\nepoch: 001 [1168/3571 ( 33%)], train_loss: 0.51318\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.67813\nepoch: 001 [1184/3571 ( 33%)], train_loss: 0.51356\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61538\nepoch: 001 [1200/3571 ( 34%)], train_loss: 0.51403\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54564\nepoch: 001 [1216/3571 ( 34%)], train_loss: 0.51359\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52723\nepoch: 001 [1232/3571 ( 35%)], train_loss: 0.51227\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52974\nepoch: 001 [1248/3571 ( 35%)], train_loss: 0.51405\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53474\nepoch: 001 [1264/3571 ( 35%)], train_loss: 0.51243\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54160\nepoch: 001 [1280/3571 ( 36%)], train_loss: 0.51328\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53975\nepoch: 001 [1296/3571 ( 36%)], train_loss: 0.51350\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54330\nepoch: 001 [1312/3571 ( 37%)], train_loss: 0.51357\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53543\nepoch: 001 [1328/3571 ( 37%)], train_loss: 0.51349\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54200\nepoch: 001 [1344/3571 ( 38%)], train_loss: 0.51256\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54046\nepoch: 001 [1360/3571 ( 38%)], train_loss: 0.51204\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52462\nepoch: 001 [1376/3571 ( 39%)], train_loss: 0.51187\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57917\nepoch: 001 [1392/3571 ( 39%)], train_loss: 0.51307\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.68610\nepoch: 001 [1408/3571 ( 39%)], train_loss: 0.51456\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.65818\nepoch: 001 [1424/3571 ( 40%)], train_loss: 0.51530\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54318\nepoch: 001 [1440/3571 ( 40%)], train_loss: 0.51349\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55362\nepoch: 001 [1456/3571 ( 41%)], train_loss: 0.51309\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.67370\nepoch: 001 [1472/3571 ( 41%)], train_loss: 0.51397\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.66972\nepoch: 001 [1488/3571 ( 42%)], train_loss: 0.51536\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55371\nepoch: 001 [1504/3571 ( 42%)], train_loss: 0.51407\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57303\nepoch: 001 [1520/3571 ( 43%)], train_loss: 0.51336\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.73486\nepoch: 001 [1536/3571 ( 43%)], train_loss: 0.51589\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.74933\nepoch: 001 [1552/3571 ( 43%)], train_loss: 0.51815\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.62948\nepoch: 001 [1568/3571 ( 44%)], train_loss: 0.51882\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56632\nepoch: 001 [1584/3571 ( 44%)], train_loss: 0.51923\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.68578\nepoch: 001 [1600/3571 ( 45%)], train_loss: 0.52332\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.80589\nepoch: 001 [1616/3571 ( 45%)], train_loss: 0.52583\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.83253\nepoch: 001 [1632/3571 ( 46%)], train_loss: 0.52757\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.77746\nepoch: 001 [1648/3571 ( 46%)], train_loss: 0.52831\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.68561\nepoch: 001 [1664/3571 ( 47%)], train_loss: 0.52950\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.64571\nepoch: 001 [1680/3571 ( 47%)], train_loss: 0.52964\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.66218\nepoch: 001 [1696/3571 ( 47%)], train_loss: 0.53159\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.65276\nepoch: 001 [1712/3571 ( 48%)], train_loss: 0.53163\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60685\nepoch: 001 [1728/3571 ( 48%)], train_loss: 0.53276\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54115\nepoch: 001 [1744/3571 ( 49%)], train_loss: 0.53459\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54610\nepoch: 001 [1760/3571 ( 49%)], train_loss: 0.53652\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55705\nepoch: 001 [1776/3571 ( 50%)], train_loss: 0.53555\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56236\nepoch: 001 [1792/3571 ( 50%)], train_loss: 0.53441\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57044\nepoch: 001 [1808/3571 ( 51%)], train_loss: 0.53247\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55272\nepoch: 001 [1824/3571 ( 51%)], train_loss: 0.53198\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54296\nepoch: 001 [1840/3571 ( 52%)], train_loss: 0.53072\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56529\nepoch: 001 [1856/3571 ( 52%)], train_loss: 0.52909\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57724\nepoch: 001 [1872/3571 ( 52%)], train_loss: 0.52899\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54091\nepoch: 001 [1888/3571 ( 53%)], train_loss: 0.52868\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54778\nepoch: 001 [1904/3571 ( 53%)], train_loss: 0.53028\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57324\nepoch: 001 [1920/3571 ( 54%)], train_loss: 0.53074\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57508\nepoch: 001 [1936/3571 ( 54%)], train_loss: 0.53094\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54663\nepoch: 001 [1952/3571 ( 55%)], train_loss: 0.53007\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53519\nepoch: 001 [1968/3571 ( 55%)], train_loss: 0.52897\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52987\nepoch: 001 [1984/3571 ( 56%)], train_loss: 0.52819\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51697\nepoch: 001 [2000/3571 ( 56%)], train_loss: 0.52832\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.50439\n1 epoch, best epoch was updated! valid_loss: 0.50439\nepoch: 001 [2016/3571 ( 56%)], train_loss: 0.52636\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52942\nepoch: 001 [2032/3571 ( 57%)], train_loss: 0.52657\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52217\nepoch: 001 [2048/3571 ( 57%)], train_loss: 0.52800\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52108\nepoch: 001 [2064/3571 ( 58%)], train_loss: 0.52903\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54672\nepoch: 001 [2080/3571 ( 58%)], train_loss: 0.52815\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57633\nepoch: 001 [2096/3571 ( 59%)], train_loss: 0.52740\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.59124\nepoch: 001 [2112/3571 ( 59%)], train_loss: 0.52717\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52336\nepoch: 001 [2128/3571 ( 60%)], train_loss: 0.52701\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52786\nepoch: 001 [2144/3571 ( 60%)], train_loss: 0.52557\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57036\nepoch: 001 [2160/3571 ( 60%)], train_loss: 0.52537\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60245\nepoch: 001 [2176/3571 ( 61%)], train_loss: 0.52620\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61180\nepoch: 001 [2192/3571 ( 61%)], train_loss: 0.52510\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61659\nepoch: 001 [2208/3571 ( 62%)], train_loss: 0.52506\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61177\nepoch: 001 [2224/3571 ( 62%)], train_loss: 0.52572\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.59928\nepoch: 001 [2240/3571 ( 63%)], train_loss: 0.52648\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57538\nepoch: 001 [2256/3571 ( 63%)], train_loss: 0.52730\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55137\nepoch: 001 [2272/3571 ( 64%)], train_loss: 0.52635\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52204\nepoch: 001 [2288/3571 ( 64%)], train_loss: 0.52612\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51437\nepoch: 001 [2304/3571 ( 65%)], train_loss: 0.52449\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51892\nepoch: 001 [2320/3571 ( 65%)], train_loss: 0.52393\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52942\nepoch: 001 [2336/3571 ( 65%)], train_loss: 0.52457\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54819\nepoch: 001 [2352/3571 ( 66%)], train_loss: 0.52431\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61709\nepoch: 001 [2368/3571 ( 66%)], train_loss: 0.52558\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56975\nepoch: 001 [2384/3571 ( 67%)], train_loss: 0.52511\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.50046\n1 epoch, best epoch was updated! valid_loss: 0.50046\nepoch: 001 [2400/3571 ( 67%)], train_loss: 0.52425\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51867\nepoch: 001 [2416/3571 ( 68%)], train_loss: 0.52290\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54481\nepoch: 001 [2432/3571 ( 68%)], train_loss: 0.52137\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56356\nepoch: 001 [2448/3571 ( 69%)], train_loss: 0.52152\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56301\nepoch: 001 [2464/3571 ( 69%)], train_loss: 0.52200\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57838\nepoch: 001 [2480/3571 ( 69%)], train_loss: 0.52169\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60103\nepoch: 001 [2496/3571 ( 70%)], train_loss: 0.52198\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.58320\nepoch: 001 [2512/3571 ( 70%)], train_loss: 0.52195\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54394\nepoch: 001 [2528/3571 ( 71%)], train_loss: 0.52108\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55268\nepoch: 001 [2544/3571 ( 71%)], train_loss: 0.52033\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.59984\nepoch: 001 [2560/3571 ( 72%)], train_loss: 0.51974\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61982\nepoch: 001 [2576/3571 ( 72%)], train_loss: 0.51991\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56812\nepoch: 001 [2592/3571 ( 73%)], train_loss: 0.51926\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52528\nepoch: 001 [2608/3571 ( 73%)], train_loss: 0.51951\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57444\nepoch: 001 [2624/3571 ( 73%)], train_loss: 0.51939\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.59271\nepoch: 001 [2640/3571 ( 74%)], train_loss: 0.51970\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53536\nepoch: 001 [2656/3571 ( 74%)], train_loss: 0.51913\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51418\nepoch: 001 [2672/3571 ( 75%)], train_loss: 0.51995\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60711\nepoch: 001 [2688/3571 ( 75%)], train_loss: 0.52056\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61123\nepoch: 001 [2704/3571 ( 76%)], train_loss: 0.52107\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51948\nepoch: 001 [2720/3571 ( 76%)], train_loss: 0.52116\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.49507\n1 epoch, best epoch was updated! valid_loss: 0.49507\nepoch: 001 [2736/3571 ( 77%)], train_loss: 0.52000\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55301\nepoch: 001 [2752/3571 ( 77%)], train_loss: 0.51933\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55345\nepoch: 001 [2768/3571 ( 78%)], train_loss: 0.51982\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.50006\nepoch: 001 [2784/3571 ( 78%)], train_loss: 0.51853\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.49883\nepoch: 001 [2800/3571 ( 78%)], train_loss: 0.51873\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.50900\nepoch: 001 [2816/3571 ( 79%)], train_loss: 0.51692\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.50619\nepoch: 001 [2832/3571 ( 79%)], train_loss: 0.51712\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51243\nepoch: 001 [2848/3571 ( 80%)], train_loss: 0.51573\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52856\nepoch: 001 [2864/3571 ( 80%)], train_loss: 0.51467\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51680\nepoch: 001 [2880/3571 ( 81%)], train_loss: 0.51454\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51455\nepoch: 001 [2896/3571 ( 81%)], train_loss: 0.51428\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52392\nepoch: 001 [2912/3571 ( 82%)], train_loss: 0.51442\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52659\nepoch: 001 [2928/3571 ( 82%)], train_loss: 0.51481\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53808\nepoch: 001 [2944/3571 ( 82%)], train_loss: 0.51438\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.59943\nepoch: 001 [2960/3571 ( 83%)], train_loss: 0.51471\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60492\nepoch: 001 [2976/3571 ( 83%)], train_loss: 0.51591\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53746\nepoch: 001 [2992/3571 ( 84%)], train_loss: 0.51693\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.54826\nepoch: 001 [3008/3571 ( 84%)], train_loss: 0.51671\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61311\nepoch: 001 [3024/3571 ( 85%)], train_loss: 0.51663\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61472\nepoch: 001 [3040/3571 ( 85%)], train_loss: 0.51662\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.56342\nepoch: 001 [3056/3571 ( 86%)], train_loss: 0.51667\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53516\nepoch: 001 [3072/3571 ( 86%)], train_loss: 0.51639\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55784\nepoch: 001 [3088/3571 ( 86%)], train_loss: 0.51591\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.63561\nepoch: 001 [3104/3571 ( 87%)], train_loss: 0.51589\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.67537\nepoch: 001 [3120/3571 ( 87%)], train_loss: 0.51695\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.62961\nepoch: 001 [3136/3571 ( 88%)], train_loss: 0.51776\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53261\nepoch: 001 [3152/3571 ( 88%)], train_loss: 0.51775\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.58150\nepoch: 001 [3168/3571 ( 89%)], train_loss: 0.51720\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.64864\nepoch: 001 [3184/3571 ( 89%)], train_loss: 0.51778\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.63764\nepoch: 001 [3200/3571 ( 90%)], train_loss: 0.51844\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53936\nepoch: 001 [3216/3571 ( 90%)], train_loss: 0.51825\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55242\nepoch: 001 [3232/3571 ( 91%)], train_loss: 0.51816\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.60766\nepoch: 001 [3248/3571 ( 91%)], train_loss: 0.51816\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.61554\nepoch: 001 [3264/3571 ( 91%)], train_loss: 0.51890\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.57981\nepoch: 001 [3280/3571 ( 92%)], train_loss: 0.51889\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51652\nepoch: 001 [3296/3571 ( 92%)], train_loss: 0.51784\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55366\nepoch: 001 [3312/3571 ( 93%)], train_loss: 0.51724\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.58380\nepoch: 001 [3328/3571 ( 93%)], train_loss: 0.51686\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55552\nepoch: 001 [3344/3571 ( 94%)], train_loss: 0.51668\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51849\nepoch: 001 [3360/3571 ( 94%)], train_loss: 0.51751\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52161\nepoch: 001 [3376/3571 ( 95%)], train_loss: 0.51648\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.55374\nepoch: 001 [3392/3571 ( 95%)], train_loss: 0.51609\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53530\nepoch: 001 [3408/3571 ( 95%)], train_loss: 0.51647\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51484\nepoch: 001 [3424/3571 ( 96%)], train_loss: 0.51522\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52495\nepoch: 001 [3440/3571 ( 96%)], train_loss: 0.51510\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53297\nepoch: 001 [3456/3571 ( 97%)], train_loss: 0.51555\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.50076\nepoch: 001 [3472/3571 ( 97%)], train_loss: 0.51541\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.49713\nepoch: 001 [3488/3571 ( 98%)], train_loss: 0.51530\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51874\nepoch: 001 [3504/3571 ( 98%)], train_loss: 0.51565\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.50966\nepoch: 001 [3520/3571 ( 99%)], train_loss: 0.51479\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.50132\nepoch: 001 [3536/3571 ( 99%)], train_loss: 0.51439\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51230\nepoch: 001 [3552/3571 ( 99%)], train_loss: 0.51484\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.52334\nepoch: 001 [3568/3571 (100%)], train_loss: 0.51451\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.53649\nepoch: 001 [3571/3571 (100%)], train_loss: 0.51452\n----Validation Results Summary----\nEpoch: [1] valid_loss: 0.51822\nepoch: 002 [  16/3571 (  0%)], train_loss: 0.33811\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50789\nepoch: 002 [  32/3571 (  1%)], train_loss: 0.29685\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50630\nepoch: 002 [  48/3571 (  1%)], train_loss: 0.31674\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.54477\nepoch: 002 [  64/3571 (  2%)], train_loss: 0.39658\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.53001\nepoch: 002 [  80/3571 (  2%)], train_loss: 0.40135\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49418\n2 epoch, best epoch was updated! valid_loss: 0.49418\nepoch: 002 [  96/3571 (  3%)], train_loss: 0.40507\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.54826\nepoch: 002 [ 112/3571 (  3%)], train_loss: 0.41253\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.60396\nepoch: 002 [ 128/3571 (  4%)], train_loss: 0.40946\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.57227\nepoch: 002 [ 144/3571 (  4%)], train_loss: 0.41265\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50029\nepoch: 002 [ 160/3571 (  4%)], train_loss: 0.40587\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49063\n2 epoch, best epoch was updated! valid_loss: 0.49063\nepoch: 002 [ 176/3571 (  5%)], train_loss: 0.39556\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50188\nepoch: 002 [ 192/3571 (  5%)], train_loss: 0.39238\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.54091\nepoch: 002 [ 208/3571 (  6%)], train_loss: 0.39655\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.53641\nepoch: 002 [ 224/3571 (  6%)], train_loss: 0.39745\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49640\nepoch: 002 [ 240/3571 (  7%)], train_loss: 0.39644\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49950\nepoch: 002 [ 256/3571 (  7%)], train_loss: 0.39469\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.58424\nepoch: 002 [ 272/3571 (  8%)], train_loss: 0.41057\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.60600\nepoch: 002 [ 288/3571 (  8%)], train_loss: 0.40919\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.56596\nepoch: 002 [ 304/3571 (  9%)], train_loss: 0.40991\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49358\nepoch: 002 [ 320/3571 (  9%)], train_loss: 0.40976\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52842\nepoch: 002 [ 336/3571 (  9%)], train_loss: 0.40463\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.57459\nepoch: 002 [ 352/3571 ( 10%)], train_loss: 0.40758\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.53053\nepoch: 002 [ 368/3571 ( 10%)], train_loss: 0.40603\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50047\nepoch: 002 [ 384/3571 ( 11%)], train_loss: 0.40405\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51590\nepoch: 002 [ 400/3571 ( 11%)], train_loss: 0.40412\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55133\nepoch: 002 [ 416/3571 ( 12%)], train_loss: 0.40627\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55800\nepoch: 002 [ 432/3571 ( 12%)], train_loss: 0.41058\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.53491\nepoch: 002 [ 448/3571 ( 13%)], train_loss: 0.40982\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51356\nepoch: 002 [ 464/3571 ( 13%)], train_loss: 0.41180\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52642\nepoch: 002 [ 480/3571 ( 13%)], train_loss: 0.40775\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51050\nepoch: 002 [ 496/3571 ( 14%)], train_loss: 0.40499\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51518\nepoch: 002 [ 512/3571 ( 14%)], train_loss: 0.40700\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52198\nepoch: 002 [ 528/3571 ( 15%)], train_loss: 0.40625\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.54621\nepoch: 002 [ 544/3571 ( 15%)], train_loss: 0.40724\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.53525\nepoch: 002 [ 560/3571 ( 16%)], train_loss: 0.40780\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52004\nepoch: 002 [ 576/3571 ( 16%)], train_loss: 0.40374\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51755\nepoch: 002 [ 592/3571 ( 17%)], train_loss: 0.40276\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50424\nepoch: 002 [ 608/3571 ( 17%)], train_loss: 0.40061\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49333\nepoch: 002 [ 624/3571 ( 17%)], train_loss: 0.39579\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51542\nepoch: 002 [ 640/3571 ( 18%)], train_loss: 0.39521\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51022\nepoch: 002 [ 656/3571 ( 18%)], train_loss: 0.39221\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50027\nepoch: 002 [ 672/3571 ( 19%)], train_loss: 0.39439\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52896\nepoch: 002 [ 688/3571 ( 19%)], train_loss: 0.39081\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.60081\nepoch: 002 [ 704/3571 ( 20%)], train_loss: 0.39391\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.57466\nepoch: 002 [ 720/3571 ( 20%)], train_loss: 0.39344\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51291\nepoch: 002 [ 736/3571 ( 21%)], train_loss: 0.39204\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49966\nepoch: 002 [ 752/3571 ( 21%)], train_loss: 0.39058\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52484\nepoch: 002 [ 768/3571 ( 22%)], train_loss: 0.38698\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50559\nepoch: 002 [ 784/3571 ( 22%)], train_loss: 0.38756\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50374\nepoch: 002 [ 800/3571 ( 22%)], train_loss: 0.38615\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.58321\nepoch: 002 [ 816/3571 ( 23%)], train_loss: 0.38714\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.63061\nepoch: 002 [ 832/3571 ( 23%)], train_loss: 0.38776\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.58586\nepoch: 002 [ 848/3571 ( 24%)], train_loss: 0.38946\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49572\nepoch: 002 [ 864/3571 ( 24%)], train_loss: 0.38685\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.53151\nepoch: 002 [ 880/3571 ( 25%)], train_loss: 0.38990\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.57574\nepoch: 002 [ 896/3571 ( 25%)], train_loss: 0.38994\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.62235\nepoch: 002 [ 912/3571 ( 26%)], train_loss: 0.39092\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.58894\nepoch: 002 [ 928/3571 ( 26%)], train_loss: 0.39244\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52707\nepoch: 002 [ 944/3571 ( 26%)], train_loss: 0.39288\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51884\nepoch: 002 [ 960/3571 ( 27%)], train_loss: 0.39411\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.54956\nepoch: 002 [ 976/3571 ( 27%)], train_loss: 0.39304\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55882\nepoch: 002 [ 992/3571 ( 28%)], train_loss: 0.39374\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50893\nepoch: 002 [1008/3571 ( 28%)], train_loss: 0.39352\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51472\nepoch: 002 [1024/3571 ( 29%)], train_loss: 0.39178\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55714\nepoch: 002 [1040/3571 ( 29%)], train_loss: 0.39287\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.54256\nepoch: 002 [1056/3571 ( 30%)], train_loss: 0.39153\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51029\nepoch: 002 [1072/3571 ( 30%)], train_loss: 0.39027\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55553\nepoch: 002 [1088/3571 ( 30%)], train_loss: 0.39072\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.61691\nepoch: 002 [1104/3571 ( 31%)], train_loss: 0.39158\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.58193\nepoch: 002 [1120/3571 ( 31%)], train_loss: 0.39149\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51522\nepoch: 002 [1136/3571 ( 32%)], train_loss: 0.39029\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50616\nepoch: 002 [1152/3571 ( 32%)], train_loss: 0.38821\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51391\nepoch: 002 [1168/3571 ( 33%)], train_loss: 0.38689\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52459\nepoch: 002 [1184/3571 ( 33%)], train_loss: 0.38689\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50810\nepoch: 002 [1200/3571 ( 34%)], train_loss: 0.38529\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49956\nepoch: 002 [1216/3571 ( 34%)], train_loss: 0.38408\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50284\nepoch: 002 [1232/3571 ( 35%)], train_loss: 0.38292\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51521\nepoch: 002 [1248/3571 ( 35%)], train_loss: 0.38207\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51754\nepoch: 002 [1264/3571 ( 35%)], train_loss: 0.38307\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49970\nepoch: 002 [1280/3571 ( 36%)], train_loss: 0.38216\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49514\nepoch: 002 [1296/3571 ( 36%)], train_loss: 0.38315\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49690\nepoch: 002 [1312/3571 ( 37%)], train_loss: 0.38198\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49626\nepoch: 002 [1328/3571 ( 37%)], train_loss: 0.38082\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50055\nepoch: 002 [1344/3571 ( 38%)], train_loss: 0.38008\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50454\nepoch: 002 [1360/3571 ( 38%)], train_loss: 0.37834\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52591\nepoch: 002 [1376/3571 ( 39%)], train_loss: 0.37857\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55800\nepoch: 002 [1392/3571 ( 39%)], train_loss: 0.37876\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55413\nepoch: 002 [1408/3571 ( 39%)], train_loss: 0.38098\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50266\nepoch: 002 [1424/3571 ( 40%)], train_loss: 0.37991\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49716\nepoch: 002 [1440/3571 ( 40%)], train_loss: 0.37898\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52616\nepoch: 002 [1456/3571 ( 41%)], train_loss: 0.37787\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51492\nepoch: 002 [1472/3571 ( 41%)], train_loss: 0.37947\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48590\n2 epoch, best epoch was updated! valid_loss: 0.48590\nepoch: 002 [1488/3571 ( 42%)], train_loss: 0.37901\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50049\nepoch: 002 [1504/3571 ( 42%)], train_loss: 0.37926\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49497\nepoch: 002 [1520/3571 ( 43%)], train_loss: 0.37931\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50091\nepoch: 002 [1536/3571 ( 43%)], train_loss: 0.37768\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48985\nepoch: 002 [1552/3571 ( 43%)], train_loss: 0.37730\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48375\n2 epoch, best epoch was updated! valid_loss: 0.48375\nepoch: 002 [1568/3571 ( 44%)], train_loss: 0.37773\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48526\nepoch: 002 [1584/3571 ( 44%)], train_loss: 0.37893\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48677\nepoch: 002 [1600/3571 ( 45%)], train_loss: 0.37897\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48953\nepoch: 002 [1616/3571 ( 45%)], train_loss: 0.37774\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49015\nepoch: 002 [1632/3571 ( 46%)], train_loss: 0.37715\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50627\nepoch: 002 [1648/3571 ( 46%)], train_loss: 0.37653\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51323\nepoch: 002 [1664/3571 ( 47%)], train_loss: 0.37638\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50716\nepoch: 002 [1680/3571 ( 47%)], train_loss: 0.37552\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49084\nepoch: 002 [1696/3571 ( 47%)], train_loss: 0.37487\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49339\nepoch: 002 [1712/3571 ( 48%)], train_loss: 0.37383\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49431\nepoch: 002 [1728/3571 ( 48%)], train_loss: 0.37327\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48871\nepoch: 002 [1744/3571 ( 49%)], train_loss: 0.37337\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48519\nepoch: 002 [1760/3571 ( 49%)], train_loss: 0.37287\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48823\nepoch: 002 [1776/3571 ( 50%)], train_loss: 0.37189\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48663\nepoch: 002 [1792/3571 ( 50%)], train_loss: 0.37130\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50516\nepoch: 002 [1808/3571 ( 51%)], train_loss: 0.37109\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52212\nepoch: 002 [1824/3571 ( 51%)], train_loss: 0.37040\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50189\nepoch: 002 [1840/3571 ( 52%)], train_loss: 0.36893\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49929\nepoch: 002 [1856/3571 ( 52%)], train_loss: 0.36902\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50640\nepoch: 002 [1872/3571 ( 52%)], train_loss: 0.36884\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51446\nepoch: 002 [1888/3571 ( 53%)], train_loss: 0.36884\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49875\nepoch: 002 [1904/3571 ( 53%)], train_loss: 0.36700\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49951\nepoch: 002 [1920/3571 ( 54%)], train_loss: 0.36631\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50524\nepoch: 002 [1936/3571 ( 54%)], train_loss: 0.36572\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50279\nepoch: 002 [1952/3571 ( 55%)], train_loss: 0.36561\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49536\nepoch: 002 [1968/3571 ( 55%)], train_loss: 0.36481\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50119\nepoch: 002 [1984/3571 ( 56%)], train_loss: 0.36542\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49854\nepoch: 002 [2000/3571 ( 56%)], train_loss: 0.36424\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49700\nepoch: 002 [2016/3571 ( 56%)], train_loss: 0.36470\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49204\nepoch: 002 [2032/3571 ( 57%)], train_loss: 0.36356\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48983\nepoch: 002 [2048/3571 ( 57%)], train_loss: 0.36454\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49163\nepoch: 002 [2064/3571 ( 58%)], train_loss: 0.36417\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49680\nepoch: 002 [2080/3571 ( 58%)], train_loss: 0.36408\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48823\nepoch: 002 [2096/3571 ( 59%)], train_loss: 0.36461\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48643\nepoch: 002 [2112/3571 ( 59%)], train_loss: 0.36444\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49099\nepoch: 002 [2128/3571 ( 60%)], train_loss: 0.36314\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51169\nepoch: 002 [2144/3571 ( 60%)], train_loss: 0.36364\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50001\nepoch: 002 [2160/3571 ( 60%)], train_loss: 0.36397\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48495\nepoch: 002 [2176/3571 ( 61%)], train_loss: 0.36313\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48383\nepoch: 002 [2192/3571 ( 61%)], train_loss: 0.36243\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48279\n2 epoch, best epoch was updated! valid_loss: 0.48279\nepoch: 002 [2208/3571 ( 62%)], train_loss: 0.36068\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48539\nepoch: 002 [2224/3571 ( 62%)], train_loss: 0.36068\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48717\nepoch: 002 [2240/3571 ( 63%)], train_loss: 0.36241\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49490\nepoch: 002 [2256/3571 ( 63%)], train_loss: 0.36308\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50278\nepoch: 002 [2272/3571 ( 64%)], train_loss: 0.36402\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48956\nepoch: 002 [2288/3571 ( 64%)], train_loss: 0.36427\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47882\n2 epoch, best epoch was updated! valid_loss: 0.47882\nepoch: 002 [2304/3571 ( 65%)], train_loss: 0.36358\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47959\nepoch: 002 [2320/3571 ( 65%)], train_loss: 0.36319\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48813\nepoch: 002 [2336/3571 ( 65%)], train_loss: 0.36261\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50799\nepoch: 002 [2352/3571 ( 66%)], train_loss: 0.36312\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52834\nepoch: 002 [2368/3571 ( 66%)], train_loss: 0.36273\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52276\nepoch: 002 [2384/3571 ( 67%)], train_loss: 0.36267\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50996\nepoch: 002 [2400/3571 ( 67%)], train_loss: 0.36320\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51455\nepoch: 002 [2416/3571 ( 68%)], train_loss: 0.36389\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51248\nepoch: 002 [2432/3571 ( 68%)], train_loss: 0.36391\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48674\nepoch: 002 [2448/3571 ( 69%)], train_loss: 0.36514\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49322\nepoch: 002 [2464/3571 ( 69%)], train_loss: 0.36526\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52761\nepoch: 002 [2480/3571 ( 69%)], train_loss: 0.36557\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52107\nepoch: 002 [2496/3571 ( 70%)], train_loss: 0.36550\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.53806\nepoch: 002 [2512/3571 ( 70%)], train_loss: 0.36609\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.58012\nepoch: 002 [2528/3571 ( 71%)], train_loss: 0.36602\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.61968\nepoch: 002 [2544/3571 ( 71%)], train_loss: 0.36649\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.61204\nepoch: 002 [2560/3571 ( 72%)], train_loss: 0.36643\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55830\nepoch: 002 [2576/3571 ( 72%)], train_loss: 0.36615\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50168\nepoch: 002 [2592/3571 ( 73%)], train_loss: 0.36570\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49349\nepoch: 002 [2608/3571 ( 73%)], train_loss: 0.36560\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50104\nepoch: 002 [2624/3571 ( 73%)], train_loss: 0.36583\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49998\nepoch: 002 [2640/3571 ( 74%)], train_loss: 0.36551\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48770\nepoch: 002 [2656/3571 ( 74%)], train_loss: 0.36523\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49455\nepoch: 002 [2672/3571 ( 75%)], train_loss: 0.36494\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50446\nepoch: 002 [2688/3571 ( 75%)], train_loss: 0.36471\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51394\nepoch: 002 [2704/3571 ( 76%)], train_loss: 0.36404\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50779\nepoch: 002 [2720/3571 ( 76%)], train_loss: 0.36379\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49141\nepoch: 002 [2736/3571 ( 77%)], train_loss: 0.36360\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49004\nepoch: 002 [2752/3571 ( 77%)], train_loss: 0.36309\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48182\nepoch: 002 [2768/3571 ( 78%)], train_loss: 0.36301\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48800\nepoch: 002 [2784/3571 ( 78%)], train_loss: 0.36331\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49094\nepoch: 002 [2800/3571 ( 78%)], train_loss: 0.36275\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48997\nepoch: 002 [2816/3571 ( 79%)], train_loss: 0.36218\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48897\nepoch: 002 [2832/3571 ( 79%)], train_loss: 0.36136\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49347\nepoch: 002 [2848/3571 ( 80%)], train_loss: 0.36219\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49157\nepoch: 002 [2864/3571 ( 80%)], train_loss: 0.36241\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47674\n2 epoch, best epoch was updated! valid_loss: 0.47674\nepoch: 002 [2880/3571 ( 81%)], train_loss: 0.36201\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48161\nepoch: 002 [2896/3571 ( 81%)], train_loss: 0.36136\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50377\nepoch: 002 [2912/3571 ( 82%)], train_loss: 0.36077\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.54283\nepoch: 002 [2928/3571 ( 82%)], train_loss: 0.36002\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.56391\nepoch: 002 [2944/3571 ( 82%)], train_loss: 0.36130\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51792\nepoch: 002 [2960/3571 ( 83%)], train_loss: 0.36106\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50331\nepoch: 002 [2976/3571 ( 83%)], train_loss: 0.36110\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52083\nepoch: 002 [2992/3571 ( 84%)], train_loss: 0.36096\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51501\nepoch: 002 [3008/3571 ( 84%)], train_loss: 0.36107\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49080\nepoch: 002 [3024/3571 ( 85%)], train_loss: 0.36082\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50430\nepoch: 002 [3040/3571 ( 85%)], train_loss: 0.36130\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52859\nepoch: 002 [3056/3571 ( 86%)], train_loss: 0.36125\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52913\nepoch: 002 [3072/3571 ( 86%)], train_loss: 0.36136\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.53052\nepoch: 002 [3088/3571 ( 86%)], train_loss: 0.36156\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51172\nepoch: 002 [3104/3571 ( 87%)], train_loss: 0.36177\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48148\nepoch: 002 [3120/3571 ( 87%)], train_loss: 0.36133\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47115\n2 epoch, best epoch was updated! valid_loss: 0.47115\nepoch: 002 [3136/3571 ( 88%)], train_loss: 0.36069\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48345\nepoch: 002 [3152/3571 ( 88%)], train_loss: 0.36056\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50974\nepoch: 002 [3168/3571 ( 89%)], train_loss: 0.36016\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.55539\nepoch: 002 [3184/3571 ( 89%)], train_loss: 0.36002\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.57695\nepoch: 002 [3200/3571 ( 90%)], train_loss: 0.36040\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52524\nepoch: 002 [3216/3571 ( 90%)], train_loss: 0.36000\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48997\nepoch: 002 [3232/3571 ( 91%)], train_loss: 0.36002\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.51197\nepoch: 002 [3248/3571 ( 91%)], train_loss: 0.36009\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52001\nepoch: 002 [3264/3571 ( 91%)], train_loss: 0.36097\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49463\nepoch: 002 [3280/3571 ( 92%)], train_loss: 0.36072\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48714\nepoch: 002 [3296/3571 ( 92%)], train_loss: 0.36052\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49621\nepoch: 002 [3312/3571 ( 93%)], train_loss: 0.36096\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.50290\nepoch: 002 [3328/3571 ( 93%)], train_loss: 0.36187\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49516\nepoch: 002 [3344/3571 ( 94%)], train_loss: 0.36156\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48358\nepoch: 002 [3360/3571 ( 94%)], train_loss: 0.36221\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47935\nepoch: 002 [3376/3571 ( 95%)], train_loss: 0.36245\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47638\nepoch: 002 [3392/3571 ( 95%)], train_loss: 0.36237\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47029\n2 epoch, best epoch was updated! valid_loss: 0.47029\nepoch: 002 [3408/3571 ( 95%)], train_loss: 0.36203\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47029\nepoch: 002 [3424/3571 ( 96%)], train_loss: 0.36285\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49223\nepoch: 002 [3440/3571 ( 96%)], train_loss: 0.36272\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49207\nepoch: 002 [3456/3571 ( 97%)], train_loss: 0.36228\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47353\nepoch: 002 [3472/3571 ( 97%)], train_loss: 0.36220\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.47739\nepoch: 002 [3488/3571 ( 98%)], train_loss: 0.36230\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.52972\nepoch: 002 [3504/3571 ( 98%)], train_loss: 0.36220\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.59128\nepoch: 002 [3520/3571 ( 99%)], train_loss: 0.36273\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.57289\nepoch: 002 [3536/3571 ( 99%)], train_loss: 0.36317\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49589\nepoch: 002 [3552/3571 ( 99%)], train_loss: 0.36309\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48821\nepoch: 002 [3568/3571 (100%)], train_loss: 0.36301\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.49794\nepoch: 002 [3571/3571 (100%)], train_loss: 0.36301\n----Validation Results Summary----\nEpoch: [2] valid_loss: 0.48188\nepoch: 003 [  16/3571 (  0%)], train_loss: 0.53202\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47824\nepoch: 003 [  32/3571 (  1%)], train_loss: 0.37962\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47822\nepoch: 003 [  48/3571 (  1%)], train_loss: 0.35313\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47576\nepoch: 003 [  64/3571 (  2%)], train_loss: 0.33075\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47829\nepoch: 003 [  80/3571 (  2%)], train_loss: 0.29896\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.50989\nepoch: 003 [  96/3571 (  3%)], train_loss: 0.32113\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.50461\nepoch: 003 [ 112/3571 (  3%)], train_loss: 0.32125\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47224\nepoch: 003 [ 128/3571 (  4%)], train_loss: 0.30919\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48402\nepoch: 003 [ 144/3571 (  4%)], train_loss: 0.30699\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.51026\nepoch: 003 [ 160/3571 (  4%)], train_loss: 0.31061\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.51252\nepoch: 003 [ 176/3571 (  5%)], train_loss: 0.31097\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47517\nepoch: 003 [ 192/3571 (  5%)], train_loss: 0.30722\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46996\n3 epoch, best epoch was updated! valid_loss: 0.46996\nepoch: 003 [ 208/3571 (  6%)], train_loss: 0.30014\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48583\nepoch: 003 [ 224/3571 (  6%)], train_loss: 0.29559\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47606\nepoch: 003 [ 240/3571 (  7%)], train_loss: 0.29638\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47069\nepoch: 003 [ 256/3571 (  7%)], train_loss: 0.29075\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46112\n3 epoch, best epoch was updated! valid_loss: 0.46112\nepoch: 003 [ 272/3571 (  8%)], train_loss: 0.29114\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47621\nepoch: 003 [ 288/3571 (  8%)], train_loss: 0.28714\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47200\nepoch: 003 [ 304/3571 (  9%)], train_loss: 0.28794\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46178\nepoch: 003 [ 320/3571 (  9%)], train_loss: 0.28690\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47928\nepoch: 003 [ 336/3571 (  9%)], train_loss: 0.28893\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47430\nepoch: 003 [ 352/3571 ( 10%)], train_loss: 0.28390\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46311\nepoch: 003 [ 368/3571 ( 10%)], train_loss: 0.28118\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47652\nepoch: 003 [ 384/3571 ( 11%)], train_loss: 0.28430\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.49469\nepoch: 003 [ 400/3571 ( 11%)], train_loss: 0.28375\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48096\nepoch: 003 [ 416/3571 ( 12%)], train_loss: 0.28314\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47042\nepoch: 003 [ 432/3571 ( 12%)], train_loss: 0.28024\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47641\nepoch: 003 [ 448/3571 ( 13%)], train_loss: 0.28095\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.49248\nepoch: 003 [ 464/3571 ( 13%)], train_loss: 0.28422\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.50097\nepoch: 003 [ 480/3571 ( 13%)], train_loss: 0.28436\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47983\nepoch: 003 [ 496/3571 ( 14%)], train_loss: 0.28069\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47504\nepoch: 003 [ 512/3571 ( 14%)], train_loss: 0.28002\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47272\nepoch: 003 [ 528/3571 ( 15%)], train_loss: 0.27974\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46955\nepoch: 003 [ 544/3571 ( 15%)], train_loss: 0.27748\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46902\nepoch: 003 [ 560/3571 ( 16%)], train_loss: 0.28016\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46972\nepoch: 003 [ 576/3571 ( 16%)], train_loss: 0.27724\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47736\nepoch: 003 [ 592/3571 ( 17%)], train_loss: 0.27736\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47419\nepoch: 003 [ 608/3571 ( 17%)], train_loss: 0.27986\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47013\nepoch: 003 [ 624/3571 ( 17%)], train_loss: 0.27852\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47009\nepoch: 003 [ 640/3571 ( 18%)], train_loss: 0.27636\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47851\nepoch: 003 [ 656/3571 ( 18%)], train_loss: 0.27675\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48106\nepoch: 003 [ 672/3571 ( 19%)], train_loss: 0.27634\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47055\nepoch: 003 [ 688/3571 ( 19%)], train_loss: 0.27310\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46800\nepoch: 003 [ 704/3571 ( 20%)], train_loss: 0.27420\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46923\nepoch: 003 [ 720/3571 ( 20%)], train_loss: 0.27314\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47034\nepoch: 003 [ 736/3571 ( 21%)], train_loss: 0.27398\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47267\nepoch: 003 [ 752/3571 ( 21%)], train_loss: 0.27278\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47220\nepoch: 003 [ 768/3571 ( 22%)], train_loss: 0.27384\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46867\nepoch: 003 [ 784/3571 ( 22%)], train_loss: 0.27282\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46781\nepoch: 003 [ 800/3571 ( 22%)], train_loss: 0.27056\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48130\nepoch: 003 [ 816/3571 ( 23%)], train_loss: 0.27036\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48078\nepoch: 003 [ 832/3571 ( 23%)], train_loss: 0.27355\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48035\nepoch: 003 [ 848/3571 ( 24%)], train_loss: 0.27339\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46895\nepoch: 003 [ 864/3571 ( 24%)], train_loss: 0.27213\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46229\nepoch: 003 [ 880/3571 ( 25%)], train_loss: 0.27133\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46418\nepoch: 003 [ 896/3571 ( 25%)], train_loss: 0.27037\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46831\nepoch: 003 [ 912/3571 ( 26%)], train_loss: 0.26988\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46380\nepoch: 003 [ 928/3571 ( 26%)], train_loss: 0.26839\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46043\n3 epoch, best epoch was updated! valid_loss: 0.46043\nepoch: 003 [ 944/3571 ( 26%)], train_loss: 0.26937\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46174\nepoch: 003 [ 960/3571 ( 27%)], train_loss: 0.26754\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47007\nepoch: 003 [ 976/3571 ( 27%)], train_loss: 0.26593\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47972\nepoch: 003 [ 992/3571 ( 28%)], train_loss: 0.26583\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46389\nepoch: 003 [1008/3571 ( 28%)], train_loss: 0.26379\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46354\nepoch: 003 [1024/3571 ( 29%)], train_loss: 0.26381\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46212\nepoch: 003 [1040/3571 ( 29%)], train_loss: 0.26297\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46064\nepoch: 003 [1056/3571 ( 30%)], train_loss: 0.26243\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45955\n3 epoch, best epoch was updated! valid_loss: 0.45955\nepoch: 003 [1072/3571 ( 30%)], train_loss: 0.26275\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45969\nepoch: 003 [1088/3571 ( 30%)], train_loss: 0.26208\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46106\nepoch: 003 [1104/3571 ( 31%)], train_loss: 0.26216\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45816\n3 epoch, best epoch was updated! valid_loss: 0.45816\nepoch: 003 [1120/3571 ( 31%)], train_loss: 0.26097\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46442\nepoch: 003 [1136/3571 ( 32%)], train_loss: 0.26052\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45760\n3 epoch, best epoch was updated! valid_loss: 0.45760\nepoch: 003 [1152/3571 ( 32%)], train_loss: 0.25896\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45955\nepoch: 003 [1168/3571 ( 33%)], train_loss: 0.25821\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46901\nepoch: 003 [1184/3571 ( 33%)], train_loss: 0.25715\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47746\nepoch: 003 [1200/3571 ( 34%)], train_loss: 0.25745\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47928\nepoch: 003 [1216/3571 ( 34%)], train_loss: 0.25680\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.49387\nepoch: 003 [1232/3571 ( 35%)], train_loss: 0.25893\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.49286\nepoch: 003 [1248/3571 ( 35%)], train_loss: 0.25909\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48613\nepoch: 003 [1264/3571 ( 35%)], train_loss: 0.25934\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48276\nepoch: 003 [1280/3571 ( 36%)], train_loss: 0.26082\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.49925\nepoch: 003 [1296/3571 ( 36%)], train_loss: 0.26087\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48312\nepoch: 003 [1312/3571 ( 37%)], train_loss: 0.26093\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45758\n3 epoch, best epoch was updated! valid_loss: 0.45758\nepoch: 003 [1328/3571 ( 37%)], train_loss: 0.26035\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47742\nepoch: 003 [1344/3571 ( 38%)], train_loss: 0.26250\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.50763\nepoch: 003 [1360/3571 ( 38%)], train_loss: 0.26360\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48352\nepoch: 003 [1376/3571 ( 39%)], train_loss: 0.26351\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46356\nepoch: 003 [1392/3571 ( 39%)], train_loss: 0.26318\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46956\nepoch: 003 [1408/3571 ( 39%)], train_loss: 0.26202\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48173\nepoch: 003 [1424/3571 ( 40%)], train_loss: 0.26267\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46403\nepoch: 003 [1440/3571 ( 40%)], train_loss: 0.26194\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46012\nepoch: 003 [1456/3571 ( 41%)], train_loss: 0.26245\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46249\nepoch: 003 [1472/3571 ( 41%)], train_loss: 0.26269\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46302\nepoch: 003 [1488/3571 ( 42%)], train_loss: 0.26305\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46638\nepoch: 003 [1504/3571 ( 42%)], train_loss: 0.26235\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46909\nepoch: 003 [1520/3571 ( 43%)], train_loss: 0.26200\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47055\nepoch: 003 [1536/3571 ( 43%)], train_loss: 0.26099\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46492\nepoch: 003 [1552/3571 ( 43%)], train_loss: 0.26096\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45944\nepoch: 003 [1568/3571 ( 44%)], train_loss: 0.26113\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46040\nepoch: 003 [1584/3571 ( 44%)], train_loss: 0.26102\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46788\nepoch: 003 [1600/3571 ( 45%)], train_loss: 0.26057\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47832\nepoch: 003 [1616/3571 ( 45%)], train_loss: 0.25926\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46400\nepoch: 003 [1632/3571 ( 46%)], train_loss: 0.25889\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46058\nepoch: 003 [1648/3571 ( 46%)], train_loss: 0.25915\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47970\nepoch: 003 [1664/3571 ( 47%)], train_loss: 0.25965\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46893\nepoch: 003 [1680/3571 ( 47%)], train_loss: 0.25968\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46663\nepoch: 003 [1696/3571 ( 47%)], train_loss: 0.25924\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48991\nepoch: 003 [1712/3571 ( 48%)], train_loss: 0.26031\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48661\nepoch: 003 [1728/3571 ( 48%)], train_loss: 0.26170\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46240\nepoch: 003 [1744/3571 ( 49%)], train_loss: 0.26155\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47988\nepoch: 003 [1760/3571 ( 49%)], train_loss: 0.26077\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.52669\nepoch: 003 [1776/3571 ( 50%)], train_loss: 0.26258\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.53275\nepoch: 003 [1792/3571 ( 50%)], train_loss: 0.26366\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.49163\nepoch: 003 [1808/3571 ( 51%)], train_loss: 0.26345\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45974\nepoch: 003 [1824/3571 ( 51%)], train_loss: 0.26350\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.52297\nepoch: 003 [1840/3571 ( 52%)], train_loss: 0.26414\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.56151\nepoch: 003 [1856/3571 ( 52%)], train_loss: 0.26474\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.53011\nepoch: 003 [1872/3571 ( 52%)], train_loss: 0.26486\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46952\nepoch: 003 [1888/3571 ( 53%)], train_loss: 0.26522\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46139\nepoch: 003 [1904/3571 ( 53%)], train_loss: 0.26489\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.50775\nepoch: 003 [1920/3571 ( 54%)], train_loss: 0.26564\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.51475\nepoch: 003 [1936/3571 ( 54%)], train_loss: 0.26610\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47326\nepoch: 003 [1952/3571 ( 55%)], train_loss: 0.26624\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45495\n3 epoch, best epoch was updated! valid_loss: 0.45495\nepoch: 003 [1968/3571 ( 55%)], train_loss: 0.26528\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48530\nepoch: 003 [1984/3571 ( 56%)], train_loss: 0.26487\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.50887\nepoch: 003 [2000/3571 ( 56%)], train_loss: 0.26485\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.49765\nepoch: 003 [2016/3571 ( 56%)], train_loss: 0.26473\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46945\nepoch: 003 [2032/3571 ( 57%)], train_loss: 0.26421\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46432\nepoch: 003 [2048/3571 ( 57%)], train_loss: 0.26429\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47380\nepoch: 003 [2064/3571 ( 58%)], train_loss: 0.26438\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47962\nepoch: 003 [2080/3571 ( 58%)], train_loss: 0.26469\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47384\nepoch: 003 [2096/3571 ( 59%)], train_loss: 0.26422\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45788\nepoch: 003 [2112/3571 ( 59%)], train_loss: 0.26383\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45665\nepoch: 003 [2128/3571 ( 60%)], train_loss: 0.26400\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47699\nepoch: 003 [2144/3571 ( 60%)], train_loss: 0.26388\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46972\nepoch: 003 [2160/3571 ( 60%)], train_loss: 0.26406\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45959\nepoch: 003 [2176/3571 ( 61%)], train_loss: 0.26352\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45626\nepoch: 003 [2192/3571 ( 61%)], train_loss: 0.26376\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45911\nepoch: 003 [2208/3571 ( 62%)], train_loss: 0.26447\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45519\nepoch: 003 [2224/3571 ( 62%)], train_loss: 0.26346\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45232\n3 epoch, best epoch was updated! valid_loss: 0.45232\nepoch: 003 [2240/3571 ( 63%)], train_loss: 0.26273\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45647\nepoch: 003 [2256/3571 ( 63%)], train_loss: 0.26256\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46345\nepoch: 003 [2272/3571 ( 64%)], train_loss: 0.26223\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45752\nepoch: 003 [2288/3571 ( 64%)], train_loss: 0.26165\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46678\nepoch: 003 [2304/3571 ( 65%)], train_loss: 0.26161\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48539\nepoch: 003 [2320/3571 ( 65%)], train_loss: 0.26152\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48448\nepoch: 003 [2336/3571 ( 65%)], train_loss: 0.26137\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46084\nepoch: 003 [2352/3571 ( 66%)], train_loss: 0.26104\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45318\nepoch: 003 [2368/3571 ( 66%)], train_loss: 0.26073\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45316\nepoch: 003 [2384/3571 ( 67%)], train_loss: 0.26008\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45524\nepoch: 003 [2400/3571 ( 67%)], train_loss: 0.26003\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46086\nepoch: 003 [2416/3571 ( 68%)], train_loss: 0.25985\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46899\nepoch: 003 [2432/3571 ( 68%)], train_loss: 0.25925\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47477\nepoch: 003 [2448/3571 ( 69%)], train_loss: 0.25936\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47666\nepoch: 003 [2464/3571 ( 69%)], train_loss: 0.25896\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47479\nepoch: 003 [2480/3571 ( 69%)], train_loss: 0.25894\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46524\nepoch: 003 [2496/3571 ( 70%)], train_loss: 0.25847\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45884\nepoch: 003 [2512/3571 ( 70%)], train_loss: 0.25853\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46019\nepoch: 003 [2528/3571 ( 71%)], train_loss: 0.25824\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45384\nepoch: 003 [2544/3571 ( 71%)], train_loss: 0.25833\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47134\nepoch: 003 [2560/3571 ( 72%)], train_loss: 0.25899\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48250\nepoch: 003 [2576/3571 ( 72%)], train_loss: 0.25901\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46593\nepoch: 003 [2592/3571 ( 73%)], train_loss: 0.25961\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45749\nepoch: 003 [2608/3571 ( 73%)], train_loss: 0.26093\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46439\nepoch: 003 [2624/3571 ( 73%)], train_loss: 0.26091\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46217\nepoch: 003 [2640/3571 ( 74%)], train_loss: 0.26102\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46065\nepoch: 003 [2656/3571 ( 74%)], train_loss: 0.26077\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46138\nepoch: 003 [2672/3571 ( 75%)], train_loss: 0.26041\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47641\nepoch: 003 [2688/3571 ( 75%)], train_loss: 0.26083\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48795\nepoch: 003 [2704/3571 ( 76%)], train_loss: 0.26087\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47319\nepoch: 003 [2720/3571 ( 76%)], train_loss: 0.26093\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47082\nepoch: 003 [2736/3571 ( 77%)], train_loss: 0.26085\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46556\nepoch: 003 [2752/3571 ( 77%)], train_loss: 0.26122\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46166\nepoch: 003 [2768/3571 ( 78%)], train_loss: 0.26126\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45834\nepoch: 003 [2784/3571 ( 78%)], train_loss: 0.26101\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45958\nepoch: 003 [2800/3571 ( 78%)], train_loss: 0.26069\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46004\nepoch: 003 [2816/3571 ( 79%)], train_loss: 0.26039\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46021\nepoch: 003 [2832/3571 ( 79%)], train_loss: 0.25975\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46115\nepoch: 003 [2848/3571 ( 80%)], train_loss: 0.26000\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46315\nepoch: 003 [2864/3571 ( 80%)], train_loss: 0.25975\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46322\nepoch: 003 [2880/3571 ( 81%)], train_loss: 0.26040\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46329\nepoch: 003 [2896/3571 ( 81%)], train_loss: 0.26004\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46587\nepoch: 003 [2912/3571 ( 82%)], train_loss: 0.25987\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46568\nepoch: 003 [2928/3571 ( 82%)], train_loss: 0.25936\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46513\nepoch: 003 [2944/3571 ( 82%)], train_loss: 0.25958\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46675\nepoch: 003 [2960/3571 ( 83%)], train_loss: 0.25953\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46070\nepoch: 003 [2976/3571 ( 83%)], train_loss: 0.25937\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45508\nepoch: 003 [2992/3571 ( 84%)], train_loss: 0.25932\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45390\nepoch: 003 [3008/3571 ( 84%)], train_loss: 0.25933\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45108\n3 epoch, best epoch was updated! valid_loss: 0.45108\nepoch: 003 [3024/3571 ( 85%)], train_loss: 0.25914\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45185\nepoch: 003 [3040/3571 ( 85%)], train_loss: 0.25922\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45002\n3 epoch, best epoch was updated! valid_loss: 0.45002\nepoch: 003 [3056/3571 ( 86%)], train_loss: 0.25913\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45078\nepoch: 003 [3072/3571 ( 86%)], train_loss: 0.25910\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45398\nepoch: 003 [3088/3571 ( 86%)], train_loss: 0.25869\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46362\nepoch: 003 [3104/3571 ( 87%)], train_loss: 0.25860\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46574\nepoch: 003 [3120/3571 ( 87%)], train_loss: 0.25860\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45384\nepoch: 003 [3136/3571 ( 88%)], train_loss: 0.25849\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45211\nepoch: 003 [3152/3571 ( 88%)], train_loss: 0.25843\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45333\nepoch: 003 [3168/3571 ( 89%)], train_loss: 0.25811\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45748\nepoch: 003 [3184/3571 ( 89%)], train_loss: 0.25849\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45852\nepoch: 003 [3200/3571 ( 90%)], train_loss: 0.25835\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46118\nepoch: 003 [3216/3571 ( 90%)], train_loss: 0.25847\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48125\nepoch: 003 [3232/3571 ( 91%)], train_loss: 0.25850\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48677\nepoch: 003 [3248/3571 ( 91%)], train_loss: 0.25896\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47907\nepoch: 003 [3264/3571 ( 91%)], train_loss: 0.25872\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46451\nepoch: 003 [3280/3571 ( 92%)], train_loss: 0.25844\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45499\nepoch: 003 [3296/3571 ( 92%)], train_loss: 0.25858\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45375\nepoch: 003 [3312/3571 ( 93%)], train_loss: 0.25874\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46917\nepoch: 003 [3328/3571 ( 93%)], train_loss: 0.25882\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46824\nepoch: 003 [3344/3571 ( 94%)], train_loss: 0.25863\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45725\nepoch: 003 [3360/3571 ( 94%)], train_loss: 0.25862\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45913\nepoch: 003 [3376/3571 ( 95%)], train_loss: 0.25884\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45906\nepoch: 003 [3392/3571 ( 95%)], train_loss: 0.25860\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45868\nepoch: 003 [3408/3571 ( 95%)], train_loss: 0.25832\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47176\nepoch: 003 [3424/3571 ( 96%)], train_loss: 0.25849\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.48108\nepoch: 003 [3440/3571 ( 96%)], train_loss: 0.25871\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45931\nepoch: 003 [3456/3571 ( 97%)], train_loss: 0.25895\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45626\nepoch: 003 [3472/3571 ( 97%)], train_loss: 0.25891\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.47425\nepoch: 003 [3488/3571 ( 98%)], train_loss: 0.25873\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46902\nepoch: 003 [3504/3571 ( 98%)], train_loss: 0.25923\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46192\nepoch: 003 [3520/3571 ( 99%)], train_loss: 0.25943\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45393\nepoch: 003 [3536/3571 ( 99%)], train_loss: 0.25917\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46281\nepoch: 003 [3552/3571 ( 99%)], train_loss: 0.25872\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.46274\nepoch: 003 [3568/3571 (100%)], train_loss: 0.25883\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45759\nepoch: 003 [3571/3571 (100%)], train_loss: 0.25884\n----Validation Results Summary----\nEpoch: [3] valid_loss: 0.45424\nepoch: 004 [  16/3571 (  0%)], train_loss: 0.45703\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45235\nepoch: 004 [  32/3571 (  1%)], train_loss: 0.28774\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45443\nepoch: 004 [  48/3571 (  1%)], train_loss: 0.24092\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45460\nepoch: 004 [  64/3571 (  2%)], train_loss: 0.21354\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45429\nepoch: 004 [  80/3571 (  2%)], train_loss: 0.21384\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45634\nepoch: 004 [  96/3571 (  3%)], train_loss: 0.20409\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45647\nepoch: 004 [ 112/3571 (  3%)], train_loss: 0.19718\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45754\nepoch: 004 [ 128/3571 (  4%)], train_loss: 0.19144\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45623\nepoch: 004 [ 144/3571 (  4%)], train_loss: 0.19519\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45467\nepoch: 004 [ 160/3571 (  4%)], train_loss: 0.19083\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45339\nepoch: 004 [ 176/3571 (  5%)], train_loss: 0.19109\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45297\nepoch: 004 [ 192/3571 (  5%)], train_loss: 0.19036\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45557\nepoch: 004 [ 208/3571 (  6%)], train_loss: 0.19329\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45156\nepoch: 004 [ 224/3571 (  6%)], train_loss: 0.18840\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45192\nepoch: 004 [ 240/3571 (  7%)], train_loss: 0.18314\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45257\nepoch: 004 [ 256/3571 (  7%)], train_loss: 0.17825\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45467\nepoch: 004 [ 272/3571 (  8%)], train_loss: 0.17820\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45718\nepoch: 004 [ 288/3571 (  8%)], train_loss: 0.17426\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45786\nepoch: 004 [ 304/3571 (  9%)], train_loss: 0.17252\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45981\nepoch: 004 [ 320/3571 (  9%)], train_loss: 0.17059\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46303\nepoch: 004 [ 336/3571 (  9%)], train_loss: 0.17214\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46723\nepoch: 004 [ 352/3571 ( 10%)], train_loss: 0.17661\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46634\nepoch: 004 [ 368/3571 ( 10%)], train_loss: 0.17685\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45534\nepoch: 004 [ 384/3571 ( 11%)], train_loss: 0.17721\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45108\nepoch: 004 [ 400/3571 ( 11%)], train_loss: 0.17456\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46739\nepoch: 004 [ 416/3571 ( 12%)], train_loss: 0.17615\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46315\nepoch: 004 [ 432/3571 ( 12%)], train_loss: 0.17729\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44893\n4 epoch, best epoch was updated! valid_loss: 0.44893\nepoch: 004 [ 448/3571 ( 13%)], train_loss: 0.18134\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45020\nepoch: 004 [ 464/3571 ( 13%)], train_loss: 0.18014\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45020\nepoch: 004 [ 480/3571 ( 13%)], train_loss: 0.18053\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45318\nepoch: 004 [ 496/3571 ( 14%)], train_loss: 0.18243\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46192\nepoch: 004 [ 512/3571 ( 14%)], train_loss: 0.18347\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45780\nepoch: 004 [ 528/3571 ( 15%)], train_loss: 0.18383\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45200\nepoch: 004 [ 544/3571 ( 15%)], train_loss: 0.18448\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46033\nepoch: 004 [ 560/3571 ( 16%)], train_loss: 0.18502\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45755\nepoch: 004 [ 576/3571 ( 16%)], train_loss: 0.18614\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45077\nepoch: 004 [ 592/3571 ( 17%)], train_loss: 0.18646\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45070\nepoch: 004 [ 608/3571 ( 17%)], train_loss: 0.18587\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46173\nepoch: 004 [ 624/3571 ( 17%)], train_loss: 0.18709\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47219\nepoch: 004 [ 640/3571 ( 18%)], train_loss: 0.18799\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46564\nepoch: 004 [ 656/3571 ( 18%)], train_loss: 0.18873\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45345\nepoch: 004 [ 672/3571 ( 19%)], train_loss: 0.18760\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45755\nepoch: 004 [ 688/3571 ( 19%)], train_loss: 0.18671\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46375\nepoch: 004 [ 704/3571 ( 20%)], train_loss: 0.18770\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46172\nepoch: 004 [ 720/3571 ( 20%)], train_loss: 0.18857\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46161\nepoch: 004 [ 736/3571 ( 21%)], train_loss: 0.18950\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45748\nepoch: 004 [ 752/3571 ( 21%)], train_loss: 0.18965\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45856\nepoch: 004 [ 768/3571 ( 22%)], train_loss: 0.19019\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46366\nepoch: 004 [ 784/3571 ( 22%)], train_loss: 0.18966\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46107\nepoch: 004 [ 800/3571 ( 22%)], train_loss: 0.18851\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45503\nepoch: 004 [ 816/3571 ( 23%)], train_loss: 0.18676\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45400\nepoch: 004 [ 832/3571 ( 23%)], train_loss: 0.18560\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46047\nepoch: 004 [ 848/3571 ( 24%)], train_loss: 0.18595\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45876\nepoch: 004 [ 864/3571 ( 24%)], train_loss: 0.18633\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45053\nepoch: 004 [ 880/3571 ( 25%)], train_loss: 0.18657\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45142\nepoch: 004 [ 896/3571 ( 25%)], train_loss: 0.18646\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45598\nepoch: 004 [ 912/3571 ( 26%)], train_loss: 0.18560\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45751\nepoch: 004 [ 928/3571 ( 26%)], train_loss: 0.18743\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45073\nepoch: 004 [ 944/3571 ( 26%)], train_loss: 0.18828\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45529\nepoch: 004 [ 960/3571 ( 27%)], train_loss: 0.18848\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45345\nepoch: 004 [ 976/3571 ( 27%)], train_loss: 0.18918\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44846\n4 epoch, best epoch was updated! valid_loss: 0.44846\nepoch: 004 [ 992/3571 ( 28%)], train_loss: 0.18889\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46145\nepoch: 004 [1008/3571 ( 28%)], train_loss: 0.18924\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47062\nepoch: 004 [1024/3571 ( 29%)], train_loss: 0.18929\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46181\nepoch: 004 [1040/3571 ( 29%)], train_loss: 0.18863\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45350\nepoch: 004 [1056/3571 ( 30%)], train_loss: 0.18877\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47275\nepoch: 004 [1072/3571 ( 30%)], train_loss: 0.19065\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.49335\nepoch: 004 [1088/3571 ( 30%)], train_loss: 0.19194\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.48737\nepoch: 004 [1104/3571 ( 31%)], train_loss: 0.19234\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46339\nepoch: 004 [1120/3571 ( 31%)], train_loss: 0.19214\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45557\nepoch: 004 [1136/3571 ( 32%)], train_loss: 0.19166\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46102\nepoch: 004 [1152/3571 ( 32%)], train_loss: 0.19190\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46848\nepoch: 004 [1168/3571 ( 33%)], train_loss: 0.19226\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45870\nepoch: 004 [1184/3571 ( 33%)], train_loss: 0.19287\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44754\n4 epoch, best epoch was updated! valid_loss: 0.44754\nepoch: 004 [1200/3571 ( 34%)], train_loss: 0.19149\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44835\nepoch: 004 [1216/3571 ( 34%)], train_loss: 0.19228\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44891\nepoch: 004 [1232/3571 ( 35%)], train_loss: 0.19161\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44898\nepoch: 004 [1248/3571 ( 35%)], train_loss: 0.19152\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44714\n4 epoch, best epoch was updated! valid_loss: 0.44714\nepoch: 004 [1264/3571 ( 35%)], train_loss: 0.19115\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45133\nepoch: 004 [1280/3571 ( 36%)], train_loss: 0.19078\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44847\nepoch: 004 [1296/3571 ( 36%)], train_loss: 0.19057\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44745\nepoch: 004 [1312/3571 ( 37%)], train_loss: 0.19019\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45175\nepoch: 004 [1328/3571 ( 37%)], train_loss: 0.18986\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45621\nepoch: 004 [1344/3571 ( 38%)], train_loss: 0.18985\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45856\nepoch: 004 [1360/3571 ( 38%)], train_loss: 0.18956\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47068\nepoch: 004 [1376/3571 ( 39%)], train_loss: 0.18927\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.48360\nepoch: 004 [1392/3571 ( 39%)], train_loss: 0.18916\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.48554\nepoch: 004 [1408/3571 ( 39%)], train_loss: 0.19027\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47257\nepoch: 004 [1424/3571 ( 40%)], train_loss: 0.19039\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45057\nepoch: 004 [1440/3571 ( 40%)], train_loss: 0.19095\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45726\nepoch: 004 [1456/3571 ( 41%)], train_loss: 0.19032\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47642\nepoch: 004 [1472/3571 ( 41%)], train_loss: 0.19060\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47377\nepoch: 004 [1488/3571 ( 42%)], train_loss: 0.19115\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45222\nepoch: 004 [1504/3571 ( 42%)], train_loss: 0.19098\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45179\nepoch: 004 [1520/3571 ( 43%)], train_loss: 0.19134\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47168\nepoch: 004 [1536/3571 ( 43%)], train_loss: 0.19203\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47770\nepoch: 004 [1552/3571 ( 43%)], train_loss: 0.19269\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45716\nepoch: 004 [1568/3571 ( 44%)], train_loss: 0.19252\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44075\n4 epoch, best epoch was updated! valid_loss: 0.44075\nepoch: 004 [1584/3571 ( 44%)], train_loss: 0.19203\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44620\nepoch: 004 [1600/3571 ( 45%)], train_loss: 0.19174\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45620\nepoch: 004 [1616/3571 ( 45%)], train_loss: 0.19148\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45529\nepoch: 004 [1632/3571 ( 46%)], train_loss: 0.19111\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45622\nepoch: 004 [1648/3571 ( 46%)], train_loss: 0.19119\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45943\nepoch: 004 [1664/3571 ( 47%)], train_loss: 0.19049\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46370\nepoch: 004 [1680/3571 ( 47%)], train_loss: 0.18998\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46220\nepoch: 004 [1696/3571 ( 47%)], train_loss: 0.19057\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45474\nepoch: 004 [1712/3571 ( 48%)], train_loss: 0.19037\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44835\nepoch: 004 [1728/3571 ( 48%)], train_loss: 0.19038\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44378\nepoch: 004 [1744/3571 ( 49%)], train_loss: 0.18997\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44523\nepoch: 004 [1760/3571 ( 49%)], train_loss: 0.18956\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45465\nepoch: 004 [1776/3571 ( 50%)], train_loss: 0.19007\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45669\nepoch: 004 [1792/3571 ( 50%)], train_loss: 0.18969\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44912\nepoch: 004 [1808/3571 ( 51%)], train_loss: 0.19001\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44378\nepoch: 004 [1824/3571 ( 51%)], train_loss: 0.19017\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44257\nepoch: 004 [1840/3571 ( 52%)], train_loss: 0.18991\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44261\nepoch: 004 [1856/3571 ( 52%)], train_loss: 0.19036\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44874\nepoch: 004 [1872/3571 ( 52%)], train_loss: 0.18954\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46313\nepoch: 004 [1888/3571 ( 53%)], train_loss: 0.18957\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46266\nepoch: 004 [1904/3571 ( 53%)], train_loss: 0.18966\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45245\nepoch: 004 [1920/3571 ( 54%)], train_loss: 0.18925\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44928\nepoch: 004 [1936/3571 ( 54%)], train_loss: 0.18955\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44775\nepoch: 004 [1952/3571 ( 55%)], train_loss: 0.18956\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44773\nepoch: 004 [1968/3571 ( 55%)], train_loss: 0.18950\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44769\nepoch: 004 [1984/3571 ( 56%)], train_loss: 0.18932\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44831\nepoch: 004 [2000/3571 ( 56%)], train_loss: 0.18875\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44841\nepoch: 004 [2016/3571 ( 56%)], train_loss: 0.18841\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44538\nepoch: 004 [2032/3571 ( 57%)], train_loss: 0.18838\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44535\nepoch: 004 [2048/3571 ( 57%)], train_loss: 0.18800\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44479\nepoch: 004 [2064/3571 ( 58%)], train_loss: 0.18783\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44435\nepoch: 004 [2080/3571 ( 58%)], train_loss: 0.18813\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44560\nepoch: 004 [2096/3571 ( 59%)], train_loss: 0.18742\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44495\nepoch: 004 [2112/3571 ( 59%)], train_loss: 0.18770\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44274\nepoch: 004 [2128/3571 ( 60%)], train_loss: 0.18757\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44435\nepoch: 004 [2144/3571 ( 60%)], train_loss: 0.18758\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45134\nepoch: 004 [2160/3571 ( 60%)], train_loss: 0.18800\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44848\nepoch: 004 [2176/3571 ( 61%)], train_loss: 0.18772\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44317\nepoch: 004 [2192/3571 ( 61%)], train_loss: 0.18744\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44165\nepoch: 004 [2208/3571 ( 62%)], train_loss: 0.18727\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44034\n4 epoch, best epoch was updated! valid_loss: 0.44034\nepoch: 004 [2224/3571 ( 62%)], train_loss: 0.18705\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43999\n4 epoch, best epoch was updated! valid_loss: 0.43999\nepoch: 004 [2240/3571 ( 63%)], train_loss: 0.18683\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44300\nepoch: 004 [2256/3571 ( 63%)], train_loss: 0.18653\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44575\nepoch: 004 [2272/3571 ( 64%)], train_loss: 0.18643\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44954\nepoch: 004 [2288/3571 ( 64%)], train_loss: 0.18631\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45153\nepoch: 004 [2304/3571 ( 65%)], train_loss: 0.18644\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45059\nepoch: 004 [2320/3571 ( 65%)], train_loss: 0.18644\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44785\nepoch: 004 [2336/3571 ( 65%)], train_loss: 0.18632\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44505\nepoch: 004 [2352/3571 ( 66%)], train_loss: 0.18621\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44334\nepoch: 004 [2368/3571 ( 66%)], train_loss: 0.18611\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44130\nepoch: 004 [2384/3571 ( 67%)], train_loss: 0.18609\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44497\nepoch: 004 [2400/3571 ( 67%)], train_loss: 0.18625\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44445\nepoch: 004 [2416/3571 ( 68%)], train_loss: 0.18598\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44581\nepoch: 004 [2432/3571 ( 68%)], train_loss: 0.18561\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44325\nepoch: 004 [2448/3571 ( 69%)], train_loss: 0.18530\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44302\nepoch: 004 [2464/3571 ( 69%)], train_loss: 0.18602\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44207\nepoch: 004 [2480/3571 ( 69%)], train_loss: 0.18631\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44327\nepoch: 004 [2496/3571 ( 70%)], train_loss: 0.18594\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44503\nepoch: 004 [2512/3571 ( 70%)], train_loss: 0.18564\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44603\nepoch: 004 [2528/3571 ( 71%)], train_loss: 0.18531\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44388\nepoch: 004 [2544/3571 ( 71%)], train_loss: 0.18501\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44267\nepoch: 004 [2560/3571 ( 72%)], train_loss: 0.18547\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44657\nepoch: 004 [2576/3571 ( 72%)], train_loss: 0.18508\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44535\nepoch: 004 [2592/3571 ( 73%)], train_loss: 0.18498\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44276\nepoch: 004 [2608/3571 ( 73%)], train_loss: 0.18496\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45043\nepoch: 004 [2624/3571 ( 73%)], train_loss: 0.18477\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45068\nepoch: 004 [2640/3571 ( 74%)], train_loss: 0.18472\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44785\nepoch: 004 [2656/3571 ( 74%)], train_loss: 0.18434\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44921\nepoch: 004 [2672/3571 ( 75%)], train_loss: 0.18457\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46373\nepoch: 004 [2688/3571 ( 75%)], train_loss: 0.18482\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.47029\nepoch: 004 [2704/3571 ( 76%)], train_loss: 0.18506\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46129\nepoch: 004 [2720/3571 ( 76%)], train_loss: 0.18567\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44593\nepoch: 004 [2736/3571 ( 77%)], train_loss: 0.18546\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45225\nepoch: 004 [2752/3571 ( 77%)], train_loss: 0.18571\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.46211\nepoch: 004 [2768/3571 ( 78%)], train_loss: 0.18591\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.45606\nepoch: 004 [2784/3571 ( 78%)], train_loss: 0.18591\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44483\nepoch: 004 [2800/3571 ( 78%)], train_loss: 0.18577\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44071\nepoch: 004 [2816/3571 ( 79%)], train_loss: 0.18569\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44810\nepoch: 004 [2832/3571 ( 79%)], train_loss: 0.18566\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44682\nepoch: 004 [2848/3571 ( 80%)], train_loss: 0.18579\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44303\nepoch: 004 [2864/3571 ( 80%)], train_loss: 0.18586\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44204\nepoch: 004 [2880/3571 ( 81%)], train_loss: 0.18582\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44146\nepoch: 004 [2896/3571 ( 81%)], train_loss: 0.18575\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44397\nepoch: 004 [2912/3571 ( 82%)], train_loss: 0.18575\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44600\nepoch: 004 [2928/3571 ( 82%)], train_loss: 0.18554\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44532\nepoch: 004 [2944/3571 ( 82%)], train_loss: 0.18521\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44291\nepoch: 004 [2960/3571 ( 83%)], train_loss: 0.18505\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44286\nepoch: 004 [2976/3571 ( 83%)], train_loss: 0.18514\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44344\nepoch: 004 [2992/3571 ( 84%)], train_loss: 0.18518\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44320\nepoch: 004 [3008/3571 ( 84%)], train_loss: 0.18527\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44334\nepoch: 004 [3024/3571 ( 85%)], train_loss: 0.18525\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44279\nepoch: 004 [3040/3571 ( 85%)], train_loss: 0.18526\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44622\nepoch: 004 [3056/3571 ( 86%)], train_loss: 0.18503\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44745\nepoch: 004 [3072/3571 ( 86%)], train_loss: 0.18505\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44791\nepoch: 004 [3088/3571 ( 86%)], train_loss: 0.18487\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44193\nepoch: 004 [3104/3571 ( 87%)], train_loss: 0.18467\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44025\nepoch: 004 [3120/3571 ( 87%)], train_loss: 0.18521\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44032\nepoch: 004 [3136/3571 ( 88%)], train_loss: 0.18523\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44345\nepoch: 004 [3152/3571 ( 88%)], train_loss: 0.18509\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44433\nepoch: 004 [3168/3571 ( 89%)], train_loss: 0.18501\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44229\nepoch: 004 [3184/3571 ( 89%)], train_loss: 0.18489\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44381\nepoch: 004 [3200/3571 ( 90%)], train_loss: 0.18489\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44109\nepoch: 004 [3216/3571 ( 90%)], train_loss: 0.18465\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43879\n4 epoch, best epoch was updated! valid_loss: 0.43879\nepoch: 004 [3232/3571 ( 91%)], train_loss: 0.18461\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43779\n4 epoch, best epoch was updated! valid_loss: 0.43779\nepoch: 004 [3248/3571 ( 91%)], train_loss: 0.18438\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43743\n4 epoch, best epoch was updated! valid_loss: 0.43743\nepoch: 004 [3264/3571 ( 91%)], train_loss: 0.18419\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43878\nepoch: 004 [3280/3571 ( 92%)], train_loss: 0.18407\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43867\nepoch: 004 [3296/3571 ( 92%)], train_loss: 0.18395\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43773\nepoch: 004 [3312/3571 ( 93%)], train_loss: 0.18369\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43995\nepoch: 004 [3328/3571 ( 93%)], train_loss: 0.18343\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43897\nepoch: 004 [3344/3571 ( 94%)], train_loss: 0.18312\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43627\n4 epoch, best epoch was updated! valid_loss: 0.43627\nepoch: 004 [3360/3571 ( 94%)], train_loss: 0.18323\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44162\nepoch: 004 [3376/3571 ( 95%)], train_loss: 0.18344\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44576\nepoch: 004 [3392/3571 ( 95%)], train_loss: 0.18311\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44204\nepoch: 004 [3408/3571 ( 95%)], train_loss: 0.18307\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43774\nepoch: 004 [3424/3571 ( 96%)], train_loss: 0.18297\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44147\nepoch: 004 [3440/3571 ( 96%)], train_loss: 0.18261\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44407\nepoch: 004 [3456/3571 ( 97%)], train_loss: 0.18292\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43905\nepoch: 004 [3472/3571 ( 97%)], train_loss: 0.18310\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43823\nepoch: 004 [3488/3571 ( 98%)], train_loss: 0.18329\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.44076\nepoch: 004 [3504/3571 ( 98%)], train_loss: 0.18335\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43988\nepoch: 004 [3520/3571 ( 99%)], train_loss: 0.18324\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43639\nepoch: 004 [3536/3571 ( 99%)], train_loss: 0.18305\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43604\n4 epoch, best epoch was updated! valid_loss: 0.43604\nepoch: 004 [3552/3571 ( 99%)], train_loss: 0.18306\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43822\nepoch: 004 [3568/3571 (100%)], train_loss: 0.18302\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43736\nepoch: 004 [3571/3571 (100%)], train_loss: 0.18294\n----Validation Results Summary----\nEpoch: [4] valid_loss: 0.43862\nepoch: 005 [  16/3571 (  0%)], train_loss: 0.42499\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43654\nepoch: 005 [  32/3571 (  1%)], train_loss: 0.25200\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44158\nepoch: 005 [  48/3571 (  1%)], train_loss: 0.19762\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45046\nepoch: 005 [  64/3571 (  2%)], train_loss: 0.18175\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44475\nepoch: 005 [  80/3571 (  2%)], train_loss: 0.16076\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43896\nepoch: 005 [  96/3571 (  3%)], train_loss: 0.15693\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44279\nepoch: 005 [ 112/3571 (  3%)], train_loss: 0.15578\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45074\nepoch: 005 [ 128/3571 (  4%)], train_loss: 0.15629\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45027\nepoch: 005 [ 144/3571 (  4%)], train_loss: 0.16112\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43842\nepoch: 005 [ 160/3571 (  4%)], train_loss: 0.16049\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44101\nepoch: 005 [ 176/3571 (  5%)], train_loss: 0.15898\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45574\nepoch: 005 [ 192/3571 (  5%)], train_loss: 0.15540\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45940\nepoch: 005 [ 208/3571 (  6%)], train_loss: 0.15659\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45111\nepoch: 005 [ 224/3571 (  6%)], train_loss: 0.15636\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43768\nepoch: 005 [ 240/3571 (  7%)], train_loss: 0.15673\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43733\nepoch: 005 [ 256/3571 (  7%)], train_loss: 0.15553\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44146\nepoch: 005 [ 272/3571 (  8%)], train_loss: 0.16015\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43824\nepoch: 005 [ 288/3571 (  8%)], train_loss: 0.15829\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43473\n5 epoch, best epoch was updated! valid_loss: 0.43473\nepoch: 005 [ 304/3571 (  9%)], train_loss: 0.15677\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44015\nepoch: 005 [ 320/3571 (  9%)], train_loss: 0.15974\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44633\nepoch: 005 [ 336/3571 (  9%)], train_loss: 0.16145\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44632\nepoch: 005 [ 352/3571 ( 10%)], train_loss: 0.15961\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44465\nepoch: 005 [ 368/3571 ( 10%)], train_loss: 0.15715\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44110\nepoch: 005 [ 384/3571 ( 11%)], train_loss: 0.15594\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44387\nepoch: 005 [ 400/3571 ( 11%)], train_loss: 0.15638\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44899\nepoch: 005 [ 416/3571 ( 12%)], train_loss: 0.15602\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44488\nepoch: 005 [ 432/3571 ( 12%)], train_loss: 0.15678\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43868\nepoch: 005 [ 448/3571 ( 13%)], train_loss: 0.15526\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44180\nepoch: 005 [ 464/3571 ( 13%)], train_loss: 0.15483\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44314\nepoch: 005 [ 480/3571 ( 13%)], train_loss: 0.15455\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44112\nepoch: 005 [ 496/3571 ( 14%)], train_loss: 0.15232\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43696\nepoch: 005 [ 512/3571 ( 14%)], train_loss: 0.15145\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44121\nepoch: 005 [ 528/3571 ( 15%)], train_loss: 0.15039\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44259\nepoch: 005 [ 544/3571 ( 15%)], train_loss: 0.15093\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43733\nepoch: 005 [ 560/3571 ( 16%)], train_loss: 0.15069\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43843\nepoch: 005 [ 576/3571 ( 16%)], train_loss: 0.14959\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44039\nepoch: 005 [ 592/3571 ( 17%)], train_loss: 0.14809\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43858\nepoch: 005 [ 608/3571 ( 17%)], train_loss: 0.14797\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44005\nepoch: 005 [ 624/3571 ( 17%)], train_loss: 0.14854\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44142\nepoch: 005 [ 640/3571 ( 18%)], train_loss: 0.14897\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43855\nepoch: 005 [ 656/3571 ( 18%)], train_loss: 0.14886\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43870\nepoch: 005 [ 672/3571 ( 19%)], train_loss: 0.14791\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44006\nepoch: 005 [ 688/3571 ( 19%)], train_loss: 0.14799\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44034\nepoch: 005 [ 704/3571 ( 20%)], train_loss: 0.14704\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44060\nepoch: 005 [ 720/3571 ( 20%)], train_loss: 0.14678\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44181\nepoch: 005 [ 736/3571 ( 21%)], train_loss: 0.14650\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44423\nepoch: 005 [ 752/3571 ( 21%)], train_loss: 0.14629\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44076\nepoch: 005 [ 768/3571 ( 22%)], train_loss: 0.14593\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43720\nepoch: 005 [ 784/3571 ( 22%)], train_loss: 0.14474\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43893\nepoch: 005 [ 800/3571 ( 22%)], train_loss: 0.14519\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44025\nepoch: 005 [ 816/3571 ( 23%)], train_loss: 0.14507\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43781\nepoch: 005 [ 832/3571 ( 23%)], train_loss: 0.14445\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43402\n5 epoch, best epoch was updated! valid_loss: 0.43402\nepoch: 005 [ 848/3571 ( 24%)], train_loss: 0.14413\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43619\nepoch: 005 [ 864/3571 ( 24%)], train_loss: 0.14305\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44546\nepoch: 005 [ 880/3571 ( 25%)], train_loss: 0.14368\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44352\nepoch: 005 [ 896/3571 ( 25%)], train_loss: 0.14280\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43617\nepoch: 005 [ 912/3571 ( 26%)], train_loss: 0.14259\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43298\n5 epoch, best epoch was updated! valid_loss: 0.43298\nepoch: 005 [ 928/3571 ( 26%)], train_loss: 0.14164\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43565\nepoch: 005 [ 944/3571 ( 26%)], train_loss: 0.14130\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43880\nepoch: 005 [ 960/3571 ( 27%)], train_loss: 0.14111\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43716\nepoch: 005 [ 976/3571 ( 27%)], train_loss: 0.14030\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43409\nepoch: 005 [ 992/3571 ( 28%)], train_loss: 0.14018\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43643\nepoch: 005 [1008/3571 ( 28%)], train_loss: 0.13988\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44779\nepoch: 005 [1024/3571 ( 29%)], train_loss: 0.13980\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44931\nepoch: 005 [1040/3571 ( 29%)], train_loss: 0.14104\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43840\nepoch: 005 [1056/3571 ( 30%)], train_loss: 0.14046\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43405\nepoch: 005 [1072/3571 ( 30%)], train_loss: 0.14041\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44652\nepoch: 005 [1088/3571 ( 30%)], train_loss: 0.14165\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45509\nepoch: 005 [1104/3571 ( 31%)], train_loss: 0.14314\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45007\nepoch: 005 [1120/3571 ( 31%)], train_loss: 0.14311\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43762\nepoch: 005 [1136/3571 ( 32%)], train_loss: 0.14372\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43746\nepoch: 005 [1152/3571 ( 32%)], train_loss: 0.14311\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44529\nepoch: 005 [1168/3571 ( 33%)], train_loss: 0.14296\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44458\nepoch: 005 [1184/3571 ( 33%)], train_loss: 0.14301\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43925\nepoch: 005 [1200/3571 ( 34%)], train_loss: 0.14231\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43535\nepoch: 005 [1216/3571 ( 34%)], train_loss: 0.14176\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43470\nepoch: 005 [1232/3571 ( 35%)], train_loss: 0.14154\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43605\nepoch: 005 [1248/3571 ( 35%)], train_loss: 0.14109\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43575\nepoch: 005 [1264/3571 ( 35%)], train_loss: 0.14084\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44133\nepoch: 005 [1280/3571 ( 36%)], train_loss: 0.14164\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44860\nepoch: 005 [1296/3571 ( 36%)], train_loss: 0.14280\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44678\nepoch: 005 [1312/3571 ( 37%)], train_loss: 0.14329\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43753\nepoch: 005 [1328/3571 ( 37%)], train_loss: 0.14271\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43500\nepoch: 005 [1344/3571 ( 38%)], train_loss: 0.14279\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43479\nepoch: 005 [1360/3571 ( 38%)], train_loss: 0.14261\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43690\nepoch: 005 [1376/3571 ( 39%)], train_loss: 0.14311\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43699\nepoch: 005 [1392/3571 ( 39%)], train_loss: 0.14291\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43591\nepoch: 005 [1408/3571 ( 39%)], train_loss: 0.14225\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43462\nepoch: 005 [1424/3571 ( 40%)], train_loss: 0.14188\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43589\nepoch: 005 [1440/3571 ( 40%)], train_loss: 0.14107\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44105\nepoch: 005 [1456/3571 ( 41%)], train_loss: 0.14106\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44237\nepoch: 005 [1472/3571 ( 41%)], train_loss: 0.14093\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43904\nepoch: 005 [1488/3571 ( 42%)], train_loss: 0.14022\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43589\nepoch: 005 [1504/3571 ( 42%)], train_loss: 0.14011\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43654\nepoch: 005 [1520/3571 ( 43%)], train_loss: 0.13967\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43697\nepoch: 005 [1536/3571 ( 43%)], train_loss: 0.13932\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43685\nepoch: 005 [1552/3571 ( 43%)], train_loss: 0.13909\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43795\nepoch: 005 [1568/3571 ( 44%)], train_loss: 0.13901\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44239\nepoch: 005 [1584/3571 ( 44%)], train_loss: 0.13892\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44589\nepoch: 005 [1600/3571 ( 45%)], train_loss: 0.13823\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44394\nepoch: 005 [1616/3571 ( 45%)], train_loss: 0.13805\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43766\nepoch: 005 [1632/3571 ( 46%)], train_loss: 0.13780\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43760\nepoch: 005 [1648/3571 ( 46%)], train_loss: 0.13782\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44021\nepoch: 005 [1664/3571 ( 47%)], train_loss: 0.13772\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43827\nepoch: 005 [1680/3571 ( 47%)], train_loss: 0.13818\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43491\nepoch: 005 [1696/3571 ( 47%)], train_loss: 0.13772\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43496\nepoch: 005 [1712/3571 ( 48%)], train_loss: 0.13750\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43393\nepoch: 005 [1728/3571 ( 48%)], train_loss: 0.13710\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43373\nepoch: 005 [1744/3571 ( 49%)], train_loss: 0.13674\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43466\nepoch: 005 [1760/3571 ( 49%)], train_loss: 0.13663\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43364\nepoch: 005 [1776/3571 ( 50%)], train_loss: 0.13657\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43278\n5 epoch, best epoch was updated! valid_loss: 0.43278\nepoch: 005 [1792/3571 ( 50%)], train_loss: 0.13634\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43400\nepoch: 005 [1808/3571 ( 51%)], train_loss: 0.13609\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43560\nepoch: 005 [1824/3571 ( 51%)], train_loss: 0.13610\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43594\nepoch: 005 [1840/3571 ( 52%)], train_loss: 0.13607\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43576\nepoch: 005 [1856/3571 ( 52%)], train_loss: 0.13601\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44101\nepoch: 005 [1872/3571 ( 52%)], train_loss: 0.13599\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44752\nepoch: 005 [1888/3571 ( 53%)], train_loss: 0.13579\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44436\nepoch: 005 [1904/3571 ( 53%)], train_loss: 0.13576\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43503\nepoch: 005 [1920/3571 ( 54%)], train_loss: 0.13541\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43195\n5 epoch, best epoch was updated! valid_loss: 0.43195\nepoch: 005 [1936/3571 ( 54%)], train_loss: 0.13523\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43685\nepoch: 005 [1952/3571 ( 55%)], train_loss: 0.13514\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43861\nepoch: 005 [1968/3571 ( 55%)], train_loss: 0.13542\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43364\nepoch: 005 [1984/3571 ( 56%)], train_loss: 0.13531\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43257\nepoch: 005 [2000/3571 ( 56%)], train_loss: 0.13517\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43522\nepoch: 005 [2016/3571 ( 56%)], train_loss: 0.13517\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44006\nepoch: 005 [2032/3571 ( 57%)], train_loss: 0.13520\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44373\nepoch: 005 [2048/3571 ( 57%)], train_loss: 0.13516\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43987\nepoch: 005 [2064/3571 ( 58%)], train_loss: 0.13499\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43442\nepoch: 005 [2080/3571 ( 58%)], train_loss: 0.13523\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43644\nepoch: 005 [2096/3571 ( 59%)], train_loss: 0.13485\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44076\nepoch: 005 [2112/3571 ( 59%)], train_loss: 0.13489\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44202\nepoch: 005 [2128/3571 ( 60%)], train_loss: 0.13482\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43652\nepoch: 005 [2144/3571 ( 60%)], train_loss: 0.13469\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43250\nepoch: 005 [2160/3571 ( 60%)], train_loss: 0.13444\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43806\nepoch: 005 [2176/3571 ( 61%)], train_loss: 0.13434\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44267\nepoch: 005 [2192/3571 ( 61%)], train_loss: 0.13428\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43929\nepoch: 005 [2208/3571 ( 62%)], train_loss: 0.13427\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43583\nepoch: 005 [2224/3571 ( 62%)], train_loss: 0.13411\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43189\n5 epoch, best epoch was updated! valid_loss: 0.43189\nepoch: 005 [2240/3571 ( 63%)], train_loss: 0.13418\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43315\nepoch: 005 [2256/3571 ( 63%)], train_loss: 0.13443\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43676\nepoch: 005 [2272/3571 ( 64%)], train_loss: 0.13404\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43753\nepoch: 005 [2288/3571 ( 64%)], train_loss: 0.13357\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43748\nepoch: 005 [2304/3571 ( 65%)], train_loss: 0.13345\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43639\nepoch: 005 [2320/3571 ( 65%)], train_loss: 0.13317\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43831\nepoch: 005 [2336/3571 ( 65%)], train_loss: 0.13366\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44191\nepoch: 005 [2352/3571 ( 66%)], train_loss: 0.13349\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44624\nepoch: 005 [2368/3571 ( 66%)], train_loss: 0.13353\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45220\nepoch: 005 [2384/3571 ( 67%)], train_loss: 0.13325\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.45389\nepoch: 005 [2400/3571 ( 67%)], train_loss: 0.13324\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44760\nepoch: 005 [2416/3571 ( 68%)], train_loss: 0.13337\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43654\nepoch: 005 [2432/3571 ( 68%)], train_loss: 0.13335\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43070\n5 epoch, best epoch was updated! valid_loss: 0.43070\nepoch: 005 [2448/3571 ( 69%)], train_loss: 0.13326\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43501\nepoch: 005 [2464/3571 ( 69%)], train_loss: 0.13319\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44007\nepoch: 005 [2480/3571 ( 69%)], train_loss: 0.13334\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.44042\nepoch: 005 [2496/3571 ( 70%)], train_loss: 0.13342\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43549\nepoch: 005 [2512/3571 ( 70%)], train_loss: 0.13327\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43099\nepoch: 005 [2528/3571 ( 71%)], train_loss: 0.13310\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43216\nepoch: 005 [2544/3571 ( 71%)], train_loss: 0.13296\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43642\nepoch: 005 [2560/3571 ( 72%)], train_loss: 0.13288\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43866\nepoch: 005 [2576/3571 ( 72%)], train_loss: 0.13259\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43694\nepoch: 005 [2592/3571 ( 73%)], train_loss: 0.13226\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43561\nepoch: 005 [2608/3571 ( 73%)], train_loss: 0.13201\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43747\nepoch: 005 [2624/3571 ( 73%)], train_loss: 0.13204\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43590\nepoch: 005 [2640/3571 ( 74%)], train_loss: 0.13169\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43433\nepoch: 005 [2656/3571 ( 74%)], train_loss: 0.13159\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43498\nepoch: 005 [2672/3571 ( 75%)], train_loss: 0.13137\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43599\nepoch: 005 [2688/3571 ( 75%)], train_loss: 0.13137\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43553\nepoch: 005 [2704/3571 ( 76%)], train_loss: 0.13137\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43313\nepoch: 005 [2720/3571 ( 76%)], train_loss: 0.13116\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43217\nepoch: 005 [2736/3571 ( 77%)], train_loss: 0.13083\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43200\nepoch: 005 [2752/3571 ( 77%)], train_loss: 0.13061\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43350\nepoch: 005 [2768/3571 ( 78%)], train_loss: 0.13071\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43446\nepoch: 005 [2784/3571 ( 78%)], train_loss: 0.13063\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43432\nepoch: 005 [2800/3571 ( 78%)], train_loss: 0.13041\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43297\nepoch: 005 [2816/3571 ( 79%)], train_loss: 0.13028\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43197\nepoch: 005 [2832/3571 ( 79%)], train_loss: 0.13041\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43159\nepoch: 005 [2848/3571 ( 80%)], train_loss: 0.13045\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43164\nepoch: 005 [2864/3571 ( 80%)], train_loss: 0.13070\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43138\nepoch: 005 [2880/3571 ( 81%)], train_loss: 0.13075\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43134\nepoch: 005 [2896/3571 ( 81%)], train_loss: 0.13078\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43081\nepoch: 005 [2912/3571 ( 82%)], train_loss: 0.13065\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43064\n5 epoch, best epoch was updated! valid_loss: 0.43064\nepoch: 005 [2928/3571 ( 82%)], train_loss: 0.13037\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43086\nepoch: 005 [2944/3571 ( 82%)], train_loss: 0.13028\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43095\nepoch: 005 [2960/3571 ( 83%)], train_loss: 0.13004\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43138\nepoch: 005 [2976/3571 ( 83%)], train_loss: 0.12999\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43163\nepoch: 005 [2992/3571 ( 84%)], train_loss: 0.12965\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43182\nepoch: 005 [3008/3571 ( 84%)], train_loss: 0.12966\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43160\nepoch: 005 [3024/3571 ( 85%)], train_loss: 0.12951\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43119\nepoch: 005 [3040/3571 ( 85%)], train_loss: 0.12976\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43062\n5 epoch, best epoch was updated! valid_loss: 0.43062\nepoch: 005 [3056/3571 ( 86%)], train_loss: 0.12938\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43039\n5 epoch, best epoch was updated! valid_loss: 0.43039\nepoch: 005 [3072/3571 ( 86%)], train_loss: 0.12939\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43040\nepoch: 005 [3088/3571 ( 86%)], train_loss: 0.12934\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43059\nepoch: 005 [3104/3571 ( 87%)], train_loss: 0.12917\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43044\nepoch: 005 [3120/3571 ( 87%)], train_loss: 0.12909\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43158\nepoch: 005 [3136/3571 ( 88%)], train_loss: 0.12912\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43136\nepoch: 005 [3152/3571 ( 88%)], train_loss: 0.12878\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43072\nepoch: 005 [3168/3571 ( 89%)], train_loss: 0.12890\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43080\nepoch: 005 [3184/3571 ( 89%)], train_loss: 0.12869\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43156\nepoch: 005 [3200/3571 ( 90%)], train_loss: 0.12871\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43239\nepoch: 005 [3216/3571 ( 90%)], train_loss: 0.12847\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43291\nepoch: 005 [3232/3571 ( 91%)], train_loss: 0.12834\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43227\nepoch: 005 [3248/3571 ( 91%)], train_loss: 0.12837\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43146\nepoch: 005 [3264/3571 ( 91%)], train_loss: 0.12837\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43262\nepoch: 005 [3280/3571 ( 92%)], train_loss: 0.12833\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43350\nepoch: 005 [3296/3571 ( 92%)], train_loss: 0.12814\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43427\nepoch: 005 [3312/3571 ( 93%)], train_loss: 0.12793\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43502\nepoch: 005 [3328/3571 ( 93%)], train_loss: 0.12794\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43551\nepoch: 005 [3344/3571 ( 94%)], train_loss: 0.12759\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43554\nepoch: 005 [3360/3571 ( 94%)], train_loss: 0.12763\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43432\nepoch: 005 [3376/3571 ( 95%)], train_loss: 0.12762\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43357\nepoch: 005 [3392/3571 ( 95%)], train_loss: 0.12762\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43403\nepoch: 005 [3408/3571 ( 95%)], train_loss: 0.12772\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43434\nepoch: 005 [3424/3571 ( 96%)], train_loss: 0.12770\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43367\nepoch: 005 [3440/3571 ( 96%)], train_loss: 0.12753\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43155\nepoch: 005 [3456/3571 ( 97%)], train_loss: 0.12730\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43027\n5 epoch, best epoch was updated! valid_loss: 0.43027\nepoch: 005 [3472/3571 ( 97%)], train_loss: 0.12721\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43025\n5 epoch, best epoch was updated! valid_loss: 0.43025\nepoch: 005 [3488/3571 ( 98%)], train_loss: 0.12708\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43041\nepoch: 005 [3504/3571 ( 98%)], train_loss: 0.12700\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43061\nepoch: 005 [3520/3571 ( 99%)], train_loss: 0.12691\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43042\nepoch: 005 [3536/3571 ( 99%)], train_loss: 0.12691\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43061\nepoch: 005 [3552/3571 ( 99%)], train_loss: 0.12680\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43170\nepoch: 005 [3568/3571 (100%)], train_loss: 0.12665\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43167\nepoch: 005 [3571/3571 (100%)], train_loss: 0.12667\n----Validation Results Summary----\nEpoch: [5] valid_loss: 0.43198\nepoch: 006 [  16/3571 (  0%)], train_loss: 0.50152\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43220\nepoch: 006 [  32/3571 (  1%)], train_loss: 0.28481\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43391\nepoch: 006 [  48/3571 (  1%)], train_loss: 0.21237\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43567\nepoch: 006 [  64/3571 (  2%)], train_loss: 0.18412\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43562\nepoch: 006 [  80/3571 (  2%)], train_loss: 0.16142\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43397\nepoch: 006 [  96/3571 (  3%)], train_loss: 0.14924\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43322\nepoch: 006 [ 112/3571 (  3%)], train_loss: 0.14172\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43274\nepoch: 006 [ 128/3571 (  4%)], train_loss: 0.13534\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43252\nepoch: 006 [ 144/3571 (  4%)], train_loss: 0.13024\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43330\nepoch: 006 [ 160/3571 (  4%)], train_loss: 0.12490\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43321\nepoch: 006 [ 176/3571 (  5%)], train_loss: 0.12046\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43263\nepoch: 006 [ 192/3571 (  5%)], train_loss: 0.12108\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43129\nepoch: 006 [ 208/3571 (  6%)], train_loss: 0.11527\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43110\nepoch: 006 [ 224/3571 (  6%)], train_loss: 0.11476\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43218\nepoch: 006 [ 240/3571 (  7%)], train_loss: 0.11314\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43271\nepoch: 006 [ 256/3571 (  7%)], train_loss: 0.11042\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43113\nepoch: 006 [ 272/3571 (  8%)], train_loss: 0.10907\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43048\nepoch: 006 [ 288/3571 (  8%)], train_loss: 0.10757\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43372\nepoch: 006 [ 304/3571 (  9%)], train_loss: 0.10652\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43641\nepoch: 006 [ 320/3571 (  9%)], train_loss: 0.10575\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43700\nepoch: 006 [ 336/3571 (  9%)], train_loss: 0.10579\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43432\nepoch: 006 [ 352/3571 ( 10%)], train_loss: 0.10458\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43245\nepoch: 006 [ 368/3571 ( 10%)], train_loss: 0.10419\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43256\nepoch: 006 [ 384/3571 ( 11%)], train_loss: 0.10320\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43331\nepoch: 006 [ 400/3571 ( 11%)], train_loss: 0.10193\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43357\nepoch: 006 [ 416/3571 ( 12%)], train_loss: 0.10270\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43192\nepoch: 006 [ 432/3571 ( 12%)], train_loss: 0.10263\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43124\nepoch: 006 [ 448/3571 ( 13%)], train_loss: 0.10180\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43286\nepoch: 006 [ 464/3571 ( 13%)], train_loss: 0.10095\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43312\nepoch: 006 [ 480/3571 ( 13%)], train_loss: 0.10060\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43118\nepoch: 006 [ 496/3571 ( 14%)], train_loss: 0.09974\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42965\n6 epoch, best epoch was updated! valid_loss: 0.42965\nepoch: 006 [ 512/3571 ( 14%)], train_loss: 0.10004\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42962\n6 epoch, best epoch was updated! valid_loss: 0.42962\nepoch: 006 [ 528/3571 ( 15%)], train_loss: 0.09912\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42976\nepoch: 006 [ 544/3571 ( 15%)], train_loss: 0.09838\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42965\nepoch: 006 [ 560/3571 ( 16%)], train_loss: 0.09791\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42958\n6 epoch, best epoch was updated! valid_loss: 0.42958\nepoch: 006 [ 576/3571 ( 16%)], train_loss: 0.09662\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42995\nepoch: 006 [ 592/3571 ( 17%)], train_loss: 0.09625\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43184\nepoch: 006 [ 608/3571 ( 17%)], train_loss: 0.09577\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43216\nepoch: 006 [ 624/3571 ( 17%)], train_loss: 0.09543\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43250\nepoch: 006 [ 640/3571 ( 18%)], train_loss: 0.09519\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43163\nepoch: 006 [ 656/3571 ( 18%)], train_loss: 0.09402\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43130\nepoch: 006 [ 672/3571 ( 19%)], train_loss: 0.09338\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43168\nepoch: 006 [ 688/3571 ( 19%)], train_loss: 0.09318\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43123\nepoch: 006 [ 704/3571 ( 20%)], train_loss: 0.09317\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43095\nepoch: 006 [ 720/3571 ( 20%)], train_loss: 0.09235\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43034\nepoch: 006 [ 736/3571 ( 21%)], train_loss: 0.09176\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43013\nepoch: 006 [ 752/3571 ( 21%)], train_loss: 0.09106\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42953\n6 epoch, best epoch was updated! valid_loss: 0.42953\nepoch: 006 [ 768/3571 ( 22%)], train_loss: 0.09109\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42924\n6 epoch, best epoch was updated! valid_loss: 0.42924\nepoch: 006 [ 784/3571 ( 22%)], train_loss: 0.09054\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42957\nepoch: 006 [ 800/3571 ( 22%)], train_loss: 0.09030\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43010\nepoch: 006 [ 816/3571 ( 23%)], train_loss: 0.09048\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42941\nepoch: 006 [ 832/3571 ( 23%)], train_loss: 0.09053\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42924\n6 epoch, best epoch was updated! valid_loss: 0.42924\nepoch: 006 [ 848/3571 ( 24%)], train_loss: 0.09013\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42890\n6 epoch, best epoch was updated! valid_loss: 0.42890\nepoch: 006 [ 864/3571 ( 24%)], train_loss: 0.08962\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42874\n6 epoch, best epoch was updated! valid_loss: 0.42874\nepoch: 006 [ 880/3571 ( 25%)], train_loss: 0.08960\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42880\nepoch: 006 [ 896/3571 ( 25%)], train_loss: 0.09029\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42891\nepoch: 006 [ 912/3571 ( 26%)], train_loss: 0.08989\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42888\nepoch: 006 [ 928/3571 ( 26%)], train_loss: 0.08979\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42916\nepoch: 006 [ 944/3571 ( 26%)], train_loss: 0.08914\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42990\nepoch: 006 [ 960/3571 ( 27%)], train_loss: 0.08925\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43004\nepoch: 006 [ 976/3571 ( 27%)], train_loss: 0.08899\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42936\nepoch: 006 [ 992/3571 ( 28%)], train_loss: 0.08871\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42888\nepoch: 006 [1008/3571 ( 28%)], train_loss: 0.08866\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42933\nepoch: 006 [1024/3571 ( 29%)], train_loss: 0.08826\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42956\nepoch: 006 [1040/3571 ( 29%)], train_loss: 0.08807\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42967\nepoch: 006 [1056/3571 ( 30%)], train_loss: 0.08804\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43071\nepoch: 006 [1072/3571 ( 30%)], train_loss: 0.08758\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43156\nepoch: 006 [1088/3571 ( 30%)], train_loss: 0.08702\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43170\nepoch: 006 [1104/3571 ( 31%)], train_loss: 0.08653\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43049\nepoch: 006 [1120/3571 ( 31%)], train_loss: 0.08623\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42955\nepoch: 006 [1136/3571 ( 32%)], train_loss: 0.08571\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42903\nepoch: 006 [1152/3571 ( 32%)], train_loss: 0.08560\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42854\n6 epoch, best epoch was updated! valid_loss: 0.42854\nepoch: 006 [1168/3571 ( 33%)], train_loss: 0.08525\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42822\n6 epoch, best epoch was updated! valid_loss: 0.42822\nepoch: 006 [1184/3571 ( 33%)], train_loss: 0.08493\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42783\n6 epoch, best epoch was updated! valid_loss: 0.42783\nepoch: 006 [1200/3571 ( 34%)], train_loss: 0.08460\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42744\n6 epoch, best epoch was updated! valid_loss: 0.42744\nepoch: 006 [1216/3571 ( 34%)], train_loss: 0.08446\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42738\n6 epoch, best epoch was updated! valid_loss: 0.42738\nepoch: 006 [1232/3571 ( 35%)], train_loss: 0.08462\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42733\n6 epoch, best epoch was updated! valid_loss: 0.42733\nepoch: 006 [1248/3571 ( 35%)], train_loss: 0.08443\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42736\nepoch: 006 [1264/3571 ( 35%)], train_loss: 0.08408\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42753\nepoch: 006 [1280/3571 ( 36%)], train_loss: 0.08405\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42757\nepoch: 006 [1296/3571 ( 36%)], train_loss: 0.08422\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42778\nepoch: 006 [1312/3571 ( 37%)], train_loss: 0.08370\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42797\nepoch: 006 [1328/3571 ( 37%)], train_loss: 0.08361\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42844\nepoch: 006 [1344/3571 ( 38%)], train_loss: 0.08367\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42867\nepoch: 006 [1360/3571 ( 38%)], train_loss: 0.08335\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42872\nepoch: 006 [1376/3571 ( 39%)], train_loss: 0.08294\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42899\nepoch: 006 [1392/3571 ( 39%)], train_loss: 0.08291\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42918\nepoch: 006 [1408/3571 ( 39%)], train_loss: 0.08261\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42936\nepoch: 006 [1424/3571 ( 40%)], train_loss: 0.08244\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42935\nepoch: 006 [1440/3571 ( 40%)], train_loss: 0.08234\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42938\nepoch: 006 [1456/3571 ( 41%)], train_loss: 0.08301\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42908\nepoch: 006 [1472/3571 ( 41%)], train_loss: 0.08297\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42889\nepoch: 006 [1488/3571 ( 42%)], train_loss: 0.08283\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42953\nepoch: 006 [1504/3571 ( 42%)], train_loss: 0.08290\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43015\nepoch: 006 [1520/3571 ( 43%)], train_loss: 0.08283\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43008\nepoch: 006 [1536/3571 ( 43%)], train_loss: 0.08287\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42937\nepoch: 006 [1552/3571 ( 43%)], train_loss: 0.08286\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42869\nepoch: 006 [1568/3571 ( 44%)], train_loss: 0.08272\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42850\nepoch: 006 [1584/3571 ( 44%)], train_loss: 0.08225\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42838\nepoch: 006 [1600/3571 ( 45%)], train_loss: 0.08202\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42849\nepoch: 006 [1616/3571 ( 45%)], train_loss: 0.08228\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42875\nepoch: 006 [1632/3571 ( 46%)], train_loss: 0.08224\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42928\nepoch: 006 [1648/3571 ( 46%)], train_loss: 0.08199\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42922\nepoch: 006 [1664/3571 ( 47%)], train_loss: 0.08189\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42899\nepoch: 006 [1680/3571 ( 47%)], train_loss: 0.08201\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42959\nepoch: 006 [1696/3571 ( 47%)], train_loss: 0.08194\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43090\nepoch: 006 [1712/3571 ( 48%)], train_loss: 0.08187\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43201\nepoch: 006 [1728/3571 ( 48%)], train_loss: 0.08177\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43204\nepoch: 006 [1744/3571 ( 49%)], train_loss: 0.08167\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43139\nepoch: 006 [1760/3571 ( 49%)], train_loss: 0.08145\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43120\nepoch: 006 [1776/3571 ( 50%)], train_loss: 0.08141\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43080\nepoch: 006 [1792/3571 ( 50%)], train_loss: 0.08169\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43045\nepoch: 006 [1808/3571 ( 51%)], train_loss: 0.08176\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43009\nepoch: 006 [1824/3571 ( 51%)], train_loss: 0.08142\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42996\nepoch: 006 [1840/3571 ( 52%)], train_loss: 0.08123\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42997\nepoch: 006 [1856/3571 ( 52%)], train_loss: 0.08135\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42979\nepoch: 006 [1872/3571 ( 52%)], train_loss: 0.08122\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42980\nepoch: 006 [1888/3571 ( 53%)], train_loss: 0.08098\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43000\nepoch: 006 [1904/3571 ( 53%)], train_loss: 0.08074\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43012\nepoch: 006 [1920/3571 ( 54%)], train_loss: 0.08058\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43064\nepoch: 006 [1936/3571 ( 54%)], train_loss: 0.08029\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43033\nepoch: 006 [1952/3571 ( 55%)], train_loss: 0.08036\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43028\nepoch: 006 [1968/3571 ( 55%)], train_loss: 0.08041\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42996\nepoch: 006 [1984/3571 ( 56%)], train_loss: 0.08031\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42962\nepoch: 006 [2000/3571 ( 56%)], train_loss: 0.08037\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42969\nepoch: 006 [2016/3571 ( 56%)], train_loss: 0.08041\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42977\nepoch: 006 [2032/3571 ( 57%)], train_loss: 0.08022\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42994\nepoch: 006 [2048/3571 ( 57%)], train_loss: 0.07998\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43003\nepoch: 006 [2064/3571 ( 58%)], train_loss: 0.07984\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43000\nepoch: 006 [2080/3571 ( 58%)], train_loss: 0.07951\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43002\nepoch: 006 [2096/3571 ( 59%)], train_loss: 0.07933\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43021\nepoch: 006 [2112/3571 ( 59%)], train_loss: 0.07916\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43054\nepoch: 006 [2128/3571 ( 60%)], train_loss: 0.07903\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43068\nepoch: 006 [2144/3571 ( 60%)], train_loss: 0.07876\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43063\nepoch: 006 [2160/3571 ( 60%)], train_loss: 0.07877\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43044\nepoch: 006 [2176/3571 ( 61%)], train_loss: 0.07855\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43092\nepoch: 006 [2192/3571 ( 61%)], train_loss: 0.07842\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43143\nepoch: 006 [2208/3571 ( 62%)], train_loss: 0.07866\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43213\nepoch: 006 [2224/3571 ( 62%)], train_loss: 0.07876\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43214\nepoch: 006 [2240/3571 ( 63%)], train_loss: 0.07854\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43149\nepoch: 006 [2256/3571 ( 63%)], train_loss: 0.07872\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43071\nepoch: 006 [2272/3571 ( 64%)], train_loss: 0.07865\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43059\nepoch: 006 [2288/3571 ( 64%)], train_loss: 0.07861\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43082\nepoch: 006 [2304/3571 ( 65%)], train_loss: 0.07873\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43102\nepoch: 006 [2320/3571 ( 65%)], train_loss: 0.07868\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43104\nepoch: 006 [2336/3571 ( 65%)], train_loss: 0.07868\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43102\nepoch: 006 [2352/3571 ( 66%)], train_loss: 0.07867\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43103\nepoch: 006 [2368/3571 ( 66%)], train_loss: 0.07856\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43107\nepoch: 006 [2384/3571 ( 67%)], train_loss: 0.07837\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43101\nepoch: 006 [2400/3571 ( 67%)], train_loss: 0.07842\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43104\nepoch: 006 [2416/3571 ( 68%)], train_loss: 0.07819\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43103\nepoch: 006 [2432/3571 ( 68%)], train_loss: 0.07811\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43094\nepoch: 006 [2448/3571 ( 69%)], train_loss: 0.07795\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43064\nepoch: 006 [2464/3571 ( 69%)], train_loss: 0.07812\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43048\nepoch: 006 [2480/3571 ( 69%)], train_loss: 0.07799\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43043\nepoch: 006 [2496/3571 ( 70%)], train_loss: 0.07806\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43038\nepoch: 006 [2512/3571 ( 70%)], train_loss: 0.07781\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43024\nepoch: 006 [2528/3571 ( 71%)], train_loss: 0.07789\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42990\nepoch: 006 [2544/3571 ( 71%)], train_loss: 0.07785\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42969\nepoch: 006 [2560/3571 ( 72%)], train_loss: 0.07769\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42954\nepoch: 006 [2576/3571 ( 72%)], train_loss: 0.07756\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42955\nepoch: 006 [2592/3571 ( 73%)], train_loss: 0.07759\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42989\nepoch: 006 [2608/3571 ( 73%)], train_loss: 0.07746\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42992\nepoch: 006 [2624/3571 ( 73%)], train_loss: 0.07750\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42996\nepoch: 006 [2640/3571 ( 74%)], train_loss: 0.07751\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42974\nepoch: 006 [2656/3571 ( 74%)], train_loss: 0.07745\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42984\nepoch: 006 [2672/3571 ( 75%)], train_loss: 0.07737\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43025\nepoch: 006 [2688/3571 ( 75%)], train_loss: 0.07739\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43065\nepoch: 006 [2704/3571 ( 76%)], train_loss: 0.07732\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43104\nepoch: 006 [2720/3571 ( 76%)], train_loss: 0.07745\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43120\nepoch: 006 [2736/3571 ( 77%)], train_loss: 0.07733\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43146\nepoch: 006 [2752/3571 ( 77%)], train_loss: 0.07725\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43158\nepoch: 006 [2768/3571 ( 78%)], train_loss: 0.07714\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43165\nepoch: 006 [2784/3571 ( 78%)], train_loss: 0.07705\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43209\nepoch: 006 [2800/3571 ( 78%)], train_loss: 0.07707\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43280\nepoch: 006 [2816/3571 ( 79%)], train_loss: 0.07698\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43442\nepoch: 006 [2832/3571 ( 79%)], train_loss: 0.07722\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43570\nepoch: 006 [2848/3571 ( 80%)], train_loss: 0.07747\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43513\nepoch: 006 [2864/3571 ( 80%)], train_loss: 0.07762\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43400\nepoch: 006 [2880/3571 ( 81%)], train_loss: 0.07773\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43208\nepoch: 006 [2896/3571 ( 81%)], train_loss: 0.07777\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43007\nepoch: 006 [2912/3571 ( 82%)], train_loss: 0.07767\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42901\nepoch: 006 [2928/3571 ( 82%)], train_loss: 0.07757\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42904\nepoch: 006 [2944/3571 ( 82%)], train_loss: 0.07750\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42973\nepoch: 006 [2960/3571 ( 83%)], train_loss: 0.07749\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43013\nepoch: 006 [2976/3571 ( 83%)], train_loss: 0.07753\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42991\nepoch: 006 [2992/3571 ( 84%)], train_loss: 0.07745\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42947\nepoch: 006 [3008/3571 ( 84%)], train_loss: 0.07744\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42896\nepoch: 006 [3024/3571 ( 85%)], train_loss: 0.07751\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42869\nepoch: 006 [3040/3571 ( 85%)], train_loss: 0.07748\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42916\nepoch: 006 [3056/3571 ( 86%)], train_loss: 0.07736\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43035\nepoch: 006 [3072/3571 ( 86%)], train_loss: 0.07740\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43137\nepoch: 006 [3088/3571 ( 86%)], train_loss: 0.07729\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43193\nepoch: 006 [3104/3571 ( 87%)], train_loss: 0.07756\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43186\nepoch: 006 [3120/3571 ( 87%)], train_loss: 0.07779\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43097\nepoch: 006 [3136/3571 ( 88%)], train_loss: 0.07768\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43029\nepoch: 006 [3152/3571 ( 88%)], train_loss: 0.07772\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42967\nepoch: 006 [3168/3571 ( 89%)], train_loss: 0.07760\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42925\nepoch: 006 [3184/3571 ( 89%)], train_loss: 0.07771\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42921\nepoch: 006 [3200/3571 ( 90%)], train_loss: 0.07776\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42954\nepoch: 006 [3216/3571 ( 90%)], train_loss: 0.07767\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43001\nepoch: 006 [3232/3571 ( 91%)], train_loss: 0.07784\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43047\nepoch: 006 [3248/3571 ( 91%)], train_loss: 0.07777\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43067\nepoch: 006 [3264/3571 ( 91%)], train_loss: 0.07768\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43072\nepoch: 006 [3280/3571 ( 92%)], train_loss: 0.07775\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43070\nepoch: 006 [3296/3571 ( 92%)], train_loss: 0.07786\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43069\nepoch: 006 [3312/3571 ( 93%)], train_loss: 0.07771\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43071\nepoch: 006 [3328/3571 ( 93%)], train_loss: 0.07770\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43087\nepoch: 006 [3344/3571 ( 94%)], train_loss: 0.07779\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43079\nepoch: 006 [3360/3571 ( 94%)], train_loss: 0.07781\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43087\nepoch: 006 [3376/3571 ( 95%)], train_loss: 0.07760\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43118\nepoch: 006 [3392/3571 ( 95%)], train_loss: 0.07753\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43119\nepoch: 006 [3408/3571 ( 95%)], train_loss: 0.07744\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43091\nepoch: 006 [3424/3571 ( 96%)], train_loss: 0.07735\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43089\nepoch: 006 [3440/3571 ( 96%)], train_loss: 0.07729\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43084\nepoch: 006 [3456/3571 ( 97%)], train_loss: 0.07731\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43045\nepoch: 006 [3472/3571 ( 97%)], train_loss: 0.07728\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.43015\nepoch: 006 [3488/3571 ( 98%)], train_loss: 0.07716\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42960\nepoch: 006 [3504/3571 ( 98%)], train_loss: 0.07717\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42923\nepoch: 006 [3520/3571 ( 99%)], train_loss: 0.07708\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42900\nepoch: 006 [3536/3571 ( 99%)], train_loss: 0.07694\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42888\nepoch: 006 [3552/3571 ( 99%)], train_loss: 0.07707\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42881\nepoch: 006 [3568/3571 (100%)], train_loss: 0.07706\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42872\nepoch: 006 [3571/3571 (100%)], train_loss: 0.07706\n----Validation Results Summary----\nEpoch: [6] valid_loss: 0.42878\nepoch: 007 [  16/3571 (  0%)], train_loss: 0.30081\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42880\nepoch: 007 [  32/3571 (  1%)], train_loss: 0.17916\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42888\nepoch: 007 [  48/3571 (  1%)], train_loss: 0.13622\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42901\nepoch: 007 [  64/3571 (  2%)], train_loss: 0.11266\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42920\nepoch: 007 [  80/3571 (  2%)], train_loss: 0.11050\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42939\nepoch: 007 [  96/3571 (  3%)], train_loss: 0.09774\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42961\nepoch: 007 [ 112/3571 (  3%)], train_loss: 0.09187\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43002\nepoch: 007 [ 128/3571 (  4%)], train_loss: 0.08972\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43045\nepoch: 007 [ 144/3571 (  4%)], train_loss: 0.08336\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43075\nepoch: 007 [ 160/3571 (  4%)], train_loss: 0.07988\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43103\nepoch: 007 [ 176/3571 (  5%)], train_loss: 0.07503\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43122\nepoch: 007 [ 192/3571 (  5%)], train_loss: 0.07268\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43115\nepoch: 007 [ 208/3571 (  6%)], train_loss: 0.07267\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43099\nepoch: 007 [ 224/3571 (  6%)], train_loss: 0.07215\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43086\nepoch: 007 [ 240/3571 (  7%)], train_loss: 0.07129\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43086\nepoch: 007 [ 256/3571 (  7%)], train_loss: 0.07013\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43096\nepoch: 007 [ 272/3571 (  8%)], train_loss: 0.06910\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43093\nepoch: 007 [ 288/3571 (  8%)], train_loss: 0.06860\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43078\nepoch: 007 [ 304/3571 (  9%)], train_loss: 0.06693\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43053\nepoch: 007 [ 320/3571 (  9%)], train_loss: 0.06546\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43031\nepoch: 007 [ 336/3571 (  9%)], train_loss: 0.06455\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43027\nepoch: 007 [ 352/3571 ( 10%)], train_loss: 0.06359\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43036\nepoch: 007 [ 368/3571 ( 10%)], train_loss: 0.06283\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43040\nepoch: 007 [ 384/3571 ( 11%)], train_loss: 0.06206\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43025\nepoch: 007 [ 400/3571 ( 11%)], train_loss: 0.06086\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43001\nepoch: 007 [ 416/3571 ( 12%)], train_loss: 0.06039\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42984\nepoch: 007 [ 432/3571 ( 12%)], train_loss: 0.05992\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42962\nepoch: 007 [ 448/3571 ( 13%)], train_loss: 0.05877\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42950\nepoch: 007 [ 464/3571 ( 13%)], train_loss: 0.05798\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42941\nepoch: 007 [ 480/3571 ( 13%)], train_loss: 0.05779\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42937\nepoch: 007 [ 496/3571 ( 14%)], train_loss: 0.05716\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42938\nepoch: 007 [ 512/3571 ( 14%)], train_loss: 0.05728\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42945\nepoch: 007 [ 528/3571 ( 15%)], train_loss: 0.05765\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42941\nepoch: 007 [ 544/3571 ( 15%)], train_loss: 0.05753\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42937\nepoch: 007 [ 560/3571 ( 16%)], train_loss: 0.05702\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42925\nepoch: 007 [ 576/3571 ( 16%)], train_loss: 0.05652\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42921\nepoch: 007 [ 592/3571 ( 17%)], train_loss: 0.05605\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42932\nepoch: 007 [ 608/3571 ( 17%)], train_loss: 0.05553\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42960\nepoch: 007 [ 624/3571 ( 17%)], train_loss: 0.05608\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42988\nepoch: 007 [ 640/3571 ( 18%)], train_loss: 0.05561\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43007\nepoch: 007 [ 656/3571 ( 18%)], train_loss: 0.05534\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43010\nepoch: 007 [ 672/3571 ( 19%)], train_loss: 0.05485\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43014\nepoch: 007 [ 688/3571 ( 19%)], train_loss: 0.05473\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43010\nepoch: 007 [ 704/3571 ( 20%)], train_loss: 0.05443\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43004\nepoch: 007 [ 720/3571 ( 20%)], train_loss: 0.05409\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43002\nepoch: 007 [ 736/3571 ( 21%)], train_loss: 0.05419\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43003\nepoch: 007 [ 752/3571 ( 21%)], train_loss: 0.05384\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43008\nepoch: 007 [ 768/3571 ( 22%)], train_loss: 0.05389\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43016\nepoch: 007 [ 784/3571 ( 22%)], train_loss: 0.05412\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43025\nepoch: 007 [ 800/3571 ( 22%)], train_loss: 0.05396\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43032\nepoch: 007 [ 816/3571 ( 23%)], train_loss: 0.05393\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43034\nepoch: 007 [ 832/3571 ( 23%)], train_loss: 0.05378\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43022\nepoch: 007 [ 848/3571 ( 24%)], train_loss: 0.05415\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43005\nepoch: 007 [ 864/3571 ( 24%)], train_loss: 0.05388\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42992\nepoch: 007 [ 880/3571 ( 25%)], train_loss: 0.05387\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42991\nepoch: 007 [ 896/3571 ( 25%)], train_loss: 0.05344\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43006\nepoch: 007 [ 912/3571 ( 26%)], train_loss: 0.05345\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43021\nepoch: 007 [ 928/3571 ( 26%)], train_loss: 0.05372\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43049\nepoch: 007 [ 944/3571 ( 26%)], train_loss: 0.05363\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43058\nepoch: 007 [ 960/3571 ( 27%)], train_loss: 0.05350\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43065\nepoch: 007 [ 976/3571 ( 27%)], train_loss: 0.05353\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43056\nepoch: 007 [ 992/3571 ( 28%)], train_loss: 0.05340\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43029\nepoch: 007 [1008/3571 ( 28%)], train_loss: 0.05417\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42994\nepoch: 007 [1024/3571 ( 29%)], train_loss: 0.05431\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42968\nepoch: 007 [1040/3571 ( 29%)], train_loss: 0.05392\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42948\nepoch: 007 [1056/3571 ( 30%)], train_loss: 0.05360\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42950\nepoch: 007 [1072/3571 ( 30%)], train_loss: 0.05352\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42971\nepoch: 007 [1088/3571 ( 30%)], train_loss: 0.05321\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42990\nepoch: 007 [1104/3571 ( 31%)], train_loss: 0.05309\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43007\nepoch: 007 [1120/3571 ( 31%)], train_loss: 0.05299\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43007\nepoch: 007 [1136/3571 ( 32%)], train_loss: 0.05284\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42988\nepoch: 007 [1152/3571 ( 32%)], train_loss: 0.05256\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42974\nepoch: 007 [1168/3571 ( 33%)], train_loss: 0.05240\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42965\nepoch: 007 [1184/3571 ( 33%)], train_loss: 0.05244\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42964\nepoch: 007 [1200/3571 ( 34%)], train_loss: 0.05231\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42966\nepoch: 007 [1216/3571 ( 34%)], train_loss: 0.05230\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42972\nepoch: 007 [1232/3571 ( 35%)], train_loss: 0.05217\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42988\nepoch: 007 [1248/3571 ( 35%)], train_loss: 0.05205\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43005\nepoch: 007 [1264/3571 ( 35%)], train_loss: 0.05205\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43023\nepoch: 007 [1280/3571 ( 36%)], train_loss: 0.05232\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43028\nepoch: 007 [1296/3571 ( 36%)], train_loss: 0.05238\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43039\nepoch: 007 [1312/3571 ( 37%)], train_loss: 0.05242\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43039\nepoch: 007 [1328/3571 ( 37%)], train_loss: 0.05253\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43028\nepoch: 007 [1344/3571 ( 38%)], train_loss: 0.05234\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.43012\nepoch: 007 [1360/3571 ( 38%)], train_loss: 0.05234\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42997\nepoch: 007 [1376/3571 ( 39%)], train_loss: 0.05214\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42975\nepoch: 007 [1392/3571 ( 39%)], train_loss: 0.05211\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42955\nepoch: 007 [1408/3571 ( 39%)], train_loss: 0.05232\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42938\nepoch: 007 [1424/3571 ( 40%)], train_loss: 0.05212\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42931\nepoch: 007 [1440/3571 ( 40%)], train_loss: 0.05207\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42931\nepoch: 007 [1456/3571 ( 41%)], train_loss: 0.05211\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42931\nepoch: 007 [1472/3571 ( 41%)], train_loss: 0.05221\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42934\nepoch: 007 [1488/3571 ( 42%)], train_loss: 0.05198\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42937\nepoch: 007 [1504/3571 ( 42%)], train_loss: 0.05195\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42937\nepoch: 007 [1520/3571 ( 43%)], train_loss: 0.05199\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42934\nepoch: 007 [1536/3571 ( 43%)], train_loss: 0.05205\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42932\nepoch: 007 [1552/3571 ( 43%)], train_loss: 0.05199\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42935\nepoch: 007 [1568/3571 ( 44%)], train_loss: 0.05191\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42943\nepoch: 007 [1584/3571 ( 44%)], train_loss: 0.05185\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42951\nepoch: 007 [1600/3571 ( 45%)], train_loss: 0.05223\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42961\nepoch: 007 [1616/3571 ( 45%)], train_loss: 0.05215\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42970\nepoch: 007 [1632/3571 ( 46%)], train_loss: 0.05208\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42974\nepoch: 007 [1648/3571 ( 46%)], train_loss: 0.05201\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42978\nepoch: 007 [1664/3571 ( 47%)], train_loss: 0.05185\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42980\nepoch: 007 [1680/3571 ( 47%)], train_loss: 0.05177\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42982\nepoch: 007 [1696/3571 ( 47%)], train_loss: 0.05166\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42984\nepoch: 007 [1712/3571 ( 48%)], train_loss: 0.05178\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42983\nepoch: 007 [1728/3571 ( 48%)], train_loss: 0.05165\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42983\nepoch: 007 [1744/3571 ( 49%)], train_loss: 0.05153\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42982\nepoch: 007 [1760/3571 ( 49%)], train_loss: 0.05140\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42982\nepoch: 007 [1776/3571 ( 50%)], train_loss: 0.05129\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42983\nepoch: 007 [1792/3571 ( 50%)], train_loss: 0.05124\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42981\nepoch: 007 [1808/3571 ( 51%)], train_loss: 0.05126\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42981\nepoch: 007 [1824/3571 ( 51%)], train_loss: 0.05121\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42978\nepoch: 007 [1840/3571 ( 52%)], train_loss: 0.05117\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42976\nepoch: 007 [1856/3571 ( 52%)], train_loss: 0.05114\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42975\nepoch: 007 [1872/3571 ( 52%)], train_loss: 0.05124\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [1888/3571 ( 53%)], train_loss: 0.05123\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [1904/3571 ( 53%)], train_loss: 0.05133\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42975\nepoch: 007 [1920/3571 ( 54%)], train_loss: 0.05150\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42979\nepoch: 007 [1936/3571 ( 54%)], train_loss: 0.05146\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42982\nepoch: 007 [1952/3571 ( 55%)], train_loss: 0.05150\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42984\nepoch: 007 [1968/3571 ( 55%)], train_loss: 0.05148\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42985\nepoch: 007 [1984/3571 ( 56%)], train_loss: 0.05147\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42985\nepoch: 007 [2000/3571 ( 56%)], train_loss: 0.05135\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42988\nepoch: 007 [2016/3571 ( 56%)], train_loss: 0.05137\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42988\nepoch: 007 [2032/3571 ( 57%)], train_loss: 0.05138\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42985\nepoch: 007 [2048/3571 ( 57%)], train_loss: 0.05124\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42983\nepoch: 007 [2064/3571 ( 58%)], train_loss: 0.05157\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42980\nepoch: 007 [2080/3571 ( 58%)], train_loss: 0.05150\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42977\nepoch: 007 [2096/3571 ( 59%)], train_loss: 0.05170\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [2112/3571 ( 59%)], train_loss: 0.05222\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42971\nepoch: 007 [2128/3571 ( 60%)], train_loss: 0.05211\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42970\nepoch: 007 [2144/3571 ( 60%)], train_loss: 0.05202\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42971\nepoch: 007 [2160/3571 ( 60%)], train_loss: 0.05191\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [2176/3571 ( 61%)], train_loss: 0.05169\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [2192/3571 ( 61%)], train_loss: 0.05169\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42972\nepoch: 007 [2208/3571 ( 62%)], train_loss: 0.05167\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42971\nepoch: 007 [2224/3571 ( 62%)], train_loss: 0.05171\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42969\nepoch: 007 [2240/3571 ( 63%)], train_loss: 0.05160\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42967\nepoch: 007 [2256/3571 ( 63%)], train_loss: 0.05156\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42964\nepoch: 007 [2272/3571 ( 64%)], train_loss: 0.05145\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42962\nepoch: 007 [2288/3571 ( 64%)], train_loss: 0.05131\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42961\nepoch: 007 [2304/3571 ( 65%)], train_loss: 0.05131\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42960\nepoch: 007 [2320/3571 ( 65%)], train_loss: 0.05122\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42959\nepoch: 007 [2336/3571 ( 65%)], train_loss: 0.05118\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42958\nepoch: 007 [2352/3571 ( 66%)], train_loss: 0.05124\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42956\nepoch: 007 [2368/3571 ( 66%)], train_loss: 0.05116\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42956\nepoch: 007 [2384/3571 ( 67%)], train_loss: 0.05103\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42955\nepoch: 007 [2400/3571 ( 67%)], train_loss: 0.05110\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42955\nepoch: 007 [2416/3571 ( 68%)], train_loss: 0.05103\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42954\nepoch: 007 [2432/3571 ( 68%)], train_loss: 0.05093\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42954\nepoch: 007 [2448/3571 ( 69%)], train_loss: 0.05078\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42954\nepoch: 007 [2464/3571 ( 69%)], train_loss: 0.05071\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42953\nepoch: 007 [2480/3571 ( 69%)], train_loss: 0.05064\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42953\nepoch: 007 [2496/3571 ( 70%)], train_loss: 0.05062\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42953\nepoch: 007 [2512/3571 ( 70%)], train_loss: 0.05065\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42953\nepoch: 007 [2528/3571 ( 71%)], train_loss: 0.05051\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42953\nepoch: 007 [2544/3571 ( 71%)], train_loss: 0.05053\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42952\nepoch: 007 [2560/3571 ( 72%)], train_loss: 0.05056\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42951\nepoch: 007 [2576/3571 ( 72%)], train_loss: 0.05048\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42951\nepoch: 007 [2592/3571 ( 73%)], train_loss: 0.05045\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42949\nepoch: 007 [2608/3571 ( 73%)], train_loss: 0.05067\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42949\nepoch: 007 [2624/3571 ( 73%)], train_loss: 0.05067\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42948\nepoch: 007 [2640/3571 ( 74%)], train_loss: 0.05065\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42948\nepoch: 007 [2656/3571 ( 74%)], train_loss: 0.05058\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42948\nepoch: 007 [2672/3571 ( 75%)], train_loss: 0.05046\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42949\nepoch: 007 [2688/3571 ( 75%)], train_loss: 0.05038\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42950\nepoch: 007 [2704/3571 ( 76%)], train_loss: 0.05026\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42951\nepoch: 007 [2720/3571 ( 76%)], train_loss: 0.05027\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42953\nepoch: 007 [2736/3571 ( 77%)], train_loss: 0.05016\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42954\nepoch: 007 [2752/3571 ( 77%)], train_loss: 0.05017\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42955\nepoch: 007 [2768/3571 ( 78%)], train_loss: 0.05013\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42956\nepoch: 007 [2784/3571 ( 78%)], train_loss: 0.05019\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42957\nepoch: 007 [2800/3571 ( 78%)], train_loss: 0.05021\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42958\nepoch: 007 [2816/3571 ( 79%)], train_loss: 0.05018\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42959\nepoch: 007 [2832/3571 ( 79%)], train_loss: 0.05005\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42960\nepoch: 007 [2848/3571 ( 80%)], train_loss: 0.05005\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42960\nepoch: 007 [2864/3571 ( 80%)], train_loss: 0.05003\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42961\nepoch: 007 [2880/3571 ( 81%)], train_loss: 0.04998\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42962\nepoch: 007 [2896/3571 ( 81%)], train_loss: 0.05003\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42962\nepoch: 007 [2912/3571 ( 82%)], train_loss: 0.04994\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42962\nepoch: 007 [2928/3571 ( 82%)], train_loss: 0.04980\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42962\nepoch: 007 [2944/3571 ( 82%)], train_loss: 0.04980\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42962\nepoch: 007 [2960/3571 ( 83%)], train_loss: 0.04973\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42963\nepoch: 007 [2976/3571 ( 83%)], train_loss: 0.04975\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42964\nepoch: 007 [2992/3571 ( 84%)], train_loss: 0.04975\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42964\nepoch: 007 [3008/3571 ( 84%)], train_loss: 0.04977\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42965\nepoch: 007 [3024/3571 ( 85%)], train_loss: 0.04976\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42966\nepoch: 007 [3040/3571 ( 85%)], train_loss: 0.04984\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42967\nepoch: 007 [3056/3571 ( 86%)], train_loss: 0.04976\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42968\nepoch: 007 [3072/3571 ( 86%)], train_loss: 0.04971\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42968\nepoch: 007 [3088/3571 ( 86%)], train_loss: 0.04964\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42969\nepoch: 007 [3104/3571 ( 87%)], train_loss: 0.04958\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42970\nepoch: 007 [3120/3571 ( 87%)], train_loss: 0.04951\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42971\nepoch: 007 [3136/3571 ( 88%)], train_loss: 0.04956\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42971\nepoch: 007 [3152/3571 ( 88%)], train_loss: 0.04956\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42972\nepoch: 007 [3168/3571 ( 89%)], train_loss: 0.04961\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42972\nepoch: 007 [3184/3571 ( 89%)], train_loss: 0.04959\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3200/3571 ( 90%)], train_loss: 0.04967\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3216/3571 ( 90%)], train_loss: 0.04959\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3232/3571 ( 91%)], train_loss: 0.04964\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42974\nepoch: 007 [3248/3571 ( 91%)], train_loss: 0.04969\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42974\nepoch: 007 [3264/3571 ( 91%)], train_loss: 0.04965\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42974\nepoch: 007 [3280/3571 ( 92%)], train_loss: 0.04959\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42974\nepoch: 007 [3296/3571 ( 92%)], train_loss: 0.04956\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42974\nepoch: 007 [3312/3571 ( 93%)], train_loss: 0.04959\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3328/3571 ( 93%)], train_loss: 0.04962\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3344/3571 ( 94%)], train_loss: 0.04967\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3360/3571 ( 94%)], train_loss: 0.04962\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3376/3571 ( 95%)], train_loss: 0.04959\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3392/3571 ( 95%)], train_loss: 0.04951\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3408/3571 ( 95%)], train_loss: 0.04957\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3424/3571 ( 96%)], train_loss: 0.04949\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3440/3571 ( 96%)], train_loss: 0.04945\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3456/3571 ( 97%)], train_loss: 0.04938\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3472/3571 ( 97%)], train_loss: 0.04930\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3488/3571 ( 98%)], train_loss: 0.04932\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3504/3571 ( 98%)], train_loss: 0.04947\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3520/3571 ( 99%)], train_loss: 0.04943\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3536/3571 ( 99%)], train_loss: 0.04935\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3552/3571 ( 99%)], train_loss: 0.04927\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3568/3571 (100%)], train_loss: 0.04932\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\nepoch: 007 [3571/3571 (100%)], train_loss: 0.04930\n----Validation Results Summary----\nEpoch: [7] valid_loss: 0.42973\n----\n","output_type":"stream"}]},{"cell_type":"markdown","source":"### Best Validation Loss per Fold","metadata":{}},{"cell_type":"code","source":"[print(\"FOLD::\", i, \"Loss:: \", fold['best_val_loss']) for i, fold in enumerate(result_list)]","metadata":{"id":"lt2jASMiPH1R","outputId":"473119ae-40c4-4b31-fa9a-b6c42feaf3fb","execution":{"iopub.status.busy":"2021-06-03T15:53:41.805514Z","iopub.status.idle":"2021-06-03T15:53:41.806043Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### OOF Prediction","metadata":{}},{"cell_type":"code","source":"oof = np.zeros(len(train))\nfor fold in tqdm(range(5), total=5):\n    model, tokenizer = make_model()\n    model.load_state_dict(\n        torch.load(f'model{fold}.bin')\n    )\n    model.cuda()\n    model.eval()\n    val_index = train[train.kfold==fold].index.tolist()\n    train_loader, val_loader = make_loader(train, tokenizer, 250, 16, fold=fold)\n    # scalar = torch.cuda.amp.GradScaler()\n    scalar = None\n    preds = []\n    for index, data in enumerate(val_loader):\n        input_ids, attention_mask, token_type_ids, labels = data['input_ids'], \\\n            data['attention_mask'], data['token_type_ids'], data['label']\n        input_ids, attention_mask, token_type_ids, labels = input_ids.cuda(), \\\n            attention_mask.cuda(), token_type_ids.cuda(), labels.cuda()\n        if scalar is not None:\n            with torch.cuda.amp.autocast():\n                outputs = model(\n                    input_ids=input_ids,\n                    attention_mask=attention_mask,\n                    token_type_ids=token_type_ids,\n                    labels=labels\n                )\n        else:\n            outputs = model(\n                input_ids=input_ids,\n                attention_mask=attention_mask,\n                token_type_ids=token_type_ids,\n                labels=labels\n            )\n        \n        loss, logits = outputs[:2]\n        preds += logits.cpu().detach().numpy().tolist()\n    oof[val_index] = preds","metadata":{"id":"_aaj35kCjxzT","outputId":"a1830cca-986e-4797-d4ee-06466df3e0d3","execution":{"iopub.status.busy":"2021-06-03T15:53:41.807475Z","iopub.status.idle":"2021-06-03T15:53:41.808419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Compute Local CV Score","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import mean_squared_error\nround(np.sqrt(mean_squared_error(train.target.values, oof)), 4)","metadata":{"id":"js2Li0srR1Cq","outputId":"bd44fd20-ccf3-4e42-eb81-677648c73105"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}