self Attention 구현(Q,K,V) 인풋

[jayden5744]

• Encoder, Decoder line에서 Q,K,V를 동일한 값 input으로 Multihead로 넘겨줌
  

  NMT_Korean_To_English/Transformer/model.py

  Line 132 in 4ea0190

  attn_outputs, attn_prob = self.enc_self_attn(enc_inputs, enc_inputs, enc_inputs, enc_self_attn_mask)

  
  

  NMT_Korean_To_English/Transformer/model.py

  Line 153 in 4ea0190

  self_att_outputs, self_self_attn = self.dec_self_attn(dec_inputs, dec_inputs, dec_inputs, dec_self_attn_mask)

• Multihead 에서 차원 변경후 Attention에 넘겨줌
  

  NMT_Korean_To_English/Transformer/model.py

  Line 88 in 4ea0190

  context, attn = self.Attention(q_s, k_s, v_s, attn_mask)

• self Attention 계산
  

  NMT_Korean_To_English/Transformer/model.py

  Line 43 in 4ea0190

  def forward(self, Q, K, V, attn_mask):