Models
The package includes the following type of models as mentioned in the paper.
This model concatenates representations of the text at different levels of abstraction, essentially chooses the most salient n-grams from the notes. We perform 1D convolutions on the input sequence with kernel sizes 1, 2, and 3. The outputs of each of these convolutions are max-pooled over time and then concatenated. This representation is fed through one or more fully connected layers and eventually with a sigmoid transformation to make a prediction. We find that adding a dense hidden layer after max pooling adds predictive value as it is able to learn interactions between the text features. We concatenate all encounter notes in the history period end to end as input to this network. We integrate lab and demographic features by concatenating the mean of each feature within the history window to the hidden state after the max pooling layer. The following figure illustrates the model architecture.
Hidden states are generated by processing one word at a time, then max-pooled across time. Lab values and demographics are concatenated to the output from Bi-LSTM and then fed into fully connected layers. The following figure shows the BiLSTM model architecture.
(--modelName Enc_CNN_LSTM)
In this model we represent all encounters as a sequence over the history window. This model first encodes each encounter into hidden states using convolutions. Each of these states is then fed into an LSTM. The final hidden state is used for predictions. Here we incorporate numerical values by concatenating the feature vector to the CNN encoder output at each encounter. Demographic values are repeated across encounters. Figure below illustrates the model architecture.
