Skip to content

Commit

Permalink
Minor HW typo corrections and clarifications
Browse files Browse the repository at this point in the history
  • Loading branch information
@cgpotts
cgpotts committed Jun 1, 2019
1 parent d26b77e commit db7e211
Show file tree
Hide file tree
Showing 3 changed files with 145 additions and 142 deletions.
39 changes: 20 additions & 19 deletions hw2_sst.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@
"name": "stderr",
"output_type": "stream",
"text": [
"Finished epoch 50 of 50; error is 2.2631479501724243"
"Finished epoch 50 of 50; error is 2.3075825870037083"
]
},
{
Expand All @@ -287,13 +287,13 @@
"text": [
" precision recall f1-score support\n",
"\n",
" negative 0.581 0.593 0.587 428\n",
" neutral 0.230 0.205 0.217 229\n",
" positive 0.624 0.646 0.635 444\n",
" negative 0.581 0.734 0.649 428\n",
" neutral 0.310 0.192 0.237 229\n",
" positive 0.656 0.619 0.637 444\n",
"\n",
" micro avg 0.534 0.534 0.534 1101\n",
" macro avg 0.479 0.482 0.480 1101\n",
"weighted avg 0.525 0.534 0.530 1101\n",
" micro avg 0.575 0.575 0.575 1101\n",
" macro avg 0.516 0.515 0.508 1101\n",
"weighted avg 0.555 0.575 0.559 1101\n",
"\n"
]
}
Expand Down Expand Up @@ -429,7 +429,7 @@
{
"data": {
"text/plain": [
"67"
"77"
]
},
"execution_count": 15,
Expand All @@ -451,15 +451,15 @@
"output_type": "stream",
"text": [
"======================================================================\n",
"It haunts you , you ca n't forget it , you admire its conception and are able to resolve some of the confusions you had while watching it .\n",
"An operatic , sprawling picture that 's entertainingly acted , magnificently shot and gripping enough to sustain most of its 170-minute length .\n",
"======================================================================\n",
"While the ideas about techno-saturation are far from novel , they 're presented with a wry dark humor .\n",
"It all adds up to good fun .\n",
"======================================================================\n",
"It deserves to be seen by anyone with even a passing interest in the events shaping the world beyond their own horizons .\n",
"A fitfully amusing romp that , if nothing else , will appeal to fans of Malcolm in the Middle and its pubescent star , Frankie Muniz .\n",
"======================================================================\n",
"My Wife Is an Actress is an utterly charming French comedy that feels so American in sensibility and style it 's virtually its own Hollywood remake .\n",
"Trademark American triteness and simplicity are tossed out the window with the intelligent French drama that deftly explores the difficult relationship between a father and son .\n",
"======================================================================\n",
"The volatile dynamics of female friendship is the subject of this unhurried , low-key film that is so off-Hollywood that it seems positively French in its rhythms and resonance .\n"
"The film 's welcome breeziness and some unbelievably hilarious moments -- most portraying the idiocy of the film industry -- make it mostly worth the trip .\n"
]
}
],
Expand Down Expand Up @@ -515,9 +515,9 @@
"source": [
"### A more powerful vector-summing baseline [3 points]\n",
"\n",
"In [Distributed representations as features](sst_03_neural_networks.ipynb#Distributed-representations-as-features), we looked at a baseline for the ternary SST problem in which each example is modeled as the sum of its 50-dimensional GloVe representations. A `LogisticRegression` model was used for prediction. A neural network might do better here, since there might be complex relationships between the input feature dimensions that a linear classifier can't learn. \n",
"In [Distributed representations as features](sst_03_neural_networks.ipynb#Distributed-representations-as-features), we looked at a baseline for the ternary SST problem in which each example is modeled as the sum of its 50-dimensional GloVe representations. A `LogisticRegression` model was used for prediction. A neural network might do better with these representations, since there might be complex relationships between the input feature dimensions that a linear classifier can't learn. \n",
"\n",
"To address this question, rerun the experiment with `torch_shallow_neural_classifier.TorchShallowNeuralClassifier` as the classifier. Specs:\n",
"To address this question, rerun the experiment with `TorchShallowNeuralClassifier` as the classifier. Specs:\n",
"* Use `sst.experiment` to conduct the experiment. \n",
"* Using 3-fold cross-validation, exhaustively explore this set of hyperparameter combinations:\n",
" * The hidden dimensionality at 50, 100, and 200.\n",
Expand All @@ -527,10 +527,10 @@
"For this problem, submit code to do the following:\n",
"\n",
"1. Your model wrapper function around `TorchShallowNeuralClassifier`. This function should implement the requisite cross-validation; see [this notebook section](sst_02_hand_built_features.ipynb#Hyperparameter-search) for examples.\n",
"1. Your average F1 score according to `sst.experiment`. \n",
"2. The optimal hyperparameters chosen in your experiment. (You can just paste in the dict that `sst._experiment` prints.)\n",
"1. The classification report as printed by `sst.experiment`. (This will print out when you run `sst.experiment`. That print-out suffices.)\n",
"2. The optimal hyperparameters chosen in your experiment. (This too will print out when you run `sst.experiment`. The print-out again suffices.)\n",
"\n",
"We're not evaluating the quality of your model. (We've specified the protocols completely, but there will still be a lot of variation in the results.) However, the primary goal of this question is to get you thinking more about this strikingly good baseline feature representation scheme for SST, so we're sort of hoping you feel compelled to try out variations on your own."
"We're not evaluating the quality of your model. (We've specified the protocols completely, but there will still be variation in the results.) However, the primary goal of this question is to get you thinking more about this strikingly good baseline feature representation scheme for SST, so we're sort of hoping you feel compelled to try out variations on your own."
]
},
{
Expand All @@ -546,7 +546,7 @@
"source": [
"### Your original system [4 points]\n",
"\n",
"Your task is to develop an original model for the SST ternary problem. There are many options. If you spend more than a few hours on this homework problem, you should consider letting it grow into your final project! Here are some relatively manageable ideas that you might try:\n",
"Your task is to develop an original model for the SST ternary problem, using only the root-level labels (again, __you cannot make any use of the subtree labels__). There are many options. If you spend more than a few hours on this homework problem, you should consider letting it grow into your final project! Here are some relatively manageable ideas that you might try:\n",
"\n",
"1. We didn't systematically evaluate the `bidirectional` option to the `TorchRNNClassifier`. Similarly, that model could be tweaked to allow multiple LSTM layers (at present there is only one), and you could try adding layers to the classifier portion of the model as well.\n",
"\n",
Expand Down Expand Up @@ -575,6 +575,7 @@
"\n",
"1. Only one evaluation is permitted.\n",
"1. No additional system tuning is permitted once the bake-off has started.\n",
"1. As noted above, __you cannot make any use of the subtree labels__.\n",
"\n",
"To enter the bake-off, upload this notebook on Canvas:\n",
"\n",
Expand Down
Loading

0 comments on commit db7e211

Please sign in to comment.