Build a complete spam/sentiment text classifier in pure NumPy: clean and tokenize text, extract fixed-vocabulary TF-IDF features, train L2-regularized logistic regression with batch gradient descent, tune the decision threshold on validation data, and deliver a predict(text) API with false-positive/false-negative error analysis.
python scaffold.py- 1. clean_text
- 2. tokenize
- 3. tokenize_corpus
- 4. split_train_val_test_indices
- 5. count_word_frequencies
- 6. build_vocabulary
- 7. tokens_to_bow
- 8. corpus_to_bow_matrix
- 9. compute_document_frequencies
- 10. compute_idf
- 11. transform_tfidf
- 12. fit_tfidf
- 13. sigmoid
- 14. logistic_predict_proba
- 15. binary_cross_entropy
- 16. logistic_gradients
- 17. initialize_logistic_params
- 18. gradient_descent_step
- 19. train_logistic_regression
- 20. predict_labels
- 21. confusion_counts
- 22. metrics_from_counts
- 23. tune_decision_threshold
- 24. evaluate_predictions
- 25. vectorize_texts
- 26. predict_text
- 27. collect_prediction_errors
vocab_size 40
final_train_loss 0.0354
best_threshold 0.66 val_f1 1.0
test_metrics {'tp': 1, 'fp': 0, 'tn': 2, 'fn': 1, 'precision': 1.0, 'recall': 0.5, 'f1': 0.667, 'accuracy': 0.75}
sample_text Free prize click here to claim now
sample_pred 1
n_false_positives 0
n_false_negatives 1