MachineLearningSamples

This repo hosts variety of examples based on Apache Spark MLIB.

Databricks Notebooks

Decision Tree

Scala IDE Based Examples

Decision Tree

A vanilla decision tree example.

Decision Tree with Stratified Sampling

How to get a stratified sample so the test and train datasets are sampled accross possible values.

Decision Tree with Categorical Feature in the DataSet

How to index and encode categorical features.

Decision Tree Multiple Categorical and Continuous Features in the DataSet

How to handle multiple categorical and continuous features on a real-life data set. Uses the Census Income data set.

Random Forest Multiple Categorical and Continuous Features in the DataSet

How to handle multiple categorical and continuous features on a real-life data set. Uses the Census Income data set.

Data Sets References

Census Income DataSet

First line from adult.test file removed for loading into Spark.

Census Income data set citation: Dua, D. and Karra Taniskidou, E. (2017). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.

Default of Credit Card Clients DataSet