The AutoGen (analyticsdf) is a Python library that allows you to generate synthetic data with any statistical characteristics desired.
This library provides a set of functionality to enable the specification and generation of a wide range of datasets with specified statistical characteristics. Specification includes the predictor matrix and the response vector.
Some common congifuration:
- High correlation and multi-collinearity among predictor variables
- Interaction effects between variables
- Skewed distributions of predictor and response variables
- Nonlinear relationships between predictor and response variables
Check the Analyticsdf documentation for more details.
- Sklearn Make Datasets functionality
- MIT Synthetic Data Vault project
- MIT Data to AI Lab
- datacebo
- 2016 IEEE conference paper, The Synthetic Data Vault.
The beta package of this library is publicly available on both PyPI and Anaconda. Install analyticsdf using pip or conda. We recommend using a virtual environment to avoid conflicts with other software on your device.
pip install analyticsdfconda install -c faye-yufan analyticsdfImport the dataset generation class from the package, and play with the class functions.
from analyticsdf.analyticsdataframe import AnalyticsDataframe
ad = AnalyticsDataframe(1000, 6)
ad.predictor_matrix.head()
The predictor matrix is initialized with all null values. Now let's update the predictors with some distributions:
for var in ['X1', 'X2', 'X3', 'X4', 'X5']:
ad.update_predictor_uniform(var, 0, 100)
ad.update_predictor_categorical('X6', ["Red", "Yellow", "Blue"], [0.3, 0.4, 0.3])
Once we have a dataframe desired and would like to visualize it, we can do:
df_visualization_bi(ad)
We plan to integrate an user interface to the library, aiming to let users configure, manipulate, and view datasets more easily.
AutoGen is released under the MIT License.