Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add profiling #74

Merged
merged 33 commits into from
Jul 28, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
b0c3535
remove docs/ and ignore
trangdata Jul 23, 2020
541024a
add example profiling htmls
trangdata Jul 23, 2020
4972093
move get_updated_datasets
trangdata Jul 23, 2020
f3f2de0
add get_updated_datasets to init
trangdata Jul 23, 2020
a5c7b12
fix missing imports
trangdata Jul 23, 2020
cb45c56
preliminary profiling.py
trangdata Jul 23, 2020
a6b6ef4
install pandas-profiling in CI
trangdata Jul 24, 2020
982d674
use minimal for high-d datasets
trangdata Jul 24, 2020
0fc8173
update CI configs
trangdata Jul 24, 2020
4d60ac9
troubleshoot deploy
trangdata Jul 24, 2020
e2f80a1
only generate htmls if there's change in the dataset
trangdata Jul 24, 2020
c81872f
add conditional to CI
trangdata Jul 24, 2020
39d5c18
minor edit
trangdata Jul 24, 2020
8d9ca54
minor edit
trangdata Jul 24, 2020
4e192aa
minor edit
trangdata Jul 25, 2020
61948d8
regenerate_profiles
trangdata Jul 25, 2020
ac9ff6b
regenerate_profiles
trangdata Jul 25, 2020
80e2fef
regenerate_profiles
trangdata Jul 25, 2020
7f16a31
[regenerate_profiles]
trangdata Jul 25, 2020
b93ff55
[regenerate_profiles]
trangdata Jul 25, 2020
c6d93b9
[regenerate_profiles]
trangdata Jul 25, 2020
9a9d8f8
[regenerate_profiles]
trangdata Jul 27, 2020
bec836f
[regenerate_profiles]
trangdata Jul 27, 2020
0a78b09
dump steps
trangdata Jul 27, 2020
83b3c93
[regenerate_profiles]
trangdata Jul 27, 2020
912c2e7
randomly select 20 predictors
trangdata Jul 27, 2020
da258ca
[regenerate_profiles]
trangdata Jul 27, 2020
99a579c
fix keep_files
trangdata Jul 27, 2020
f76dff0
clean up and [regenerate_profiles]
trangdata Jul 27, 2020
db4e848
add plotly page
trangdata Jul 28, 2020
cec8482
[regenerate_profiles]
trangdata Jul 28, 2020
804f7b9
test keep_files
trangdata Jul 28, 2020
455cf5b
fix browse-datasets.md
trangdata Jul 28, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified .DS_Store
Binary file not shown.
44 changes: 18 additions & 26 deletions .github/workflows/pythonapp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,33 +27,25 @@ jobs:
python-version: 3.8
- name: Install dependencies
run: |
pip install nose numpy scipy tabulate pandas pyyaml requests parameterized matplotlib seaborn mkdocs
pip install nose numpy scipy tabulate pandas pyyaml requests parameterized matplotlib seaborn mkdocs pandas-profiling
- name: Test with nose
run: |
nosetests -s -v
- name: Set deploy variables
run: |
BRANCH_REF=${{github.head_ref || github.ref}}
echo "::set-env name=BRANCH::${BRANCH_REF##*/}"
git config --local user.name 'github-actions[bot]'
git config --local user.email 'github-actions[bot]@users.noreply.github.com'
- name: Deploy GitHub Pages

- name: Generate profiling reports
run: |
mkdocs gh-deploy

#- name: Generate readme figures
#run: |
#python -m pmlb.update_dataset_readmes
#- name: Set deploy variables
#run: |
#BRANCH_REF=${{github.head_ref || github.ref}}
#echo "::set-env name=BRANCH::${BRANCH_REF##*/}"
#- name: Deploy
#uses: peaceiris/actions-gh-pages@v3
#with:
# external_repository: ${{ github.repository }}
#github_token: ${{ secrets.GITHUB_TOKEN }}
#publish_dir: .
#publish_branch: ${{ env.BRANCH }}
# without keep_files, .github/workflows/pythonapp.yml was removed
#keep_files: true
python -m pmlb.profiling

- name: Build
run: mkdocs build

- run: ls -lhR docs/

- name: Deploy
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: docs
# keep existing files if no changes, unless commit message has regenerate_profiles
keep_files: ${{ env.regenerate_profiles == null }}

12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,15 @@ ENV/

# test notebook
test*.ipynb
.Rproj.user

# test R scripts
r-scripts/
.DS_Store
.Rhistory
penn-ml-benchmarks.Rproj
datasets/.DS_Store

# mkdocs
docs/
docs_sources/profile/
Binary file modified datasets/.DS_Store
Binary file not shown.
143 changes: 0 additions & 143 deletions docs/404.html

This file was deleted.

Loading