Skip to content

Add GPU benchmarks support to readme #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Apr 15, 2021
Merged

Add GPU benchmarks support to readme #59

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
672c8cc
Update readme
Mar 30, 2021
19bc2e7
Merge branch 'master' of https://github.com/IntelPython/scikit-learn_…
Apr 13, 2021
53b4200
Update GPU readme
Apr 13, 2021
cbd99b1
Add hyperlink
Apr 13, 2021
becfbde
Fix comments
Apr 14, 2021
9874958
Update readme
Apr 14, 2021
0fe0ab4
Fix hyperlink
Apr 14, 2021
9ef18ab
Update comments
Apr 14, 2021
ece84d9
Update link
Apr 14, 2021
6568e48
Fix comments
Apr 15, 2021
9f133ef
Apply comments
Apr 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,10 @@ We publish blogs on Medium, so [follow us](https://medium.com/intel-analytics-so

## Table of content

* [Prerequisites](#prerequisites)
* [How to create conda environment for benchmarking](#how-to-create-conda-environment-for-benchmarking)
* [How to enable daal4py patching for scikit-learn benchmarks](#how-to-enable-daal4py-patching-for-scikit-learn-benchmarks)
* [Running Python benchmarks with runner script](#running-python-benchmarks-with-runner-script)
* [Supported algorithms](#supported-algorithms)
* [Benchmark supported algorithms](#benchmark-supported-algorithms)
* [Intel(R) Extension for Scikit-learn* support](#intelr-extension-for-scikit-learn-support)
* [Algorithms parameters](#algorithms-parameters)

## How to create conda environment for benchmarking
Expand Down Expand Up @@ -67,7 +66,7 @@ Run `python runner.py --configs configs/config_example.json [--output-file resul

runner options:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
runner options:
Options:

* ``configs`` : configuration files paths
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* ``configs`` : configuration files paths
* ``--configs``: specify the path to a configuration file.

* ``no-intel-optimized`` : using Scikit-learn without Intel(R) Extension for Scikit-learn*. Now avalible for scikit-learn benchmarks. Default starts with using Intel(R) Extension for Scikit-learn*.
* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench/tree/master/sklearn_bench). Default running with using Intel(R) Extension for Scikit-learn.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now available for [scikit-learn benchmarks] -- why do you need this info?

Suggested change
* ``no-intel-optimized`` : using Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench/tree/master/sklearn_bench). Default running with using Intel(R) Extension for Scikit-learn.
* ``--no-intel-optimized``: use Scikit-learn without [Intel(R) Extension for Scikit-learn*](#intelr-extension-for-scikit-learn-support). Now available for [scikit-learn benchmarks](https://github.com/IntelPython/scikit-learn_bench/tree/master/sklearn_bench). By default, the runner uses Intel(R) Extension for Scikit-learn.

* ``output-file``: output file name for result benchmarks. Default is `result.json`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* ``output-file``: output file name for result benchmarks. Default is `result.json`
* ``--output-file``: output file name for the benchmark result. The default name is `result.json`.

* ``report``: create an Excel report based on benchmarks results. Need library `openpyxl`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* ``report``: create an Excel report based on benchmarks results. Need library `openpyxl`.
* ``--report``: create an Excel report based on benchmark results. The `openpyxl` library is required.

* ``dummy-run`` : run configuration parser and datasets generation without benchmarks running.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* ``dummy-run`` : run configuration parser and datasets generation without benchmarks running.
* ``--dummy-run``: run configuration parser and dataset generation without benchmarks running.

Expand Down Expand Up @@ -108,6 +107,17 @@ The configuration of benchmarks allows you to select the frameworks to run, sele
|**[GradientBoostingClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html)**|gbt|:x:|:x:|:x:|:white_check_mark:|
|**[GradientBoostingRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)**|gbt|:x:|:x:|:x:|:white_check_mark:|

## Intel(R) Extension for Scikit-learn support

By default scikit-learn benchmark launches using [Intel(R) Extension for Scikit-learn](https://github.com/intel/scikit-learn-intelex) on the CPU (use ``no-intel-optimized`` option to run without extention). Some benchmarks have a GPU support:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this is correct sentence. Maybe change by "The launches of scikit-learn benchmark use ... by default"?


* dbscan
* kmeans
* linear
* log_reg

A configuration file that contains all these benchmarks can be found [here](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json). You can use this file to run these benchmarks on both CPU and GPU.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the command to run this config and get a report

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A configuration file that contains all these benchmarks can be found [here](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json). You can use this file to run these benchmarks on both CPU and GPU.
You may use the [configuration file for these benchmarks](https://github.com/IntelPython/scikit-learn_bench/blob/master/configs/skl_xpu_config.json) to run them on both CPU and GPU.


## Algorithms parameters

You can launch benchmarks for each algorithm separately.
Expand Down
1 change: 1 addition & 0 deletions configs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Refer to the tables below for descriptions of all fields in the configuration fi
|data-order| array[string] | **REQUIRED** input data order. Data order: *C* (row-major, default) or *F* (column-major) |
|dtype| array[string] | **REQUIRED** input data type. Data type: *float64* (default) or *float32* |
|check-finitness| array[] | Check finiteness in sklearn input check(disabled by default) |
|device| array[string] | For scikit-learn only. The list of devices to run the benchmarks on. It can be *None* (default, run on CPU without sycl context) or one of the types of sycl devices: *cpu*, *gpu*, *host*. Please reffer to [SYCL specification](https://www.khronos.org/files/sycl/sycl-2020-reference-guide.pdf) for details|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
|device| array[string] | For scikit-learn only. The list of devices to run the benchmarks on. It can be *None* (default, run on CPU without sycl context) or one of the types of sycl devices: *cpu*, *gpu*, *host*. Please reffer to [SYCL specification](https://www.khronos.org/files/sycl/sycl-2020-reference-guide.pdf) for details|
|device| array[string] | For scikit-learn only. The list of devices to run the benchmarks on. It can be *None* (default, run on CPU without sycl context) or one of the types of sycl devices: *cpu*, *gpu*, *host*. Please refer to [SYCL specification](https://www.khronos.org/files/sycl/sycl-2020-reference-guide.pdf) for details|


### Case Object

Expand Down
0 configs/skl_with_context_config.json → configs/skl_xpu_config.json
View file
Open in desktop
File renamed without changes.