forked from microsoft/qlib
-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Supporting Arctic Backend Provider & Orderbook, Tick Data Example (mi…
…crosoft#744) * change weight_decay & batchsize * del weight_decay * big weight_decay * mid weight_decay * small layer * 2 layer * full layer * no weight decay * divide into two data source * change parse field * delete some debug * add Toperator * new format of arctic * fix cache bug to arctic read * fix connection problem * add some operator * final version for arcitc * clear HZ cache * remove not used function * add topswrappers * successfully import data and run first test * A simpler version to support arctic * Successfully run all high-freq expressions * Black format and fix add docs * Add docs for download and test data * update scripts and docs * Add docs * fix bug * Refine docs * fix test bug * fix CI error * clean code Co-authored-by: bxdd <bxddream@gmail.com> Co-authored-by: wangwenxi.handsome <wangwenxi.handsome@gmail.com> Co-authored-by: Young <afe.young@gmail.com>
- Loading branch information
4 people
authored
Jan 18, 2022
1 parent
f0cd1df
commit ef28d71
Showing
16 changed files
with
923 additions
and
90 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # Introduction | ||
|
|
||
| This example tries to demonstrate how Qlib supports data without fixed shared frequency. | ||
|
|
||
| For example, | ||
| - Daily prices volume data are fixed-frequency data. The data comes in a fixed frequency (i.e. daily) | ||
| - Orders are not fixed data and they may come at any time point | ||
|
|
||
| To support such non-fixed-frequency, Qlib implements an Arctic-based backend. | ||
| Here is an example to import and query data based on this backend. | ||
|
|
||
| # Installation | ||
|
|
||
| Please refer to [the installation docs](https://docs.mongodb.com/manual/installation/) of mongodb. | ||
| Current version of script with default value tries to connect localhost **via default port without authentication**. | ||
|
|
||
| Run following command to install necessary libraries | ||
| ``` | ||
| pip install pytest | ||
| ``` | ||
|
|
||
| # Importing example data | ||
|
|
||
|
|
||
| 1. (Optional) Please follow the first part of [this section](https://github.com/microsoft/qlib#data-preparation) to **get 1min data** of Qlib. | ||
| 2. Please follow following steps to download example data | ||
| ```bash | ||
| cd examples/orderbook_data/ | ||
| wget http://fintech.msra.cn/stock_data/downloads/highfreq_orderboook_example_data.tar.bz2 | ||
| tar xf highfreq_orderboook_example_data.tar.bz2 | ||
| ``` | ||
|
|
||
| 3. Please import the example data to your mongo db | ||
| ```bash | ||
| cd examples/orderbook_data/ | ||
| python create_dataset.py initialize_library # Initialization Libraries | ||
| python create_dataset.py import_data # Initialization Libraries | ||
| ``` | ||
|
|
||
| # Query Examples | ||
|
|
||
| After importing these data, you run `example.py` to create some high-frequency features. | ||
| ```bash | ||
| cd examples/orderbook_data/ | ||
| pytest -s --disable-warnings example.py # If you want run all examples | ||
| pytest -s --disable-warnings example.py::TestClass::test_exp_10 # If you want to run specific example | ||
| ``` | ||
|
|
||
|
|
||
| # Known limitations | ||
| Expression computing between different frequencies are not supported yet |
Oops, something went wrong.