如何将财务数据dump到qlib中，并且可以每天更新

[louis-xuy]

❓ Questions and Help

We sincerely suggest you to carefully read the documentation of our library as well as the official paper. After that, if you still feel puzzled, please describe the question clearly under this issue.

[bbbzhai]

I think you can append your financial data and do a dump_update.

[louis-xuy]

dump_bin script only supply dump all data，dump update by date, and dump update by stock。i want to dump some features add.

[zhupr]

dump_bin script only supply dump all data，dump update by date, and dump update by stock。i want to dump some features add.

@louis-xuy

• initially dump all data: python dump_bin.py dump_all --csv_path <all data> --qlib_dir <qlib data> --exclude_fields xxx
• dump update(append data): python dump_bin.py dump_update --csv_path <new/all data> --qlib_dir <old qlib data> --exclude_fields xxx

[louis-xuy]

dump_bin script only supply dump all data，dump update by date, and dump update by stock。i want to dump some features add.

@louis-xuy

• initially dump all data: python dump_bin.py dump_all --csv_path <all data> --qlib_dir <qlib data> --exclude_fields xxx
• dump update(append data): python dump_bin.py dump_update --csv_path <new/all data> --qlib_dir <old qlib data> --exclude_fields xxx

@zhupr

if i use dump_bin.py dump_all to add financial data original data. Will the original data be affected? Especially the calendars data, in my opinion, the calendar date calculate by the dump data . and the financial data only one data per quarter.

[zhupr]

dump_bin script only supply dump all data，dump update by date, and dump update by stock。i want to dump some features add.

@louis-xuy

• initially dump all data: python dump_bin.py dump_all --csv_path <all data> --qlib_dir <qlib data> --exclude_fields xxx
• dump update(append data): python dump_bin.py dump_update --csv_path <new/all data> --qlib_dir <old qlib data> --exclude_fields xxx

@zhupr

if i use dump_bin.py dump_all to add financial data original data. Will the original data be affected? Especially the calendars data, in my opinion, the calendar date calculate by the dump data . and the financial data only one data per quarter.

• user need todump_all the volume data (close/open/...) to qlib data
• align the financial data to the calendar of qlib
• dump the financial data to qlib by dump_fix

PIT storage for financial data is under development

[vitorying]

dump_bin script only supply dump all data，dump update by date, and dump update by stock。i want to dump some features add.

@louis-xuy

• initially dump all data: python dump_bin.py dump_all --csv_path <all data> --qlib_dir <qlib data> --exclude_fields xxx
• dump update(append data): python dump_bin.py dump_update --csv_path <new/all data> --qlib_dir <old qlib data> --exclude_fields xxx

@zhupr
if i use dump_bin.py dump_all to add financial data original data. Will the original data be affected? Especially the calendars data, in my opinion, the calendar date calculate by the dump data . and the financial data only one data per quarter.

• user need todump_all the volume data (close/open/...) to qlib data
• align the financial data to the calendar of qlib
• dump the financial data to qlib by dump_fix

PIT storage for financial data is under development

Hi @zhupr
Is it possible to provide an example of adding financial data? thanks

[zhupr]

dump_bin script only supply dump all data，dump update by date, and dump update by stock。i want to dump some features add.

@louis-xuy

• initially dump all data: python dump_bin.py dump_all --csv_path <all data> --qlib_dir <qlib data> --exclude_fields xxx
• dump update(append data): python dump_bin.py dump_update --csv_path <new/all data> --qlib_dir <old qlib data> --exclude_fields xxx

@zhupr
if i use dump_bin.py dump_all to add financial data original data. Will the original data be affected? Especially the calendars data, in my opinion, the calendar date calculate by the dump data . and the financial data only one data per quarter.

• user need todump_all the volume data (close/open/...) to qlib data
• align the financial data to the calendar of qlib
• dump the financial data to qlib by dump_fix

PIT storage for financial data is under development

Hi @zhupr Is it possible to provide an example of adding financial data? thanks

@vitorying Hi,

1. The user can align financial data to the daily frequency calendar:

import qlib
from qlib.data import D
qlib.init(xxx)
calendar = D.calendar(freq="day")
df = pd.read_csv("sh600000_financial_data.csv")
df.set_index("datetime", inplace=True)
# FIXME: the datetime of the financial data may not be in the calendar
df = df.reindex(calendar)
df = df.fillna(method="ffill")
# df.to_csv(xxx)
# dump csv to qlib data; dump_bin.py:DumpDataFix

2. Using PIT： Support Point-in-time Data Operation #343

[github-actions]

This issue is stale because it has been open for three months with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days