Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DDG-DA paper code #743

Merged
merged 47 commits into from
Jan 10, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
48f8694
Merge data selection to main
wendili-cs Jul 1, 2021
5bb06cd
Update trainer for reweighter
wendili-cs Jul 1, 2021
4f442f5
Typos fixed.
wendili-cs Jul 8, 2021
da013fd
Merge branch 'main' into ds
you-n-g Jul 30, 2021
81b4383
update data selection interface
you-n-g Aug 9, 2021
aa2699f
successfully run exp after refactor some interface
you-n-g Aug 13, 2021
d17aaac
data selection share handler & trainer
you-n-g Aug 20, 2021
82b4115
fix meta model time series bug
you-n-g Aug 22, 2021
5b118c4
fix online workflow set_uri bug
you-n-g Sep 13, 2021
3b073f7
fix set_uri bug
you-n-g Sep 26, 2021
384b670
Merge remote-tracking branch 'origin/main' into ds
you-n-g Sep 26, 2021
b0850b0
updawte ds docs and delay trainer bug
you-n-g Sep 27, 2021
051b261
Merge remote-tracking branch 'wd_ds/ds' into ds
you-n-g Oct 9, 2021
f10d726
Merge branch 'main' into ds
you-n-g Nov 14, 2021
cdcfe30
Merge remote-tracking branch 'origin/main' into ds
you-n-g Nov 14, 2021
6d61ad0
Merge remote-tracking branch 'origin/main' into ds
you-n-g Nov 16, 2021
f32a7ad
docs
you-n-g Nov 16, 2021
8fb37b6
resume reweighter
you-n-g Nov 16, 2021
21baead
add reweighting result
you-n-g Nov 16, 2021
12afe61
fix qlib model import
you-n-g Nov 17, 2021
1d9732b
make recorder more friendly
you-n-g Nov 17, 2021
20a8fe5
fix experiment workflow bug
you-n-g Nov 18, 2021
faf3e03
commit for merging master incase of conflictions
you-n-g Dec 9, 2021
76d1bd9
Merge remote-tracking branch 'origin/main' into ds
you-n-g Dec 9, 2021
3bc4030
Successful run DDG-DA with a single command
you-n-g Dec 11, 2021
49c4074
remove unused code
you-n-g Dec 11, 2021
ce66d9a
asdd more docs
you-n-g Dec 13, 2021
cea134d
Update README.md
you-n-g Dec 13, 2021
a4a2b32
Update & fix some bugs.
demon143 Jan 8, 2022
8241832
Update configuration & remove debug functions
wendili-cs Jan 8, 2022
e1b079d
Update README.md
wendili-cs Jan 9, 2022
6a3f471
Modfify horizon from code rather than yaml
wendili-cs Jan 9, 2022
c3364cd
Update performance in README.md
wendili-cs Jan 9, 2022
b3d1081
Merge remote-tracking branch 'origin/main' into ds
you-n-g Jan 9, 2022
fa2d047
fix part comments
you-n-g Jan 9, 2022
efab5cb
Remove unfinished TCTS.
wendili-cs Jan 10, 2022
5a184eb
Fix some details.
wendili-cs Jan 10, 2022
8fee1b4
Update meta docs
wendili-cs Jan 10, 2022
a31a4d5
Update README.md of the benchmarks_dynamic
wendili-cs Jan 10, 2022
ca3fe76
Merge branch 'main' into ds
you-n-g Jan 10, 2022
97f61d5
Update README.md files
wendili-cs Jan 10, 2022
2726560
Merge branch 'ds' of wd_git:you-n-g/qlib into ds
wendili-cs Jan 10, 2022
da68103
Add README.md to the rolling_benchmark baseline.
wendili-cs Jan 10, 2022
7e1183b
Refine the docs and link
you-n-g Jan 10, 2022
b0857c2
Rename README.md in benchmarks_dynamic.
wendili-cs Jan 10, 2022
38b83dd
Remove comments.
wendili-cs Jan 10, 2022
34f5bd2
auto download data
you-n-g Jan 10, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
updawte ds docs and delay trainer bug
  • Loading branch information
you-n-g committed Sep 27, 2021
commit b0850b07051f1c0fa81dda62b2421fff90d0391b
3 changes: 2 additions & 1 deletion qlib/data/dataset/weight.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ def __len__(self):
class SampleReweighter(Reweighter):
"""
The sample-wise reweighter. It aims to reweight by the given weight of each sample.
The samples are indexed in a pandas way
"""

def __init__(self, sample_weights: pd.Series, *args, **kwargs):
Expand All @@ -82,7 +83,7 @@ def _sample_reweight_DataFrame(self, samples: Union[pd.Series, pd.DataFrame], *a
weight.update(self.weights)
return weight

def _sample_reweight_TSDataSampler(self, sampler: TSDataSampler, *args, **kwargs):
def _sample_reweight_TSDataSampler(self, sampler: TSDataSampler, *args, **kwargs) -> WeightSampler:
"""
This function processes the prepared data with TSDataSampler type.

Expand Down
16 changes: 14 additions & 2 deletions qlib/model/meta/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@

class MetaModel(metaclass=abc.ABCMeta):
"""
The meta-model controls the training process.
The meta-model guiding the model learning.

The word `Guiding` can be categorized into two types based on the stage of model learning
- The definition of learning tasks: Please refer to docs of `MetaTaskModel`
- Controlling the learning process of models: Please refer to the docs of `MetaGuideModel`
"""

@abc.abstractmethod
Expand All @@ -21,9 +25,14 @@ def fit(self, *args, **kwargs):
pass

@abc.abstractmethod
def inference(self, *args, **kwargs):
def inference(self, *args, **kwargs) -> object:
"""
The inference process of the meta-model.

Returns
-------
object:
Some information to guide the model learning
"""
pass

Expand All @@ -38,6 +47,9 @@ def prepare_task(self, task: MetaTask) -> dict:
"""
Input a meta task and output a task with qlib format

When modifying the model tasks, the meta model will leverage `self.inference` to get some necessary
information.

Parameters
----------
task : MetaTask
Expand Down
6 changes: 5 additions & 1 deletion qlib/model/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -497,13 +497,17 @@ def train(self, tasks: list, train_func=None, experiment_name: str = None, **kwa
tasks = [tasks]
if len(tasks) == 0:
return []
return super().train(
_skip_run_task = self.skip_run_task
self.skip_run_task = False # The task preparation can't be skipped
res = super().train(
tasks,
train_func=train_func,
experiment_name=experiment_name,
after_status=TaskManager.STATUS_PART_DONE,
**kwargs,
)
self.skip_run_task = _skip_run_task
return res

def end_train(self, recs, end_train_func=None, experiment_name: str = None, **kwargs) -> List[Recorder]:
"""
Expand Down