add existing TS BO by gezelligheid · Pull Request #6 · TimeEval/GutenTAG

gezelligheid · 2023-02-06T19:44:10Z

One can now add synthetic anomalies to existing univariate TS.

removed obsolete comment

*anonymized data file names *changed naming custom process to custom input *update requirements.txt *removed notebooks *replaced assertion with try except

add .venv variants to ignore

fixed import naming

codecov · 2023-02-15T14:32:56Z

Codecov Report

Merging #6 (fdbd6dd) into main (0a756c4) will increase coverage by 0.15%.
The diff coverage is 96.29%.

@@            Coverage Diff             @@
##             main       #6      +/-   ##
==========================================
+ Coverage   89.57%   89.73%   +0.15%     
==========================================
  Files          52       53       +1     
  Lines        2034     2084      +50     
==========================================
+ Hits         1822     1870      +48     
- Misses        212      214       +2

Flag	Coverage Δ
unittests	`89.73% <96.29%> (+0.15%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
gutenTAG/utils/compatibility.py	`75.00% <ø> (ø)`
gutenTAG/utils/default_values.py	`100.00% <ø> (ø)`
gutenTAG/base_oscillations/custom_input.py	`94.59% <94.59%> (ø)`
gutenTAG/base_oscillations/__init__.py	`100.00% <100.00%> (ø)`
gutenTAG/base_oscillations/interface.py	`94.20% <100.00%> (+0.35%)`	⬆️
gutenTAG/base_oscillations/random_walk.py	`96.96% <100.00%> (ø)`
gutenTAG/consolidator.py	`94.73% <100.00%> (+0.19%)`	⬆️
gutenTAG/generator/timeseries.py	`66.36% <100.00%> (ø)`
gutenTAG/utils/global_variables.py	`100.00% <100.00%> (ø)`

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

To do: -errors from type checker -testing all anomaly types -document working of the TS input file path -(optional) second path for train data

Previous to dos + creating updated dummy data + create test config representing all custom input compatible anomaly types

+ add dummy input

+ document input file path structure

remaining to dos: - test all anomaly types - resolve type checking errors

visualizations to verify correctness of adding anomaly types

dbroeh

Hi VANRIJNA,

thanks for providing this PR. From my end, this looks good 👍 I did spot only a few minor things. I did not run this though.

Cheers, David

SebastianSchmidl · 2023-03-14T07:57:44Z

@gezelligheid Thank you very much for the effort you put into this PR!

There are still some tests failing. However, I think despite the mentioned minor issues, this PR reached a good state. Let's try to fix the failing tests and type checks and, then, we can merge this into main 👍🏼

You can also run the tests locally:

python setup.py typecheck
python setup.py test

gezelligheid · 2023-03-14T09:37:34Z

typecheck won't run giving the following message:
running typecheck
.venv/lib/python3.11/site-packages/numpy/init.pyi:641: error: Positional-only parameters are only supported in Python 3.8 and greater
def seek(self, offset: int, whence: int, /) -> object: ...
^
Found 1 error in 1 file (errors prevented further checking)

Which puzzles me as I believe I run 3.11.2

SebastianSchmidl · 2023-03-14T15:02:58Z

Which puzzles me as I believe I run 3.11.2

This might be the problem and solution likewise. We perform our type checks in Python 3.7 to ensure backwards-compatibility.

typechecking

supevised parameters

typechecking Co-authored-by: Sebastian Schmidl <CodeLionX@users.noreply.github.com>

*Reset file mode of pycharm-create-schema-mapping.png to 755 *Remove unused generation configs *Add custom_input unittest

…ntag-custom-time-series into existing_ts_bo

SebastianSchmidl · 2023-03-28T15:48:27Z

Please fix the CI issues.

gezelligheid · 2023-04-05T12:39:54Z

Thank you for your patience and support @CodeLionX . Is it ok like this?

SebastianSchmidl · 2023-04-05T12:53:23Z

Thank you for your patience and support @CodeLionX . Is it ok like this?

Yes, this looks like it fixed the typing and test issues. 👍🏼

However, your latest commit 3211be3 also changed the line ending of many files. Please revert to standard \n (LF) line endings for all files!

gezelligheid · 2023-04-05T15:01:09Z

I forgot to save al CRLF to LF changes before I committed and then corrupted my repo with a reckless bash command. I hope to get to doing it right later today

SebastianSchmidl

Please check the line endings again. I still found the following files with CRLF:

gutenTAG/config/parser.py
gutenTAG/config/schema/anomaly.guten-tag-generation-config.schema.yaml
gutenTAG/config/schema/oscillation.guten-tag-generation-config.schema.yaml
gutenTAG/utils/compatibility.py
gutenTAG/utils/default_values.py
gutenTAG/utils/global_variables.py

…ntag-custom-time-series into existing_ts_bo

SebastianSchmidl

lgtm

VANRIJNA added 2 commits February 6, 2023 20:11

support existing timeseries

d9dd925

Update custom_process.py

8ed357a

removed obsolete comment

SebastianSchmidl assigned gezelligheid Feb 7, 2023

SebastianSchmidl mentioned this pull request Feb 7, 2023

Use existing TS file as base oscillation #9

Open

Sebastian Schmidl and others added 6 commits February 7, 2023 15:43

Merge branch 'main' into existing_ts_bo

b66a8c2

fixed style issues

73a90b3

*anonymized data file names *changed naming custom process to custom input *update requirements.txt *removed notebooks *replaced assertion with try except

updated ignore and example configs

dca2458

add .venv variants to ignore

Update __init__.py

43fc588

fixed import naming

deleted example data

4abe883

Merge branch 'main' into existing_ts_bo

a7f2927

wenig reviewed Feb 15, 2023

View reviewed changes

Comment thread gutenTAG/base_oscillations/custom_input.py Outdated

SebastianSchmidl requested changes Feb 15, 2023

View reviewed changes

fixed issues and column specification

aa2d3ac

To do: -errors from type checker -testing all anomaly types -document working of the TS input file path -(optional) second path for train data

SebastianSchmidl reviewed Feb 21, 2023

View reviewed changes

Comment thread gutenTAG/base_oscillations/custom_input.py Outdated

SebastianSchmidl reviewed Feb 21, 2023

View reviewed changes

Comment thread gutenTAG/utils/dataframe_operations.py Outdated

gezelligheid added 7 commits February 21, 2023 14:35

corrected input file row length check

08c0e00

Previous to dos + creating updated dummy data + create test config representing all custom input compatible anomaly types

removed obsolete file

039a72e

+ add dummy input

fixed custom_input params

4afa0e4

+ document input file path structure

update import custom_input

22c339e

add train and test path custom input

088b90c

remaining to dos: - test all anomaly types - resolve type checking errors

added visualizations

b07dc5a

visualizations to verify correctness of adding anomaly types

Merge branch 'main' into existing_ts_bo

ddbfb17

dbroeh reviewed Mar 3, 2023

View reviewed changes

Comment thread gutenTAG/api/bo.py Outdated

Comment thread doc/introduction/base-oscillations.md Outdated

Comment thread gutenTAG/base_oscillations/custom_input.py Outdated

Comment thread gutenTAG/base_oscillations/random_walk.py

Comment thread gutenTAG/base_oscillations/random_walk.py

SebastianSchmidl reviewed Mar 14, 2023

View reviewed changes

Comment thread gutenTAG/api/bo.py Outdated

SebastianSchmidl reviewed Mar 14, 2023

View reviewed changes

Comment thread tests/configs/example-config-custom-input-all-compatible-types.yaml Outdated

fix rw and misc

f9d94e6

gezelligheid and others added 3 commits March 14, 2023 17:18

Update custom_input.py

37b3afb

typechecking

undo reversions

7d2a955

supevised parameters

Merge branch 'main' into existing_ts_bo

4a86d5c

SebastianSchmidl requested changes Mar 22, 2023

View reviewed changes

Comment thread doc/images/pycharm-create-schema-mapping.png

Comment thread doc/images/pycharm-select-schema.png

Comment thread gutenTAG/base_oscillations/custom_input.py Outdated

Comment thread tests/configs/example-config-custom-input-all-compatible-types.yaml Outdated

gezelligheid and others added 4 commits March 22, 2023 09:17

Update gutenTAG/base_oscillations/custom_input.py

7a84248

typechecking Co-authored-by: Sebastian Schmidl <CodeLionX@users.noreply.github.com>

preparation merge existing_ts_bo

c125f5b

*Reset file mode of pycharm-create-schema-mapping.png to 755 *Remove unused generation configs *Add custom_input unittest

Merge branch 'existing_ts_bo' of https://github.com/gezelligheid/gute…

6da7dad

…ntag-custom-time-series into existing_ts_bo

Merge branch 'main' into existing_ts_bo

3cd9e31

gezelligheid requested a review from SebastianSchmidl March 28, 2023 14:50

fixed tests and typecheck

3211be3

Revert line endings to standard \n (LF)

04f57aa

Fixed revert line endings to standard \n (LF)

43742fb

SebastianSchmidl requested changes Apr 6, 2023

View reviewed changes

Sebastian Schmidl and others added 3 commits April 6, 2023 08:12

Merge branch 'main' into existing_ts_bo

ae6a014

Fixed remaining revert line endings to standard \n (LF)

4e009ab

Merge branch 'existing_ts_bo' of https://github.com/gezelligheid/gute…

fdbd6dd

…ntag-custom-time-series into existing_ts_bo

gezelligheid requested a review from SebastianSchmidl April 6, 2023 07:20

SebastianSchmidl approved these changes Apr 6, 2023

View reviewed changes

SebastianSchmidl merged commit 42816ec into TimeEval:main Apr 6, 2023

gezelligheid deleted the existing_ts_bo branch April 6, 2023 17:05

Conversation

gezelligheid commented Feb 6, 2023

Uh oh!

codecov Bot commented Feb 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dbroeh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SebastianSchmidl commented Mar 14, 2023

Uh oh!

gezelligheid commented Mar 14, 2023

Uh oh!

SebastianSchmidl commented Mar 14, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SebastianSchmidl commented Mar 28, 2023

Uh oh!

gezelligheid commented Apr 5, 2023

Uh oh!

SebastianSchmidl commented Apr 5, 2023

Uh oh!

gezelligheid commented Apr 5, 2023

Uh oh!

SebastianSchmidl left a comment

Choose a reason for hiding this comment

Uh oh!

SebastianSchmidl left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov Bot commented Feb 15, 2023 •

edited

Loading