Add Geometric Time Series Chains

[seanlaw]

There has been recent work to help improve the robustness of time series chains.

See this published conference paper

See this supplementary site

The data for Fig 12 can be found here

See this short YouTube video

[NimaSarajpoor]

Hi Seanlaw,

Thanks for creating this wonderful package. I was wondering if I could help you with implementing this.

I was using TSC17 for my data and got some weird result, and, in my case study, the problem was curse of dimensionality and the possible noises. Then I realized there is this new algorithm that can enhance the quality of the discovered chains.

According to the paper, we need to implement Table 4 and 5 and a few evaluation metrics. I haven't contributed to any project before (in terms of adding a code to a project), so it would be great if you could provide me with a link that shows the rules/styles I should follow. Your package already provides the left matrix profile, which is referred to as "backward matrix profile index" in the Table 4. So, I think we have things ready.

UPDATE:
I implemented this.

[seanlaw]

@ninimama Thank you for your interest in contributing to STUMPY! We recommend going over this Contributor's Guide just to familiarize yourself with the general process.

As for Geometric Time Series Chains, I haven't yet gone over the paper in detail so we certainly welcome contributions! As you can see with all of our Tutorials, we always try to reproduce one or more examples directly from the published work (and using their datasets too) by reproducing one or more of the published figures in the paper. This serves to ensure that what we are doing is consistent with the published work and it allows the user to go back to the paper for more details beyond the tutorial.

So, having said that, I usually recommend forking the main repo, creating a branch, and simply starting by creating a Jupyter notebook in docs/Tutorial_Geometric_chains.ipynb and reproducing one or more of the figures in the original paper. After we're confident that we are able to reproduce the published work accurately (sometimes we can't) then we can start thinking about whether it makes sense for it to stay as a tutorial or if it should be included as native STUMPY functions. I also recommend looking through the codebase and understanding all of the variable naming conventions (i.e., you'll notice that the window size, m, is called the same thing EVERYWHERE and so we'd want to do the same thing where possible rather than coming up with yet another new variable name).

However, I also prefer to not let "perfect be the enemy of good". I think the first thing is to get something working that reproduces one of the published figures and do this inside of a Jupyter notebook. Then, I can provide the relevant feedback and we can proceed from there. Also, please do not hesitate to submit pull requests (PRs) for work in progress.

[seanlaw]

I implemented this.

I am excited to take a look! Would you mind going through the contributor's guide and submitting a PR? Please let me know if I can help walk you through the process.

[seanlaw]

@ninimama I just wanted to point out these 2 Tutorials that I am working on that are still a work-in-progress. Both started as notebook-only implementations and then I migrated some of the parts into STUMPY:

1. https://github.com/TDAmeritrade/stumpy/blob/main/docs/Tutorial_Pan_Matrix_Profile.ipynb
2. https://github.com/TDAmeritrade/stumpy/blob/main/docs/Tutorial_Time_Series_Snippets.ipynb

[NimaSarajpoor]

@seanlaw
Thanks for your response. Sure, I will go through the contributor's guide and submit a PR within the next few days. I will try to find the data used in the original paper and see if I can reproduce it. Thanks for the tutorial as well.

[seanlaw]

I am here to help!

[NimaSarajpoor]

@seanlaw:
I am about to start working on this. I have a couple of questions and I would truly appreciate it if you could explain it to me:

1. Going through the steps provided in the project (fork, clone, create virtual environment, create branch, install dependencies from source and then install the package from the current directory), I faced the following messages when I tried to run:

./setup.sh && ./test.sh

Why does it try to remove the stumpy? Is it ok?

(stumpy_project_env_01) nimas (geometric_chains) stumpy
$ ./setup.sh && ./test.sh
Found existing installation: stumpy 1.8.0
Uninstalling stumpy-1.8.0:
Would remove:
c:\users\nimas\anaconda3\envs\stumpy_project_env_01\lib\site-packages\stumpy-1.8.0.dist-info*
c:\users\nimas\anaconda3\envs\stumpy_project_env_01\lib\site-packages\stumpy*
Proceed (y/n)? Successfully uninstalled stumpy-1.8.0
Processing e:+machine_learning_journey\contributions\stumpy_project\stumpy
Requirement already satisfied: numpy>=1.15 in c:\users\nimas\anaconda3\envs\stumpy_project_env_01\lib\site-packages (from stumpy==1.8.0) (1.20.3)
Requirement already satisfied: scipy>=1.5 in c:\users\nimas\anaconda3\envs\stumpy_project_env_01\lib\site-packages (from stumpy==1.8.0) (1.6.3)
Requirement already satisfied: numba>=0.48 in c:\users\nimas\anaconda3\envs\stumpy_project_env_01\lib\site-packages (from stumpy==1.8.0) (0.53.1)
Requirement already satisfied: llvmlite<0.37,>=0.36.0rc1 in c:\users\nimas\anaconda3\envs\stumpy_project_env_01\lib\site-packages (from numba>=0.48->stumpy==1.8.0) (0.36.0)
Requirement already satisfied: setuptools in c:\users\nimas\anaconda3\envs\stumpy_project_env_01\lib\site-packages (from numba>=0.48->stumpy==1.8.0) (49.6.0.post20210108)
Building wheels for collected packages: stumpy
Building wheel for stumpy (setup.py): started
Building wheel for stumpy (setup.py): finished with status 'done'
Created wheel for stumpy: filename=stumpy-1.8.0-py3-none-any.whl size=109207 sha256=89ecc598513da3680f564a2782760a52b261c01e3d74b0ccaea221f7388a68c2
Stored in directory: C:\Users\nimas\AppData\Local\Temp\pip-ephem-wheel-cache-5g189ppz\wheels\08\17\1e\f2d537c2d4765fba444dddb4f288da590a8b1151b8a7444d56
Successfully built stumpy
Installing collected packages: stumpy
Successfully installed stumpy-1.8.0
Cleaning Up
Checking Black Code Formatting

================================================================

2. Then, it continues the testing procedure, at the end, I got the following message:

============================= 34 passed in 49.34s =============================
============================= test session starts =============================
platform win32 -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: E:+Machine_Learning_Journey\contributions\stumpy_project\stumpy, configfile: pytest.ini
collected 9 items

tests\test_stimp.py ......... [100%]

============================= 9 passed in 28.31s ==============================
Cleaning Up
Disabling Numba JIT and CUDA Compiled Functions
Testing Code Coverage
./test.sh: line 131: coverage: command not found
Error: pytest encountered exit code 127

=======================================================================
Thanks,
Nima

[seanlaw]

Why does it try to remove the stumpy? Is it ok?

Good question. If you look inside of setup.sh, you'll find that it's a bash shell script that helps you set up install stumpy. However, there are times when the user may have already installed stumpy from conda-forge/PyPI and so the first thing that we do is ALWAYS uninstall any version of stumpy that may be currently installed. That is why you are seeing that message. Then, it re-installs stumpy but using the local, cloned, development version.

Then, as you make code changes, it is again best to uninstall and reinstall stumpy (just in case) before running the unit tests. This helps to ensure that your local changes are being used and incorporated.

From what I can tell, things installed properly. Great!

Then, it continues the testing procedure, at the end, I got the following message:

So, while stumpy has only 3 main package dependencies (numpy, scipy, and numba). There are several more dependencies that you'll need to install to run the unit tests. In fact, the last part of the error is relevant:

Testing Code Coverage
./test.sh: line 131: coverage: command not found
Error: pytest encountered exit code 127

More specifically line 131: coverage: command not found suggests that you have not installed the pytest-cov package. So, considering that you are installing the development version of stumpy in a conda environment, I have a suggestion:

1. Create a new conda env
2. cd machine_learning_journey\contributions\stumpy_project\stumpy
3. ./conda.sh
4. ./setup.sh && ./test.sh

So, the conda.sh script is something that I use regularly to automatically make sure that all required packages are installed in your conda environment and the environment is up-to-date before setup.sh is executed. My guess is that everything should work thereafter. I regularly delete my anaconda directory, reinstall it, and then run the conda.sh scriptbefore setting upstumpy` and testing it in a clean environment using these scripts.

Please let me know if you have any questions and I'd be happy to clarify them.

Based on your wonderful questions, it is clear to me that the documentation is missing some information and we should update it! Would you mind submitting a PR for this?

[NimaSarajpoor]

Thanks for the thorough response. I get it now. I tried to create the env using git bash in windows. Then I realized I should have done the command "conda init bash" and close/open the bash and then activate the env. This issue is discussed here:
stack overflow

However,first time I thought I activated the env, but I didn't notice. That'a why the stumpy was installed in the global env. Then I deleted the local git repo and forgot to uninstall the stumpy. That's why it was uninstalled in my second attempt of going through the whole process.

I will try to go through the steps you provided.

I can try to clarify these things in the document. I am somehow novice but I hope I can do something useful. 😊

[NimaSarajpoor]

@seanlaw
Thanks to your clarification, everything works fine now. I was wondering if you could elaborate one thing for now:

I regularly delete my anaconda directory, reinstall it, and then run the conda.sh script before setting up stumpy and testing it in a clean environment using these scripts.

Particularly, the part where you said you are regularly delete and reinstall the anaconda. Because, we do all the stuff in the local env now. Right? Is there any reason for bothering ourselves with reinstalling the anaconda ?

[seanlaw]

Because, we do all the stuff in the local env now. Right? Is there any reason for bothering ourselves with reinstalling the anaconda ?

In your case you have created a separate env so, no, there is no need. I am probably less disciplined (or I don't like or am too lazy to track so many different envs) and so I do not create separate envs for everything. Instead, I reinstall from scratch whenever I switch projects. Just a different approach.

[NimaSarajpoor]

@seanlaw:
I reproduced two figures from the supplementary materials and also got the same result for one part of the paper. I will work on one more figure (I am just getting warmed up 😃).

I will check the name of variables and styles. Then I will submit PR. Then, I will try to add text to the tutorial notebook.

I haven't done any unit test in my life 😄; so, I hope you can help me with creating one for this if you are thinking of adding it to the package.

[seanlaw]

@ninimama That sounds great! I want to make one small suggestion. In software development, it is often better to submit an initial (small) PR. Then, as you get feedback and make changes, you can push your new updates to the same PR and the history is preserved and kept together.

The key thing is (unlike writing an academic paper where you'd hand in a completed manuscript for review), it is best to submit a small PR (or updates) in small bite size chunks. So, preferably, each Git commit is for "one small, related thing". Of course, what constitutes as "small" is somewhat subjective but the goal is to make each commit document the development process and capture some of the logic over time. Versus handing a single giant commit of the final product (this is less desirable). The shorter commits that you share also ensures that we have a tighter feedback loop where I can help confirm you initial approach, steer you in a different direction, or we can also decide to stop early. The goal is avoid wasting yours/my precious time when a simple review for feedback and resolve things.

Let me know if it makes sense. Again, let's not let "perfect be the enemy of good". I appreciate your help