notebooks updates

[JamesAnstey]

Testing notebooks in Binder, suggesting minor edits

[JamesAnstey]

@rigoudyg in HowTo-01_Import_and_Load_the_DR.ipynb this line

DR = dr.DataRequest.from_separated_inputs(**content_dic)

gave me the error:

Could not find element type modelling_realm_-_primaries in the vocabulary server.

But if I use v1.2 content instead:

content_dic = dt.get_transformed_content(version='v1.2')

then DR = dr.DataRequest.from_separated_inputs(**content_dic) was ok. Some issue with v1.2.1 content?

Same thing in other notebooks, e.g. HowTo-02_Discover_What_is_in_DR.ipynb: ok with dt.get_transformed_content(version='v1.2') but not for dt.get_transformed_content() (i.e. latest version of dreq content, v1.2.1).

[JamesAnstey]

In HowTo-03b_Find_Experiments_and_Variables_for_given_Opportunities.ipynb we shouldn't use check_core_variables=False. It seems that when this is True (the default) then some experiments end up with no BCVs included unless the BCVs opportunity is included. I'm not sure if this behaviour is a bug or intended (I can't remember!) so I'll look into it. But since the BCVs should always be included, I've put it in to the used_opps list, and at least for now this fixes the problem.

[rigoudyg]

@rigoudyg in HowTo-01_Import_and_Load_the_DR.ipynb this line

DR = dr.DataRequest.from_separated_inputs(**content_dic)

gave me the error:

Could not find element type modelling_realm_-_primaries in the vocabulary server.

But if I use v1.2 content instead:

content_dic = dt.get_transformed_content(version='v1.2')

then DR = dr.DataRequest.from_separated_inputs(**content_dic) was ok. Some issue with v1.2.1 content?

Same thing in other notebooks, e.g. HowTo-02_Discover_What_is_in_DR.ipynb: ok with dt.get_transformed_content(version='v1.2') but not for dt.get_transformed_content() (i.e. latest version of dreq content, v1.2.1).

@JamesAnstey Thank you for testing.
I saw there is a discussion about it here: #116
But I don't know why there is an issue.
If I test the current code interactively, it works well.
The issue may come from a cache issue. If the two files have been created before the update of the code (with a former version), you have this issue. Maybe, the simplies would be to ensure the cache in removed between two sessions (if it is possible)

[rigoudyg]

or you can use the option force_retrieve=True of the get_transformed_content function

[mpmoine]

@JamesAnstey : Thanks for pointing this out! I reproduces the error as well in binder. I can correct the notebooks with one of Gaëlle's suggestions and test in the binder of my fetched GitHub repo before adding to this PR if it solves the problem.

[mpmoine]

@rigoudyg : I have tested the 2 solutions (cache cleaning and force_retrieve=True) but it does not solve the issue. In fact, all goes well when using DR content v1.1, v1.2 but the error raised by @JamesAnstey and in #116 arises when using v1.2.1.

[rigoudyg]

The issue seems to come from the version available in binder. It seems not to be the last commit of the main branch.
The version get from data_request_api is v1.2.0.
This explains why it does not work

[mpmoine]

Thanks @rigoudyg for the explaination. Do you know if there any manual operation to synchronize binder with the last Software version?

[rigoudyg]

no, I don't know. Dan @wolfiex do you know how to update the code?

[sol1105]

@rigoudyg I installed the latest software release (1.2.1) in binder (!pip install --upgrade CMIP7-data-request-api), then restarted the kernel and made sure it is really version 1.2.1 loaded now. The issue unfortunately remains.

In binder/environment.yml one could force the version to be >=1.2.1 or install it from the main branch. It usually should use the latest version by default (?!), maybe it is some caching issue.

[rigoudyg]

@sol1105 Thanks for testing Martin.
Using the main branch, I managed to make it runs but I had to specify that I use the main branch and update the package in the kernel before using it
If I test the notebook with @JamesAnstey branch, it works. But maybe it's because I have generated the files before and they already exists?

[rigoudyg]

@JamesAnstey I did not have time to add the example because I had an unexpected behaviour with some example. I will try to fix it for next week.

[mpmoine]

from TISG meeting 38:

• issue with binder should be fixed now but leave this PR unmerged until all notebooks re-tested

[sol1105]

To open binder from a certain branch:
https://mybinder.org/v2/gh/CMIP-Data-Request/CMIP7_DReq_Software/notebooks_update?filepath=notebooks?force=1

When I then opened a terminal (File -> New -> Terminal) to make sure, however, I saw the checked out branch is main... so I do not think it works as intended. It may also be an issue with the "fix" I put in place to have our latest pip version installed. But you can just switch the branch and install again:

cd data_request_api
git stash # if there are any local changes
git switch notebooks_update
pip install -e .