Automatically download indexes, if missing, in gRPC Init call

[cmaglie]

Please check if the PR fulfills these requirements

See how to contribute

• The PR has no duplicates (please search among the Pull Requests
  before creating one)
• The PR follows
  our contributing guidelines
• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)
• UPGRADING.md has been updated with a migration guide (for breaking changes)
• configuration.schema.json updated if new parameters are added.

What kind of change does this PR introduce?

This PR changes the behavior of the Init function of the gRPC API.

What is the current behavior?

Previously if the main indexes package_index.json or library_index.json were missing (for example after installing on a clean system) the gRPC client had to call UpdateIndex or UpdateLibrariesIndex to download the indexes and re-Init the gRPC instance.

What is the new behavior?

The indexes download is performed automatically.

Does this PR introduce a breaking change, and is titled accordingly?

It changes some golang API calls as described in the UPGRADING.md.
The gRPC is backward compatible, but the old clients may not be optimized and may force the download of the index twice.

Other information

Fix #1529

[codecov]

Codecov Report

Patch coverage: 91.25% and project coverage change: +0.03 🎉

Comparison is base (1877431) 62.87% compared to head (01b444c) 62.90%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2119      +/-   ##
==========================================
+ Coverage   62.87%   62.90%   +0.03%     
==========================================
  Files         218      218              
  Lines       19474    19452      -22     
==========================================
- Hits        12244    12237       -7     
+ Misses       6146     6134      -12     
+ Partials     1084     1081       -3     

Flag	Coverage Δ
unit	62.90% <91.25%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
internal/cli/lib/search.go	54.91% <66.66%> (-5.89%)	⬇️
commands/instances.go	60.96% <88.63%> (+2.51%)	⬆️
arduino/resources/index.go	49.01% <91.66%> (+3.65%)	⬆️
commands/core/list.go	86.95% <100.00%> (ø)
commands/daemon/daemon.go	28.97% <100.00%> (+1.19%)	⬆️
internal/cli/arguments/completion.go	87.67% <100.00%> (ø)
internal/cli/arguments/reference.go	100.00% <100.00%> (ø)
internal/cli/core/list.go	93.61% <100.00%> (ø)
internal/cli/core/search.go	74.69% <100.00%> (-0.89%)	⬇️
internal/cli/core/update_index.go	100.00% <100.00%> (ø)
... and 4 more

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[kittaakos]

This is looking very promising. 😍 The sister PR in the IDE2 repo is here: arduino/arduino-ide#2023.

A few remarks:

• IDE2 would like to recover the 3rd party package indexes on startup. Is it possible? It does not work currently: IDE2 cannot recover 3rd party package index if the package_${3RD_PARTY_PACKAGE_NAME}_index.json is deleted between sessions arduino-ide#2021.
• IDE2 does a periodic (every 4 hours) index update. To correctly calculate the four hours timeframe, IDE2 must know if the index has been recovered during the app startup. With this PR, it's impossible, as IDE2 does not know if the index has been updated.
• TBD (I am going to update this message with additional items...)

[cmaglie]

I'm still not fully convinced this is the right thing to do, BTW I've implemented this one:

IDE2 would like to recover the 3rd party package indexes on startup. Is it possible? It does not work currently: arduino/arduino-ide#2021.

so we can try it.

The visible change to the user is that now, as soon as you add a 3rd party package_index URL to the config, the CLI will automatically download the index, if not already present, at the first use (you don't need to do the first core update-index anymore).
BTW you still need to do update-index if you want to update it.

IDE2 does a periodic (every 4 hours) index update. To correctly calculate the four hours timeframe, IDE2 must know if the index has been recovered during the app startup. With this PR, it's impossible, as IDE2 does not know if the index has been updated.

The extra auto-update should not happen (only at the first startup maybe? or if the user explicitly deletes an index from the data dir... how often this happens?).
BTW In the worst-case scenario, the only visible effect would be the two index updates in a row, not a big deal IMHO.

[kittaakos]

so we can try it.

Great. I will update the IDE2 branch soon.

how often this happens?).
the only visible effect would be the two index updates in a row, not a big deal IMHO.

I completely agree. Let's not overcomplicate the index update strategies.