Development scripts

[klacabane]

Summary

When working with a package I'd like to automate recurring operations like attaching an integration to the running agent or setting up resources needed for my testing on the running services (example in elastic/integrations#4033). These operations could be done in scripts that would be included in the package's _dev folder to avoid bundling them in the artifact.

Adding a _dev/scripts directory is however not supported by the linter:

[2022-08-24T16:20:47.598Z] Error: checking package failed: linting package failed: found 1 validation error:

[2022-08-24T16:20:47.598Z]    1. item [scripts] is not allowed in folder [/var/lib/jenkins/workspace/est-manager_integrations_PR-4033/src/github.com/elastic/integrations/packages/elasticsearch/_dev]

Should we allow a dedicated place in _dev for such scripts ?

[jsoriano]

Without being opposed to the proposal of having a place where to put custom scripts, I wonder if there could be other ways to include the functionality you need.

Looking to the scripts in elastic/integrations#4033:

• generate-logs.sh is used to generate logs in the running elasticsearch, would it make sense to include it in the docker compose scenario? For example you could have a container running this script, and it would be started on system tests or when using elastic-package service.
• install-package.sh is used to install the package. I think that this kind of functionality makes sense in elastic-package install, as is a common operation. Is this subcommand missing something that would be needed in your case?
  • We have an open issue about testing packages with different options: [Testing] Add contract testing for agent policies #947.

[klacabane]

For example you could have a container running this script

Neat, this approach would solve most of the use cases I had in mind so far. Last time I tried though, only the container named similarly as the package was attached to the elastic-package stack network and that would prevent the extra container from easily reaching the stack services. Otoh that's probably not an issue if you keep your testing to the services started by elastic-package service

I think that this kind of functionality makes sense in elastic-package install

I wanted to create an issue for that but figured we'd eventually discuss it here. Extending install to also attach an integration would be very useful. One thing is that it would need to be general purpose and what the script does is harcode all the input configuration of the package with the service values (hostname, log directory), but that could be solved by a cli prompt.

[klacabane]

So I'll create an ER to have the option to attach a package integration to the running Elastic Agent in elastic-package install. I'll also try the suggested approach of having a container running the script, a put the scripts in _dev/deploy if that works

[jsoriano]

Sounds good 👍 we can leave this issue also open by now, maybe it still makes sense to have scripts or other dev files for other use cases.