Run all examples as part of E2E tests

[akihikokuroda]

Is your feature request related to a problem? Please describe.
The examples/agents/bee got broken multiple times in the last a few days. #55 #60

Describe the solution you'd like
Add e2e tests that runs the example so that the issue is detected in the pull request.

Describe alternatives you've considered
Run the example manually before creating the pull requests.

Additional context
Add any other context or screenshots about the feature request here.

[akihikokuroda]

I want to work on this. Please assign to me if it's OK. Thanks!

[Tomas2D]

I appreciate your dedication. However, there are a few things you need to consider.

• Ollama is not available in Github Actions/
• Don't test against a specific output.
• Feel free to check the current E2E implementation. Some event names may changed or may not be complete.

[Tomas2D]

It would be great to be able to run all code snippets in examples and exclude some of them if necessary.
I would like to be able to run standalone E2E tests and examples.

yarn test:examples # all examples
yarn test:examples examples/agents/bee.ts # single example

It would be great to use test for that, with the ability to include/exclude examples on the fly. A typical contributor can't run all the examples because they may not have access to certain environments, etc.

Feel free to come up with something or ask for guidance.

[akihikokuroda]

I need some guidance. I'm still learning TypeScript. The examples are excluded in /tsconfig.json" and are under "/examples/tsconfig.json". There is another "/tsconfig.examples.json" that extends "/examples/tsconfig.json" and referred by scripts in "package.json".

I tried to import the examples from the tests but so far I couldn't figure out how. I probably can copy the examples in another location under "src", tweak the import statements and import from the test program in the execution of the tests but it's not nice.

Is there any good way to import the example from the test program?
How about create a separate package of examples?

Thanks!

[Tomas2D]

Let me clarify.

• tsconfig.json root TS configuration
• examples/tsconfig.json - scoped TS configuration that we are using only because it enables imports from bee-agent-framework instead of @ so the user can easily copy-paste and run the code without any hustle.

Don't import tests directly into' test`. It would be easier to run them as standalone scripts (they are scripts; they do not export anything).

Look at how we implemented this in IBM Generative AI Python SDK - test examples script.