feat: initial oas generators support; algod http clients with tests covering transaction endpoints; refinements in python ffi package

[aorumbayev]

Proposed changes

This PR introduces the infrastructure for generating Algorand API clients using the OpenAPI Generator tool. This approach automates the creation of clients in various languages (initially focusing on Python and TypeScript for algokit-utils) from a single OpenAPI specification, ensuring consistency and reducing maintenance overhead.

Motivation:

As outlined in docs/research/openapi-generators.md, manually maintaining API clients across multiple languages is time-consuming and error-prone. Using OpenAPI generators allows us to:

• Generate consistent clients from a single source of truth (the OpenAPI spec).
• Easily propagate API changes to all supported languages.
• Leverage a mature, well-maintained open-source tool.

Changes:

1. Directory Structure: Introduced new directories and files under /api to support the generation process:
   • api/specs/: Contains the OpenAPI specifications (v2 and v3).
   • api/api_clients/: Will hold the generated client code.
   • api/oas_templates/: Stores custom Mustache templates for tailoring generated code.
   • api/scripts/: Includes utility scripts (generate-api-clients.ts) for managing the generation process.
   • Configuration files (openapitools.json, package.json, tsconfig.json, openapi-config.yaml) to configure the generator and manage dependencies (like bun for running the generation script).
2. Specification Handling:
   • Uses the v3 OpenAPI spec derived from the official Go Algorand source for better generator compatibility.
   • Includes minor improvements to the spec for clarity and generator compatibility (e.g., formatting, fixing AvmValue schema for Python).
3. Generation Script: Added scripts/generate-api-clients.ts (run via bun run) to orchestrate the client generation, including handling .openapi-generator-ignore files to protect custom code.
4. Initial Client Focus: The initial setup targets core transaction endpoints (get pending txns, get tx params, post raw txn) needed for algokit-utils integration.
5. Testing Strategy: Adopted a custom testing approach (as detailed in the research doc) rather than modifying generated tests, allowing for more robust and maintainable tests tailored to our environment (e.g., Sandbox dependencies).
6. Documentation: Added docs/research/openapi-generators.md detailing the approach, structure, rationale, and future work.
7. Dependency Updates: Updated poetry.lock and pyproject.toml in packages/python/algokit_transact reflecting changes potentially related to CFFI or other dependencies needed for the generated clients or testing infrastructure. (Specific updates include cffi, maturin, packaging, pluggy, pytest, tomli, and adjustments to dev dependencies).

This PR lays the baseline for a more scalable and maintainable approach to managing Algorand API clients across different languages within the monorepo project.

[joe-p]

Haven't reviewed in-depth yet, but would it be possible make the Python changes in a separate PR? We have merge conflicts anyways after #61 and having that work in a separate PR will make it a bit easier to review and iterate.

[aorumbayev]

@joe-p moved python ffi changes to #65

[joe-p]

Could we add extension to the moustache template files to make them a bit easier to digest? For example, instead of conftest.moustache it could be conftest.py.moustache

[aorumbayev]

@joe-p adding extensions to mustache templates, while possible with mustache templating in general, wouldn't fit well to our use case. The base template was generated via oas generator command that stores the base files to the filesystem so developers can then extend them. The problem is that extension mapping for template files included in the base template are all postfixed with .mustache, if we are to preserve the extensions from original base templates we would have to manually update the files and then original generator would threat them as supporting external files instead of recognizing as files from base templates. Whenever potential updates in base templates arrive it would be hard for us to manually keep track of changes and remapping against latest templates we maintain with the file extensions.

Once we have CD for clients, we can add the generated clients back to version control for tracking output stability, at that point it would be fairly easy to traceback from generated file back to mustache template by the filename. In most cases those would be custom template files for tests and those also contain explicit filename specification in the spec files per template folder. https://github.com/algorandfoundation/algokit-core/blob/spike/oas_clients/api/oas_templates/python/openapi-config.yaml

[engineering-ci]

🎉 This PR is included in version 1.0.0-alpha.1 🎉

The release is available on:

• GitHub release
• npm package (@main dist-tag)

Your semantic-release bot 📦🚀

[engineering-ci]

🎉 This PR is included in version 1.0.0-alpha.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[engineering-ci]

🎉 This PR is included in version 1.0.0-alpha.1 🎉

The release is available on:

• GitHub release
• npm package (@main dist-tag)

Your semantic-release bot 📦🚀

[engineering-ci]

🎉 This PR is included in version 1.0.0-alpha.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀