Skip to content

[pigeon] Add end-to-end integration_test tests for all generators #111505

Closed
flutter/packages
#4484
@stuartmorgan-g

Description

@stuartmorgan-g

Currently the Pigeon testing is using unit tests on the Dart and native sides; this is useful, but leaves open the potential for unit tests on each side to pass, but for the protocol being unit tested to mis-align slightly such that the end-to-end call doesn't actually work. #111083 is an example of a case that should have been trivially caught by basic integration tests, but has to be more explicitly tested correctly with unit tests based on details of the serialization format on each end. (We also don't have that unit test on the Swift side, which is a problem, but a simple test would be even better.)

I would suggest that we rework the platform_tests directory so that instead of having a combination (depending on language) of bespoke tests and per-language dummy plugins, we just have one single test_plugin plugin that supports every platform/language (or I guess, two plugins: one for ObjC and Java, and one for everything else, since they can't really co-exist with their alternate languages in the same plugin). That plugin would have:

  • Unit tests for each language, using exactly the same structure that native unit tests in flutter/plugins use. That's already how Windows works, and the others could be converted.
  • integration_test tests that would drive full end-to-end calls for a variety of core use cases.

(Eventually I would like to revisit using the plugin repo tooling to drive all of this; I temporarily abandoned that when setting up Windows because it would involve more tool changes than I had time to make, but it would make things more sustainable long term that maintaining lots of run_tests.{dart,sh} logic. Using this structure would have the added benefit of setting ourselves up for that being easier to add later.)

/cc @gaaclarke @tarrinneal

Checklist:

  • Add Swift macOS integration tests.
  • Add Swift iOS integration tests.
    • These are mostly redundant with macOS, but there are features like background platform channels that we don't currently support on desktop. And the added cost of having both should be low. (I don't want to do iOS-only, because for local iteration a macOS integration test is way faster.)
  • Add Windows integration tests.
  • Add Java Android integration tests, local-only (not in CI)
  • Add Kotlin Android integration tests, local-only (not in CI)
  • Update [pigeon] Consolidate platform test harnesses part 3.2 - iOS ObjC packages#2816 to include adding Obj-C integration tests
  • Migrate the handful of (Obj-C-only) tests that are currently in the e2e_tests directory to the new structure, and eliminate e2e_tests.
  • Enable Java and Kotlin integration tests in CI
    • This is currently hard, which is why it's a separate step:
      • We could use FTL, like we do in flutter/plugins, but that's a lot of complexity/setup to bring into the tooling (or we need to adapt the repo tooling to work with Pigeon's unusual structure), and:
      • I'm hopeful that we can use the new (WIP?) LUCI Android emulator support instead, but that will require us to get that part of the testing moved over to LUCI, and emulator support wired in.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority issues at the top of the work lista: tests"flutter test", flutter_test, or one of our testsc: contributor-productivityTeam-specific productivity, code health, technical debt.p: pigeonrelated to pigeon messaging codegen toolpackageflutter/packages repository. See also p: labels.team-ecosystemOwned by Ecosystem teamtriaged-ecosystemTriaged by Ecosystem team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions