Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow combinatorics on provider::Baked #3935

Open
sffc opened this issue Aug 24, 2023 · 5 comments
Open

Allow combinatorics on provider::Baked #3935

sffc opened this issue Aug 24, 2023 · 5 comments
Assignees
Labels
C-data-infra Component: provider, datagen, fallback, adapters question Unresolved questions; type unclear

Comments

@sffc
Copy link
Member

sffc commented Aug 24, 2023

Users should be able to chain provider::Baked providers to make a multi-crate AnyProvider.

@Manishearth suggested aked.add(otherprovider::Baked) making a fork provider.

@sffc sffc added C-data-infra Component: provider, datagen, fallback, adapters discuss Discuss at a future ICU4X-SC meeting labels Aug 24, 2023
@sffc sffc added this to the Priority Backlog ⟨P3⟩ milestone Aug 24, 2023
@robertbastian
Copy link
Member

robertbastian commented Aug 25, 2023

crate::provider::Baked is an implementation detail in the unstable provider modules. Users can use compiled data by depending on the data crates and creating their own baked providers:

struct MyBakedProvider;

icu_datetime_data::impl_datetime_chinese_datelengths_v1!(MyBakedProvider);
icu_datetime_data::impl_datetime_chinese_datesymbols_v1!(MyBakedProvider);
icu_list_data::impl_and_list_v1!(MyBakedProvider);

@sffc sffc changed the title All combinatorics on provider::Baked Allow combinatorics on provider::Baked Aug 25, 2023
@sffc
Copy link
Member Author

sffc commented Sep 5, 2023

Discuss with:

@sffc sffc added the discuss-triaged The stakeholders for this issue have been identified and it can be discussed out-of-band label Sep 5, 2023
@sffc
Copy link
Member Author

sffc commented Oct 31, 2023

Discussion: revisit this in the context of #3947, when we remove AnyProvider. Leaving some breadcrumbs for my mental model of this and #1893:

  • We should have a macro impl_forking_keyed_provider!([key1, key2, key3], Provider1, Provider2) that generates a impl<M: KeyedDataMarker> DataProvider<M> where the list of keys go to Provider1 and any key not in the list goes to Provider2. Note that DCE should work fine through this layer because the match arm is statically known based on M::KEY.
  • The provider::Baked impls can have a function that internally contains this match statement for all of the keys it supports.

@sffc sffc removed discuss Discuss at a future ICU4X-SC meeting discuss-triaged The stakeholders for this issue have been identified and it can be discussed out-of-band labels Oct 31, 2023
@sffc
Copy link
Member Author

sffc commented Nov 6, 2023

Another use case here is if you want your language packs to contain only non-singleton keys and get the singleton keys from compiled data with the non-singleton keys from dynamically loaded data.

A few approaches here:

  1. Use with_any_provider constructors and a registry macro that forks between the singleton compiled data and the blob dynamic data
  2. Use _unstable constructors (but then we don't handle older blob data)
  3. Use a new type of constructor called _unstable_with_backwards_compatibility
  4. Use _with_buffer_provider and export the singleton keys as blobs from the data crates
  5. In 2.0 make _with_provider constructors that take an enum between BufferProvider and AnyProvider (or equivalent) and fork between the singleton static data and the blob data (and use this opportunity to reduce the number of constructors -- rather than deleting _with_any_provider we merge it in with the buffer provider constructor, which slightly increases the buffer provider constructor code size but only by the size of a ZeroFrom)

@sffc
Copy link
Member Author

sffc commented Mar 14, 2024

Probably doesn't block 2.0 so removing it from that milestone, but it is still important.

@sffc sffc self-assigned this Mar 14, 2024
@sffc sffc added the question Unresolved questions; type unclear label Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-data-infra Component: provider, datagen, fallback, adapters question Unresolved questions; type unclear
Projects
None yet
Development

No branches or pull requests

2 participants