Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split DataProvider into ResourceProvider and DynProvider #1554

Merged
merged 66 commits into from
Feb 3, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
238a44c
Initial ResourceMarker and ResourceProvider definitions
sffc Jan 20, 2022
0c7f4f2
Checkpoint
sffc Jan 20, 2022
b60b909
Remove redundant module annotation
sffc Jan 24, 2022
a9e1b8d
Big refactor around new traits
sffc Jan 27, 2022
10dbe6f
cargo check --all-targets --all-features in core crate
sffc Jan 27, 2022
600cfbf
icu_provider_blob building
sffc Jan 27, 2022
d2ce258
icu_provider_fs building
sffc Jan 27, 2022
3b0b5c9
Delete blanket impls
sffc Jan 27, 2022
e591f9a
Add DataProvider::cast method
sffc Jan 28, 2022
aaa8132
icu_properties via DynProvider
sffc Jan 28, 2022
9a19c8c
Fixing up some components
sffc Jan 28, 2022
9098d3e
Merge remote-tracking branch 'upstream/main' into resourcemarker
sffc Jan 28, 2022
c1af44d
More migrations
sffc Jan 28, 2022
70b6821
More migrations
sffc Jan 28, 2022
86b1e91
ResourceProvider in plurals
sffc Jan 28, 2022
f7412eb
Split plural rules constructors
sffc Jan 28, 2022
f330a8a
Migrate call sites to type-specific PluralRules constructors
sffc Jan 28, 2022
579179b
fmt
sffc Jan 28, 2022
beaca9c
Add basic tests to icu_provider_macros
sffc Jan 29, 2022
4174efd
Add basic ResourceMarker to macro
sffc Jan 29, 2022
e1ecce8
Improve test
sffc Jan 29, 2022
54a6a94
Add multi-key support
sffc Jan 29, 2022
7077b75
Make docs more consistent
sffc Jan 29, 2022
d72d419
Add docs example
sffc Jan 29, 2022
f858b38
Add namespace to ZeroCopyFrom in yoke derive
sffc Jan 29, 2022
1e6b178
Use new data_struct attribute across components
sffc Jan 29, 2022
d97f077
Clean up imports
sffc Jan 29, 2022
c885e29
uprops building
sffc Jan 29, 2022
1418c81
Migrating cldr
sffc Jan 29, 2022
c0e3444
ICU4X cargo check --all-features
sffc Jan 29, 2022
91e0f5e
Fixing some tests
sffc Jan 29, 2022
f36c233
ICU4X cargo check --all-targets --all-features
sffc Jan 29, 2022
5a3374f
Merge remote-tracking branch 'upstream/main' into resourcemarker
sffc Jan 29, 2022
8a40399
fmt
sffc Jan 29, 2022
edd600e
cargo quick
sffc Jan 29, 2022
fda78d1
Work around Rust bug rust-lang/rust#93470
sffc Jan 31, 2022
87fb440
Merge remote-tracking branch 'upstream/main' into resourcemarker
sffc Feb 1, 2022
c0262c1
Fix resource_path_to_string
sffc Feb 1, 2022
c112f5b
Fixing docs tests in icu_provider
sffc Feb 1, 2022
2f34269
Fix failing bench test
sffc Feb 1, 2022
a08a744
More docs tests fixes
sffc Feb 1, 2022
bb1479a
Another docs test down
sffc Feb 1, 2022
4010e7a
Re-write icu_provider crate-level docs
sffc Feb 1, 2022
7150659
Doc link fixes
sffc Feb 1, 2022
285cccf
diplomat regen
sffc Feb 2, 2022
7217ea7
A few more docs updates + ForkByKey for DynProvider
sffc Feb 2, 2022
0d93723
icu_provider links fixed
sffc Feb 2, 2022
0f010c0
Fixing remaining docs links
sffc Feb 2, 2022
68b45cd
list_formatter fixups
sffc Feb 2, 2022
e04ba20
clippy & tidy
sffc Feb 2, 2022
6e4bac1
Docs for DynProvider and ResourceProvider
sffc Feb 2, 2022
33498b9
Fix plural rules FFI test
sffc Feb 2, 2022
bdce762
Fix FFI test take 2
sffc Feb 2, 2022
860e21f
Merge remote-tracking branch 'upstream/main' into resourcemarker
sffc Feb 2, 2022
9772515
Fix build
sffc Feb 2, 2022
842215a
Remove unneeded type info
sffc Feb 2, 2022
127aaf0
Fix another compile error
sffc Feb 2, 2022
099ab34
try_langid -> get_langid
sffc Feb 2, 2022
b2c8f7d
Update data_struct macro to require explicit marker symbol path
sffc Feb 2, 2022
8a47ffa
Migrate call sites to new macro
sffc Feb 2, 2022
d2902af
Make PluralRulesV1Marker public
sffc Feb 2, 2022
14fefcc
Remove ResourcePath and DataRequestOld
sffc Feb 2, 2022
51cf962
Remove unused file
sffc Feb 2, 2022
8f992bf
Update provider/macros/src/lib.rs
sffc Feb 2, 2022
1217eed
Merge branch 'main' into resourcemarker
sffc Feb 2, 2022
16120c6
fmt
sffc Feb 3, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions provider/core/src/data_provider.rs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ pub struct DataRequestOld {
#[non_exhaustive]
pub struct DataRequestMetadata;

#[derive(Default)]
sffc marked this conversation as resolved.
Show resolved Hide resolved
pub struct DataRequest {
pub options: ResourceOptions,
pub metadata: DataRequestMetadata,
Expand Down
65 changes: 24 additions & 41 deletions provider/core/src/data_provider/test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ impl DataMarker for HelloAltMarker {
type Yokeable = HelloAlt;
}

impl ResourceMarker for HelloAltMarker {
sffc marked this conversation as resolved.
Show resolved Hide resolved
const KEY: ResourceKey = HELLO_ALT_KEY;
}

#[derive(Serialize, Deserialize, Debug, Clone, Default, PartialEq)]
struct HelloCombined<'data> {
#[serde(borrow)]
Expand All @@ -47,12 +51,11 @@ struct DataWarehouse {
hello_alt: HelloAlt,
}

impl DataProvider<HelloWorldV1Marker> for DataWarehouse {
fn load_payload(
impl ResourceProvider<HelloWorldV1Marker> for DataWarehouse {
fn load_resource(
&self,
req: &DataRequest,
_: &DataRequest,
) -> Result<DataResponse<HelloWorldV1Marker>, DataError> {
req.resource_path.key.match_key(key::HELLO_WORLD_V1)?;
Ok(DataResponse {
metadata: DataResponseMetadata::default(),
payload: Some(DataPayload::from_owned(self.hello_v1.clone())),
Expand All @@ -76,22 +79,20 @@ impl From<DataWarehouse> for DataProvider2 {
}
}

impl DataProvider<HelloWorldV1Marker> for DataProvider2 {
fn load_payload(
impl ResourceProvider<HelloWorldV1Marker> for DataProvider2 {
fn load_resource(
&self,
req: &DataRequest,
_: &DataRequest,
) -> Result<DataResponse<HelloWorldV1Marker>, DataError> {
req.resource_path.key.match_key(key::HELLO_WORLD_V1)?;
Ok(DataResponse {
metadata: DataResponseMetadata::default(),
payload: Some(DataPayload::from_owned(self.data.hello_v1.clone())),
})
}
}

impl DataProvider<HelloAltMarker> for DataProvider2 {
fn load_payload(&self, req: &DataRequest) -> Result<DataResponse<HelloAltMarker>, DataError> {
req.resource_path.key.match_key(HELLO_ALT_KEY)?;
impl ResourceProvider<HelloAltMarker> for DataProvider2 {
fn load_resource(&self, _: &DataRequest) -> Result<DataResponse<HelloAltMarker>, DataError> {
Ok(DataResponse {
metadata: DataResponseMetadata::default(),
payload: Some(DataPayload::from_owned(self.data.hello_alt.clone())),
Expand Down Expand Up @@ -123,34 +124,16 @@ fn get_warehouse(data: &'static str) -> DataWarehouse {
}
}

fn get_payload_v1<P: DataProvider<HelloWorldV1Marker> + ?Sized>(
fn get_payload_v1<P: ResourceProvider<HelloWorldV1Marker> + ?Sized>(
provider: &P,
) -> Result<DataPayload<HelloWorldV1Marker>, DataError> {
provider.load_payload(&get_request_v1())?.take_payload()
provider.load_resource(&Default::default())?.take_payload()
}

fn get_payload_alt<P: DataProvider<HelloAltMarker> + ?Sized>(
d: &P,
fn get_payload_alt<P: ResourceProvider<HelloAltMarker> + ?Sized>(
provider: &P,
) -> Result<DataPayload<HelloAltMarker>, DataError> {
d.load_payload(&get_request_alt())?.take_payload()
}

fn get_request_v1() -> DataRequest {
DataRequest {
resource_path: ResourcePath {
key: key::HELLO_WORLD_V1,
options: Default::default(),
},
}
}

fn get_request_alt() -> DataRequest {
DataRequest {
resource_path: ResourcePath {
key: HELLO_ALT_KEY,
options: Default::default(),
},
}
provider.load_resource(&Default::default())?.take_payload()
}

#[test]
Expand Down Expand Up @@ -180,7 +163,7 @@ fn test_warehouse_owned_dyn_erased() {
#[test]
fn test_warehouse_owned_dyn_generic() {
let warehouse = get_warehouse(DATA);
let hello_data = get_payload_v1(&warehouse as &dyn DataProvider<HelloWorldV1Marker>).unwrap();
let hello_data = get_payload_v1(&warehouse as &dyn ResourceProvider<HelloWorldV1Marker>).unwrap();
assert!(matches!(
hello_data.get(),
HelloWorldV1 {
Expand Down Expand Up @@ -240,7 +223,7 @@ fn test_provider2_dyn_erased_alt() {
fn test_provider2_dyn_generic() {
let warehouse = get_warehouse(DATA);
let provider = DataProvider2::from(warehouse);
let hello_data = get_payload_v1(&provider as &dyn DataProvider<HelloWorldV1Marker>).unwrap();
let hello_data = get_payload_v1(&provider as &dyn ResourceProvider<HelloWorldV1Marker>).unwrap();
assert!(matches!(
hello_data.get(),
HelloWorldV1 {
Expand All @@ -253,7 +236,7 @@ fn test_provider2_dyn_generic() {
fn test_provider2_dyn_generic_alt() {
let warehouse = get_warehouse(DATA);
let provider = DataProvider2::from(warehouse);
let hello_data = get_payload_alt(&provider as &dyn DataProvider<HelloAltMarker>).unwrap();
let hello_data = get_payload_alt(&provider as &dyn ResourceProvider<HelloAltMarker>).unwrap();
assert!(matches!(hello_data.get(), HelloAlt { .. }));
}

Expand All @@ -263,7 +246,7 @@ fn test_mismatched_types() {
let provider = DataProvider2::from(warehouse);
// Request is for v2, but type argument is for v1
let response: Result<DataResponse<HelloWorldV1Marker>, DataError> =
AnyProvider::load_any(&provider.as_any_provider(), &get_request_alt())
AnyProvider::load_any(&provider.as_any_provider(), HELLO_ALT_KEY, &Default::default())
.unwrap()
.downcast();
assert!(matches!(
Expand All @@ -277,15 +260,15 @@ fn test_mismatched_types() {

fn check_v1_v2<P>(d: &P)
where
P: DataProvider<HelloWorldV1Marker> + DataProvider<HelloAltMarker> + ?Sized,
P: ResourceProvider<HelloWorldV1Marker> + ResourceProvider<HelloAltMarker> + ?Sized,
{
let v1: DataPayload<HelloWorldV1Marker> = d
.load_payload(&get_request_v1())
.load_resource(&Default::default())
.unwrap()
.take_payload()
.unwrap();
let v2: DataPayload<HelloAltMarker> = d
.load_payload(&get_request_alt())
.load_resource(&Default::default())
.unwrap()
.take_payload()
.unwrap();
Expand Down
6 changes: 3 additions & 3 deletions provider/core/src/serde/de.rs
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ impl DataPayload<BufferMarker> {
}
}

impl<P, M> DataProvider<M> for DeserializingBufferProvider<'_, P>
impl<P, M> DynProvider<M> for DeserializingBufferProvider<'_, P>
where
M: DataMarker,
P: BufferProvider + ?Sized,
Expand All @@ -115,8 +115,8 @@ where
for<'de> YokeTraitHack<<M::Yokeable as Yokeable<'de>>::Output>: Deserialize<'de>,
{
/// Converts a buffer into a concrete type by deserializing from a supported buffer format.
fn load_payload(&self, req: &DataRequest) -> Result<DataResponse<M>, DataError> {
let old_response = BufferProvider::load_buffer(self.0, req)?;
fn load_payload(&self, key: ResourceKey, req: &DataRequest) -> Result<DataResponse<M>, DataError> {
let old_response = BufferProvider::load_buffer(self.0, key, req)?;
if let Some(old_payload) = old_response.payload {
let buffer_format = old_response
.metadata
Expand Down