Skip to content

Allow customizable "reading" of JSON maps #1045

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion _test_yaml/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ dev_dependencies:
build_runner: ^2.0.0
build_verify: ^2.0.0
checked_yaml: any
json_annotation: ^4.3.0
json_annotation: ^4.4.0
json_serializable: any
test: ^1.6.0
yaml: ^3.0.0
Expand Down
5 changes: 4 additions & 1 deletion json_annotation/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
## 4.3.1-dev
## 4.4.0-dev

- Added `JsonKey.readValue`.
- Non-breaking updates to `checkedCreate` and `checkedConvert` to support
`JsonKey.readValue`.
- Improved `toString` in included exceptions.

## 4.3.0
Expand Down
26 changes: 21 additions & 5 deletions json_annotation/lib/src/checked_helpers.dart
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,22 @@ typedef _CastFunction<R> = R Function(Object?);
T $checkedCreate<T>(
String className,
Map map,
T Function(S Function<S>(String, _CastFunction<S>) converter) constructor, {
T Function(
S Function<S>(
String,
_CastFunction<S>, {
Object? Function(Map, String)? readValue,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should readValue also return a T here to give better static checking? In general the relationship here between the _CastFunction and readValue params is a bit murky to me.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will investigate...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is a bit weird.

readValue is meant to be pre-converter. So it should return a JSON-ish result. that then goes through any associated converters, etc.

If one wants readValue to return the field type, they have to also create a converter.

I pondered doing the logic where readValue just takes over ALL conversion logic, too. But that gets tricky.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya, that is what I was wondering about. This would cover some number of use cases, where one or more json fields can be merged into a new json-like field, but its a bit indirect for others. For instance you might have people actually returning a Map from readValue, built from some other fields, and then deserializing that map into some class in the regular conversion function.

Ultimately I think this choice is probably fine, but I would clarify things in the documentation?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added more docs. PTAL

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pondered doing the logic where readValue just takes over ALL conversion logic, too. But that gets tricky.

Tricky for users or for this library?

It feels like a simpler concept to document if readValue does all the work instead of coordinating with some other uncoupled concept with no static types in between...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be both. Imagine a case where I just want to rename a field foo -> bar but the type is something complex like Map<int, List<Duration?>>? – that's a bear to rewrite for the user.

}),
)
constructor, {
Map<String, String> fieldKeyMap = const {},
}) {
Q _checkedConvert<Q>(String key, _CastFunction<Q> convertFunction) =>
$checkedConvert<Q>(map, key, convertFunction);
Q _checkedConvert<Q>(
String key,
_CastFunction<Q> convertFunction, {
Object? Function(Map, String)? readValue,
}) =>
$checkedConvert<Q>(map, key, convertFunction, readValue: readValue);

return $checkedNew(
className,
Expand Down Expand Up @@ -69,9 +80,14 @@ T $checkedNew<T>(
/// `JsonSerializableGenerator.checked` is `true`.
///
/// Should not be used directly.
T $checkedConvert<T>(Map map, String key, T Function(dynamic) castFunc) {
T $checkedConvert<T>(
Map map,
String key,
T Function(dynamic) castFunc, {
Object? Function(Map, String)? readValue,
}) {
try {
return castFunc(map[key]);
return castFunc(readValue == null ? map[key] : readValue(map, key));
} on CheckedFromJsonException {
rethrow;
} catch (error, stack) {
Expand Down
16 changes: 16 additions & 0 deletions json_annotation/lib/src/json_key.dart
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,21 @@ class JsonKey {
/// If `null`, the field name is used.
final String? name;

/// Specialize how a value is read from the source JSON map.
///
/// Typically, the value corresponding to a given key is read directly from
/// the JSON map using `map[key]`. At times it's convenient to customize this
/// behavior to support alternative names or to support logic that requires
/// accessing multiple values at once.
///
/// The provided, the [Function] must be a top-level or static within the
/// using class.
///
/// Note: using this feature does not change any of the subsequent decoding
/// logic for the field. For instance, if the field is of type [DateTime] we
/// expect the function provided here to return a [String].
final Object? Function(Map, String)? readValue;

/// When `true`, generated code for `fromJson` will verify that the source
/// JSON map contains the associated key.
///
Expand Down Expand Up @@ -108,6 +123,7 @@ class JsonKey {
this.ignore,
this.includeIfNull,
this.name,
this.readValue,
this.required,
this.toJson,
this.unknownEnumValue,
Expand Down
6 changes: 6 additions & 0 deletions json_serializable/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
## 6.1.0-dev

- Support `JsonKey.readValue` to allow customized reading of values from source
JSON map objects.
- Require `json_annotation` `'>=4.4.0 <4.5.0'`.

## 6.0.1

- Don't require `json_annotation` in `dependencies` if it's just used in tests.
Expand Down
16 changes: 8 additions & 8 deletions json_serializable/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,14 +199,14 @@ targets:
[`Enum`]: https://api.dart.dev/stable/dart-core/Enum-class.html
[`int`]: https://api.dart.dev/stable/dart-core/int-class.html
[`Iterable`]: https://api.dart.dev/stable/dart-core/Iterable-class.html
[`JsonConverter`]: https://pub.dev/documentation/json_annotation/4.3.0/json_annotation/JsonConverter-class.html
[`JsonEnum`]: https://pub.dev/documentation/json_annotation/4.3.0/json_annotation/JsonEnum-class.html
[`JsonKey.fromJson`]: https://pub.dev/documentation/json_annotation/4.3.0/json_annotation/JsonKey/fromJson.html
[`JsonKey.toJson`]: https://pub.dev/documentation/json_annotation/4.3.0/json_annotation/JsonKey/toJson.html
[`JsonKey`]: https://pub.dev/documentation/json_annotation/4.3.0/json_annotation/JsonKey-class.html
[`JsonLiteral`]: https://pub.dev/documentation/json_annotation/4.3.0/json_annotation/JsonLiteral-class.html
[`JsonSerializable`]: https://pub.dev/documentation/json_annotation/4.3.0/json_annotation/JsonSerializable-class.html
[`JsonValue`]: https://pub.dev/documentation/json_annotation/4.3.0/json_annotation/JsonValue-class.html
[`JsonConverter`]: https://pub.dev/documentation/json_annotation/latest/json_annotation/JsonConverter-class.html
[`JsonEnum`]: https://pub.dev/documentation/json_annotation/latest/json_annotation/JsonEnum-class.html
[`JsonKey.fromJson`]: https://pub.dev/documentation/json_annotation/latest/json_annotation/JsonKey/fromJson.html
[`JsonKey.toJson`]: https://pub.dev/documentation/json_annotation/latest/json_annotation/JsonKey/toJson.html
[`JsonKey`]: https://pub.dev/documentation/json_annotation/latest/json_annotation/JsonKey-class.html
[`JsonLiteral`]: https://pub.dev/documentation/json_annotation/latest/json_annotation/JsonLiteral-class.html
[`JsonSerializable`]: https://pub.dev/documentation/json_annotation/latest/json_annotation/JsonSerializable-class.html
[`JsonValue`]: https://pub.dev/documentation/json_annotation/latest/json_annotation/JsonValue-class.html
[`List`]: https://api.dart.dev/stable/dart-core/List-class.html
[`Map`]: https://api.dart.dev/stable/dart-core/Map-class.html
[`num`]: https://api.dart.dev/stable/dart-core/num-class.html
Expand Down
2 changes: 1 addition & 1 deletion json_serializable/lib/src/check_dependencies.dart
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import 'package:pubspec_parse/pubspec_parse.dart';

const _productionDirectories = {'lib', 'bin'};
const _annotationPkgName = 'json_annotation';
final requiredJsonAnnotationMinVersion = Version.parse('4.3.0');
final requiredJsonAnnotationMinVersion = Version.parse('4.4.0');

Future<void> pubspecHasRightVersion(BuildStep buildStep) async {
final segments = buildStep.inputId.pathSegments;
Expand Down
34 changes: 26 additions & 8 deletions json_serializable/lib/src/decode_helper.dart
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,24 @@ abstract class DecodeHelper implements HelperCore {
..write('''
final val = ${data.content};''');

for (final field in data.fieldsToSet) {
for (final fieldName in data.fieldsToSet) {
sectionBuffer.writeln();
final safeName = safeNameAccess(accessibleFields[field]!);
final fieldValue = accessibleFields[fieldName]!;
final safeName = safeNameAccess(fieldValue);
sectionBuffer
..write('''
\$checkedConvert($safeName, (v) => ''')
..write('val.$field = ')
..write(_deserializeForField(accessibleFields[field]!,
checkedProperty: true))
..write(');');
..write('val.$fieldName = ')
..write(
_deserializeForField(fieldValue, checkedProperty: true),
);

final readValueFunc = jsonKeyFor(fieldValue).readValueFunctionName;
if (readValueFunc != null) {
sectionBuffer.writeln(',readValue: $readValueFunc,');
}

sectionBuffer.write(');');
}

sectionBuffer.write('''\n return val;
Expand Down Expand Up @@ -182,6 +190,8 @@ abstract class DecodeHelper implements HelperCore {
}
}

/// If [checkedProperty] is `true`, we're using this function to write to a
/// setter.
String _deserializeForField(
FieldElement field, {
ParameterElement? ctorParam,
Expand All @@ -192,6 +202,7 @@ abstract class DecodeHelper implements HelperCore {
final contextHelper = getHelperContext(field);
final jsonKey = jsonKeyFor(field);
final defaultValue = jsonKey.defaultValue;
final readValueFunc = jsonKey.readValueFunctionName;

String deserialize(String expression) => contextHelper
.deserialize(
Expand All @@ -206,14 +217,21 @@ abstract class DecodeHelper implements HelperCore {
if (config.checked) {
value = deserialize('v');
if (!checkedProperty) {
value = '\$checkedConvert($jsonKeyName, (v) => $value)';
final readValueBit =
readValueFunc == null ? '' : ',readValue: $readValueFunc,';
value = '\$checkedConvert($jsonKeyName, (v) => $value$readValueBit)';
}
} else {
assert(
!checkedProperty,
'should only be true if `_generator.checked` is true.',
);
value = deserialize('json[$jsonKeyName]');

value = deserialize(
readValueFunc == null
? 'json[$jsonKeyName]'
: '$readValueFunc(json, $jsonKeyName)',
);
}
} on UnsupportedTypeError catch (e) // ignore: avoid_catching_errors
{
Expand Down
10 changes: 10 additions & 0 deletions json_serializable/lib/src/json_key_utils.dart
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,13 @@ KeyConfig _from(FieldElement element, ClassConfig classAnnotation) {
}
}

String? readValueFunctionName;
final readValue = obj.read('readValue');
if (!readValue.isNull) {
final objValue = readValue.objectValue.toFunctionValue()!;
readValueFunctionName = objValue.name;
}

return _populateJsonKey(
classAnnotation,
element,
Expand All @@ -217,6 +224,7 @@ KeyConfig _from(FieldElement element, ClassConfig classAnnotation) {
ignore: obj.read('ignore').literalValue as bool?,
includeIfNull: obj.read('includeIfNull').literalValue as bool?,
name: obj.read('name').literalValue as String?,
readValueFunctionName: readValueFunctionName,
required: obj.read('required').literalValue as bool?,
unknownEnumValue: _annotationValue('unknownEnumValue', mustBeEnum: true),
);
Expand All @@ -230,6 +238,7 @@ KeyConfig _populateJsonKey(
bool? ignore,
bool? includeIfNull,
String? name,
String? readValueFunctionName,
bool? required,
String? unknownEnumValue,
}) {
Expand All @@ -249,6 +258,7 @@ KeyConfig _populateJsonKey(
includeIfNull: _includeIfNull(
includeIfNull, disallowNullValue, classAnnotation.includeIfNull),
name: name ?? encodedFieldName(classAnnotation.fieldRename, element.name),
readValueFunctionName: readValueFunctionName,
required: required ?? false,
unknownEnumValue: unknownEnumValue,
);
Expand Down
3 changes: 3 additions & 0 deletions json_serializable/lib/src/type_helpers/config_types.dart
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,15 @@ class KeyConfig {

final String? unknownEnumValue;

final String? readValueFunctionName;

KeyConfig({
required this.defaultValue,
required this.disallowNullValue,
required this.ignore,
required this.includeIfNull,
required this.name,
required this.readValueFunctionName,
required this.required,
required this.unknownEnumValue,
});
Expand Down
2 changes: 1 addition & 1 deletion json_serializable/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ dependencies:

# Use a tight version constraint to ensure that a constraint on
# `json_annotation` properly constrains all features it provides.
json_annotation: '>=4.3.0 <4.4.0'
json_annotation: '>=4.4.0 <4.5.0'
meta: ^1.3.0
path: ^1.8.0
pub_semver: ^2.0.0
Expand Down
15 changes: 14 additions & 1 deletion json_serializable/test/kitchen_sink/kitchen_sink.dart
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,18 @@ class _Factory implements k.KitchenSinkFactory<String, dynamic> {
JsonConverterTestClass.fromJson(json);
}

Object? _valueAccessor(Map json, String key) {
if (key == k.trickyKeyName) {
return json[k.trickyKeyName] ?? json['STRING'];
}

if (key == 'iterable') {
return json['iterable'] ?? json['theIterable'];
}

return json[key];
}

@JsonSerializable()
class KitchenSink implements k.KitchenSink {
// NOTE: exposing these as Iterable, but storing the values as List
Expand Down Expand Up @@ -115,6 +127,7 @@ class KitchenSink implements k.KitchenSink {

BigInt? bigInt;

@JsonKey(readValue: _valueAccessor)
Iterable? get iterable => _iterable;

Iterable<dynamic> get dynamicIterable => _dynamicIterable;
Expand Down Expand Up @@ -152,7 +165,7 @@ class KitchenSink implements k.KitchenSink {
// Handle fields with names that collide with helper names
Map<String, bool> val = _defaultMap();
bool? writeNotNull;
@JsonKey(name: r'$string')
@JsonKey(name: k.trickyKeyName, readValue: _valueAccessor)
String? string;

SimpleObject simpleObject = _defaultSimpleObject();
Expand Down
4 changes: 2 additions & 2 deletions json_serializable/test/kitchen_sink/kitchen_sink.g.dart

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 14 additions & 1 deletion json_serializable/test/kitchen_sink/kitchen_sink.g_any_map.dart
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,18 @@ class _Factory implements k.KitchenSinkFactory<dynamic, dynamic> {
JsonConverterTestClass.fromJson(json);
}

Object? _valueAccessor(Map json, String key) {
if (key == k.trickyKeyName) {
return json[k.trickyKeyName] ?? json['STRING'];
}

if (key == 'iterable') {
return json['iterable'] ?? json['theIterable'];
}

return json[key];
}

@JsonSerializable(
anyMap: true,
)
Expand Down Expand Up @@ -115,6 +127,7 @@ class KitchenSink implements k.KitchenSink {

BigInt? bigInt;

@JsonKey(readValue: _valueAccessor)
Iterable? get iterable => _iterable;

Iterable<dynamic> get dynamicIterable => _dynamicIterable;
Expand Down Expand Up @@ -152,7 +165,7 @@ class KitchenSink implements k.KitchenSink {
// Handle fields with names that collide with helper names
Map<String, bool> val = _defaultMap();
bool? writeNotNull;
@JsonKey(name: r'$string')
@JsonKey(name: k.trickyKeyName, readValue: _valueAccessor)
String? string;

SimpleObject simpleObject = _defaultSimpleObject();
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading