Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: pattern properties not being accounted for #1006

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
d13c267
Fixing dependencies
jonaslagoni Sep 27, 2022
c46e70f
fixed renderer
jonaslagoni Sep 27, 2022
8e6150b
fixed blackbox
jonaslagoni Sep 27, 2022
f103f82
add test
jonaslagoni Sep 27, 2022
dd4ddab
Fixed linter
jonaslagoni Sep 27, 2022
cdaf662
fixed dependencies
jonaslagoni Sep 27, 2022
9bf54a7
Fixed unique implementation
jonaslagoni Sep 27, 2022
6f1d6c6
fixed linter
jonaslagoni Sep 27, 2022
bb673d7
Merge branch 'next' into feature/fix_all_blackbox_tests
jonaslagoni Sep 27, 2022
c7e8c1d
Merge remote-tracking branch 'origin/feature/fix_python_generator_wit…
jonaslagoni Sep 27, 2022
80af36d
Merge remote-tracking branch 'origin/feature/fix_self_dependencies' i…
jonaslagoni Sep 27, 2022
c475eee
WIP
jonaslagoni Sep 28, 2022
fff0d08
Merge branch 'next' into feature/fix_all_blackbox_tests
jonaslagoni Oct 3, 2022
e57f7c5
Merge branch 'next' into feature/fix_all_blackbox_tests
jonaslagoni Oct 4, 2022
e31e2bc
Fixed implementation and test
jonaslagoni Oct 5, 2022
c1ccf69
add fix and implementation
jonaslagoni Oct 5, 2022
a7d7649
update snapshot
jonaslagoni Oct 5, 2022
061a31b
wip
jonaslagoni Oct 4, 2022
cca613a
Merge remote-tracking branch 'origin/feature/fix_typescript_array_typ…
jonaslagoni Oct 5, 2022
a9dce75
Merge remote-tracking branch 'origin/feature/fix_javascript_splits_ou…
jonaslagoni Oct 5, 2022
4c7e6c3
add implementation and test
jonaslagoni Oct 5, 2022
aed9512
fixed linter
jonaslagoni Oct 6, 2022
921da11
Merge remote-tracking branch 'origin/feature/fix_java_maps_cannot_hav…
jonaslagoni Oct 6, 2022
89a94e0
wip
jonaslagoni Oct 17, 2022
35239a2
Merge branch 'next' into feature/fix_all_blackbox_tests
jonaslagoni Nov 9, 2022
61aec5f
fixed rest of the problems
jonaslagoni Nov 10, 2022
a3ba013
removed unused test file
jonaslagoni Nov 10, 2022
7c2501f
fixed lint
jonaslagoni Nov 10, 2022
5395956
separated blackbox tests and scripts
jonaslagoni Nov 10, 2022
cf0d339
re-added files
jonaslagoni Nov 10, 2022
40e965c
Merge branch 'next' into feature/fix_all_blackbox_tests
jonaslagoni Nov 10, 2022
2c4de23
updated packagelock
jonaslagoni Nov 10, 2022
7911d28
rename
jonaslagoni Nov 10, 2022
71001e6
fixed linter
jonaslagoni Nov 10, 2022
92bac61
fixed more problems
jonaslagoni Nov 15, 2022
3fe7b78
wip
jonaslagoni Nov 17, 2022
74e7b87
Merge branch 'next' into feature/fix_all_blackbox_tests
jonaslagoni Nov 17, 2022
1fb9f15
update
jonaslagoni Nov 17, 2022
19fa426
renamed parameter
jonaslagoni Nov 17, 2022
8ff1ade
fixed incorrect test scripts was run
jonaslagoni Nov 17, 2022
2114dd2
removed unused file
jonaslagoni Nov 18, 2022
1792c05
added comments
jonaslagoni Nov 23, 2022
24e2651
fixed tests and implementation
jonaslagoni Nov 23, 2022
c6abd12
Merge branch 'next' into feature/fix_pattern_properties_problem
jonaslagoni Nov 23, 2022
af13a3c
removed unneccessary newline
jonaslagoni Nov 23, 2022
30da8fd
Merge branch 'feature/fix_pattern_properties_problem' into feature/fi…
jonaslagoni Nov 23, 2022
c7efd32
Revert "Merge branch 'feature/fix_pattern_properties_problem' into fe…
jonaslagoni Nov 23, 2022
cab7de9
Merge branch 'feature/fix_all_blackbox_tests' into feature/fix_patter…
jonaslagoni Nov 23, 2022
f69c212
fix common model conversion
jonaslagoni Nov 23, 2022
9c2ec21
Merge branch 'next' into feature/fix_pattern_properties_problem
jonaslagoni Nov 29, 2022
903ab1e
add wrong merges
jonaslagoni Nov 29, 2022
b1edfe4
fixed model implementation
jonaslagoni Nov 29, 2022
3d649ea
fixed test
jonaslagoni Nov 29, 2022
de6419c
update snapshots
jonaslagoni Nov 29, 2022
4d28554
update test and implementation
jonaslagoni Nov 29, 2022
397eeeb
update snapshots
jonaslagoni Nov 30, 2022
0c8893a
fixed tests and implementation
jonaslagoni Nov 30, 2022
2989917
fixed lint
jonaslagoni Nov 30, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert "Merge branch 'feature/fix_pattern_properties_problem' into fe…
…ature/fix_all_blackbox_tests"

This reverts commit 30da8fd, reversing
changes made to 1792c05.
  • Loading branch information
jonaslagoni committed Nov 23, 2022
commit c7efd32a96ea50634b156b7c8aaad32c7e5c1e29
54 changes: 3 additions & 51 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The Acceptance Criteria for _adding new features_ requires a few things in order
1. **No documentation, no feature:** If a user cannot understand a new feature, that feature basically doesn't exist! Remember to make sure that any and all relevant [documentation](./) is consistently updated.
- New features such as new presets, generators or inputs, etc, need associated use case documentation along side [examples](../examples). This is not only to showcase the feature, but to ensure it will always work. Checkout our [adding examples](#-adding-examples) doc for more information on how to do this.

### Adding examples
### Adding Examples
The Acceptance Criteria Process for _adding examples_ is not only something we use to showcase features, but also to ensure those features always work. _(This is important since it is picked up by [our CI system](#What-does–the-CI-system-do-when-I-create-a-PR).)_

Adding examples is quite straight forward, so don't feel shy! Here's how to do it:
Expand All @@ -37,55 +37,7 @@ Adding examples is quite straight forward, so don't feel shy! Here's how to do i

Aaaand you are done! :tada:

### Adding a new preset
Presets are for when you want to customize the generated output, they work like middleware that layers on top of each other, you can read more [about presets here](./presets.md).

Here is how you add a new preset:
1. All presets are located under `src/generators/${language}/presets`, either duplicate an existing preset and adapt it or create an empty TypeScript file.
2. The preset file has the syntax:
```ts
export const LANGUAGE_MY_PRESET: LanguagePreset = {
class: {
// Add preset hooks here
},
// enum: {
// Add preset hooks here
// }
};
```
Replace `LANGUAGE` with the generator the preset is for (for example `TYPESCRIPT`), and replace `LanguagePreset` with the generator the preset is for (for example `TypeScriptPreset`). It is optional which models you add preset hooks for, i.e. you can add preset hooks for `enum` alongside for `class`, but it's not required. Each generator has a set of outputs you can change, read more [about the presets here](./presets.md).

3. Add your preset to the `src/generators/${language}/presets/index.ts` file.
4. Add an [example](#adding-examples) to showcase your new feature.
5. Add documentation to [the language docs](./languages/) that explain the use case and links to your new example.
6. In most cases you want to add specific tests for edge cases or simply to test the preset. To do this add a new test file in `test/generators/${language}/presets/MyPreset.spec.ts` and replace `MyPreset` with your preset name. Now add a test using the following syntax:
```ts
describe('LANGUAGE_MY_PRESET', () => {
let generator: LanguageGenerator;
beforeEach(() => {
generator = new LanguageGenerator({ presets: [LANGUAGE_MY_PRESET] });
});

test('should render xxx', async () => {
const input = {
$id: 'Clazz',
type: 'object',
properties: {
min_number_prop: { type: 'number' },
max_number_prop: { type: 'number' },
},
};
const models = await generator.generate(input);
expect(models).toHaveLength(1);
expect(models[0].result).toMatchSnapshot();
});
});
```
Remember to replace `LANGUAGE` and `Language` with the appropriate values.

Aaaand you are done! :tada:

### Adding a new input processor
### New input processor
Input processors are the translators from inputs to MetaModel (read more about [the input processing here](./input-processing.md)).

Here is how you can add a new input processor:
Expand All @@ -105,7 +57,7 @@ Thats it for the code and tests, now all that remains is docs and examples! :fir

Aaaand you are done! :tada:

### Adding a new generator
### New Generators
Generators sits as the core of Modelina, which frames the core concepts of what you can generate. Therefore it's also no small task to create a new one, so dont get discourage, we are here to help you!

To make it easier to contribute a new generator, and to avoid focusing too much of the internals of Modelina, we created a template generator to get you started. If you encounter discreprencies with the following guide or templates, make sure to raise it as an issue so it can be fixed!
Expand Down
3 changes: 1 addition & 2 deletions docs/inputs/JSON_Schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The order of interpretation:
- `true` boolean schema infers all model types (`object`, `string`, `number`, `array`, `boolean`, `null`, `integer`) schemas.
- `type` infers the initial model type.
- `required` are interpreted as is.
- `patternProperties` are interpreted as is, where duplicate patterns for the model are [merged](#Merging-models).
- `patternProperties` are merged together with any additionalProperties, where duplicate additionalProperties are [merged](#Merging-models).
- `additionalProperties` are interpreted as is, where duplicate additionalProperties for the model are [merged](#Merging-models). If the schema does not define `additionalProperties` it defaults to `true` schema.
- `additionalItems` are interpreted as is, where duplicate additionalItems for the model are [merged](#Merging-models). If the schema does not define `additionalItems` it defaults to `true` schema.
- `items` are interpreted as ether tuples or simple array, where more than 1 item are [merged](#Merging-models). Usage of `items` infers `array` model type.
Expand Down Expand Up @@ -49,7 +49,6 @@ Because of the recursive nature of the interpreter (and the nested nature of JSO

If only one side has a property defined, it is used as is, if both have it defined they are merged based on the following logic (look [here](./input_processing.md#Internal-model-representation) for more information about the CommonModel and its properties):
- `additionalProperties` if both models contain it the two are recursively merged together.
- `patternProperties` if both models contain a pattern the corresponding models are recursively merged together.
- `properties` if both models contain the same property the corresponding models are recursively merged together.
- `items` are merged together based on a couple of rules:
- If both models are simple arrays those item models are merged together as is.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,12 @@ public class Root

public class RootConverter : JsonConverter<Root>
{
public override Root ReadJson(JsonReader reader, System.Type objectType, Root existingValue, bool hasExistingValue, JsonSerializer serializer)
public override Root ReadJson(JsonReader reader, Type objectType, Root existingValue, bool hasExistingValue, JsonSerializer serializer)
{
JObject jo = JObject.Load(reader);
Root value = new Root();

if(jo[\\"email\\"] != null) {
if(jo[\\"email\\" != null) {
value.Email = jo[\\"email\\"].ToObject<string>(serializer);
}

Expand All @@ -32,7 +32,7 @@ public class RootConverter : JsonConverter<Root>
{
JObject jo = new JObject();

if (value.Email != null)
if (value.email != null)
{
jo.Add(\\"email\\", JToken.FromObject(value.Email, serializer));
}
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 1 addition & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@asyncapi/modelina",
"version": "1.0.0-next.28",
"version": "1.0.0-next.26",
"description": "Library for generating data models based on inputs such as AsyncAPI, OpenAPI, or JSON Schema documents",
"license": "Apache-2.0",
"homepage": "https://github.com/asyncapi/modelina",
Expand Down Expand Up @@ -80,10 +80,8 @@
"docker:test": "npm run docker:build && docker run asyncapi/modelina npm run test",
"docker:test:blackbox": "npm run docker:build && docker run asyncapi/modelina npm run test:blackbox",
"test": "npm run test:library && npm run test:examples",
"test:update": "npm run test:library -- -u && npm run test:examples:update",
"test:library": "cross-env CI=true jest --coverage --testPathIgnorePatterns ./test/blackbox --testPathIgnorePatterns ./examples",
"test:examples": "npm run test:examples:regular && npm run test:examples:websites",
"test:examples:update": "npm run test:examples:regular -- -u && npm run test:examples:websites -- -u",
"test:examples:regular": "cross-env CI=true jest ./examples --testPathIgnorePatterns ./examples/integrate-with-react",
"test:examples:websites": "cd ./examples/integrate-with-react && npm i && npm run test",
"test:blackbox": "concurrently --group -n csharp,go,java,javascript,python,rust,typescript \"npm run test:blackbox:csharp\" \"npm run test:blackbox:go\" \"npm run test:blackbox:java\" \"npm run test:blackbox:javascript\" \"npm run test:blackbox:python\" \"npm run test:blackbox:rust\" \"npm run test:blackbox:typescript\"",
Expand Down
7 changes: 3 additions & 4 deletions src/generators/csharp/presets/NewtonsoftSerializerPreset.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ var jsonStringCompliant = stringEnumValue == "True" || stringEnumValue == "False
var jsonToken = JToken.Parse(jsonStringCompliant);
jo.Add("${prop.unconstrainedPropertyName}", jsonToken);`;
}
return `if (value.${propertyAccessor} != null)
return `if (value.${prop.propertyName} != null)
{
${toJson}
}`;
Expand Down Expand Up @@ -70,7 +70,7 @@ function renderDeserialize({ model }: {
&& prop.property.ref instanceof ConstrainedEnumModel) {
toValue = `${prop.property.type}Extensions.To${prop.property.type}(jo["${prop.unconstrainedPropertyName}"])`;
}
return `if(jo["${prop.unconstrainedPropertyName}"] != null) {
return `if(jo["${prop.unconstrainedPropertyName}" != null) {
value.${propertyAccessor} = ${toValue};
}`;
});
Expand All @@ -92,7 +92,7 @@ function renderDeserialize({ model }: {
{
${unwrapDictionaryRead.join('\n')}
}` : '';
return `public override ${model.name} ReadJson(JsonReader reader, System.Type objectType, ${model.name} existingValue, bool hasExistingValue, JsonSerializer serializer)
return `public override ${model.name} ReadJson(JsonReader reader, Type objectType, ${model.name} existingValue, bool hasExistingValue, JsonSerializer serializer)
{
JObject jo = JObject.Load(reader);
${model.name} value = new ${model.name}();
Expand All @@ -115,7 +115,6 @@ export const CSHARP_NEWTONSOFT_SERIALIZER_PRESET: CSharpPreset<CSharpOptions> =
renderer.addDependency('using Newtonsoft.Json;');
renderer.addDependency('using Newtonsoft.Json.Linq;');
renderer.addDependency('using System.Collections.Generic;');
renderer.addDependency('using System.Linq;');

const deserialize = renderDeserialize({ model });
const serialize = renderSerialize({ model });
Expand Down
19 changes: 17 additions & 2 deletions src/generators/rust/constrainer/EnumConstrainer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,22 @@ export function defaultEnumKeyConstraints(customConstraints?: Partial<ModelEnumK
}

export function defaultEnumValueConstraints(): EnumValueConstraint {
return ({enumValue}) => {
return enumValue;
return ({ enumValue }) => {
switch (typeof enumValue) {
case 'boolean':
return 'bool';
case 'bigint':
return 'i64';
case 'number': {
return 'f64';
}
case 'object': {
return 'HashMap<String, serde_json::Value>';
}
case 'string':
default: {
return 'String';
}
}
};
}
29 changes: 3 additions & 26 deletions src/generators/rust/renderers/EnumRenderer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,38 +43,15 @@ ${additionalContent}`;
runStructMacroPreset(): Promise<string> {
return this.runPreset('structMacro');
}

/**
* Returns the type for the JSON value
*/
renderEnumValueType(value: any): string {
switch (typeof value) {
case 'boolean':
return 'bool';
case 'bigint':
return 'i64';
case 'number': {
return 'f64';
}
case 'object': {
return 'HashMap<String, serde_json::Value>';
}
case 'string':
default: {
return 'String';
}
}
}
}

export const RUST_DEFAULT_ENUM_PRESET: EnumPresetType<RustOptions> = {
self({ renderer }) {
return renderer.defaultSelf();
},
item({ item, renderer }) {
const typeOfEnumValue = renderer.renderEnumValueType(item.value);
if (typeOfEnumValue === 'HashMap<String, serde_json::Value>') {
return `${item.key}(${typeOfEnumValue})`;
item({ item }) {
if (item.value === 'HashMap<String, serde_json::Value>') {
return `${item.key}(${item.value})`;
}
return `${item.key}`;
},
Expand Down
25 changes: 4 additions & 21 deletions src/helpers/CommonModelToMetaModel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -219,31 +219,14 @@ export function convertToObjectModel(jsonSchemaModel: CommonModel, name: string,
metaModel.properties[String(propertyName)] = propertyModel;
}

if(jsonSchemaModel.additionalProperties !== undefined || jsonSchemaModel.patternProperties !== undefined) {
if (jsonSchemaModel.additionalProperties !== undefined) {
let propertyName = 'additionalProperties';
while (metaModel.properties[String(propertyName)] !== undefined) {
propertyName = `reserved_${propertyName}`;
}
const keyModel = new StringModel(propertyName, jsonSchemaModel.originalInput);
const modelsAsValue = new Map<string, MetaModel>();
if (jsonSchemaModel.additionalProperties !== undefined) {
const additionalPropertyModel = convertToMetaModel(jsonSchemaModel.additionalProperties, alreadySeenModels);
modelsAsValue.set(additionalPropertyModel.name, additionalPropertyModel);
}

if (jsonSchemaModel.patternProperties !== undefined) {
for (const patternModel of Object.values(jsonSchemaModel.patternProperties)) {
const patternPropertyModel = convertToMetaModel(patternModel);
modelsAsValue.set(patternPropertyModel.name, patternPropertyModel);
}
}
let valueModel: MetaModel;
if (modelsAsValue.size === 1) {
valueModel = Array.from(modelsAsValue.values())[0];
} else {
valueModel = new UnionModel(propertyName, jsonSchemaModel.originalInput, Array.from(modelsAsValue.values()));
}
const dictionaryModel = new DictionaryModel(propertyName, jsonSchemaModel.originalInput, keyModel, valueModel, 'unwrap');
const keyModel = new StringModel(propertyName, jsonSchemaModel.additionalProperties.originalInput);
const valueModel = convertToMetaModel(jsonSchemaModel.additionalProperties, alreadySeenModels);
const dictionaryModel = new DictionaryModel(propertyName, jsonSchemaModel.additionalProperties.originalInput, keyModel, valueModel, 'unwrap');
const propertyModel = new ObjectPropertyModel(propertyName, false, dictionaryModel);
metaModel.properties[String(propertyName)] = propertyModel;
}
Expand Down
4 changes: 2 additions & 2 deletions src/interpreter/InterpretPatternProperties.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ import { Interpreter, InterpreterOptions, InterpreterSchemaType } from './Interp
*/
export default function interpretPatternProperties(schema: InterpreterSchemaType, model: CommonModel, interpreter : Interpreter, interpreterOptions: InterpreterOptions = Interpreter.defaultInterpreterOptions): void {
if (typeof schema === 'boolean') {return;}
for (const [pattern, patternSchema] of Object.entries(schema.patternProperties || {})) {
for (const [,patternSchema] of Object.entries(schema.patternProperties || {})) {
const patternModel = interpreter.interpret(patternSchema as any, interpreterOptions);
if (patternModel !== undefined) {
model.addPatternProperty(pattern, patternModel, schema);
model.addAdditionalProperty(patternModel, schema);
}
}
}
Loading