Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Dart] - Rework Dart client generator to be flutter-compatible #7418

Merged
merged 36 commits into from
Jan 25, 2018

Conversation

joernahrens
Copy link
Contributor

@joernahrens joernahrens commented Jan 16, 2018

PR checklist

  • Read the contribution guidelines.
  • Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh and ./bin/security/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\.
  • Filed the PR against the correct branch: 3.0.0 branch for changes related to OpenAPI spec 3.0. Default: master.
  • Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

Resolves #7367

The dart code generated by the DartClientCodegen is not working with Flutter due to the usage of dartson, which uses dart-mirrors. This PR introduces a major change to the Dart generator which generates flutter-compatible Dart code.

This is a new PR, but based on my first try, where I created a new flutter generator (based on the one for Dart).
The existing test cases for web are still working, but I didn't work for me with dartdevc, I am getting compile errors when trying that:

$ pub build --all --web-compiler=dartdevc
Loading source assets...
Building petstore_client...
Building dartdevc modules... (6.1s)
Error compiling dartdevc module:guinness|lib/lib__guinness.js

[error] super call must be last in an initializer list (see https://goo.gl/EY6hDP): 'super(name, parent, excluded: excluded, exclusive: exclusive, pending: fn == null)'. (package:guinness/src/common/model.dart, line 35, col 9)
[error] The return type 'List' isn't a 'Iterable<BeforeEach>', as defined by the method 'beforeEachFns'. (package:guinness/src/common/model.dart, line 57, col 12)
[error] The return type 'List' isn't a 'Iterable<AfterEach>', as defined by the method 'afterEachFns'. (package:guinness/src/common/model.dart, line 68, col 12)
[error] The argument type 'Iterable<BeforeEach>' can't be assigned to the parameter type 'List'. (package:guinness/src/common/model.dart, line 82, col 34)
[error] The argument type 'Iterable<AfterEach>' can't be assigned to the parameter type 'List'. (package:guinness/src/common/model.dart, line 83, col 33)
[error] Invalid override. The type of 'UnitTestMatchers.toThrow' ('(dynamic, [Pattern]) → void') isn't a subtype of 'Matchers.toThrow' ('(dynamic, dynamic) → void'). (package:guinness/src/common/unittest_backend.dart, line 94, col 3)

Please fix all errors before compiling (warnings are okay).

Build failed.

PTAL @ircecho

Sample

In /samples/client/petstore/flutter I created a flutter app which is basically doing some petstore API calls and logging the output.

This is not to see as a complete solution, I didn't test every feature of swagger, for example I left out authentication completely for now. There might be still a decent amount of work to be done for that.
But the sample shows that there are some things working already. If you want to test that you will have to setup the Flutter SDK. Afterwards running the app from the folder /samples/client/petstore/flutter/flutter_petstore should be working and show you some API responses in the console output.

Output

$ flutter run
Launching lib/main.dart on Pixel in debug mode...
Initializing gradle...                                1.0s
Resolving dependencies...                             1.3s
Running 'gradlew assembleDebug'...                    8.5s
Built build/app/outputs/apk/app-debug.apk (22.9MB).
Installing build/app/outputs/apk/app.apk...           6.0s
I/FlutterActivityDelegate( 3435): onResume setting current activity to this
Syncing files to device Pixel...
I/flutter ( 3435): fetching pets by status...
I/flutter ( 3435): fetching inventory...
I/flutter ( 3435): fetching user1...
I/flutter ( 3435): user received:
I/flutter ( 3435): User[id=0, username=user1, firstName=FN, lastName=LN, email=user1@ssgcard.com, password=password, phone=string, userStatus=0, ]
I/flutter ( 3435): inventory received:
I/flutter ( 3435): {available%')) RLIKE SLEEP(4) AND (('%'=': 1, available%')) RLIKE (SELECT * FROM (SELECT(SLEEP(4)))rGyM) AND (('%'=': 1, unavailable: 36, available;SELECT COUNT(*) FROM DOMAIN.DOMAINS AS T1,DOMAIN.COLUMNS AS T2,DOMAIN.TABLES AS T3# DENYALLjCod: 1, '--: 1, available) AND 3610=(SELECT COUNT(*) FROM SYSMASTER:SYSPAGHDR)--: 1, 21121212212: 1, aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

🔥  To hot reload your app on the fly, press "r". To restart the app entirely, press "R".
An Observatory debugger and profiler on Pixel is available at: http://127.0.0.1:8106/
For a more detailed help message, press "h". To quit, press "q".
I/flutter ( 3435): pets received:
I/flutter ( 3435): [Pet[id=9123612807670197307, category=Category[id=0, name=string, ], name=<IMG SRC=&#x6A&#x61&#x76&#x61&#x73&#x63&#x72&#x69&#x70&#x74&#x3A&#x61&#x6C&#x65&#x72&#x74&#x28&#x27&#x58&#x53&#x53&#x27&#x29>, photoUrls=[string], tags=[Tag[id=0, name=string, ]], status=available, ], Pet[id=9123612807670197310, category=Category[id=0, name=string, ], name=<IMG SRC="jav&#x0A;ascript:alert('XSS');">, photoUrls=[string], tags=[Tag[id=0, name=string, ]], status=available, ], Pet[id=9123612807670197312, category=Category[id=0, name=string, ], name=perl -e 'print "<IMG SRC=java\0script:alert(\"XSS\")>";' > out, photoUrls=[string], tags=[Tag[id=0, name=string, ]], status=available, ], Pet[id=9123612807670197313, category=Category[id=0, name=string, ], name=perl -e 'print "<SCR\0IPT>alert(\"XSS\")</SCR\0IPT>";' > out, photoUrls=[string], tags=[Tag[id=0, name=string, ]], status=available, ], Pet[id=9123612807670197314, category=Category[id=0, name=string, ], name=<IMG SRC=" &#14;  javascript:alert('XSS');">, photoUrls=[string]
I/flutter ( 3435): Order[id=0, petId=9123612807670197307, quantity=1, shipDate=null, status=null, complete=false, ]

Btw, the output of the inventory looks really weird, but it looks the same when using Swagger UI

@joernahrens
Copy link
Contributor Author

joernahrens commented Jan 16, 2018

I don't understand the error log of AppVeyor. Since I removed the flutter generator it shouldn't search for that. 🤔
Any help appreciated.

@joernahrens
Copy link
Contributor Author

I don't understand the error log of AppVeyor.

OK, nevermind, I think I found the problem

@ircecho
Copy link
Contributor

ircecho commented Jan 16, 2018

Very good indeed! I want to merge this as soon as possible!

Maybe you try to remove all the dev_dependencies, because in my understanding neither "browser" nor "guinness" is used anywhere at all. And the dartdevc problem seems to originate in the guiness library.

@joernahrens
Copy link
Contributor Author

Thanks @ircecho, I'm happy to hear that! You are talking about these lines, right?

These are needed for the testcases in https://github.com/swagger-api/swagger-codegen/tree/master/samples/client/petstore/dart/petstore/test

Instructions for running these tests: https://github.com/swagger-api/swagger-codegen/blob/master/samples/client/petstore/dart/petstore/README.md

@ircecho
Copy link
Contributor

ircecho commented Jan 16, 2018

Yes, that is true! It is very unfortunate, that the tests have been written with a now unmaintained library, which is no longer usable with dartdevc. One day, they will have to be rewritten with proper dart testing libraries. However, I think we should leave the situation alone and ignore dartdevc for the moment, and when there is time, fix it in another pull request.

However, I think that the dependencies on browser and guinness can safely be dropped from pubspec.mustache and thereby from the library.

Other than that, I think this is merge-ready. @wing328

@ircecho
Copy link
Contributor

ircecho commented Jan 18, 2018

Unfortunately there is a problem when serializing maps.

        "strings": {
          "type": "object",
          "additionalProperties": {
            "$ref": "#/definitions/ArticleSpecificationStrings"
          }
        },

This apparently creates a map in the dart code which is correct:

  Map<String, ArticleSpecificationStrings> strings = {};

However, the serialization/deserialization code is erroneous. @joernahrens could you fix this, please?

There's another thing as well, which makes fixing this problem a lot easier.

You should rename the method toMap to toJson, and the encoder will automatically use it when encoding complex objects, which should save you a lot of complexity because

      'articleRepresentations': articleRepresentations == null ? null : ArticleRepresentation.toMapList(articleRepresentations),

can become

      'articleRepresentations': articleRepresentations,

and you can throw away all the toMapList and so on methods and JSON.encode(.) should take care of everything. I did a small proof-of-concept and it seems to work, however, there may be problems when implementing it in the whole library.

To deserialize the maps I created a method like this

  static Map<String, ProductStrings> mapFromJson(Map<String, Map<String, dynamic>> json) {
    var map = new Map<String, ProductStrings>();
    if (json != null && json.length > 0) {
      json.forEach((String key, Map<String, dynamic> value) => map[key] = new ProductStrings.fromJson(value));
    }
    return map;
  }

It has to be used in the right places, but that should fix everything.

@joernahrens
Copy link
Contributor Author

@ircecho I already was expecting something.. 😉
Sure, I will take a look!

@ghost ghost force-pushed the replace-dart-with-flutter-impl branch from 379615c to fdecd30 Compare January 18, 2018 21:06
@joernahrens
Copy link
Contributor Author

Made some changes @ircecho, thx for the toJson hint, that makes the code a lot easier. The mapFromJson looks good, I added it, but I didn't have a test at hand.
Could you please have a look again? Thanks in advance!

@wing328
Copy link
Contributor

wing328 commented Jan 20, 2018

@joernahrens thanks for the quick update based on the feedback.

@ircecho I wonder if you can take another look and I'll merge it if it looks good to you.

@@ -103,7 +93,7 @@ class ApiClient {
} else if (obj is String) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if, as a whole, should be unnecessary. JSON.convert(.) should be able to cope with all those types out of the box.

{{name}} =
{{#complexType}}
{{#isListContainer}}{{complexType}}.listFromJson(json['{{name}}']){{/isListContainer}}{{^isListContainer}}
{{#isMapContainer}}new {{complexType}}.mapFromJson(json['{{name}}']){{/isMapContainer}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This leads to a problem because mapFromJson rightly is not a constructor.

Example code generated:

    strings =
        
        new ProductStrings.mapFromJson(json['strings'])
        
    ;

Problems detected:

  error • A value of type 'ProductStrings' can't be assigned to a variable of type 'Map<String, ProductStrings>' at lib/model/product_representation.dart:57:9 • invalid_assignment
  error • The class 'ProductStrings' doesn't have a constructor named 'mapFromJson' at lib/model/product_representation.dart:57:28 • new_with_undefined_constructor

new needs to be removed. We should also add a test that uses maps.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi again, sorry, that new was wrong of course..

I fixed that, and also removed the line for string "serialization". How would you add a test for this in general? All the testing is based on the petstore.yml at the moment. Would be best, to extend that API to include an API-call including a Map, or did you have a simpler test in mind @ircecho ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

petstore.yaml only covers the basic while petstore-with-fake-endpoints-models-for-testing.yaml contains a lot more edge cases.

You may consider updating ./bin/dart-petstore.sh to use petstore-with-fake-endpoints-models-for-testing.yaml.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wing328 Thanks, for the hint, I could have found this by myself 😁
I gave it a short try and saw, that there are a LOT of errors to fix. There are things like:

  • Classes starting with numbers
  • Other names not matching dart requirements
  • Maps in maps don't work at the moment
  • maybe many more..

I started this a bit, but would provide a further PR if I have the time to fix these.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes absolutely! Better fix the crazy stuff later!

@ircecho
Copy link
Contributor

ircecho commented Jan 20, 2018

Oh and I found another bug:

        "debugInformation" : {
          "type" : "object"
        },

When I'm using a field like this, which correctly leads to field

  Object debugInformation = null;

but Object does not have a toJson or fromJson method.

    debugInformation =
        
        
        new Object.fromJson(json['debugInformation'])
    ;

@wing328 How do you think we should handle this situation? Because we cannot properly deserialize or serialze an object if we do not know what type it is.

@wing328
Copy link
Contributor

wing328 commented Jan 21, 2018

@wing328 How do you think we should handle this situation? Because we cannot properly deserialize or serialze an object if we do not know what type it is.

We encounter similar issues with other generators based on the same reason that there's no way the API client can serialize/deserialize the payload without knowing anything (e.g. properties) about the object. My suggestion is to avoid using type: object and to define the object properly with the properties clearly defined.

@ghost ghost force-pushed the replace-dart-with-flutter-impl branch from a79bb8b to d99040e Compare January 22, 2018 13:58
@joernahrens joernahrens reopened this Jan 22, 2018
@joernahrens joernahrens reopened this Jan 22, 2018
@ircecho
Copy link
Contributor

ircecho commented Jan 22, 2018

@wing328 That is exactly that kind of behaviour from a code generator that bums people out. If it is valid code in the specification, it should also produce valid code in the generator.

@joernahrens On the curious case of

        "debugInformation" : {
          "type" : "object"
        },

I suggest we use dynamic instead of Object, e.g.

  dynamic debugInformation = null;

and in the fromJson method just use

    debugInformation = json['debugInformation'];

And then I hope we can finally merge this PR! This will fix so many problems!

@wing328
Copy link
Contributor

wing328 commented Jan 22, 2018

@wing328 That is exactly that kind of behaviour from a code generator that bums people out. If it is valid code in the specification, it should also produce valid code in the generator.

Yup, I agreed the spec is correct but most generators do not work well in this case and users may find it surprising.

I'll try to add an FAQ to educate the users about clearly documenting the "object" with properties.

@joernahrens
Copy link
Contributor Author

@ircecho I'd agree with the solution, but what would be the best way to handle this object case in the mustache files? I don't see how to achieve this.

@ircecho
Copy link
Contributor

ircecho commented Jan 22, 2018

@joernahrens Is there no way to check, if it is a plain object? I assumed as much. Well if it is not that simple, I guess we can keep it as it is for the moment and fix it later.

@wing328
Copy link
Contributor

wing328 commented Jan 23, 2018

Agreed with fixing it later.

If there's no other pending item, I'll merge this PR tomorrow.

@joernahrens
Copy link
Contributor Author

The circleci build failed, but I don't really see if this PR caused that (I think not). Otherwise, yes, sounds good, thanks for all feedback.

@wing328
Copy link
Contributor

wing328 commented Jan 23, 2018

No worry. It fails more frequently for other jobs too. I'll restart it later

@cputoaster
Copy link

Thanks a lot for this PR, I would also like to use Flutter. Was just trying this, and got a problem when using enums. Its starting to use dartson again and does some strange quoting:


@Entity()
class ACTION {
  /// The underlying value of this enum member.
  final  value;

  const ACTION._internal(this.value);

  static const ACTION nONE_ = const ACTION._internal(&quot;NONE&quot;);
  static const ACTION lOGOUT_ = const ACTION._internal(&quot;LOGOUT&quot;);
  static const ACTION uPGRADE_ = const ACTION._internal(&quot;UPGRADE&quot;);
  static const ACTION rESETCLIENTDB_ = const ACTION._internal(&quot;RESET_CLIENT_DB&quot;);
  static const ACTION rELOADPAGE_ = const ACTION._internal(&quot;RELOAD_PAGE&quot;);
  static const ACTION rEGISTER_ = const ACTION._internal(&quot;REGISTER&quot;);
}

class ACTIONTypeTransformer extends TypeTransformer<ACTION> {

  @override
  dynamic encode(ACTION data) {
    return data.value;
  }

  @override
  ACTION decode(dynamic data) {
    switch (data) {
      case &quot;NONE&quot;: return ACTION.nONE_;
      case &quot;LOGOUT&quot;: return ACTION.lOGOUT_;
      case &quot;UPGRADE&quot;: return ACTION.uPGRADE_;
      case &quot;RESET_CLIENT_DB&quot;: return ACTION.rESETCLIENTDB_;
      case &quot;RELOAD_PAGE&quot;: return ACTION.rELOADPAGE_;
      case &quot;REGISTER&quot;: return ACTION.rEGISTER_;
      default: throw('Unknown enum value to decode: $data');
    }
  }
}```

Am I doing something wrong or is this just not there yet?

@joernahrens
Copy link
Contributor Author

@cputoaster Sorry, I didn't put work into enums. As far I understood this also wasn't completely supported before. Since I didn't change the mustache template for it, this weird &quot; was like this before 😐

@wing328 wing328 merged commit a5e26a4 into swagger-api:master Jan 25, 2018
@wing328
Copy link
Contributor

wing328 commented Jan 25, 2018

@cputoaster please open a separate issue to track it.

The &quot; issue can be resolved by replacing {{ ... }} with {{{ ... }}}.

@cputoaster cputoaster mentioned this pull request Jan 25, 2018
while-loop added a commit to while-loop/swagger-codegen that referenced this pull request Sep 18, 2018
string enums need to generated with triple handlebars. swagger-api#7418 (comment)
while-loop added a commit to while-loop/swagger-codegen that referenced this pull request Sep 18, 2018
string enums need to generated with triple handlebars. swagger-api#7418 (comment)
while-loop added a commit to while-loop/swagger-codegen that referenced this pull request Sep 19, 2018
string enums need to generated with triple handlebars. swagger-api#7418 (comment)
while-loop added a commit to while-loop/swagger-codegen that referenced this pull request Sep 19, 2018
string enums need to generated with triple handlebars. swagger-api#7418 (comment)
@aroraenterprise
Copy link

Hey, i am just curious and a little confused. This PR definitely seems to be something I am looking for which is to generate a Flutter 1.0 compatible Swagger Client Library. I am unsure which branch/jar of swagger-codgen-cli to use to be able to use the work the that has been done in this PR. Thank you.

@SimonLammer
Copy link

Any update on the question from @aroraenterprise ?

@vitorTheDev
Copy link

vitorTheDev commented Aug 15, 2019

Just add the option -DbrowserClient=false.
e.g.
java -jar swagger-codegen-cli-2.4.2.jar generate -l dart -i openapi.json -o swagger -DbrowserClient=false

After that, follow the install instructions in the generated folder README.md

Source

@krishnakirana
Copy link

DbrowserClient

Where to run this command?

@vitorTheDev
Copy link

vitorTheDev commented Aug 16, 2019

DbrowserClient

Where to run this command?

Easiest way is to use the online generators.
Or you can download and run this command (you must have java installed correctly):
java -jar path/to/swagger-codegen-cli-2.4.7.jar generate -l dart -i http://petstore.swagger.io/v2/swagger.json -o /path/to/petstore_dart_client -DbrowserClient=false
For more info, just read the docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Dart/Flutter] Dart files generated not working on flutter platform
8 participants