Differences between LB3 and LB4 req-res cycle

[hacksparrow]

Differences between LB3 and LB4 req-res cycle. Addresses #4836.

[raymondfeng]

LoopBack object? Do you mean LoopBack application object?

[raymondfeng]

We should mention rest-crud here to mimic creating REST CRUD APIs from a model and a datasource by convention. See https://github.com/strongloop/loopback-next/blob/master/docs/site/Creating-CRUD-REST-apis.md

[hacksparrow]

Ah yes, thanks!

[hacksparrow]

Added:

For those who are uncomfortable with the concept of having to creating a
repository and a controller for a model, we have a component @loopback/rest-crud
; with a little bit of configuration, a model file is all you will need to
create the REST endpoints. Once your requirements outgrow what
@loopback/rest-crud provides, you can implement your REST endpoints the
idiomatic way.

[agnes512]

I think we can use page https://loopback.io/doc/en/lb4/Creating-crud-rest-apis.html on the site

[hacksparrow]

@loopback/rest-crud links to https://loopback.io/doc/en/lb4/Creating-crud-rest-apis.html .

[dhmlau]

In LoopBack 4, you cannot do it anymore

Maybe we can make this more positive. :). Some suggestion:
In LoopBack 4, a custom Express router can be mounted on a LoopBack app using the RestApplication.mountExpressRouter() API.

[hacksparrow]

Facts should be presented as clearly and straightforwardly as possibly, so that expectations are set right there are no misconceptions. If we don't tell a certain popular way of doing something is not supported anymore, we are adding ambiguity. Let's not do that.

[bajtos]

I agree with Diana that "you cannot do it anymore" leaves negative impression. We have plans to support Express middleware, see #1293 and #2035. It is also possible to mount a LB4 app in a new top-level Express app, that's our currently recommended way for installing Express middleware - see https://loopback.io/doc/en/lb4/express-with-lb4-rest-tutorial.html

I am proposing to rephrase the sentence as "you cannot do it yet" and provide users with details about the current status (workaround) and future plans.

[bajtos]

I am proposing to rephrase the sentence as "you cannot do it yet"

Even better: "LoopBack 4 does not provide first-class support for mounting application-wide Express middleware yet." (and so on)

[dhmlau]

just wondering... does middleware.json related to the paragraph below?

Suggested change:

LoopBack 4 does not have
[middleware.json](https://loopback.io/doc/en/lb3/middleware.json.html)
as LoopBack 3 because it is not required because of the architectural changes.

[hacksparrow]

Updated:

In LoopBack 4 middleware.json is not require anymore because of architectural changes.

[dhmlau]

models files automatically created -> model files automatically create?

[dhmlau]

I'm thinking whether we can put it something along the line:
In LoopBack 3, the model and datasources are tightly coupled. Therefore, once both are specified, the corresponding REST APIs can be created.
In LoopBack 4, there is a separation of concerns where models and datasources are not tightly coupled. The Repository does xxxxx. The Controller does xxxxxx. With the rest-crud package, a similar mechanism can be achieved.

[hacksparrow]

Edited:

In LoopBack 3, models files automatically create the corresponding REST API
endpoints and the database query machinery (using the configured datasource).

Apart from that, I'd like to keep it as such.

[agnes512]

I don't know if we have any tutorial/blog that puts everything together. It'd be good to link it here.

[hacksparrow]

That link https://loopback.io/doc/en/lb4/Creating-components.html is supposed to be it.

[emonddr]

Are the contents of Request/response pathway forthcoming` ?

[hacksparrow]

Yes :D

[bajtos]

Thank you @hacksparrow for sharing the early draft. I like that you are covering various aspects of migration.

I find it difficult to orientate myself in the text, I am lacking at least a bit of structure - it feels like a raw dump of ideas to me.

[bajtos]

I agree with Diana that "you cannot do it anymore" leaves negative impression. We have plans to support Express middleware, see #1293 and #2035. It is also possible to mount a LB4 app in a new top-level Express app, that's our currently recommended way for installing Express middleware - see https://loopback.io/doc/en/lb4/express-with-lb4-rest-tutorial.html

I am proposing to rephrase the sentence as "you cannot do it yet" and provide users with details about the current status (workaround) and future plans.

[bajtos]

Since you are using a link for Repository, please use a link for Controller too. https://loopback.io/doc/en/lb4/Controllers.html

[hacksparrow]

Controllers already linked two paragraphs up. Will link, anyway.

[bajtos]

I feel the components are out of scope of request/response life cycle. We have a standalone doc page to cover migration of components - see https://loopback.io/doc/en/lb4/migration-extensions.html

Unless there is something specific to the way how components interact (or contribute to) request/response life cycle, then I am proposing to remove this and the next paragraph.

[hacksparrow]

As someone who wrote a component to add routes to the app, who is going through the docs to see what has changed in LB4 from the req-res perspective, this info will definitely help me; instead of having to browse through the docs, with no certainty when and where I will find the info I need.

[bajtos]

Migration of boot scripts are covered by https://loopback.io/doc/en/lb4/migration-boot-scripts.html.

In the context of this page, I would like to see instructions on migrating custom Express routes contributed by a boot script. I think the solution is to convert such Express routes to regular LB4 Controllers? Maybe that's something belonging to our docs for boot script migration? IDK.

[hacksparrow]

Will add an example of migrating a simple route, using a controller.

[jannyHou]

agreed, the bootscript migration doc contains the guide of retrieving different artifacts, but not adding artifacts like endpoints.

server.get('/hello', (req, res) => {
    res.send('Hello!');
  });

is not covered yet, but we should add an entry for it, I mean in the bootscript migration guide.

[bajtos]

I don't see how is this relevant to request/response life cycle? IMO, this belongs to https://loopback.io/doc/en/lb4/migration-extensions.html

[hacksparrow]

Imagine someone wrote LB3 components to add routes to the app. It would help them to know what happens to those components from the context of the changes in the req-res cycle.

[hacksparrow]

I find it difficult to orientate myself in the text, I am lacking at least a bit of structure - it feels like a raw dump of ideas to me.

Maybe the lack of sub headers. I struggled with it too, but as the size of the content is growing, I think they can be contained within sub headers, and it will look much better.

[hacksparrow]

@bajtos I have applied your feedback and added sub headers.

[jannyHou]

👍 Good start! I like the beginning part of section "Request/response infrastructure", it would be good if it has brief outline about how the other artifact comparisons(other sections afterwards) are related/connected to "the means of adding request/response infrastructure".

[jannyHou]

Suggested change

	is not require anymore because of architectural changes.
	is not required anymore because of architectural changes.

[hacksparrow]

@jannyHou

Let's take a look at how the means of adding request/response infrastructure
have changed in LoopBack 4.

Doesn't that hint that the following sub-sections are about the the means of adding request/response infrastructure?

[jannyHou]

Doesn't that hint that the following sub-sections are about the the means of adding request/response infrastructure?

@hacksparrow oh let me explain, you have 4 sections as

• Express middleware and routers
• Models
• Components
• Boot Scripts

I was suggesting add a brief sentence describing how they are related to the Req/Res cycle.

[bajtos]

Nice progress!

The content starting from "Request/response pathway" is exactly what I was looking for in this page! 👏

I feel the initial sections (up to but excluding "Request/response pathway") are duplicating content from other parts of the migration guide, but I can live with that 🤷

I found few places that deserves further clarification, PTAL at my comments below 👇

[bajtos]

I feel this section is misplaced here. As I understand the content, it describes how LoopBack 4 validates model data at data-access (Repository/ORM) layer. Please note that we are not using AJV there!

Now this page is about REST layer. I agree it's a good idea to describe how validation (and coercion) works at the REST layer, so let's improve/rework the validation section to be more accurate.

In LB3, this part is handled by strong-remoting - see https://github.com/strongloop/strong-remoting/tree/master/lib/types:

• The validation is very minimal, we check only that the provided value matches the expected type (e.g. a string is not allowed for a numeric parameter, but any object will be accepted for object parameters). See validate methods in individual types.
• Coercion is pretty comprehensive and applies different rules for values coming from JSON (implemented by fromTypeValue) or from string sources like the url query & HTTP headers (implemented by fromSloppyValue)

In LB4, we use OpenAPI to describe the type & constraints of request parameters. In a typical controller, we build OpenAPI Schemas describing model data from the metadata provided by LoopBack Models. Non-model parameters (e.g. id in findById(id), filter in find(filter)) are described using inline schema.

At runtime, parameter handling involves several steps:

1. We start with lightweight coercion & validation at OpenAPI parameter level, see https://github.com/strongloop/loopback-next/blob/master/packages/rest/src/coercion/coerce-parameter.ts and https://github.com/strongloop/loopback-next/blob/master/packages/rest/src/coercion/validator.ts
2. Then we convert OpenAPI Schema to JSON Schema and run AJV to validate values against the schema, see https://github.com/strongloop/loopback-next/tree/master/packages/rest/src/validation. I think we are not using AJV-based coercion, but I am not sure. Also at the moment, schema-based validation is implemented for request bodies only, see Validate parameter values against their schema #1573

[hacksparrow]

@bajtos this section is not about validation at the Repository/ORM layer, it is more about how to do something (validation) in LB4, what you did in LB3 - as closely as possible.

I have updated the content and rephrased some sentences, I hope it makes more sense now.

[bajtos]

I see. I am afraid I find the new content still misleading.

Specifically, the following paragraph refers to validation rules like validatesPresenceOf that are not applied by the HTTP/REST layer at all.

LoopBack 3 models come with
 [validation methods](https://loopback.io/doc/en/lb3/Validating-model-data.html)
 . The validation rules are derived from the model definition files or are
 applied by calling the validation methods in the model's JavaScript file.

I am missing a mention of the validation & coercion rules applied by strong-remoting.

I think it would be really helpful for our users to capture more content & pointer from ea52b6f#r408201456 in the docs, this is the kind of information that's almost impossible to find otherwise.

I am proposing the following high-level structure for this section:

1. In LB3, we have strong-remoting validation defined by accepts metadata and data-access-level validation defined by model methods like validatesPresenceOf.
2. Briefly describe how strong-remoting validates & coerces data, add links to source code. This way we don't have to go into too much detail now, but still provide useful links to anybody wanting to dig deeper (including us in the future).
3. In LB4, we do full JSON Schema validation at REST layer and property-based validation at data-access level.
4. Briefly describe how LB4 REST layer validates & coerces data, add links to source code.
5. LB4 does not validate non-body parameters against schema yet, + include a link to GH issue.
6. In LB4, it's possible to apply additional validation of model data at REST layer via jsonSchema model setting, which can partially replace advanced validations provided by LB3 at data-access level. Explain the caveats.
7. LB4 does not yet offer advanced validation rules at data-access layer like LB3 does, + include a link to GH issue.

[bajtos]

Nice! This is showing how to provide custom validations that will be applied at REST layer only. Please note this is not a direct replacement for validatesLenghtOf() and friends, because those validations are applied only at data-access level in LB3.

I feel it's important to be very clear about which validation rules are applied at which layer.

1. validatesLengthOf() is defined in LB3 models and executed by the data-access layer, it's ignored by REST API layer (strong-remoting). This constraint is enforced always, both for REST API requests and when modifying data via JS API.
2. jsonSchema field in LB4 property definition is executed by REST API layer and is ignored by the data-access layer. These constraints are enforced only for REST API requests, they can be violated/by-passed when modifying data via JS/TS Repository APIs.

[emonddr]

Nice writeup, @hacksparrow ! :)

[agnes512]

Good progress. I like the examples you have to show the differences. Just have a few comments.

[jannyHou]

👏 Good stuff I like the diagrams showing LB3 vs LB4 req/res cycle. A few suggestions and comments.

And I think interceptor also plays an important role in the cycle because they are invoked right before/after the controller methods and have a chance to process the req/res. Maybe mention it in section "Access to the request/response object" or "Request/response pathway"?

[jannyHou]

👍

[jannyHou]

nitpick: we usually use relative links like ./express-with-lb4-rest-tutorial.md(the path may not be accurate).

[jannyHou]

Suggested change

	[mount a LoopBack 4](https://loopback.io/doc/en/lb4/express-with-lb4-rest-tutorial.html)
	[mount a LoopBack 4 application](https://loopback.io/doc/en/lb4/express-with-lb4-rest-tutorial.html)

[jannyHou]

Suggested change

	If you want to mount a Express router, you can use the
	If you want to mount an Express router in a LoopBack 4 application, you can use the

[jannyHou]

Suggested change

	and controllers to the app using dependency injection.
	and [controllers](the_link_of_controller) to the app using dependency injection.

[jannyHou]

agreed, the bootscript migration doc contains the guide of retrieving different artifacts, but not adding artifacts like endpoints.

server.get('/hello', (req, res) => {
    res.send('Hello!');
  });

is not covered yet, but we should add an entry for it, I mean in the bootscript migration guide.

[jannyHou]

sth maybe worth mentioning here: a sequence consists of a series of actions. Developers can add custom actions to process the request and response.

[hacksparrow]

Let's leave that to the sequence doc, which is linked.

[jannyHou]

nitpick: adding links for the 3 artifacts like the previous content.

[jannyHou]

Suggested change

	Controllers, services, and repositories are LoopBack 4 components that
	Controllers, services, and repositories are LoopBack 4 artifacts that

[jannyHou]

We now have the validation docs explaining the 3 layers data validation in lb4, see https://github.com/strongloop/loopback-next/blob/master/docs/site/Validation.md (maybe not published yet)
You probably want to add it as a reference here :p

[bajtos]

Much better!

I feel some of the changed sections still need more attention, see my comments below.

[bajtos]

Please replace absolute URLs with relative paths.

Suggested change

	[routing table](https://loopback.io/doc/en/lb4/apidocs.rest.routingtable.md).
	[routing table](./apidocs/apidocs.rest.routingtable.md).

The same comment applies to all links starting with https://loopback.io/doc/en/lb4. Please search for them yourself.

You can run the following command to generate files in docs/site/apidocs:

$ (cd docs && npm run prepack)

[hacksparrow]

Oh, actually I added them back manually as I imagined those missing files will be a problem.

[bajtos]

I see. I am afraid I find the new content still misleading.

Specifically, the following paragraph refers to validation rules like validatesPresenceOf that are not applied by the HTTP/REST layer at all.

LoopBack 3 models come with
 [validation methods](https://loopback.io/doc/en/lb3/Validating-model-data.html)
 . The validation rules are derived from the model definition files or are
 applied by calling the validation methods in the model's JavaScript file.

I am missing a mention of the validation & coercion rules applied by strong-remoting.

I think it would be really helpful for our users to capture more content & pointer from ea52b6f#r408201456 in the docs, this is the kind of information that's almost impossible to find otherwise.

I am proposing the following high-level structure for this section:

1. In LB3, we have strong-remoting validation defined by accepts metadata and data-access-level validation defined by model methods like validatesPresenceOf.
2. Briefly describe how strong-remoting validates & coerces data, add links to source code. This way we don't have to go into too much detail now, but still provide useful links to anybody wanting to dig deeper (including us in the future).
3. In LB4, we do full JSON Schema validation at REST layer and property-based validation at data-access level.
4. Briefly describe how LB4 REST layer validates & coerces data, add links to source code.
5. LB4 does not validate non-body parameters against schema yet, + include a link to GH issue.
6. In LB4, it's possible to apply additional validation of model data at REST layer via jsonSchema model setting, which can partially replace advanced validations provided by LB3 at data-access level. Explain the caveats.
7. LB4 does not yet offer advanced validation rules at data-access layer like LB3 does, + include a link to GH issue.

[bajtos]

Suggested change

	[default CRUD repository](https://loopback.io/doc/en/lb4/apidocs.repository.defaultcrudrepository.html)
	[default CRUD repository](apidocs/apidocs.repository.defaultcrudrepository.md)

[bajtos]

LGTM 👏

I have few minor suggestions to consider, see below.

[bajtos]

Suggested change

	[this issue](https://github.com/strongloop/loopback-next/issues/1573)
	[loopback-next#1573](https://github.com/strongloop/loopback-next/issues/1573)

[bajtos]

(The last review was meant to be an approval.)