Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shim ObjectMapper and add diagnostic version info to exceptions #23441

Merged
merged 17 commits into from
Sep 14, 2021

Conversation

lmolkova
Copy link
Member

@lmolkova lmolkova commented Aug 9, 2021

This is proof of concept for shimming ObjectMapper

  • moves all configuration and custom serializers to azure.core.implementation.jackson package
  • it only exposes ObjectMapperShim that is used by JacksonAdapter and other core classes that use mapper
  • ObjectMapperShim, when LinkageError is detected, adds diagnostic version info to exception (we'll add link to docs here)

Here's how adapter = new JacksonAdapter() looks like with Jackson 2.9.0:

Logs:

[main] ERROR com.azure.core.implementation.jackson.JacksonVersion - Version '2.9.0' of package 'jackson-annotations' is not supported (too old), please upgrade.
[main] ERROR com.azure.core.implementation.jackson.JacksonVersion - Version '2.9.0' of package 'jackson-core' is not supported (too old), please upgrade.
[main] ERROR com.azure.core.implementation.jackson.JacksonVersion - Version '2.9.0' of package 'jackson-databind' is not supported (too old), please upgrade.
[main] ERROR com.azure.core.implementation.jackson.JacksonVersion - Version '2.9.0' of package 'jackson-dataformat-xml' is not supported (too old), please upgrade.
[main] ERROR com.azure.core.implementation.jackson.JacksonVersion - Version '2.9.0' of package 'jackson-datatype-jsr310' is not supported (too old), please upgrade.
[main] INFO com.azure.core.implementation.jackson.JacksonVersion - Package versions: jackson-annotations=2.9.0, jackson-core=2.9.0, jackson-databind=2.9.0, jackson-dataformat-xml=2.9.0, jackson-datatype-jsr310=2.9.0, azure-core=1.19.0-beta.2, critical errors found: true
[main] ERROR com.azure.core.implementation.jackson.ObjectMapperShim - com/fasterxml/jackson/databind/cfg/MapperBuilder
Package versions: jackson-annotations=2.9.0, jackson-core=2.9.0, jackson-databind=2.9.0, jackson-dataformat-xml=2.9.0, jackson-datatype-jsr310=2.9.0, azure-core=1.19.0-beta.2, critical errors found: true

Exception:

com.azure.core.implementation.jackson.JacksonVersionMismatchError: com/fasterxml/jackson/databind/cfg/MapperBuilder
Package versions: jackson-annotations=2.9.0, jackson-core=2.9.0, jackson-databind=2.9.0, jackson-dataformat-xml=2.9.0, jackson-datatype-jsr310=2.9.0, azure-core=1.19.0-beta.2, critical errors found: true

	at com.azure.core.implementation.jackson.ObjectMapperShim.createSimpleMapper(ObjectMapperShim.java:53)
	at com.azure.core.util.serializer.JacksonAdapter.<init>(JacksonAdapter.java:64)
	at com.meskazu.dephell.Tests.nullProps(Tests.java:16)
	...
Caused by: java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/cfg/MapperBuilder
	at com.azure.core.implementation.jackson.ObjectMapperShim.createSimpleMapper(ObjectMapperShim.java:50)
	... 67 more
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.cfg.MapperBuilder
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
	... 68 more

Please share any early feedback.

@ghost ghost added the Azure.Core azure-core label Aug 9, 2021
Copy link
Member

@JonathanGiles JonathanGiles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like great progress!


public static SemanticVersion parse(String version) {
Objects.requireNonNull(version, "'version' cannot be null.");
String[] parts = version.split("\\.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a capturing regex would be better here

Pattern VERSION_PATTERN = Pattern.compile("(\d+)\\.(\d+)\\.(\d+)(?:-beta\\.\d+)?");

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a full real semver regex ^([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?(?:\+[0-9A-Za-z-]+)? , I'd prefer to keep it simple since we don't really need all parts

Comment on lines 74 to 82
if (!version.isValid()) {
logger.warning("Could not find version of '{}'.", packageName);
return false;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused on this, below both using a too old or too new version will return true (which I'm taking as it failed the version check?) but this returns false.

Copy link
Member Author

@lmolkova lmolkova Aug 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we were not able to get version (e.g. it may happen when you debug locally), i.e. we can't conclusively say if this version is compatible or not, i.e. we don't indicate the critical error.
Maybe since the critical error is just a flag on the log, I can remove it - by looking at versions we should be able to deduce it ourselves.

@check-enforcer
Copy link

This pull request is protected by Check Enforcer.

What is Check Enforcer?

Check Enforcer helps ensure all pull requests are covered by at least one check-run (typically an Azure Pipeline). When all check-runs associated with this pull request pass then Check Enforcer itself will pass.

Why am I getting this message?

You are getting this message because Check Enforcer did not detect any check-runs being associated with this pull request within five minutes. This may indicate that your pull request is not covered by any pipelines and so Check Enforcer is correctly blocking the pull request being merged.

What should I do now?

If the check-enforcer check-run is not passing and all other check-runs associated with this PR are passing (excluding license-cla) then you could try telling Check Enforcer to evaluate your pull request again. You can do this by adding a comment to this pull request as follows:
/check-enforcer evaluate
Typically evaulation only takes a few seconds. If you know that your pull request is not covered by a pipeline and this is expected you can override Check Enforcer using the following command:
/check-enforcer override
Note that using the override command triggers alerts so that follow-up investigations can occur (PRs still need to be approved as normal).

What if I am onboarding a new service?

Often, new services do not have validation pipelines associated with them, in order to bootstrap pipelines for a new service, you can issue the following command as a pull request comment:
/azp run prepare-pipelines
This will run a pipeline that analyzes the source tree and creates the pipelines necessary to build and validate your pull request. Once the pipeline has been created you can trigger the pipeline using the following comment:
/azp run java - [service] - ci

@lmolkova
Copy link
Member Author

/azp run java - core - tests

@azure-pipelines
Copy link

Azure Pipelines failed to run 1 pipeline(s).

sdk/core/azure-core-jackson-tests/pom.xml Outdated Show resolved Hide resolved
Comment on lines 373 to 375
if (cache.size() >= CACHE_SIZE_LIMIT) {
cache.clear();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clearing an input cache can be misleading as the caller would not have expected the cache to be cleared. Cache eviction should be handled by the class/owner declaring the cache to give more control over the lifetime of cache entries.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not part of the current change, this code was just moved as-is from JacksonAdapter.

cache.clear();
}

return cache.computeIfAbsent(key, compute);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before clearing the cache, should we check to see if the entry exists and clearing may not be required?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still, I'd like to preserve existing behavior since this is not part of this change.

Copy link
Member

@srnagar srnagar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lmolkova lmolkova merged commit bf82b20 into Azure:main Sep 14, 2021
*/
private void checkVersion(SemanticVersion version, String packageName) {
if (!version.isValid()) {
logger.warning("Could not find version of '{}'.", packageName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this have a return below it?

and not sure if this needs to be a warning, just because you can't figure out the version doesn't mean you need to spam my logs 😢

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

azure-sdk pushed a commit to azure-sdk/azure-sdk-for-java that referenced this pull request Apr 6, 2023
Azure Orbital - swagger fix for api-version 2022-11-01 (Azure#23441)

* added example for the new field

* updated the double ref that was causing build failure
azure-sdk pushed a commit to azure-sdk/azure-sdk-for-java that referenced this pull request Apr 13, 2023
Review request for Microsoft.ContainerInstance to add version stable/2023-05-01 (Azure#23485)

* Adds base for updating Microsoft.ContainerInstance from version preview/2022-10-01-preview to version 2023-05-01

* Updates readme

* Updates API version in new specs and examples

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23166)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23169)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* use altest api version as default for Avocado

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23170)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* use altest api version as default for Avocado

* update host to eastus2euap endpoint to check manifest

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23452)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* use altest api version as default for Avocado

* update host to eastus2euap endpoint to check manifest

* use old api versionf or operations

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23453)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* use altest api version as default for Avocado

* update host to eastus2euap endpoint to check manifest

* use old api versionf or operations

* revert Operations example api version

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23471)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* use altest api version as default for Avocado

* update host to eastus2euap endpoint to check manifest

* use old api versionf or operations

* revert Operations example api version

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23473)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* use altest api version as default for Avocado

* update host to eastus2euap endpoint to check manifest

* MGRP S360 Vuln (Azure#22832)

* Add blockchain to latest profile

* Add additional types

* Fix Swagger issues

* Solve validation

---------

Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>

* use old api versionf or operations

* Azure Orbital - swagger fix for api-version 2022-11-01 (Azure#23441)

* added example for the new field

* updated the double ref that was causing build failure

* Fixed PrometheusRuleGroups examples (Azure#23390)

* Fixed PrometheusRuleGroups examples

* One more fix

* Remvoe flattern (Azure#23460)

Co-authored-by: Will Huang <huangwill@microsoft.com>

* Mvad update (Azure#23434)

* Add default value 10 for topContributorCount

* Update AnomalyDetector typespec to latest typespec and Azure.Core versions and fix all warnings

* Update TypeSpec config

* Add back language emitter options

* Fix cspell and model validation errors

---------

Co-authored-by: Chunlei Wang <chuwan@microsoft.com>

* revert Operations example api version

---------

Co-authored-by: ramoka178 <57157576+ramoka178@users.noreply.github.com>
Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>
Co-authored-by: Stuti Kumar <113545470+stuti-1807@users.noreply.github.com>
Co-authored-by: giladsu <43436811+giladsu@users.noreply.github.com>
Co-authored-by: will <koyasu221b@gmail.com>
Co-authored-by: Will Huang <huangwill@microsoft.com>
Co-authored-by: Mike Kistler <mikekistler@microsoft.com>
Co-authored-by: Chunlei Wang <chuwan@microsoft.com>

* use management.auzre.com endpoint

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23483)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* use altest api version as default for Avocado

* update host to eastus2euap endpoint to check manifest

* MGRP S360 Vuln (Azure#22832)

* Add blockchain to latest profile

* Add additional types

* Fix Swagger issues

* Solve validation

---------

Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>

* use old api versionf or operations

* Azure Orbital - swagger fix for api-version 2022-11-01 (Azure#23441)

* added example for the new field

* updated the double ref that was causing build failure

* Fixed PrometheusRuleGroups examples (Azure#23390)

* Fixed PrometheusRuleGroups examples

* One more fix

* Remvoe flattern (Azure#23460)

Co-authored-by: Will Huang <huangwill@microsoft.com>

* Mvad update (Azure#23434)

* Add default value 10 for topContributorCount

* Update AnomalyDetector typespec to latest typespec and Azure.Core versions and fix all warnings

* Update TypeSpec config

* Add back language emitter options

* Fix cspell and model validation errors

---------

Co-authored-by: Chunlei Wang <chuwan@microsoft.com>

* revert Operations example api version

* use management.auzre.com endpoint

---------

Co-authored-by: ramoka178 <57157576+ramoka178@users.noreply.github.com>
Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>
Co-authored-by: Stuti Kumar <113545470+stuti-1807@users.noreply.github.com>
Co-authored-by: giladsu <43436811+giladsu@users.noreply.github.com>
Co-authored-by: will <koyasu221b@gmail.com>
Co-authored-by: Will Huang <huangwill@microsoft.com>
Co-authored-by: Mike Kistler <mikekistler@microsoft.com>
Co-authored-by: Chunlei Wang <chuwan@microsoft.com>

* fix package version in readme

* Fnuarnav containerinstance microsoft.container instance 2023 05 01 (Azure#23484)

* add container security context property

* update readme tag to previous stable version

* add example with security context

* avocado fix default tag should contain all endpoints

* add back spot priority changes

* use previous stable version as default tag

* prettier fix

* use spaces

* fix error

* use spaces

* fix CI errors

* use altest api version as default for Avocado

* update host to eastus2euap endpoint to check manifest

* MGRP S360 Vuln (Azure#22832)

* Add blockchain to latest profile

* Add additional types

* Fix Swagger issues

* Solve validation

---------

Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>

* use old api versionf or operations

* Azure Orbital - swagger fix for api-version 2022-11-01 (Azure#23441)

* added example for the new field

* updated the double ref that was causing build failure

* Fixed PrometheusRuleGroups examples (Azure#23390)

* Fixed PrometheusRuleGroups examples

* One more fix

* Remvoe flattern (Azure#23460)

Co-authored-by: Will Huang <huangwill@microsoft.com>

* Mvad update (Azure#23434)

* Add default value 10 for topContributorCount

* Update AnomalyDetector typespec to latest typespec and Azure.Core versions and fix all warnings

* Update TypeSpec config

* Add back language emitter options

* Fix cspell and model validation errors

---------

Co-authored-by: Chunlei Wang <chuwan@microsoft.com>

* revert Operations example api version

* add codeowners for Compute Instance swagger (Azure#23437)

Co-authored-by: Naman Agarwal <naagarw@microsoft.com>

* [Hub Generated] Review request for Microsoft.DevHub to add version preview/2022-10-11-preview (Azure#22828)

* Adds base for updating Microsoft.DevHub from version preview/2022-04-01-preview to version 2022-10-11-preview

* Updates readme

* Updates API version in new specs and examples

* start 10-11 preview

* add words

* fix readme version

* update swagger version

* add second putworkflow example

* fix generatepreviewartifactsresponse

* align generate preview artifacts example

* update param locations that got changed

* add x-ms-client-flatten for artifact properties

* Adding WorkflowRunStatus

* Fixing enum name

* add namespace to example

---------

Co-authored-by: Brandon Foley <brandonfoley13@gmail.com>

* Update readme.python.md (Azure#23208)

* fixing async response type for machinelearningservices-2023-02-01-preview (Azure#23105)

* fixing regex pattern and async response type

* remove update to regex

* Fix lint error for Datadog RP (Azure#23477)

* Fix link error for Datadog RP

* Fix version

* merging billing fix to public repo (Azure#23424)

Co-authored-by: Gaurav Bang <gauravbang@microsoft.com>

* use management.auzre.com endpoint

* fix package version in readme

---------

Co-authored-by: ramoka178 <57157576+ramoka178@users.noreply.github.com>
Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>
Co-authored-by: Stuti Kumar <113545470+stuti-1807@users.noreply.github.com>
Co-authored-by: giladsu <43436811+giladsu@users.noreply.github.com>
Co-authored-by: will <koyasu221b@gmail.com>
Co-authored-by: Will Huang <huangwill@microsoft.com>
Co-authored-by: Mike Kistler <mikekistler@microsoft.com>
Co-authored-by: Chunlei Wang <chuwan@microsoft.com>
Co-authored-by: Naman Agarwal <namanag16@gmail.com>
Co-authored-by: Naman Agarwal <naagarw@microsoft.com>
Co-authored-by: David Gamero <david340804@gmail.com>
Co-authored-by: Brandon Foley <brandonfoley13@gmail.com>
Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com>
Co-authored-by: Karishma Daga <karishmadaga@microsoft.com>
Co-authored-by: vikotha <81368129+vikotha@users.noreply.github.com>
Co-authored-by: Gaurav <bang.gourav@gmail.com>
Co-authored-by: Gaurav Bang <gauravbang@microsoft.com>

* fix tag in readme

* update example with capabilities example

* fix typo

---------

Co-authored-by: ramoka178 <57157576+ramoka178@users.noreply.github.com>
Co-authored-by: Mark Cowlishaw <markcowl@microsoft.com>
Co-authored-by: Stuti Kumar <113545470+stuti-1807@users.noreply.github.com>
Co-authored-by: giladsu <43436811+giladsu@users.noreply.github.com>
Co-authored-by: will <koyasu221b@gmail.com>
Co-authored-by: Will Huang <huangwill@microsoft.com>
Co-authored-by: Mike Kistler <mikekistler@microsoft.com>
Co-authored-by: Chunlei Wang <chuwan@microsoft.com>
Co-authored-by: Naman Agarwal <namanag16@gmail.com>
Co-authored-by: Naman Agarwal <naagarw@microsoft.com>
Co-authored-by: David Gamero <david340804@gmail.com>
Co-authored-by: Brandon Foley <brandonfoley13@gmail.com>
Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com>
Co-authored-by: Karishma Daga <karishmadaga@microsoft.com>
Co-authored-by: vikotha <81368129+vikotha@users.noreply.github.com>
Co-authored-by: Gaurav <bang.gourav@gmail.com>
Co-authored-by: Gaurav Bang <gauravbang@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Azure.Core azure-core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants