[Not Swagger] API profile map for the profile version 2017-03-09-profile #2895

bganapa · 2018-04-18T05:00:12Z

This is not a swagger file. This PR is to discuss the API-Profile map format. Added the map for the existing azure stack specific api-profile 2017-03-09-profile

AutorestCI · 2018-04-18T05:00:14Z

Automation for azure-sdk-for-node

Unable to detect any generation context from this PR.

AutorestCI · 2018-04-18T05:00:14Z

Automation for azure-sdk-for-python

Unable to detect any generation context from this PR.

AutorestCI · 2018-04-18T05:00:14Z

Automation for azure-libraries-for-java

Unable to detect any generation context from this PR.

AutorestCI · 2018-04-18T05:00:14Z

Automation for azure-sdk-for-go

Unable to detect any generation context from this PR.

hovsepm · 2018-04-18T17:54:55Z

api-profiles/2017-03-09-profile.json

+            "types": {}
+        },
+        "microsoft.subscription": {
+            "version": "2016-02-01"


there is subscription section under microsoft.resources with the different API version. Aren't they gona conflict?

Its a mistake. I am fixing. Subscriptions is not a resource type under resources namespace. Going to remove from here. Thanks for taking a look. I am going to send a mail after adding a readme.md on the structure

bganapa · 2018-04-18T16:39:09Z

api-profiles/2017-03-09-profile.json

+            "version": "2016-02-01",
+            "types": {
+                "links": "2016-09-01",
+                "subscriptions": "2016-06-01"


subscriptions is not a RT under microsoft.resources

bganapa · 2018-04-18T17:46:27Z

api-profiles/2017-03-09-profile.json

@@ -0,0 +1,48 @@
+{


The file should probably get in to the existing profiles folder

bganapa · 2018-04-18T21:01:42Z

profile/2017-03-09-profile.json

+        },
+        "microsoft.compute": {
+            "version": "2016-03-30",
+            "types": {}


Do we need to mandate specifying empty types: {}

There is no schema for the profile AFAIK. What docs did you use to create this file?

we dont have schema yet. This is just a result of internal discussion with SDK team

@hovsepm this PR will by default define the schema. I'm +1 on allowing skipping empty properties.

Keeping empty types{} could be helpful for easy parsing , Im keeping it for now. we can discuss this in our meeting

hovsepm · 2018-04-18T21:11:26Z

@joshgav could you review this PR?

hovsepm · 2018-04-18T21:12:39Z

profile/readme.md

+An API profile represents a map of resource provider namespaces and their API versions. It is a representation of an Azure Platform declaring a set of APIs to be supported across all our clouds.Instead of specifying individual api-versions for each resource type across each namespace, the developer can just align the application to a profile and the tools/SDKs will themselves revert to the right api-versions as directed by the profile. With API Profiles developers can specify a profile version that applies to an entire template and, at runtime, ARM will pick the right versions of the resources. This way, the customers don’t have to worry about which are the right resource versions to specific clouds. 
+
+
+<<Insert detailed documentation link here>>


this seems to be temporary placeholders. Could you add the actual link?

added the link

bganapa · 2018-04-18T21:24:44Z

profile/readme.md

+|    |   +---"version":"xxxx-xx-xx",
+|    |   +---"types":{
+|    |        +---"resourcetype3":"xxxx-xx-xx",
+|    |        +---"resourcetype4":"xxxx-xx-xx"


How do we handle nested resource type supporting a different api-version than the parent resource type? This is not a concern for now, but our format should be able to handle this

Should use the syntax of the providers call:
"resourceType": "storageAccounts/services/metricDefinitions"

knithinc · 2018-04-19T22:10:28Z

profile/2017-03-09-profile.json

+        },
+        "microsoft.network": {
+            "version": "2015-06-15",
+            "types": {}


I would rather be more verbose than less. List out the individual resource types here explicitly.

Personally I fine to be less explicit, for the point I made here just after.

I expect this map to be used to create the schemas that Visual Studio can consume and aid (intellisense, etc) in authoring templates within the IDE, along with SDKs generation. I want this to be a generic map which could serve multiple uses. In that case, if it is not explicitly verbose, i.e., if all the resource types and corresponding API versions are not listed, I am not sure how useful this is.

Also, if sometime down the line ARM decides to provide an API to retrieve cloud capabilities based on profile versions, this map could serve as a starting manifest or this in itself could become a manifest.

Either way is okay with me, people looking to this file for guidance can learn to skip or minimize those sections. For purposes of excluding types, though, as discussed below, it may be best to list them all minus the excluded ones.

This is Open Issue 1

bganapa · 2018-04-19T22:16:09Z

@joshgav added 2018-03-01-hybrid json

bganapa · 2018-04-19T22:17:26Z

profile/2018-03-01-hybrid.json

+            "version": "2018-02-01",
+            "types": {
+                "extensionsMetadata": "2015-01-01",
+                "offers": "2015-01-01"


/offers is specific to azurestack and i do not think we need to include it here since we dont have swaggers yet

Endpoint availability and Swagger are two differents things. If offers exists, and then this file is an accurate description of the endpoint, it's nice to have it here. Swagger can come later, and will match that.

I think without swagger, it is not goign to be meaningful to keep it here as SDKs would refer this json and likely fail as it is not goign to find a swagger?

They are not supposed to fail if the Swagger doesn't exist. It depends how this file is interpreted, and again, what is the purpose of this file. Is this a Stack only file? A generic file? etc.

Profiles on the whole are generic, though the two in this PR are only for use with Stack. There will be others not intended for Stack soon. The purpose of profile specs is to provide a canonical definition which all downstream things can turn to - e.g. SDKs and docs.

bganapa · 2018-04-20T22:09:12Z

profile/2017-03-09-profile.json

+        },
+        "microsoft.network": {
+            "version": "2015-06-15",
+            "types": {}


How do we handle the Not supported resource type in an api-version? e.g though azurestack supports 2015-06-15 api version for microsoft.network, it does not support applicationGateway resource type

It depends how you interpret this file IMHO. Is this a Stack only file or not?
If not, and we're saying "Everything that is under Microsoft.Network should be called with 2015-06-15" and that's fine for Public Azure. Stack gateway will complain that the RT doesn't exist, and it's a fine error as well.

If someone builds an app on this profile it should work with both Stack and Azure - that's the promise of profiles. Profiles themselves are generic, Stack happens to snap to a specific profile.

An app developer might build an app with a profile on public Azure, then expect it to run on Stack. For that case I think we need to somehow spec that a resource type isn't supported in a given profile; that's only part of the problem though, the SDKs aren't designed to exclude use of certain types I don't think.

This is an Open issue 2

knithinc · 2018-04-27T06:00:45Z

@devigned could you please review the profile map and provide feedback.

lmazuel · 2018-04-27T20:12:50Z

profile/2017-03-09-profile.json

+        },
+        "microsoft.compute": {
+            "version": "2016-03-30",
+            "types": {}


We must have virtualMachineScaleSets here with 2015-06-15 (the Network ApiVersion). This file should not replicate the Swagger mistake of considering that virtualMachineScaleSets is a Network RP.

I retract that comment, after discussing with @johanste

joshgav · 2018-04-27T20:16:52Z

@johanste @sphibbs @devigned @lmazuel @marstr @salameer this PR describes the last two hybrid profiles in a machine-readable way. We will use this for reference and automatic generation. Does this proposal meet those use cases for all languages? Thanks!

lmazuel · 2018-04-27T20:17:58Z

profile/2017-03-09-profile.json

+            "version": "2016-01-01",
+            "types": {}
+        },
+        "microsoft.subscription": {


This is false, Microsoft.Subscription exists, but it's not what you mean here:
https://github.com/Azure/azure-rest-api-specs/blob/master/specification/subscription/resource-manager/Microsoft.Subscription/preview/2018-03-01-preview/subscriptions.json#L19

This is tough because the URL is https://management.azure.com/subscriptions?api-version=2016-06-01 that does not involve RP.
When we list providers, how this one is listed?

If you do az provider list, you get:

"namespace": "Microsoft.Resources", ... "apiVersions": [ "2018-02-01", "2018-01-01", "2017-08-01", "2017-06-01", "2017-05-10", "2017-05-01", "2017-03-01", "2016-09-01", "2016-07-01", "2016-06-01", "2016-02-01", "2015-11-01", "2015-01-01", "2014-04-01-preview" ], "locations": [], "properties": null, "resourceType": "subscriptions"

which is probably what I would use for consistency

Microsoft.Subscription is REALLY something else:

"namespace": "Microsoft.Subscription", "registrationState": "NotRegistered", "resourceTypes": [ { "aliases": null, "apiVersions": [ "2017-11-01-preview" ], "locations": [ "West US" ], "properties": null, "resourceType": "SubscriptionDefinitions" },

I am going to remove microsoft.subscription from this list and we should model Subscription as a resource type under Microsoft.Resources

thanks @bganapa

lmazuel · 2018-04-27T20:21:44Z

profile/2018-03-01-hybrid.json

+            "version": "2018-02-01",
+            "types": {
+                "extensionsMetadata": "2015-01-01",
+                "offers": "2015-01-01"


Endpoint availability and Swagger are two differents things. If offers exists, and then this file is an accurate description of the endpoint, it's nice to have it here. Swagger can come later, and will match that.

lmazuel · 2018-04-27T20:24:03Z

profile/readme.md

+|    |   +---"version":"xxxx-xx-xx",
+|    |   +---"types":{
+|    |        +---"resourcetype3":"xxxx-xx-xx",
+|    |        +---"resourcetype4":"xxxx-xx-xx"


Should use the syntax of the providers call:
"resourceType": "storageAccounts/services/metricDefinitions"

marstr · 2018-04-27T20:24:45Z

profile/2017-03-09-profile.json

@@ -0,0 +1,47 @@
+{


I'd like to see a metadata property in this object, to remove any semantics from the name of this file/structure of the repository its living in.

For example:

"info":{ "title":"2017-03-09", "description":"<A brief overview of whom it is that is expected to consume this profile, and under which circumstances.>" }, "resource-manager":{ ... }

this wouldn't hurt IMHO. there are probably other properties we'll want here too

Added a info node

Sorry to bother you again, but could you include a title or name property in the info block?

marstr · 2018-04-27T20:29:05Z

profile/2017-03-09-profile.json

+                "locks":"2015-01-01",
+                "policyassignments": "2015-10-01-preview",
+                "policydefinitions": "2015-10-01-preview"
+            }


My interpretation of the API Versions above is that operations not associated with a particular type should be populated from "2015-07-01" and type specific operations should come from the API Version specified. In this case:

locks -> "2015-01-01"

policyassignements -> "2015-10-01-preview"

policydefinitions -> "2015-10-01-preview"

Is my interpretation correct?

Yes, That is correct

marstr · 2018-04-27T20:30:36Z

profile/readme.md

+The files in the folder describe the map of each resource provider's resource types and their supported api versions in the api profile.
+
+## Basics
+An API profile represents a map of resource provider namespaces and their API versions. It is a representation of an Azure Platform declaring a set of APIs to be supported across all our clouds.Instead of specifying individual api-versions for each resource type across each namespace, the developer can just align the application to a profile and the tools/SDKs will themselves revert to the right api-versions as directed by the profile. With API Profiles developers can specify a profile version that applies to an entire template and, at runtime, ARM will pick the right versions of the resources. This way, the customers don’t have to worry about which are the right resource versions to specific clouds. 


I don't like the word "revert" here, it sounds negative. How about "select"?

marstr · 2018-04-27T20:32:05Z

profile/readme.md

+## File Structure
+<<Todo: Define a json schema file if it helps>>
+
+The profile is an one to one map. The api version specified is a single version for the resource type, although the reource type could support other older api versions as well.


Grammar: "The profile is a one to one map"

joshgav · 2018-05-15T21:12:48Z

Hi @bganapa - looks good to me! Can you address @marstr and @lmazuel's comments as you've proposed so we can merge? I think this remains:

we may need to add all resource types so that unsupported ones can be excluded
add a "info" object for metadata
fix items clarified by @lmazuel

Thanks!

azuresdkciprbot · 2018-05-16T00:18:27Z

AutoRest linter results for ARM Related Validation Errors/Warnings

These errors are reported by the ARM team's validation tools, reachout to ARM RP API Review directly for any questions or concerns.

AutoRest Linter Guidelines | AutoRest Linter Issues | Send feedback

Thanks for your co-operation.

azuresdkciprbot · 2018-05-16T00:18:27Z

AutoRest linter results for SDK Related Validation Errors/Warnings

These errors are reported by the SDK team's validation tools, reachout to ADX Swagger Reviewers directly for any questions or concerns.

AutoRest Linter Guidelines | AutoRest Linter Issues | Send feedback

Thanks for your co-operation.

joshgav · 2018-05-16T19:13:56Z

Thanks @bganapa! I believe the only remaining question is whether to list all resource types explicitly. I'm +1 on that cause it will allow us to exclude unsupported types. @lmazuel @johanste @marstr thoughts?

marstr

Approving so that my status isn't blocking, but I'd still like to see a name/title property added.

joshgav · 2018-05-23T20:33:19Z

Hi @bganapa! Can you please add the individual resource types under their provider? We all agreed to this in another thread. Can you also add a "name" property to the metadata section per @marstr's comment? Thank you!

AutorestCI · 2018-05-23T20:37:36Z

Automation for azure-sdk-for-ruby

Unable to detect any generation context from this PR.

bganapa · 2018-05-23T20:46:34Z

@joshgav Added the resource types explicitly . Note that this changes the file format quite a bit. we can discuss in today's meeting. This needs another commit as i am yet to update the readme and Microsoft.web resource types, I am also waiting on confiration for the exact set of resource types that we support for the new 2018* profile

joshgav · 2018-05-30T20:40:36Z

Thanks @bganapa! I just checked the versions referenced in the 2018-03-01 profile and find many of the versions don't exist in the specifications/ directory in this repo, which is a prerequisite for generating libs. Do these versions exist in public Azure APIs? If not, the value of a hybrid profile is lost. If yes, can you add specs for them to the repo?

Thank you!

FYI @johanste

joshgav · 2018-05-30T21:05:38Z

profile/2018-03-01-hybrid.json

+			]
+        },
+        "microsoft.resources": {
+			"2017-10-01": [


version doesn't exist

joshgav · 2018-05-30T21:05:48Z

profile/2018-03-01-hybrid.json

+				"subscriptions/tagNames/tagValues",
+				"tenants"
+			],
+			"2015-01-01": [


version doesn't exist

decided in f2f meeting that we should remove this version completely since types are only for Stack

joshgav · 2018-05-30T21:20:52Z

remaining: verify Web entries are correct, remove Stack-specific types, and update README.

then merge!!

jianghaolu · 2018-05-31T23:31:47Z

profile/2018-03-01-hybrid.json

+    "resource-manager": {
+        "microsoft.authorization": {
+			"2017-04-01": [
+				"locks"


There doesn't seem to be a 2017-04-01 version of locks: https://github.com/Azure/azure-rest-api-specs/tree/master/specification/resources/resource-manager

…for the changes

jianghaolu · 2018-06-01T00:42:48Z

profile/2018-03-01-hybrid.json

+			"2016-09-01": [
+				"locks"
+			],
+			"2015-07-01": [


I don't see this API version here either: https://github.com/Azure/azure-rest-api-specs-pr/tree/master/specification/resources/resource-manager.

@jianghaolu Its here https://github.com/Azure/azure-rest-api-specs/tree/master/specification/resources/resource-manager/Microsoft.Authorization/stable/2016-09-01

if you are looking roleassignments, they are here.
Microsoft.Azuthorization specs are in two different places :( (they are owned by two different teams)

https://github.com/Azure/azure-rest-api-specs/tree/master/specification/authorization/resource-manager/Microsoft.Authorization/stable/2015-07-01

Ahh gotcha. Sorry about the confusion

jianghaolu · 2018-06-01T00:43:06Z

profile/2018-03-01-hybrid.json

+				"policyAssignments",
+				"policyDefinitions"
+			],
+			"2015-07-01-preview": [


Not this one either.

It seems azure team has left out specifying this providerOperations resource type for this api-version. This is not critical resource type and i am removing it from the profile map

jianghaolu · 2018-06-01T00:44:19Z

profile/2018-03-01-hybrid.json

+			]        
+		},
+        "microsoft.web": {
+				"2015-02-01":[


I cannot find 2015-02-01 here: https://github.com/Azure/azure-rest-api-specs/tree/master/specification/web/resource-manager. Is it somewhere else?

jianghaolu · 2018-06-01T01:13:16Z

profile/2018-03-01-hybrid.json

+				"providers",
+				"resourceGroups",
+				"resources",
+				"subscriptions",


subscriptions seem to be in different API versions: https://github.com/Azure/azure-rest-api-specs/tree/master/specification/resources/resource-manager#tag-package-subscriptions-2016-06

…ings under Web, picking up available swagger versions for /subscriptions

bganapa · 2018-06-04T18:59:52Z

@jianghaolu please let me know if you find any other issue. @joshgav This should be good to merge..

joshgav · 2018-06-05T17:09:43Z

Thanks @bganapa! Working on merging this now.

azuresdkci assigned hovsepm Apr 18, 2018

azuresdkci requested a review from hovsepm April 18, 2018 05:01

hovsepm suggested changes Apr 18, 2018

View reviewed changes

bganapa commented Apr 18, 2018

View reviewed changes

bganapa force-pushed the api-profiles branch from f345912 to a92c63f Compare April 18, 2018 21:00

bganapa commented Apr 18, 2018

View reviewed changes

hovsepm assigned joshgav Apr 18, 2018

hovsepm removed their assignment Apr 18, 2018

hovsepm reviewed Apr 18, 2018

View reviewed changes

hovsepm approved these changes Apr 18, 2018

View reviewed changes

bganapa commented Apr 18, 2018

View reviewed changes

knithinc reviewed Apr 19, 2018

View reviewed changes

bganapa commented Apr 19, 2018

View reviewed changes

bganapa commented Apr 20, 2018

View reviewed changes

lmazuel reviewed Apr 27, 2018

View reviewed changes

lmazuel requested changes Apr 27, 2018

View reviewed changes

marstr requested changes Apr 27, 2018

View reviewed changes

lmazuel approved these changes May 16, 2018

View reviewed changes

marstr approved these changes May 16, 2018

View reviewed changes

joshgav reviewed May 30, 2018

View reviewed changes

profile/2018-03-01-hybrid.json Outdated

]

},

"microsoft.resources": {

"2017-10-01": [

Copy link

joshgav May 30, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

version doesn't exist

joshgav reviewed May 30, 2018

View reviewed changes

jianghaolu reviewed May 31, 2018

View reviewed changes

bganapa added 6 commits May 31, 2018 17:32

API profile map for the profile version 2017-03-09-profile

0152129

added 2018-03-01-hybrid profile json

75223b3

Addressed the review comments

49087fb

Adding supported resource types explicitly

d622166

Microsoft.Web resource types and fixing resources apiversion

aa4c7de

Fixing Microsoft.Web, Fixing Locks resource type and updating readme …

fdb2a3e

…for the changes

bganapa force-pushed the api-profiles branch from bc5178b to fdb2a3e Compare June 1, 2018 00:35

jianghaolu reviewed Jun 1, 2018

View reviewed changes

Removing the providerOperations under Auth and pricingTiers,quotasett…

b03a201

…ings under Web, picking up available swagger versions for /subscriptions

marstr merged commit b29373c into Azure:master Jun 5, 2018

		An API profile represents a map of resource provider namespaces and their API versions. It is a representation of an Azure Platform declaring a set of APIs to be supported across all our clouds.Instead of specifying individual api-versions for each resource type across each namespace, the developer can just align the application to a profile and the tools/SDKs will themselves revert to the right api-versions as directed by the profile. With API Profiles developers can specify a profile version that applies to an entire template and, at runtime, ARM will pick the right versions of the resources. This way, the customers don’t have to worry about which are the right resource versions to specific clouds.


		<<Insert detailed documentation link here>>

[Not Swagger] API profile map for the profile version 2017-03-09-profile #2895

[Not Swagger] API profile map for the profile version 2017-03-09-profile #2895

Conversation

bganapa commented Apr 18, 2018 • edited by azuresdkci Loading

AutorestCI commented Apr 18, 2018 • edited Loading

Automation for azure-sdk-for-node

AutorestCI commented Apr 18, 2018 • edited Loading

Automation for azure-sdk-for-python

AutorestCI commented Apr 18, 2018 • edited Loading

Automation for azure-libraries-for-java

AutorestCI commented Apr 18, 2018 • edited Loading

Automation for azure-sdk-for-go

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hovsepm commented Apr 18, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bganapa commented Apr 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

knithinc commented Apr 27, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joshgav commented Apr 27, 2018

Choose a reason for hiding this comment

lmazuel Apr 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joshgav commented May 15, 2018

azuresdkciprbot commented May 16, 2018

AutoRest linter results for ARM Related Validation Errors/Warnings

azuresdkciprbot commented May 16, 2018

AutoRest linter results for SDK Related Validation Errors/Warnings

joshgav commented May 16, 2018

marstr left a comment

Choose a reason for hiding this comment

joshgav commented May 23, 2018

AutorestCI commented May 23, 2018 • edited Loading

Automation for azure-sdk-for-ruby

bganapa commented May 23, 2018

joshgav commented May 30, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joshgav commented May 30, 2018

bganapa commented Apr 18, 2018 •

edited by azuresdkci

Loading

AutorestCI commented Apr 18, 2018 •

edited

Loading

AutorestCI commented Apr 18, 2018 •

edited

Loading

AutorestCI commented Apr 18, 2018 •

edited

Loading

AutorestCI commented Apr 18, 2018 •

edited

Loading

lmazuel Apr 27, 2018 •

edited

Loading

AutorestCI commented May 23, 2018 •

edited

Loading