Missing CloudFormationResourceSpecification.json Files in This Repository

[ScriptAutomate]

Referencing the following documentation:
https://github.com/awsdocs/aws-cloudformation-user-guide/blob/master/doc_source/cfn-resource-specification.md

Can the CloudFormationResourceSpecification.json files be versioned in this repository? Or are they versioned elsewhere?

I have run into bugs when generating modules from the specifications, with things like broken Documentation property links. I think it would be beneficial to have the JSON files versioned here to make it easier to open issues on them and update/fix.

It would also make life easier for toolmakers by tracking the files in this repo (troposphere, sparkleformation, etc.), and being able to see the commits that have modified the json source files.

EDIT (04/28/18) Here is the published documentation page that includes the JSON specification templates I am referencing, but are not versioned within this repo: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-resource-specification.html

[AWSRandall]

Hello, and thanks for using CloudFormation!

I'm discussing your request with the engineering team. We do not currently have plans to include the resource specification JSON files in this repo.

Is your request to have the file in this repo so that you can track changes between each version?

[andreabedini]

hey @AWSRandall, re: CloudFormationResourceSpecification.json: the version for ap-southeast-2 is mysteriously missing CloudBuild 🤷‍♂️ I already reported it on the website but I haven't heard back anything.

[ScriptAutomate]

@AWSRandall -- Here are some reasons why this would be beneficial:

• Easily see changes between new additions to CloudFormation
• Easily see what regions differ, and to create tools that could give instant feedback on supported regions for a stack
• To automatically create CloudFormation modules for languages like PowerShell Core, and automatically create help documentation within the module based on certain Documentation sub-property links (in the past, I have found many broken links -- of which I do not know if they had been resolved)
• To automatically kick-off Continuous Integration (CI) tools and build pipelines based off of updates / changes to the JSON configurations
• Versioning the JSON configuration files here would improve the feedback loop for those consuming the JSON configurations
• To open issues here in the GitHub repository when there are problems with the JSON configuration files (such as broken documentation links, and other things), and to see issues raised by others such as @andreabedini as it creates an added layer of visibility to AWS CloudFormation customers to be aware of and contribute further to the conversation :)

[andreabedini]

We do have CloudFormation tooling built automatically from that specifications document so, yeah, I subscribe to everything that @ScriptAutomate says

[jakejscott]

@ScriptAutomate @andreabedini Not sure if your using C# and dotnet core at all, but I maintain a library that has all the cloudformation api generated here: https://github.com/jakejscott/Humidifier You might be able to call that from Powershell?

[cmmeyer]

Hi @ScriptAutomate -- the S3 bucket behind the CDN does use object prefixes to track versions (alongside "latest" which is linked from the URL in the docs).

I've written some tools myself to track changes across versions. For instance, the code snippet below can grab all the versions currently available along with their timestamps. Is that useful or is the problem closer to being able to track when "latest" shifts to a new version/which version is currently active?

from boto3 import client
from boto3 import resource

cfn_resource_spec_bucket='cfn-resource-specifications-us-east-1-prod'
cfn_resource_spec_key='CloudFormationResourceSpecification.json'

def resource_spec_versions():
    versions = set()

    # Create a client
    conn = client('s3', region_name='us-east-1')

    # Create a reusable Paginator
    paginator = conn.get_paginator('list_objects')

    # Create a PageIterator from the Paginator
    page_iterator = paginator.paginate(Bucket=cfn_resource_spec_bucket)

    for page in page_iterator:
        for key in page['Contents']:
            prefix,file = key['Key'].split('/',1)
            versions.add(prefix)
    return(versions)

def resource_spec_date(version):
    key = version + '/' + cfn_resource_spec_key
    s3 = resource('s3',region_name='us-east-1')
    obj = s3.Object(cfn_resource_spec_bucket, key)
    return(obj.last_modified.date())

for version in resource_spec_versions():
    spec_date = resource_spec_date(version)
    print(version + ': ' + str(spec_date))

[PatMyron]

Related, these few are missing "Properties": {} in the specification for the regions I checked:
AWS::AppSync::GraphQLApi.Tags
AWS::AppSync::GraphQLApi.AdditionalAuthenticationProviders
AWS::CodeBuild::Project.FilterGroup
AWS::Glue::SecurityConfiguration.S3Encryptions
AWS::Transfer::User.SshPublicKey
AWS::EC2::LaunchTemplate.CapacityReservationPreference
The fields that would be inside "Properties": {} are still there for the first four, but the different formatting on these could lead to incorrect automated parsing for someone unaware of the difference

[ScriptAutomate]

Last week I noticed this was still open, so I have created this repository backup of versioned CFN resource specification files:

• ScriptAutomate/aws-cfn-resource-specs

Thanks to @cmmeyer for the code example, which I started from!

This is what I had in mind when it comes to having it versioned somewhere, and managed by AWS. Issues like what @PatMyron and @andreabedini are talking about could then be easily tracked and referenced to the specification json files themselves, and closed out upon resolution/fixes in resolved new versions of spec files, instead of people needing to open an issue in this repository or only sending support tickets directly to AWS Support. My initial justification statement above, to @AWSRandall, still stands.

The build code I have in a separate repo that needs some cleaning up, but it's automated enough that the repo will continue to be updated automatically if I deploy it with some monitoring triggers.

[joshbean]

Closing this issue or pull request in advance of archiving this repo. For more information about the decision to archive this repo (and others in the "awsdocs" org), see the announcement on the AWS News Blog.