proposal: build: define standard way to recognize machine-generated files

[dmitshur]

Latest Status

Edit in 2018: This is a large issue with many comments (so many, that GitHub hides most of them by default). Here is the summary of the latest status.

This proposal has been accepted and implemented by Rob Pike. The final format can be seen here (it links to a comment in this thread by Rob Pike, with the final format that was chosen):

https://golang.org/s/generatedcode

By now, most of the generated Go code uses a comment that matches the format that's described there.

The original proposal is below.

---

Abstract

I propose Go creates a standardized format, which would enable code-generating tools to reliably communicate to humans and other machine tools that the output is in fact a generated file. Additionally, Go should add a recommended style for a simple code generated disclaimer (which satisfies the above criteria).

Proposed Definition

A file is considered to be "generated" if and only if the maintainer(s) of the project consider it a non-canonical source. In order to make long-term changes to such files, another source must be modified, and the file in question is then fully (re)generated by a reproducible machine tool.

A distinguishable property of generated files is that they can be deleted and re-generated with a zero diff.

Background

One of the strong values that Go brings are conventions and best practices that reduce bikeshedding, increase consistency and readability across diverse teams of Go programmers. Having a well defined convention, format, or standard for things that are unimportant to the key task, but need to have some value saves time.

During Gopherfest SV 2015, Rob Pike gave a talk Go Proverbs (video, bullet-point summary) that mentioned:

Gofmt's style is no one's favorite, yet gofmt is everyone's favorite.

To expand on that, there are many examples for things that have have a recommend format/style in Go that let you simply reuse that and not force you (and other people) to invent your own style:

• gofmt for Go code formatting. https://golang.org/cmd/gofmt/
• Variable naming. https://github.com/golang/go/wiki/CodeReviewComments#mixed-caps
• Package comments. https://github.com/golang/go/wiki/CodeReviewComments#package-comments
• Package names. https://github.com/golang/go/wiki/CodeReviewComments#package-names
• Example func naming. https://godoc.org/testing#hdr-Examples
• Build constraints. https://godoc.org/go/build#hdr-Build_Constraints
• Struct tag strings. https://godoc.org/reflect#StructTag

Description

There is one type of comment which is commonly used, but has no existing well-defined officially suggested style recommended by Go.

It is a comment that most tools that generate Go code tend to write somewhere at the top of the code.

There are currently many variations of such disclaimer headers in the wild, and they often vary insignificantly (in spacing, punctuation, etc.). New variations come to be when authors look at how other tools do this, see a large variance, end up picking their favorite and tweaking it.

Consider the following examples in the wild:

// generated by stringer -type Pill pill.go; DO NOT EDIT

// Code generated by "stringer -type Pill pill.go"; DO NOT EDIT

// Code generated by vfsgen; DO NOT EDIT

// Created by cgo -godefs - DO NOT EDIT

/* Created by cgo - DO NOT EDIT. */

// Generated by stringer -i a.out.go -o anames.go -p ppc64
// Do not edit.

// DO NOT EDIT
// generated by: x86map -fmt=decoder ../x86.csv

// DO NOT EDIT.
// Generate with: go run gen.go -full -output md5block.go

// generated by "go run gen.go". DO NOT EDIT.

// DO NOT EDIT. This file is generated by mksyntaxgo from the RE2 distribution.

// GENERATED BY make_perl_groups.pl; DO NOT EDIT.

// generated by mknacl.sh - do not edit

// DO NOT EDIT ** This file was generated with the bake tool ** DO NOT EDIT //

// Generated by running
//  maketables --tables=all --data=http://www.unicode.org/Public/8.0.0/ucd/UnicodeData.txt --casefolding=http://www.unicode.org/Public/8.0.0/ucd/CaseFolding.txt
// DO NOT EDIT

/*
* CODE GENERATED AUTOMATICALLY WITH github.com/ernesto-jimenez/gogen/unmarshalmap
* THIS FILE SHOULD NOT BE EDITED BY HAND
*/

This creates 2 problems.

1. It's a problem for authors of code generator tools. Such authors need to spend time figuring out what format of a disclaimer header they want to use; there is no canonical standardized format.
2. It's a problem for authors of tools that want to be able to use the information (in a helpful way) whether certain files are generated or not. There is no simple implementation that will catch all of the variations above, and there is no standardized machine-readable format that they can detect.

This leads to circular arguments and PRs/CLs. For example, see the discussion and the change itself at https://go-review.googlesource.com/#/c/15073/. It started from https://github.com/github/linguist/blob/473282d/lib/linguist/generated.rb#L241, which led to CL 15073. That lead to shurcooL/vfsgen@b2aab1c and shurcooL/go@43b2166. But the initial GitHub behavior came from protobuf disclaimers.

I've created the following func to try to answer the question if a file is generated. At this time, it uses heuristics and best-effort to tell if a file is generated. https://github.com/shurcooL/go/blob/c661e953e604ba4a84a3c4e458462a481bd6ce72/analysis/generated_detection.go If there was a well defined requirement for tools to follow, this code can be made simpler and more reliable. Ideally, that helper should be moved into external library for people to reuse, and for generator tools that wish to be compliant to be able to use it for verification.

Goals

The goals of this proposal are twofold.

Primarily, to resolve the current impossibility of reliable communication between code generator tool output, and tools that try to determine if a file is code generated.

There should be a way for code generator tool authors to be able to express in their generated output that the file is generated, such that it's possible to reliably detect if a file is generated by other tools.

Secondarily, for code generator authors that simply don't care about what their disclaimer header looks like, provide a recommended style (that satisfies the first condition) template to use.

The implementation details should be defined in a design doc.

Non-goal

It is a non-goal to figure out how existing tools should choose to use or not use the fact whether a given file is generated.

There is some fear that if it's possible to determine if a file is generated reliably, then tools that display code differences will hide generated code differences. That is absolutely the choice of the tool, and in my opinion it should not enforce any behavior that users are unhappy with.

Having additional information (whether a file is generated or not) should enable tools to offer better user experiences - it should not cause tools offer worse experiences than currently.

This proposal focuses solely on enabling code generator tool authors that wish to use a standard disclaimer header to do without forcing them to invent their own format, and for tool authors that wish to make use of information whether a file is generated or not to be able to use that information as they wish. Details of how they do that is outside the scope.

Conclusion

By not standardizing a way for those two types of tools to communicate, it leads to ad-hoc solutions that are sub-optimal emerging, as can be seen above. Go has an opportunity to de-fragment this space and create a recommended standard format that will resolve the needs above, and allow people to migrate existing tools to use the specified format.

Once there's a standard, it's easy to begin updating existing tools towards it over time, and new generator/other tools can start relying on it.

Meta-disclaimer

I expect coming up with a recommended style may likely cause a lot of bikeshedding. However, I think it's a cost that's worth incurring, to go through this process once, so that we can avoid having to continuously suffer it while there's no standard at all. I personally don't care too much about what the actual format is (as much as I do about resolving the higher level problems described); I'm okay with whatever Go authors come up with. Any standard is better than no standard at all.

[dmitshur]

This proposal comes from the result of discussion in CL 15073. /cc @bradfitz @dominikh @minux

[zellyn]

How about // DO NOT EDIT. Generated by $GENERATOR_NAME_OR_COMMANDLINE on the first line of the file?

[bradfitz]

@zellyn, I think we need to agree that this is a good idea to standardize before we pick paint colors. :)

[zellyn]

@bradfitz Agreed. Just curious why the proposal didn't include a strawman proposal for format. :-)

[dmitshur]

I think a good and simple way to approach it would be to add somewhere (Go style guide, or a more internal document) a section that says something like this:

If a code generator tool wishes to mark a file as "generated" in a way so it can be reliably understood by other tools to be generated, then it must satisfy these criteria:

1. {{Simple rule to follow.}}
2. {{Another simple rule that must be followed.}}

As an example, the following disclaimer will satisfy both the criteria above:

{{Example disclaimer template.}}

That way, people who don't want to be creative can just copy/paste that example disclaimer template and use it.

However, if someone wants to add more details or make their disclaimer different, they can still do that. They just need to follow rules 1 and 2 so that it can still be recognized by tools as a generated disclaimer header.

To detect if a given file is generated, you'd only need to check if those described rules are satisfied - no need for any other heuristics. (If there are some false negatives, then the generator can be updated to satisfy the rules, instead of muddying the detection algorithm.)

I think putting it that way makes this workable. I don't think it's possible to enforce a single header format that will satisfy everyone's needs/wants and have people be okay with it. But providing simple rules that the header will follow and giving an example would work splendidly.

[rakyll]

/cc @hyangah for gobind generated files.

[adg]

This proposal sounds like a good idea. We should use the same string that is recognized by GitHub.