x/pkgsite: add support for type parameters

[findleyr]

As the proposal for go/ast has stabilized (#47781), I think it is time to start adding support for type parameters in pkgsite.

There is one major obstacle for this, which is that App Engine will not support 1.18 until some time after the 1.18 release, and we want to support type parameters well before then.

My thoughts on how to do this:

• Write a tool to maintain copies of stdlib packages in pkgsite. I made a working prototype of this in https://golang.org/cl/322411
• Copy the following packages to pkgsite: go/ast, go/build/constraint, go/format, go/internal/typeparams, go/parser, go/token, go/printer, go/scanner.
• Wire these in to pkgsite, which already uses a modified copy of go/doc.

I tested that this was possible in https://golang.org/cl/321949.

Once this is done, we should be able to sync these copied packages to tip using go generate. Out of the box this should resolve parsing errors for generic code, and we can proceed with adding any additional features that make it easier to browse code with type parameters in pkgsite. After App Engine supports 1.18, we can delete these copies.

I can do this, but will hold of for a week or two to see if anyone has concerns or better ideas.

CC @julieqiu @jba @jamalc

[dmitshur]

I understand the scope of this issue is x/pkgsite, and I don't mean to change that.

I'll point out that a similar issue may apply to x/website, though golang.org serves documentation under a significantly reduced set of use cases as of #44356. It similarly has a directory with ported stdlib packages (https://go.googlesource.com/website/+/refs/heads/master/internal/backport/).

[jba]

I'm in favor. There is an alternative: move pkgsite to Cloud Run. That would probably be more work, and certainly more risk, but I just wanted to put it out there.

One point you don't mention: none of those packages would be able to use 1.18 features (including generics) for a while. And we won't know that they do until we copy them and recompile pkgsite. On the other hand, I don't imagine they'll change much once generics are done.

[findleyr]

One point you don't mention: none of those packages would be able to use 1.18 features

Right, that's a good point. In https://golang.org/cl/321949 I had to add a stub io/fs for this reason.

IIRC @griesemer and I had discussed and thought this was OK for these packages in particular. We also have a history of maintaining ports like x/tools/go/internal/gcimporter, but that's more work.

[earthboundkid]

FWIW, as a user, I'm writing some generic packages against gotip, and I find it really annoying that I can't read HTML formatted docs yet. If this can ship before 1.18, it would be great for me.

[findleyr]

@carlmjohnson this will definitely ship before 1.18.

Since no objections and #47781 has been accepted, I will start soon.

[toothrot]

Re-milestoning, as we will not be blocking the Go 1.18 release on this issue.