Add annotations when consumer test fails (#1857)

* Add annotation step to test_types workflow

* Remove branch restriction from `pull_request` event

* Make tsconfig.build.json fail consumer test

* Add always() to if clause in test_types workflow

* Use clearer failed() instead of always()

* Remove brackets from workflow if

* s/status/conclusion

* s/failed/failure

* Add longer annotation

* Format workflow echo

* Attempt to use cat for multi-line annotation

* Rename to consumer test

* Add issue context links

* Nicer links in consumer test readme

* Revert "Make tsconfig.build.json fail consumer test"

This reverts commit f6a3678.

* Add consumer-test to eslintrc ignore

.eslintrc.json

@@ -23,7 +23,7 @@
     "lib/**/*",
     "lib-*/**/*",
     "types/**/*",
-    "test-types/**/*"
+    "consumer-test/**/*"
   ],
   "globals": {
     "__DEV__": "readonly"

.github/workflows/test_types.yml → .github/workflows/consumer_test.yml

@@ -1,12 +1,12 @@
-name: Test types
+name: Consumer test
 on:
   push: {branches: main}
-  pull_request: {branches: main}
+  pull_request:
   workflow_dispatch:
 
 jobs:
-  test-types:
-    name: Test types
+  consumer-test:
+    name: Consumer test
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
@@ -38,8 +38,14 @@ jobs:
         run: npm link
 
       - name: Link and test
-        working-directory: test-types
+        id: link-and-test
+        working-directory: consumer-test
         run: |
           npm install
           npm link @primer/react
           npm run check
+
+      - name: Add annotation
+        if: failure() && steps.link-and-test.conclusion == 'failure'
+        run: |
+          echo "::error file=tsconfig.build.json::Test package could not build. See https://github.com/primer/react/blob/main/consumer-test"

consumer-test/README.md

@@ -0,0 +1,25 @@
+# Primer React Consumer Test
+
+This directory is used to run a simple test that asserts that a consumer of
+Primer React can build their own project with strict TypeScript options enabled,
+including `"skipLibCheck": false`.
+
+During Primer React's build process, we run the TypeScript compiler and output
+`.d.ts` declaration files for consumers of Primer React that are using
+TypeScript. If the build script runs with a TypeScript configuration that has
+any files in its `types` or `typeRoots` that import any of our development
+dependencies, it's possible for our build output to be polluted by interface
+augmentations in those dependencies, or in transitive dependencies.
+
+The best way to avoid this is to ensure that any files that import development
+dependencies are excluded in our `tsconfig.build.json` file we use to build
+Primer React.
+
+If a mistake is made and a file is omitted, we will catch those when we attempt
+to build this consumer library, which has `"skipLibCheck": false` in its
+TypeScript configuration.
+
+For historical context, see these issues:
+
+- [v27.0.0 breaks TypeScript typings](https://github.com/primer/react/issues/1163)
+- [Storybook dependency changes types in build output](https://github.com/primer/react/issues/1849)