-
SummaryThis is an extension of the already existing RFC: gatsby-config/node typescript support We want to introduce an Ahead of Time (AOT) compilation of Gatsby's config files (gatsby-config and gatsby-node) to support use cases like TypeScript. In this first phase TypeScript will also be the only use case, however this will enable writing ESModules Syntax / You can try out a Proof of Concept with: npm install gatsby@latestThen you'll be able to write and use How can I help?
Detailed designYou can find the code of the current Pull Request here: #34776 It uses https://parceljs.org/ to compile Then the internal pieces of Gatsby don't look for Parcel is super fast and also supports using That's pretty much the technical details to make this work. Some of the current limitations that should be supported will warrant some more changes, but fundamentally the technical solution is: Use Parcel to compile the files to a directory and point Gatsby to that new directory. Drawbacks / AlternativesThe RFC gatsby-config/node typescript support already explained AOT and JIT in some detail (and favored JIT). Drawbacks we saw with JIT:
Drawbacks we saw with AOT:
Why we chose AOT:
Rollout
What's next?This is not the end of Gatsby's TypeScript support initiative! Ideas that people voiced:
These will follow as additional RFCs and/or features. I'm also aware that the feature in its current form is lacking any form of customization aside from |
Beta Was this translation helpful? Give feedback.
Replies: 29 comments 93 replies
-
|
Thanks for all the work on this @LekoArts. We'll definitely be trying out the PoC and giving feedback. |
Beta Was this translation helpful? Give feedback.
-
|
What about |
Beta Was this translation helpful? Give feedback.
-
|
Updated the version, everything still works! 🙂 Tested a full build on my machine and on Gatsby Cloud. I didn't need to make any adjustements. |
Beta Was this translation helpful? Give feedback.
-
|
Cool! Since we don't have any e2e-tests in place yet I just want to make sure we don't regress. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Hi - I'm using version ERROR #10124 CONFIG
It looks like you were trying to add the config file? Please rename "gatsby-config.ts" to "gatsby-config.js"However when I use ts-node wrapper that we use in our project - build and develop succeeds, but I cannot serve the project since we've always used Could that be something in our tsconfig? Here is also fragment of |
Beta Was this translation helpful? Give feedback.
-
|
@LekoArts |
Beta Was this translation helpful? Give feedback.
-
|
@WiktorG Please put this in a new reply to this discussion. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
local plugins would be great, is that coming soon ? |
Beta Was this translation helpful? Give feedback.
-
|
💯 ... I don't have a lot of time to dedicate to it, so it may be difficult to coordinate. It looks like I'll be clocking some overtime this weekend, so it may be a week or two before I can get into it. If it's not already done by then, I will reach out |
Beta Was this translation helpful? Give feedback.
-
|
@tyhopp will be working on this during this week 👍 |
Beta Was this translation helpful? Give feedback.
-
|
@LekoArts my local plugin works with version |
Beta Was this translation helpful? Give feedback.
-
|
I posted below here on how I use Gatsby with full TS support including custom plugin.. with a simple ts-node command see post (#34613 (comment)) |
Beta Was this translation helpful? Give feedback.
-
|
@cameronbraid Support for local plugins is in now. Can you try it with @krismorf I don't think we meant the same thing then :) Local plugins are e.g. |
Beta Was this translation helpful? Give feedback.
-
|
Saw this pop up a few days ago, and thought that it would be a boon in this project: https://kdy1.dev/posts/2022/1/tsc-go A bundler with Typescript support + Typechecking, so you’re not sacrificing type safety, running natively 🚀. Unfortunately, it will probably be in development for a while yet. Besides that, I did have a question: has built in Typechecking been considered? If parcel allows certain hooks, I imagine it would be possible to fork a type checker during transpilation/bundling. Obviously this would be a convenience feature, since we can always run our own type checker in parallel with Gatsby, but it would be nice if the work for it was already handled, and integrated directly into the build. Additionally, will it be possible to configure Parcel from userland? There’s various features supported in the existing Typescript universe that I would like to keep, but that may need to be reimplemented in a way that will integrate with Parcel… but to plug them in, we would need to be able to add our own configurations. |
Beta Was this translation helpful? Give feedback.
-
|
That's fantastic, thanks @LekoArts. I'll start exploring what can be done with Parcel here pretty soon. |
Beta Was this translation helpful? Give feedback.
-
|
We had some more talks internally and for transparency sake here's the current status on this: Since we want to move more/everything really to Parcel within Gatsby, exposing the ability to use As a follow-up I want to expose a public API surface to configure this, plus defining a |
Beta Was this translation helpful? Give feedback.
-
That would be great to have. Some of the functionality that I imagine could be added via plugins would be:
Do you envision an api surface that separates plugins/custom configurations based on build area, such as gatsby-config/node vs later bootstrap/build functionality?
Can you elaborate a bit more on this? |
Beta Was this translation helpful? Give feedback.
-
tbh, I didn't have any time yet to think about this API surface as getting this out of the door was more pressing :)
By that I meant that Parcel already looks for a |
Beta Was this translation helpful? Give feedback.
-
|
Hey folks, I'd like to be able to force the transpilation of node_modules that are part of the same monorepo, which I think would mean extending the Parcel config, with the includeNodeModules option.
Related issue, but only covering plugins: #35032 |
Beta Was this translation helpful? Give feedback.
-
Just for clarification: This link seems to point to a PR fixing a type hint for a CLI option. I'm assuming it was supposed to be parcel-bundler/parcel#6925? |
Beta Was this translation helpful? Give feedback.
-
|
Yep, thanks, that was a typo — fixed it :) |
Beta Was this translation helpful? Give feedback.
-
|
I am getting an issue with gatsby@npm:4.7.0-alpha-ts-aot.6 when running gatsby build |
Beta Was this translation helpful? Give feedback.
-
|
Can you share your project or a minimal reproduction (https://gatsby.dev/reproduction) where you see this? Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Please see #34613 (reply in thread) |
Beta Was this translation helpful? Give feedback.
-
|
And gatsby@4.8.0-next.1 doesn't detect ts :( |
Beta Was this translation helpful? Give feedback.
-
|
Only |
Beta Was this translation helpful? Give feedback.
-
|
does |
Beta Was this translation helpful? Give feedback.
-
|
Please see #34613 (reply in thread) |
Beta Was this translation helpful? Give feedback.
-
|
Hey, Firstly, this version seems to have problems writing fileson the first run of gatsby build, and on consecutive ones if i don't manually run gatsby clean before that. This happens for a barebones test project, only consisting of and with just gatsby installed. This is testing under windows, but i can assure you that i own the directory and all files within it and have full access. Additionally,if i first run clean the project builds, but not before spamming the console with error messages formed like this starts to happen after Thirdly, placingexport default {
plugins: [
{
resolve: "gatsby-plugin-page-creator",
options: {
path: "./src/pages",
ignore: [
"**/__generated__/*",
],
}
},
]
}in the gatsby-config to override the default behavior of the page creator results in which does not happen in dockerI'm sure at this stage you have better things to do than tracking down some non descript error, but building a more complicated project inside of docker |
Beta Was this translation helpful? Give feedback.
-
|
Hi, thanks for giving this a try! To be honest, we never tested this so far on Windows so happy to get your input here. The first two issues shouldn't be related to this version as those things weren't changed. I'd expect them to also happen on current The third one seems a bit weird, as the published package references the version you mentioned: https://unpkg.com/browse/gatsby@4.7.0-alpha-ts-aot.11/package.json About the segfaults: I've added a note to the current doc (limitations), it's unfortunately a bug in Parcel/LMDB which is looked at. There are workarounds we can use if necessary. |
Beta Was this translation helpful? Give feedback.
-
|
To fix the issue with |
Beta Was this translation helpful? Give feedback.
-
|
Ok, the weird dependency resolution is a problem currently with the canary versions, this shouldn't happen once we have it in I'll research what's up with the webpack logs, thanks! |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
I haven't seen this package mentioned before so I just thought you might like to discover it: https://github.com/egoist/bundle-require |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the link! I don't think I'll switch to this but good to know there's at least some prior art to this I can use :) |
Beta Was this translation helpful? Give feedback.
-
|
Hi @LekoArts , I'm reading through the proposal, but I'm unsure of the complexity that comes with AOT... and the introduction of another build tool (parcel, along with its dependencies), but I'll keep reading to get a better grasp. BUT, I have been using Gatsby with full typescript support with this wrapper CLI I wrote around https://github.com/newrade/newrade-core/blob/master/packages/core-cli/src/commands/gatsby.ts I can use all config files in
Local plugins work 100% in TS:
The TL;DR is
It works like a charm... any thoughts on this approach? Thanks, Phil |
Beta Was this translation helpful? Give feedback.
-
|
Regarding:
I have not seen any memory problem in my builds so far, I'm yet to have experienced any errors with |
Beta Was this translation helpful? Give feedback.
-
|
Hi, thanks for your comment! I know that JIT approach ála But the AOT (with Parcel) approach fits better with what we have in mind with Gatsby in the future and will already be helpful with the current architecture. Multiple parts of gatsby require/bundle the configuration and parts of Again, thanks for the comment but so far the AOT approach is working great and as an end-user you shouldn't see a difference at the end. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you @LekoArts for the work on this, and I'm 100% for the AOT approach. One frustration I've had with trying to use TypeScript with Gatsby and a JIT approach with Stack TracesI notice that using ERROR Failed to compile gatsby files. Missing initializer in const declaration
Error: Missing initializer in const declaration
not finished compile gatsby files - 1.122sIntellisense for Gatsby Config OptionsI wanted Intellisense for the Gatsby plugins I'm providing options for in One possibility would be extracting types from plugins that provide Joi validation schemas using Gatsby's built-in API. That would have the advantage of avoiding duplication and keeping types and validation in sync. Many plugins don't provide validation, so a manual approach to providing typings would still be required. There are also some Joi methods that plugins are using and that don't translate (either automatically or well) to TS types like One issue I haven't resolved with providing manual typings is that Gatsby configuration allows either a plugin name if the plugin option defaults are used (or there are no options), and an object with const gatsby_plugin_theme_ui: ThemeUIConfig = {
resolve: `gatsby-plugin-theme-ui`,
options: {
preset: `./src/styles/theme`,
prismPreset: `vs-dark`,
},
}
const GatsbyConfig = {
siteMetadata,
plugins: [
...
gatsby_plugin_theme_ui,
...
],
}
export default GatsbyConfigOne really nice aside is that I get type checking and Intellisense for my Theme UI theme CSS:sparkles:. I'm building an MDX site, and it uses all of the plugins set up in @LekoArts Minimal Blog theme. Here's a gist of the typings I worked out. There's no tests, and I'm sure there's a mistake or three. It seems to me the best approach for typing Gatsby plugin configuration options is to have a standard named export for their types, and encourage plugin authors to include them in the plugin documentation. It could then be imported into import { GatsbyConfig as ThemeUiConfig } from `gatsby_plugin_theme_ui`And then mention a best practice to use DefinitelyTyped for community-provided config option typings with a standard naming convention, like Gatsby Node API TypingsThe Gatsby project manual typings for Local PluginsI'm looking forward to local plugin support! My particular use case atm involves Docs on Parcel Compiler LimitationsI saw the note with a link to Parcel about "Parcel Limitations" in the OP. The eventual documentation should note that Parcel requires and is implicitly setting the TSConfig LintingCode editors (at least VS Code and Jetbrains) will automatically pull in JSON schema for
Gratitude:sparkling_heart: :fire: :zap: :fireworks:Thank you @LekoArts for the work on this. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, thanks for the comment!
Thanks for the note, we'll see to get the usual more verbose logging working, too.
Also thanks for that, I'll also note this here & internally as a possible follow-up improvement for this overall initiative. I think I've looked into the Joi => TS in the past already and yeah, as you already mentioned it has some limitation. So about this + the API how people would use the types in gatsby-config will be a bit of a longer endeavour I think.
Can you give specific examples? :)
As the typings are manually written at the moment, not auto-generated they are certainly not perfect. PRs to improve it are always welcome!
What you're describing is not local plugin by our definition: https://www.gatsbyjs.com/docs/creating-a-local-plugin/
Yep, will do. Thanks!
So far the tsconfig we scaffold here: https://github.com/gatsbyjs/gatsby-starter-minimal-ts/blob/master/tsconfig.json is not really specific to Gatsby. I'd say it has things defined that every normal React project would have. What benefit would our own JSON schema give us? Happy to see you're excited about this and keep the comments coming, appreciate it! |
Beta Was this translation helpful? Give feedback.
-
|
@webstackdev Can you try the latest version of the canary? ( |
Beta Was this translation helpful? Give feedback.
-
|
Awesome on 4.9 w/TS built-in! On typings, I need to dig through the project I'm working on and document the issues I've encountered. I'll open an Issue on Gatsby with them and maybe try to push a PR (though I don't feel really confident I have the right answers on the problems that have come up for me). Some of the issues are:
A more general issue is that since the types are manual and there's no tests for them, they're out of sync with the code in places. I'm hoping it's on the roadmap to type the core Gatsby code - a lot of the hard work's been done with creating the manual typings I think. |
Beta Was this translation helpful? Give feedback.
-
|
Edit: As requested - I'm putting this in a new comment 😁 Hi - I'm using version ERROR #10124 CONFIG
It looks like you were trying to add the config file? Please rename "gatsby-config.ts" to "gatsby-config.js"However when I use ts-node wrapper that we use in our project - build and develop succeeds, but I cannot serve the project since we've always used Could that be something in our tsconfig? Here is also fragment of |
Beta Was this translation helpful? Give feedback.
-
|
As described in https://parceljs.org/languages/typescript/ Parcel currently doesn't support |
Beta Was this translation helpful? Give feedback.
-
|
@LekoArts - yes, this was the case! 😁 Beside what I wrote about. We were using ts-node to run our project and during So we managed to migrate very larg project! |
Beta Was this translation helpful? Give feedback.
-
|
It's something that needs to be added to Parcel so I can't give any ETA on this sadly. |
Beta Was this translation helpful? Give feedback.
-
|
I thought this was added :D Or is the thing a little bit more complicated? |
Beta Was this translation helpful? Give feedback.
-
|
That's the old documentation, this here is up-to-date: https://parceljs.org/languages/typescript/#resolution |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for linking to this RFC in the 4.8 release notes; hopefully future release notes can mention when workspace support is ready so we can give it a try. Being able to write these files in TypeScript is exciting; being able to import TypeScript and ESM into them is even more exciting. This will de-cruft a lot of our codebase. I saw an earlier request for built-in type checking. I think that makes sense for smaller projects, but larger projects probably already disable Gatsby's built-in linting in favour of standalone ESLint and TSC configs, as we do. I would suggest that linting could be moved to a removable plugin, and type checking could be implemented there as well. This would go a long way toward cutting down the number of dependencies in today's monolithic The same thing goes for GraphQL type generation. I would suggest you first expose your schema builder for integration with GraphQL Codegen, then implement it in a removable plugin. This would still improve the out-of-the-box experience, but it would help larger-scale projects, too (we currently need to run Other projects have moved in this direction, and our dev experience has benefited greatly from enhanced control of transient dependencies. Please don't grow the monolith! 🪨 |
Beta Was this translation helpful? Give feedback.
-
|
This feature is now available in |
Beta Was this translation helpful? Give feedback.
-
|
Some comments after GA release 👇 (Edited message) |
Beta Was this translation helpful? Give feedback.
-
|
Ah yeah, switching Sorry for the noise / incomplete report! |
Beta Was this translation helpful? Give feedback.
-
|
Trying 4.9.0 and getting This is a local plugin in a monorepo, is there any way to tell Gatsby to include those modules as well? |
Beta Was this translation helpful? Give feedback.
-
|
@LekoArts I read the note about But what about the resolve for local plugin: Can we use Edit: correct path was Now I'm getting the first error: |
Beta Was this translation helpful? Give feedback.
-
|
Update created an issue with repro: #35032 |
Beta Was this translation helpful? Give feedback.
-
|
For the error with More details about the ESM incompatibility with Gatsby here: |
Beta Was this translation helpful? Give feedback.
-
|
This is now generally available in 4.9: https://www.gatsbyjs.com/docs/reference/release-notes/v4.9/ |
Beta Was this translation helpful? Give feedback.
-
|
Good work, I managed to successfully upgrade from v3 However I do get regular segfaults :(. I'm working on a standalone repro for that, though its been hard to isolate |
Beta Was this translation helpful? Give feedback.
-
|
@LekoArts I see the note
Is there an issue or discussion I can track for this, maybe upstream with Parcel? |
Beta Was this translation helpful? Give feedback.
-
|
For now I want to track any incoming bug reports/regressions for the first week or so, and then (here on GitHub) convert the "What's next" list in the start post into issues/discussions to individually track and evaluate them |
Beta Was this translation helpful? Give feedback.
-
|
@LekoArts strong work! I converted a 1k+ I suspect there's a better way to do what I'm doing, and hopefully--as the types are surfaced--said way will become clear. Again, strong work! |
Beta Was this translation helpful? Give feedback.
-
|
@cameronbraid If you're hitting segfaults, can you check which versions of npm list lmdbOr yarn why lmdb |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
You'll want to update/delete your lock file then and make sure that you have at least 2.2.2 of lmdb installed: parcel-bundler/parcel#7702 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
it was cashing more than it wasn't, I tried adding |
Beta Was this translation helpful? Give feedback.
-
|
Resolutions shouldn't be necessary as |
Beta Was this translation helpful? Give feedback.
-
|
that seems to have done the trick. thanks heaps.. that was very frustrating |
Beta Was this translation helpful? Give feedback.
-
|
So how does GraphQL codegen support going? I'm ready to release gatsby-plugin-typegen v3 and it works quite smoothly. I decided to continue the project again because I have some ideas to improve the core DX of my Gatsby projects, I'm curious what the Gatsby core team's plan is. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @cometkim 👋 Thanks for asking! We're fleshing out the details of user stories but it's safe to say that we want to focus on built-in GraphQL codegen support as a next feature :) So we want to bring this into gatsby as a plugin, activate it by default (but people can disable it). You and @d4rekanguok did awesome work on this and I'd like to collaborate with you on this. |
Beta Was this translation helpful? Give feedback.
-
|
Really glad that TypeScript is coming to I'm trying to convert this entry, and it doesn't appear to be working. The original: Into this: There are no complaints, but when I run Have I converted it wrong? Thanks |
Beta Was this translation helpful? Give feedback.
-
|
Are you sure the file is being found / is running? Eg. try adding a |
Beta Was this translation helpful? Give feedback.
-
|
You are correct. It (onCreateNode) wasn't even being called. I had to do a I have another issue with |
Beta Was this translation helpful? Give feedback.
-
|
Ran into another cryptic build issue (works fine in develop) with gatsby@next and ts : (looks related to the one above, but not missing module but this time a accessing of .path on undefined |
Beta Was this translation helpful? Give feedback.
-
|
I think this comes after my changing my gatsby-node to look like this before I structured like this for example |
Beta Was this translation helpful? Give feedback.
-
|
As a workaround, the following works fine : |
Beta Was this translation helpful? Give feedback.
-
|
This seems unrelated to this TS work as it fails later in the overall process. It fails at https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/utils/babel/babel-plugin-remove-api.ts#L119 - if you can create a minimal reproduction, please open a new bug issue :) |
Beta Was this translation helpful? Give feedback.
-
|
@LekoArts Just a quick to say I hit this issue and it does seem to be when you export straight from another file, it hits this function: Im happy to put a minimal reproduction if one has not been created yet |
Beta Was this translation helpful? Give feedback.
-
|
@ashhitch yeah, please do. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for helping me solve the The error is Any idea how I should replace Thanks in advance |
Beta Was this translation helpful? Give feedback.
-
|
Ok, if it helps anyone, the solution appears to be to simply remove |
Beta Was this translation helpful? Give feedback.
-
|
This is one of the known limitations at the moment: https://www.gatsbyjs.com/docs/how-to/custom-configuration/typescript/#requireresolve |
Beta Was this translation helpful? Give feedback.
-
|
Probably not the right place for this, but Gatsby Cloud CMS Preview fails on TypeScript validation errors, whereas it succeeds in production.
|
Beta Was this translation helpful? Give feedback.
-
|
hi @njbmartin could you create an issue with a reproduction? so we can help you with this issue |
Beta Was this translation helpful? Give feedback.
-
|
One issue I ran into which was causing hours of pain, was parcel would compile successfully but none of the gatsby-node apis would run. The underlying cause was because I use a yarn link: dependency as follows : Then in code I would What this did was make parcel think that "src" was a real module leaving it untranspiled. However when gatsby ran no error was thrown or logged and all my gatsby-node apis would be ignored completely. The solution was to configure parcel via It might be worth adding this to the docs somewhere. |
Beta Was this translation helpful? Give feedback.
-
|
I created a standalone repro at https://github.com/cameronbraid/gatsby-typescript-yarn-link-breaks-parcel |
Beta Was this translation helpful? Give feedback.
-
|
My Parcel PR to mitigate the Also, please read the new RFC: #35420
|
Beta Was this translation helpful? Give feedback.
-
|
Once #35446 is merged and released the |
Beta Was this translation helpful? Give feedback.
-
|
@LekoArts You mention “Future ESM support possible” – I also see questions around the current state in this issue (maybe you could share some more info?): #31599 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
Hi! A couple questions for everyone we'd like your feedback on:
|
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Hey folks, This specifically happens when I import a TypeScript package from within a PNPM monorepo. These packages works well out of the box when imported within pages (it seems that as a default, external packages are transpiled, contrary to Next.js that will use a "transpilePackages" option?), but not in the config. The code is open source though not minimal, this is the codebase that powers the State of JS survey, it's visible in this pull request, in the "results" app. I'll try to craft a more minimal repro and open a bug ticket but I am not sure this is a bug in the first place or I just missed some configuration? The architecture of the monorepo is like this:
|
Beta Was this translation helpful? Give feedback.
-
|
Exact same problem here with |
Beta Was this translation helpful? Give feedback.
-
|
Only solution is to setup a build system for shared external packages as far as I can tell |
Beta Was this translation helpful? Give feedback.
-
So I assume that using |
Beta Was this translation helpful? Give feedback.
and others
This is now generally available in 4.9: https://www.gatsbyjs.com/docs/reference/release-notes/v4.9/