Conditional Page Builds do not work

[styfle]

Description

Following the steps from Conditional Page Builds, all pages are still built.

Steps to reproduce

npx gatsby-cli new gatsby-example
cd gatsby-example
yarn install
yarn build
# change "build" script to "GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true gatsby build --log-pages"
yarn build
# modify the `<h1>` contents in `src/pages/page-2.js`
yarn build

Expected result

Expected only src/pages/page-2.js to be rebuilt.

Actual result

All pages were rebuilt.

info Done building in 10.813733398 sec
info Built pages:
Updated page: /404/
Updated page: /
Updated page: /page-2/
Updated page: /using-typescript/
Updated page: /404.html
✨  Done in 11.46s.

Environment

  System:
    OS: macOS 10.15.5
    CPU: (4) x64 Intel(R) Core(TM) i5-8210Y CPU @ 1.60GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 12.18.3 - /var/folders/v7/6kzxdgb93x328tk9pg633s200000gn/T/yarn--1598975468259-0.611812660811949/node
    Yarn: 1.22.4 - /var/folders/v7/6kzxdgb93x328tk9pg633s200000gn/T/yarn--1598975468259-0.611812660811949/yarn
    npm: 6.14.6 - /usr/local/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 85.0.4183.83
    Firefox: 77.0.1
    Safari: 13.1.1
  npmPackages:
    gatsby: ^2.24.50 => 2.24.50 
    gatsby-image: ^2.4.16 => 2.4.16 
    gatsby-plugin-manifest: ^2.4.24 => 2.4.24 
    gatsby-plugin-offline: ^3.2.24 => 3.2.24 
    gatsby-plugin-react-helmet: ^3.3.10 => 3.3.10 
    gatsby-plugin-sharp: ^2.6.28 => 2.6.28 
    gatsby-source-filesystem: ^2.3.25 => 2.3.25 
    gatsby-transformer-sharp: ^2.5.13 => 2.5.13 

[herecydev]

If you try further builds is it consistently not working?

I couldn't recreate unfortunately;

The only page that updates (every minute or so) is the /using-typescript because of the timestamp. Other pages never rebuild

[styfle]

If I run yarn build without modifying any files, then I see what you see: only /using-typescript/ is built.

However, when I modify only static content (for example <h1>) in src/pages/page-2.js, then build again, all pages are built. This is unexpected because only one page was modified so I would expect only one page to be built.

Here's the complete log:

$ vim src/pages/page-2.js
$ yarn build
yarn run v1.22.4
$ GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true gatsby build --log-pages
success open and validate gatsby-configs - 0.077s
success load plugins - 1.553s
success onPreInit - 0.073s
info One or more of your plugins have changed since the last time you ran Gatsby. As
a precaution, we're deleting your site's cache to ensure there's no stale data.
success initialize cache - 0.091s
success copy gatsby files - 0.147s
success onPreBootstrap - 0.017s
success createSchemaCustomization - 0.029s
success Checking for changed pages - 0.004s
success source and transform nodes - 0.114s
success building schema - 0.474s
success createPages - 0.003s
success Checking for changed pages - 0.007s
success createPagesStatefully - 0.144s
success update schema - 0.060s
success onPreExtractQueries - 0.009s
success extract queries from components - 0.588s
success write out redirect data - 0.006s
success Build manifest and related icons - 0.238s
success onPostBootstrap - 0.249s
info bootstrap finished - 13.040s
success run static queries - 0.118s - 3/3 25.40/s
success run page queries - 0.037s - 5/5 136.91/s
success write out requires - 0.007s
success Building production JavaScript and CSS bundles - 17.778s
success Rewriting compilation hashes - 0.022s
success Building static HTML for pages - 3.470s - 5/5 1.44/s
success Delete previous page data - 0.007s
success onPostBuild - 0.004s
info Done building in 34.734892582 sec
info Built pages:
Updated page: /404/
Updated page: /
Updated page: /page-2/
Updated page: /using-typescript/
Updated page: /404.html
✨  Done in 35.63s.

[herecydev]

Right, sorry I understand. Basically any changes to js/css files will cause a full rebuild. "Conditional page builds" only work for data changes

[styfle]

I see now. The documentation is confusing because its called Conditional Page Builds but changing one page builds all pages.

This sounds like it would work:

it can save time on the HTML-generation step by not re-rendering HTML for pages with unchanged data

The part thats not well defined is "data". It wasn't until the second paragraph that I found an answer:

If there are any changes to code (JS, CSS) the bundling process returns a new webpack compilation hash which causes all pages to be rebuilt.

But this still doesn't define "data" or how to modify the data to make a "Conditional page builds" work properly.

Can "data" be a json file? How can I try this out with the gatsby example?

[lucis]

@styfle I think you could say that "data" is anything that comes from the GraphQL data layer. You can read more about it here. In some cases it's markdown content, in others is API-fetched data. It's actually quite common on Gatsby projects to have a "data" part.

You can try set it up with a simple mdx plugin to see it working.

Do you think the issue is solved?

[herecydev]

The "Data" in question is anything from page queries. Ideally it would work with static queries as well, but there's a bug with it that needs fixing.

I've put together a pr that adds some additional logging so hopefully that will at least help show when Gatsby is having to rebuild all the pages.

[styfle]

I couldn't get the gatsby-plugin-mdx to work using the linked docs so I tried a different data source using an existing example:

https://github.com/gatsbyjs/gatsby/tree/master/examples/using-gatsby-with-json-yaml

git clone https://github.com/gatsbyjs/gatsby
cd ./examples/using-gatsby-with-json-yaml
yarn install
vim package.json    # update "build" script
yarn build
vim content/My-JSON-Content.json # update title
yarn build

See this git diff screenshot to see what I changed.

This still built all the pages. I would expect only /json-at-buildtime/ to be rebuilt.

success onPostBuild - 0.003s
info Done building in 12.978111875 sec
info Built pages:
Updated page: /page1
Updated page: /page2
Updated page: /page3
Updated page: /page4
Updated page: /page5
Updated page: /
Updated page: /json-at-buildtime/
Updated page: /yml-at-buildtime/
✨  Done in 13.54s.

Seems like a bug to me.

[wardpeet]

Hey,

Everything already got mentioned but I'll give a tl;dr and close it afterward.

Conditional Page builds is an experimental feature that only works with data updates. Data updates in Gatsby means, data that comes from the GraphQL layer, in this case only page queries are supported. It can work with JSON updates when using gatsby-transformer-json and gatsby-source-filesystem.

Gatsby at this point in time is not going to invest in this feature and it will probably stay experimental without any development.

For more info about our roadmap see https://www.gatsbyjs.com/blog/announcing-unified-gatsby