gatsby-source-drupal: Incremental build breaks backlinks

[kszot-ref]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

Example scenario:

• You source your data from drupal JSON:API.
• You get two nodes: "node--vacancy" and "node--office".
• "node--vacancy" has a relationship to a "node--office", but an office doesn't have one to vacancies (on drupal side).
• You run a full build ("npm run build").
• During the build, a backlink is created (a new relationship is created - "node--office" will contain an array of related "node--vacancy" nodes).
• Now you update the "node--office" in drupal, doesn't matter which field, the change needs to appear in Gatsby Fastbuild Log.
• You run a new production build ("npm run build") with fastbuilds enabled.
• You get the new office from the fastbuild endpoint, and it doesn't have a relationship to any "node--vacancy".
• The backlinks are lost - your "node--vacancy" still has a connection to the "node--office", but the "node--office" connection to that "node--vacancy" is deleted during an update.

The issue at hand seems to be related to the way references are handled during an update.

There are two lookup tables related to this issue:

gatsby/packages/gatsby-source-drupal/src/utils.js

Lines 12 to 13 in d03fe39

	const backRefsNamesLookup = new WeakMap()
	const referencedNodesLookup = new WeakMap()

While updating a node, the code tries to get data from one of them:

gatsby/packages/gatsby-source-drupal/src/utils.js

Lines 242 to 249 in d03fe39

	const backRefsNames = backRefsNamesLookup.get(oldNode)
	if (backRefsNames) {
	backRefsNamesLookup.set(newNode, backRefsNames)
	backRefsNames.forEach(backRefFieldName => {
	newNode.relationships[backRefFieldName] =
	oldNode.relationships[backRefFieldName]
	})
	}

The problem is, there's no possible way in this scenario for "backRefsNamesLookup" to actually have information about the backlinks, so the code inside that if block is never called and the backlink is removed.

The "backRefsNamesLookup" data is sourced in the "handleReferences" function, which is called for every node during a full build, but for an incremental one it's only called on the updated NEW NODE (so in our example that would be the "node--office" node that already doesn't have a backlink):

gatsby/packages/gatsby-source-drupal/src/utils.js

Lines 82 to 104 in d03fe39

	if (referencedNodes.length) {
	const nodeFieldName = `${node.internal.type}___NODE`
	referencedNodes.forEach(nodeID => {
	const referencedNode = getNode(nodeID)
	if (!referencedNode.relationships[nodeFieldName]) {
	referencedNode.relationships[nodeFieldName] = []
	}
	if (!referencedNode.relationships[nodeFieldName].includes(node.id)) {
	referencedNode.relationships[nodeFieldName].push(node.id)
	}
	let backRefsNames = backRefsNamesLookup.get(referencedNode)
	if (!backRefsNames) {
	backRefsNames = []
	backRefsNamesLookup.set(referencedNode, backRefsNames)
	}
	if (!backRefsNames.includes(nodeFieldName)) {
	backRefsNames.push(nodeFieldName)
	}
	})
	}

The "referencedNodes" array won't contain the backlink as it only contains the data from a new node, so line 97 for this connection is never called.

The issue doesn't seem to happen in dev mode, as it's a continous process so the lookup tables still have data in it from the full build.

Reproduction Link

Steps to Reproduce

Expected Result

The backlink would be preserved after an incremental build.

Actual Result

The backlink is removed.

Environment

System:
    OS: Linux 5.4 Ubuntu 20.04.2 LTS (Focal Fossa)
    CPU: (8) x64 Intel(R) Core(TM) i7-10610U CPU @ 1.80GHz
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 14.17.3 - ~/.nvm/versions/node/v14.17.3/bin/node
    Yarn: 1.22.10 - /mnt/c/Program Files/nodejs/yarn
    npm: 6.14.13 - ~/.nvm/versions/node/v14.17.3/bin/npm
  npmPackages:
    gatsby: ^3.10.2 => 3.10.2
    gatsby-background-image: ^1.5.3 => 1.5.3
    gatsby-plugin-create-client-paths: ^3.10.0 => 3.10.0
    gatsby-plugin-emotion: ^6.10.0 => 6.10.0
    gatsby-plugin-google-tagmanager: ^3.10.0 => 3.10.0
    gatsby-plugin-image: ^1.10.1 => 1.10.1
    gatsby-plugin-loadable-components-ssr: ^3.2.0 => 3.2.0
    gatsby-plugin-manifest: ^3.10.0 => 3.10.0
    gatsby-plugin-meta-redirect: ^1.1.1 => 1.1.1
    gatsby-plugin-react-helmet: ^4.10.0 => 4.10.0
    gatsby-plugin-react-helmet-canonical-urls: ^1.4.0 => 1.4.0
    gatsby-plugin-robots-txt: ^1.6.8 => 1.6.8
    gatsby-plugin-sass: ^4.10.0 => 4.10.0
    gatsby-plugin-sharp: ^3.10.2 => 3.10.2
    gatsby-plugin-sitemap: ^4.6.0 => 4.6.0
    gatsby-plugin-typegen: ^2.2.4 => 2.2.4
    gatsby-plugin-use-query-params: ^1.0.1 => 1.0.1
    gatsby-plugin-webpack-bundle-analyser-v2: ^1.1.24 => 1.1.24
    gatsby-query-params: ^1.2.1 => 1.2.1
    gatsby-source-drupal: ^4.14.0 => 4.14.0
    gatsby-source-filesystem: ^3.10.0 => 3.10.0
    gatsby-transformer-sharp: ^3.10.0 => 3.10.0

Config Flags

No response