Skip to content

Commit

Permalink
[json-loader] Only log file events if we're past bootstrap (#4826)
Browse files Browse the repository at this point in the history
* Don't emit new file node until previous is finished processing

This is an experiment to use
[xstate](http://davidkpiano.github.io/xstate/docs/#/) to setup state
machines to better handle complex state changes as we sometimes have.

Ideally this happens in core and then gatsby-source-filesystem
just has a simple queue and emits a new file node every time
the system returns to idle.

In a future refactor we'll do that plus refactor other parts of core
that should be handled in a state machine e.g. pages-query-runner.js

This PR also reinforced the need for us to implement
[tracing](https://github.com/jaegertracing/jaeger) in core / plugins
as that'd make it far far easier to understand what's happening and
when.

* Document state machine and remove extraneous Chokidar states

* Remove console.log

* Only log file events if we're past bootstrap
  • Loading branch information
KyleAMathews authored Apr 3, 2018
1 parent abcb58c commit e374818
Showing 1 changed file with 32 additions and 5 deletions.
37 changes: 32 additions & 5 deletions packages/gatsby-source-filesystem/src/gatsby-node.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,20 @@ const fsMachine = Machine({
CHOKIDAR_NOT_READY: {
on: {
CHOKIDAR_READY: "CHOKIDAR_WATCHING",
BOOTSTRAP_FINISHED: "CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED",
},
},
CHOKIDAR_WATCHING: {
on: {
BOOTSTRAP_FINISHED: "CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED",
CHOKIDAR_READY: "CHOKIDAR_WATCHING",
},
},
CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED: {
on: {
CHOKIDAR_READY: "CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED",
},
},
},
},
PROCESSING: {
Expand Down Expand Up @@ -152,20 +159,32 @@ See docs here - https://www.gatsbyjs.org/packages/gatsby-source-filesystem/

watcher.on(`add`, path => {
if (currentState.value.CHOKIDAR !== `CHOKIDAR_NOT_READY`) {
reporter.info(`added file at ${path}`)
if (
currentState.value.CHOKIDAR === `CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED`
) {
reporter.info(`added file at ${path}`)
}
createAndProcessNode(path).catch(err => reporter.error(err))
} else {
pathQueue.push(path)
}
})

watcher.on(`change`, path => {
reporter.info(`changed file at ${path}`)
if (
currentState.value.CHOKIDAR === `CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED`
) {
reporter.info(`changed file at ${path}`)
}
createAndProcessNode(path).catch(err => reporter.error(err))
})

watcher.on(`unlink`, path => {
reporter.info(`file deleted at ${path}`)
if (
currentState.value.CHOKIDAR === `CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED`
) {
reporter.info(`file deleted at ${path}`)
}
const node = getNode(createNodeId(path))
// It's possible the file node was never created as sometimes tools will
// write and then immediately delete temporary files to the file system.
Expand All @@ -177,15 +196,23 @@ See docs here - https://www.gatsbyjs.org/packages/gatsby-source-filesystem/

watcher.on(`addDir`, path => {
if (currentState.value.CHOKIDAR !== `CHOKIDAR_NOT_READY`) {
reporter.info(`added directory at ${path}`)
if (
currentState.value.CHOKIDAR === `CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED`
) {
reporter.info(`added directory at ${path}`)
}
createAndProcessNode(path).catch(err => reporter.error(err))
} else {
pathQueue.push(path)
}
})

watcher.on(`unlinkDir`, path => {
reporter.info(`directory deleted at ${path}`)
if (
currentState.value.CHOKIDAR === `CHOKIDAR_WATCHING_BOOTSTRAP_FINISHED`
) {
reporter.info(`directory deleted at ${path}`)
}
const node = getNode(createNodeId(path))
deleteNode(node.id, node)
})
Expand Down

0 comments on commit e374818

Please sign in to comment.