Conversation
| if (fs.existsSync(fullPath)) { | ||
| if (['yml', 'yaml', 'toml', 'json'].includes(path.extname(fullPath).substring(1))) { | ||
| encodedFiles[i].data = mergeFile(fullPath, encodedFiles[i].data); | ||
| } | ||
| fileContentChanged = !_.isEqual(encodedFiles[i].data, fs.readFileSync(fullPath, 'utf8')); | ||
| } |
There was a problem hiding this comment.
🚨 @rodikh @davbree an important notes about merging data here...
Originally the reason for this kind of merging in stackbit-pull, and also in container here, was done because we didn't model the full structure of Jekyll/Hugo config files.
For example, while Jekyll config could be something like this:
baseurl: "/"
title: "Site Title"
include: [".htaccess"]
ollections:
posts:
output: true
kramdown:
auto_ids: trueWe only modeled the baseurl and title. Meaning that data stored in Contentful had only the baseurl and title fields. The rest of the fields were left in the config.yaml.
That means that when stackbit-pull fetches the content, it should not override everything but assign the additional values stored in CMS.
I admit, this was a huge mistake :)
- First of all, this should only happen to config models because only they are modeled partially, other files should be always modeled fully
- Second, this presents a huge bug - if user deletes a field in Contentful, and we fetch the data and merge it with older file that has the field, then we didn't actually delete the field.
One of the possible solution was this task:
https://www.notion.so/stackbit/Themes-Move-all-config-yaml-params-to-a-separate-file-91517dbceafb40a4a21c2fc01de4319d#764577d127764bfd925d9d157449ad98
There was a problem hiding this comment.
Sounds like we should fix it! :)
Created this task https://www.notion.so/stackbit/Fix-stackbit-pull-merging-behavior-4c99c343ed704a3097ca4784bdedf9f5
There was a problem hiding this comment.
Yes, this should be fixed, but only after we update our themes according to the previous task, and also we need to make sure it is backward compatible with existing projects.
Basically, the merge should be made only if the model is of type config - meaning the object in CMS has stackbit_model_type=config.
There was a problem hiding this comment.
@smnh I actually mentioned this exact issue to David on slack just before you commented here.
In any case, let's release this totally unrelated PR and deal with this new issue separately.
3 participants
Improve underlying SSG performance by only writing files that actually changed.