Deploy failed in build step

[jamestagal]

Hi @jimafisk

It's been awhile since i make any changes to my site and today after making some changes and tried git push I first got an error about not having the permission and a reference to a (publickey) and that might have been because I did change the .ssh/config file when you helped me set up and test that GitLab Plenti site. Anyway I think i fixed the .ssh issue by just authenticating with a GitHub host for the repo with HTTS instead. But after it pushed the repo from my local folder, I got a failed deploy instead. see link below.
Would you mind having a look and see what the issue might be when you can pls.

https://github.com/jamestagal/edtechdesigner/runs/7052483872?check_suite_focus=true

Does the reference in the error to the _blueprint.json file mean that I just need to install your latest binary? (v0.5.8)
Regards,
Ben

[jimafisk]

I see what's going on here, this error is getting thrown in your plenti build:

Error in DataSource build step
Could not get all content from virtual theme 
Error getting content from content/pages/_blueprint.json 

That's because we're finally starting to use blueprints in our builds for the CMS functionality. Basically these files can't be blank because that's not valid json, so even if you're not actively using them, you want to make sure they have empty curly brackets {}. This seems to be coming from the theme you're using, so it might make the most sense to make the change there. I just update the compendium theme to fix the empty pages blueprint: plenti-themes/compendium@0d8efd1

You should be able to just rerun your broken build to pull the change in. Can you give that a shot when you get a chance and see if it's cleared up for you?

[jamestagal]

Hi @jimafisk
Thanks. So does that mean the theme gets overridden in the build step? Anyway I tried re-running that action in Github and it didn't work. I thought I would have to git pull in the change from compendium theme to get this change locally then git push for any change to take affect?

[jimafisk]

You're 100% correct @jamestagal, my mistake. You'll have to pull the theme updates into your project first: plenti theme update compendium. Then you should be able to kick off a new build by pushing back to your project: git push

I was going to make a quick PR for you, but a couple of things are happening. First, plenti theme update compendium was throwing an error so I had to run plenti theme add https://github.com/plenti-themes/compendium instead, which effectively do the same thing (get you the newest version of the theme).

Then the build throws these errors:

Error in DataSource build step
Can't create props for content/404.json 
Can't render htmlComponent for layouts_content_404_svelte: TypeError: Cannot convert undefined or null to object

I'm going to need to do some debugging to figure out exactly what's going on here.

[jimafisk]

A quick fix you in the meantime would be locking your CI build to an older version of Plenti that still works for your project. So in https://github.com/jamestagal/edtechdesigner/blob/main/.github/workflows/netlify-deploy.yml#L15 change docker://plentico/plenti:latest to docker://plentico/plenti:v0.5.5

[jimafisk]

@jamestagal I tried reverting locally to Plenti v0.5.5, v0.5.4, and then v0.5.3 but all of those builds on this repo are failing with similar errors. Looks like your last passing build was on 6/5 so that should have used v0.5.5. Not sure what's going on here yet...

[jimafisk]

It's very strange, when I delete all the subfolders in your project's content folder, the build passes. I can also recopy the folders from the theme's content folder and the build works just fine that way too. There must be an invisible character or something being inserted into your content source or something. Which text editor do you use?

[jamestagal]

Hi @jimafisk
Sorry I haven't had a chance to update the theme etc etc. But I use VS Code as my editor of choice.

Ben

[jimafisk]

I've been trying to debug this repo locally, the build errors appear to start on 3/6/22 at Commits from Roberto 20Feb 7da5fa3

I'm not sure exactly what was introduced at that commit, but something in there is breaking all the builds from that point forward on my machine. I'm not exactly sure how you were getting passing builds after that commit in CI: https://github.com/jamestagal/edtechdesigner/actions.

[jamestagal]

Hi @jimafisk
Yes it is a bit strange because I have been able to run plenti serve locally without an issue recently so I mean even before I pushed that most recent update to Github so I am thinking it has to do with the binary because I only updated that yesterday....BTW actually I didn't even realise that there were a few builds that failed :)

[jimafisk]

A quick fix is deleting all the type folders (catgs, pages, posts, projs, tags) in the content source of either the project or the theme. Once I've done the quick fix to get things to a working state, I can once again reproduce the error by defining a catgs but not referencing it any posts.

For example:

1. Delete all content source to get passing build: rm -R content/catgs content/pages content/posts content/projs content/tags
2. Creating a new category in the theme breaks the build themes/compendium/content/catgs/test.json:

{
  "name": "Test",
  "route": "categories/design"
}

Note: only the name key matters

3. Then I can fix the build again by editing a post themes/compendium/content/posts/post-01.json and adding Test to the categories array like so: "categories": ["Photography", "Test"],

It seems like there's some fragile logic here that is breaking the entire build if you try to add a category value that is not being referenced in a post somewhere. This should probably be refactored so the theme can allow unused categories to exist.

[jimafisk]

This is all making sense to me now. Your recent category & tag updates (edf6e07) have broken the build per my explanation above.

The builds were passing previously because you were using an older version of Plenti (v0.5.5) that didn't care about empty _blueprint.json files. Locally if I reverted your repo to builds that passed your CI, it was still failing for me because I'm using a newer version of Plenti (v0.5.8) that does require valid JSON in _blueprint.json files. I didn't realize it initially, but I was getting different errors in that scenario:

Error in DataSource build step
Could not get all content from virtual theme 
Error getting content from content/pages/_blueprint.json 
Unable to read content because: unexpected end of JSON input

The solution is to match your categories and tags to what is referenced in your content source exactly. Optionally you may want to petition Roberto to update the theme so it's less strict about this.

[jimafisk]

Another thing that was throwing me off was .gitignore had the themes dir in it. Most theme files were being tracked, but the following were ignored:

• themes/compendium/content/catgs
• themes/compendium/content/tags

I found this with:

• git check-ignore -v themes/compendium/content/catgs/
• git check-ignore -v themes/compendium/content/tags/

Then adjusted the project's .gitignore to track all theme files.

I also updated Plenti's error handling again to produce a JS stacktrace:

Error in DataSource build step
Can't create props for content/404.json 
Can't render htmlComponent for layouts_content_404_svelte: TypeError: Cannot convert undefined or null to object
TypeError: Cannot convert undefined or null to object
    at Function.values (<anonymous>)
    at layouts_scripts_catgs_tags_svelte_catgs_tags (create_ssr:58:18)
    at create_ssr:31:14
    at $$render (create_ssr:1555:22)
    at Object.render (create_ssr:1563:26)
    at create_ssr:1:58

The layouts_scripts_catgs_tags_svelte_catgs_tags component signature let me know the issue was in themes/compendium/layouts/scripts/catgs_tags.svelte. I also knew the error was being thrown at line 58, but I had to look to the built output (public/spa/scripts/catgs_tags.js) to figure out exactly which line that was.

Turns out there are scenarios when tagsObj[tagContent[index].fields.name] and catgObj[catContent[index].fields.name] are null, so we just need to check for them before trying to use them. I didn't dive any deeper into why these are sometimes null.

[jamestagal]

Thanks @jimafisk for all this investigating and problem solving. Hopefully it hasn't taken up too much of your time but from these post I am afraid it has.

I will look at merging the changes you added tonight. From memory I first Merge pull request #30 and then do git pull in my local repo to see that change?
thanks
Ben

[jamestagal]

Hi @jimafisk
The changes you made fixed that error and built successfully which is great. Thank you.

However now the problem is that since you adjusted the project's .gitignore to track all following theme files. I get all of them (Compendium's) in my project with Null values. See below.

• themes/compendium/content/catgs
• themes/compendium/content/tags

Any thoughts? Should I add themes/ back to the .gitignore file?

[jimafisk]

It's not actually the tracking of the themes dir that is causing that, so I recommend leaving that in. Those null values are showing up because those specific catgs and tags are defined in the content source of the theme, but since you are overriding all the posts they are no longer being referenced anywhere. The null value is actually the fix I added, before this was just throwing an error and breaking the build. The best path forward here is to use an exclude in your theme_config, but there's a bug that Roberto found that's preventing us from doing that. I think I've identified what the issue is and I'm working on a fix currently.

[jimafisk]

@jamestagal if you merge in this PR it should resolve your null values problem: #31. Let me know if it gives you any problems. Thanks!

[jamestagal]

Hi @jimafisk
That change built successfully in Git and to Netlify but after running a git pull origin main I got an error locally.
Pull request:

git pull origin main
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 11 (delta 6), reused 9 (delta 6), pack-reused 0
Unpacking objects: 100% (11/11), 1.73 KiB | 253.00 KiB/s, done.
From https://github.com/jamestagal/edtechdesigner
 * branch            main       -> FETCH_HEAD
   53ca60b..468aefe  main       -> origin/main
Updating 53ca60b..468aefe
Fast-forward
 content/404.json             | 9 +++++++++
 plenti.json                  | 5 ++++-
 themes/compendium/.gitignore | 2 +-
 3 files changed, 14 insertions(+), 2 deletions(-)
 create mode 100644 content/404.json

Serve step error:

❯ plenti serve
███▒▒▒▒▒▒▒ Building... 2022/07/02 10:49:40 build.go:155: 
Error in DataSource build stepCould not create props: ReferenceError: layouts_content_404_svelte is not defined /home/runner/work/plenti/plenti/cmd/build/data_source.go on line 402

[jimafisk]

Hi @jamestagal, I think your CI is passing because it's pulling the latest version of Plenti (which I release this morning), but locally you may still be using v0.5.8? Can you try upgrading to Plenti v0.5.9 and see if that fixes this? It looks like that same theme bug that I was running into earlier. Thanks!

[jamestagal]

Hi @jimafisk
Yes thank you. updating plenti to v0.5.9 worked :) so I don't get that error now.

Thanks again.

[jimafisk]

Wonderful, happy to help!