Add golden tests

[chalin]

As a followup to #540 (comment), and in conjunction with #725, I believe that docsy could benefit from some CI testing. It would probably be easiest to add some golden tests. That is, have test site(s) containing test pages, then generate the site files and compare the results against the golden files. This would help us avoid features being overwritten or just plain reverted by mistake.

Thoughts?

Related:

• Add unit/integration tests  #286
• Semver tagging commits #619

/cc @emckean @nate-double-u @celestehorgan

Tasks

• Add agent-friendly support plan and phase 1 implementation of Markdown output #2597
• ...

[emckean]

Thanks Patrice! This is an excellent idea! I'm working on setting up screenshot tests w/Percy -- you can see my in-progress stuff here, I'd love feedback! https://github.com/emckean/docsy/actions

[chalin]

Cool!

I was going to suggest that the golden files be HTML (generated test site files) rather than screenshots. That makes it much easier to review.

[emckean]

I've been poking around to see if there's a GitHub action for easily diffing the generated HTML—do you have a tool you recommend?

I agree that HTML makes it easier to see what changed in the markup, but I think the screenshots make it slightly easier to notice CSS weirdness?

[chalin]

I've been poking around to see if there's a GitHub action for easily diffing the generated HTML—do you have a tool you recommend?

What I've done before is to use a simple script to regenerate the golden files and then use git status (during CI) to detect if anything has changed -- and fail the build if something has changed.

I agree that HTML makes it easier to see what changed in the markup, but I think the screenshots make it slightly easier to notice CSS weirdness?

Right, both are useful.