The test system is currently broken

[SohamG]

The test system for scheme on exercism.io is currently broken. Broken as in it gives the "something went wrong" error and basically doesn't run any actual tests.

Let me know if I can gather any more information that can help fix this

[SohamG]

I have since cloned this repo and tried to run it with the run-in-docker script. It seems to be running fine. Perhaps theres an issue with how it communicates with the exercism website (aka via results.json)?

[guygastineau]

Perhaps theres an issue with how it communicates with the exercism website (aka via results.json)?

@SohamG that seems likely to me too. Maybe it has to do with #8
I will look into it some this week. I didn't originally work on this repo, but I'll give it a shot.

[guygastineau]

@SohamG your issue might actually be caused by the issue addressed in #22

I have reviewed the PR, and assuming the author is responsive we might have a fix soon. If that is the issue, then it means we need tests to catch it, because obviously our current test harness failed to do its job.

[guygastineau]

@SohamG my PR #32 should fix whatever failure you are experiencing. In addition to implementing the V3 interface for test runners. Failure during test execution is more robust, and a fail while loading/compiling the user solution is reported as a syntax error now.

[SohamG]

@SohamG my PR #32 should fix whatever failure you are experiencing. In addition to implementing the V3 interface for test runners. Failure during test execution is more robust, and a fail while loading/compiling the user solution is reported as a syntax error now.

Hello, sorry for the delayed response.

I don't know if I'm supposed to do anything to get the new tester but I still see the same error message?

[guygastineau]

The new test-runner was just pushed to main a few minutes ago. You can try again. I would love to see if you get a different response. I don't know how long it takes for pushes to main to be live on the website though. Also, please keep in mind that errors will be more vague with the old tests, but until exercism/scheme#262 is merged you won't be able to get the new tests via upgrading the exercise in exercism.

Please let me know how it goes, and we can work out any new details together.

[guygastineau]

As with the other related issue where you are active, I will give you a couple days to try things out and report results here. I am confident the issue is resolved though, and I will close the issue if there is no response in a few days.

[guygastineau]

Nevermind, we are getting syntax errors reported by the test-runner in exercism, but not in the tests anywhere.

[guygastineau]

An anomaly in the production docker containers at least alerted me to a bug that I also introduced into the new test runner. Everything should be working now (and we have tested it manually). Please check when you have time. I will probably close this issue in a few days if there is no more noise.

[guygastineau]

I am taking the silence to mean no new issues were encountered.

[SohamG]

Hello @guygastineau the code runner does seem to be working. I'm sorry for being silent. My interest for learning on exercism comes in erratic bursts + uni work :) Thanks for your help