Exercise tests should be generated

[tommyschaefer]

I recently came across #381, and saw that @kytrinyx suggesting converting all existing exercises to use generated tests as a first step.

On PR #394, @Insti recommended creating an issue to track progress towards this goal.

Here's a list of exercises and whether they generate their tests:

Converted

• acronym
• all-your-base
  • Generator at all-your-base: add test generator #510
• alphametics
• anagram
• beer song
• binary
• bowling
• bracket-push
• clock
• connect
• custom-set
• difference-of-squares
• dominoes
• etl
• gigasecond
• grains
  • Generator at grains: add test generator #514
• hamming
• hello-world
• isogram
• largest-series-product
• leap
• luhn
• nth-prime
• ocr-numbers
• pangram
• pig-latin
• queen-attack
• raindrops
• rna-transcription
• roman-numerals
• run-length-encoding
• say
• sieve
• space-age (space-age: Add generator #583)
• tournament
• transpose
• triangle
• two-bucket
• word-count
• wordy
  • Generator at wordy: Generate tests #509

In Progress

Has Canonical Data in x-common

• allergies
• atbash-cipher
• binary-search
• bob
• change
• crypto-square
• flatten-array
• food-chain
• forth
• grep
• markdown
• meetup
• minesweeper
• pascals-triangle
• pov
• rail-fence-cipher
• react
• rectangles
• robot-simulator
• scrabble-score
• sublist
• sum-of-multiples

Needs Canonical Data

• accumulate
• binary-search-tree
• circular-buffer
• diamond
• grade-school
• hexadecimal
• house
• kindergarten-garden
• linked-list
• list-ops
• matrix
• nucleotide-count
• octal
• palindrome-products
• perfect-numbers
• phone-number
• point-mutations
• poker
• prime-factors
• protein-translation
• proverb
• pythagorean-triplet
• robot-name
• saddle-points
• scale-generator
• secret-handshake
• series
  • Issue at Add generator for "series" #531
• simple-cipher
• simple-linked-list
• strain
• trinary
• twelve-days

[kotp]

Also see this list in todo repository.

[kytrinyx]

@tommyschaefer @exercism/ruby What do you think of generating individual issues for each of these?

Ideally an issue like that would explain in detail exactly what the process would be to do the work, and then we could label them with "good first patch" (and we wouldn't have to remember to check things off a todo list).

If there's interest for this, we should probably hash out the text of the issue here so that we can get it right before making a gazillion copies of it.

[Insti]

I'd like to have a go at changing the way generators work.
This probably needs its own discussion issue, but I want to change some file names/locations, add a superclass with common methods, add auto version bump detection, change generator invocation script.
But it will hopefully make it easier to write and run generators.

It would be good to be able to do this before we start spending a lot of time writing guides.

Once that is done, individual issues might be the way to go.
Has that proven effective in other instances?

[kytrinyx]

I'd like to have a go at changing the way generators work.
[...]
it will hopefully make it easier to write and run generators.

Cool! Yeah, would you open a separate discussion for that?

It would be good to be able to do this before we start spending a lot of time writing guides.

Agreed.

Once that is done, individual issues might be the way to go.
Has that proven effective in other instances?

Yeah, it has been easier to manage than checkbox lists, especially if we have one top-level issue that each of the child issues links to, which gives us a visual "todo list" that shows how many of the sub-things are completed.

[tommyschaefer]

@kytrinyx Ah, yes! That makes a lot of sense to me!

Especially:

Ideally an issue like that would explain in detail exactly what the process would be to do the work, and then we could label them with "good first patch" (and we wouldn't have to remember to check things off a todo list).

Because some of the exercises already have shared test data in exercism/x-common, I'm thinking those might be especially well suited as a "good first patch".

[kytrinyx]

Because some of the exercises already have shared test data in exercism/x-common, I'm thinking those might be especially well suited as a "good first patch".

Yeah, agreed. We have a bunch of issues for them over in "todo" but I'm starting to think that the todo repo is not a great idea afterall. What we really need to do is get all the other repositories under control :) https://github.com/exercism/todo/issues?q=is%3Aissue+is%3Aopen+label%3Ashared-test-data