Handling duplicate kata

[hobovsky]

One of long lasting problems affecting Codewars are duplicate kata and difficulties with handling them. But as new functions appear and CW maintainers create more and more tools, it can be finally possible to get rid of them, or at least limit amount of duplicated kata. Mods can unpublish betas, content-issues repository can be used to track and discuss problematic instances, etc. That's why I think it would be good to discuss the ideas, options, and possibilities of handling duplicate kata. For example, what is your opinion on following topics?

1. Do you think duplicate kata are a problem? Is it bad to have similar kata, or is it good?
2. When are two kata duplicate?
   1. Are kata which present different tasks or problems, which can be solved by the same algorithm or approach, a duplicate?
3. Can/should duplicates be prevented? If so, how?
4. Should the CW community work out some common method or guidelines for handling duplicates, or they should be always approached on a case-by-case basis?
5. Should existing duplicates be handled somehow? How? Let them be? Retire/unpublish? Handle betas and leave approved ones? Should authors or existing solutions be considered in any way?
6. How to de-duplicate kata which overlap only partially and differ in terms of available languages, difficulty, or one of them requires some additional pre- or post-processing? Which one to leave, which to remove?
7. What would be your ideas for using existing tools, or what tools would be needed to make handling of duplicate kata easier?

Please feel free to share any opinion or ideas on the topic.

Thanks!

[hobovsky]

To get the discussion started, here are some of my personal opinions. Please remember that they are in no way authoritative, I just wat to share them with others :)

Ad 1: I have a somewhat mixed feelings, but I am leaning towards: strict duplicates are bad, similar white kata are ok, similar yellow and harder kata are bad. The most commonly raised argument in favour of duplicates is that newbies benefit them, and I agree with it.

Ad 2: I think two kata are duplicates when they can easily be brought down to the same problem statement, with eventually some additional differences like parsing the input, validation, formatting the output, etc. I think that two kata are not duplicates if they present different problems, but can be solved with the same algorithm. Sometimes getting from the task to the algorithm is not obvious, and two different problems (think: descriptions) which boil down to the same approach might not constitute a duplicate.

Ad 3: They are bad, so they should be prevented. Good thing is, reviewers are doing usually a good job identifying them. Bad thing is, they often fail to report this fact clearly. When a reviewer finds out that a beta kata is a duplicate, it would be good if they backed the claim with a link to existing, similar kata. Another problem is that kata search works as it does, making searching for existing, similar kata difficult.

Ad 4: I don't know. I think it's difficult to create a good general method of handling existing duplicates. There's many of them, and they are sometimes difficult to handle because of differences between them. Maybe when we solve two or three such cases, more ideas will appear.

Ad 5: Good thing is that duplicates can be handled: beta kata can be unpublished by a mod, approved kata can be retired by an admin, and kazk agreed he could step in and handle specific instances when a community comes with a clear agreement how to do it. It's also planned to hand out this privilege (of retiring approved kata) to mods, under some restrictions.
I think that while handling duplicates, quality of content should be prioritized over personal gains of users, so the existing solutions, points rewarded to the author, etc. should not be a deciding factor. I am not sure what would happen with rewards (rank and honor) to people who solved, authored and translated a kata which would be removed, but I personally would be OK with losing them.

Ad 6: My general rule would be: when in doubt, aim for quality. Differences in an input and output do not make two kata different for me, so it does not really matter for me if one kata accepts ony ints, and another one accepts floats or mixed types: if they boil down to the same task, any of them can be selected to stay. When kata differ in difficulty ranking, I would choose the one which comes closer to my estimation. When details of requirements cause two similar kata to be of different difficulty, I would pick the harder one ;) When kata differ in terms of available languages, I'd like it to be not a deciding factor: i'd pick the one of better quality and translate it to missing languages (preferrably before retiring remaining duplicates).

Ad 7: `content-issues repository on Github is public and has an issue board where issues, discussions, and project kanbans can be created. It would be possible to create issues and discuss specific instances of the problem, create tasks for translators and maintainers, etc. Repo's wiki can be used to create some related documentation or lists (like this one: https://github.com/Codewars/content-issues/wiki/List-of-Possible-Duplicate-Kata). Some support for mods to retire existing kata would be useful, to not engage admins unnecessarily. Kata search could be improved, but I have no good idea how.

Thanks for listening!

[JohanWiltink]

Ad (1) If you allow duplicate kata at any rank, I have a hundred fizzbuzz kata for you. It's not the existence of one or two duplicates that is the problem, it's the unbridled duplication that will follow as sure as night follows day.

Any, especially new, kata author can play the "but duplicates already exist, so mine should be allowed as well" card. And some will, and then what are you going to say?

I would actually be in favour of some duplication in lower ranks, but I see a problem keeping it contained.

Ad (2) If you allow identical solutions with different descriptions, I see the same problem: endless "map/reduce/filter my list" kata. Some duplication is good, lots of it is bad, where do you draw the line and how do you defend that line against authors who just want their kata to survive?

I would actually be in favour of quite a lot of duplication here, because a lot of interesting problems actually come down to the same solution. But how do you keep it in check?

[JohanWiltink]

Ad (5) Approved kata should not be retired, especially not beyond a certain age ( older than a year or so ). Beta kata are fair game.

This because of being a duplicate. Being of bad quality is a different matter, but not the subject here.

[XRFXLP]

Ad (5) Approved kata should not be retired, especially not beyond a certain age

Why?

[hobovsky]

Approved kata should not be retired, especially not beyond a certain age ( older than a year or so ).

May I ask why exactly?

My main concern with approved duplicates is that they are a maintenance cost: each such kata attracts translations, induces issues, and requires maintenance. I see more benefits from merging them than keeping them separate.

If one of them is of a bad quality, would it be ok to remove it as not worth fixing because another, better one already exists?
If all of them ae of bad quality, would it be ok to select one to fix, and remove others?

I am not talking exactly of retirement, but with current technical means it would be probably the easiest option. But maybe something else would be better, I do not know, maybe "locking" the kata (to keep exposure, points and solutions), and linking to the "original" one in the description?

[JohanWiltink]

To preserve history.

Approved kata need not attract new "duplicate" issues; those can be restricted to new beta kata. If you don't want maintenance, don't have kata ?!? I don't find that argument very convincing. Approval should count for something.

Approved kata that are just of bad quality could be returned to Beta. If it's not worth the effort to bring it up to current standards, it could even get retired, as a Beta kata. But not just for being a duplicate, not when it's approved. Approval should not be possible after an hour and two votes either, the way it is now ( exaggerating a little here ).

So it's not an absolute; there could still be a way to maintain quality. But historical duplicates can be grandfathered in IMO.

Would you retire an old kata just for not having random tests? I'm not talking about kata with two fixed tests here, but there are old, approved kata with very decent fixed testing, just no random tests, the original author long gone, nobody willing to add random tests, and not really a problem with hardcoded solutions. I consider them time documents, and don't see the problem really. ( And raising issues on them without fixing it yourself does not make you part of the solution. )

You would not get away with that now, but one could then. The publication date is on them for all to see. It would not lower current standards.

[hobovsky]

OK I think I see what you mean, but I believe that having a set of duplicate kata and all of them being of an acceptable quality is quite uncommon scenario. I think much more common case is when all considered kata have some shortcomings in their quality, every of them some different ones. When deduplicating such set, the community would have to decide which ones to get rid of, and which ones to keep and eventually fix.

I also think such occurrences should be inspected case by case, so maybe I will try to pick something from this list, discuss it, and see how it would go.

[XRFXLP]

but I believe that having a set of duplicate kata and all of them being of an acceptable quality is quite uncommon scenario

Hard sudoku solver 2kyu and 3kyu are exact duplicates, both have plenty number of random tests.

[dolamroth]

IMHO

1. (Semi)-exact duplicates are bad, but in general each case those should be treated individually. There are newer "duplicate" katas, that have better description, better test cases, and in general are less cringy that old stuff that was approved years ago for no reason. So, having similar katas is not entirely bad. However, having similar easy katas is bad (7-8 kyu).

2. 2 katas are duplicates, if they can be solved with same approach and a beginner-level programmer can determine, that they can be solved with similar approach. If it is not obvious from the first glance, such cases should be reconsidered.

3. Yes, they should -- by deleting katas that have worse description, title and test cases, however old they are and however many people have solved them. Furthermore, authors should be enforced to use more informative tags, title and description (i.e., that should be pre-written in description section when creating new kata). I.e., incorrect tags and poor description may now be considered "Issue", not just "Suggestion".

4. Case-by-case

5. Current cases should be handled, by leaving the better kata (description, title, test cases). Old katas should also be re-ranked.

6. If duplicate kata is in beta, suggest improvements and re-publish.

7. Having reserved dialog in Gitter and reserved page on Github would be nice. Tags should be used to determine more instances of "possible duplicates" to authors of new katas.

[Blind4Basics]

(note that most tags are automatic, and almost all of the wrong tags are those)

[hobovsky]

3. incorrect tags and poor description may now be considered "Issue", not just "Suggestion"

Even currently, bad description usually should be considered an issue, but actual label might depend on the severity.

About tags, it's difficult to enforce this requirement because tags do not work to well on CW. But getting them fixed would improve searching for kata, and, as a result, would make finding duplicates easier, and it's a very good point. I still see some issues related to tagging systems in general, but it's a thing for another discussion.

[dolamroth]

(note that most tags are automatic, and almost all of the wrong tags are those)

That falls into point (7) and probably should be fixed. I've also noticed, that not only useless tags are automatically added, but some of my tags are omitted. For example, here I have added tag "travelling salesman problem", but it's not visible https://www.codewars.com/kata/601b1d0c7de207001c81980b

[KovDimaY]

I would like to contribute as well, if possible:

1. I think duplicate katas is definitely a problem. Similar katas are a problem too, but depends how similar they are. Anyway, if one can solve 10 katas in a row during 10 mins just copypasting his solution (maybe with little params tweaks) it is not good in my opinion.

2. Is not easy to explain it here, sorry. But there is one case that for me is obviously a duplicate and it concerns me a lot since today morning and is not considered as a duplicate at all 🤯 (I think is a big mistake).
   Here is an example:

   1. https://www.codewars.com/kata/574e890e296e412a0400149c
   2. https://www.codewars.com/kata/574be65a974b95eaf40008da

   Exactly the same katas with exactly the same description, just the difficulty of the tests is different.
   Why do I think it is a problem? Because when you solve a kata you can see the solutions of others and someone can post his already performant solution in the simple kata so everyone who solves the simple kata can automatically solve the difficult one because he knows the approach already. And it makes the difficult kata be just free points, and a lot of points as normally they are very difficult with a high rank! 😕

3. I think duplicates should be prevented, how to do it is another question. Once we have duplicates, I would suggest removing all the duplicates except only one version that is considered to be the best of them

4. I would say in general should be handled case by case, but some common cases can be described using guidelines

5. As I suggested in the point 3, we should remove duplicates. Maybe we can define some logic for ranking duplicates in order to define which is better/worse (based on age, people who solved this kata, number of translations, etc etc) and then remove all the duplicates and keep the one with the highest rank

6. Using the idea of ranking from the previous point, we could merge several katas in 1 (for example move translations from one kata to another if the set of translations differs, etc) and remove all the duplicates with the lower rank

7. Maybe we could use the feature that matches solutions in groups. For example, if the algorithm detects that 2 katas have a shared solution that solves both of them it can put a warning that will force moderators to review the case. 🤔

But the point 2 is really concerning me, idk why this was not detected before, but now you can solve both 4kyu and 2kyu katas by just solving only 4kyu kata 🤷‍♂️

[hobovsky]

Thank you for your insight, it's definitely going to be useful.

About spoilers and the example you gave, it's also a recognized problem which is currently possible to be handled in some way. I do not think "tiered" kata should be considered duplicates, but spoilers should be handled somehow. There is a couple of ideas for this, and I can start separate discussion on the topic, but even now you can report spoiler solutions and they can be hidden by a mod: no points are withdrawn, no one loses any reward, but hidden solution is not visible on solutions list of an easy kata. Just catch a mod on the Gitter and give them a link to the solution.

[KovDimaY]

Perfect! Thank you! If it is handled somehow then I am less concerned 😌

[hobovsky]

I like your idea of ranking the kata from a set of duplicates, I wonder how it could look like.

[ejini6969]

1. Exact duplicate -> Yes(sum of list, average, fibonacci, factorial, replace string.........) , similar duplicate -> depends on the task (direct map, filter, reduce, dict manipulation with no brainstorming SHOULD BE RETIRED, but for performance wise katas and myjinxin thinking & testing that requires self digging to find the solution SHOULD BE KEPT thoo they have similar algorithms as other direct katas)

2. EXACTLY THE SAME in terms of user solution & task description thoo with different input (ie. sum of list & sum of numerous arguments are basically the same). AND can be solved by same algo and at the same difficulty level -> DUPLICATE(ie. look and say sequence in the form of list and string input are not considered DUPLICATE as algo is somewhat different ~~)

3. EXACT ONES OF COURSE. if it's in beta, retire ~~ if approved, send back to beta and (fix the description, test cases, language conflict in terms of task to make it a bit different from other original ones if possible or else just let community and mods decide if it's worth the value of existing in DB)

4. There should DEFINITELY be a set of agreed-upon guidelines for defining what's a duplicate, how likely is it close to being a duplicate, duplicate percentage(Implement an algo that can compare with DB's kata that contain similar task description & user solution -> NOT EASY i know XD) . At the same time, case by case basis should be done at the same time but with less attention(should not be done when everyone clearly understands the standards & guidelines of being a duplicate)

5. Already mentioned at 1 and 3. BUT I dont agree with admins retiring an approved kata directly without any reasoned evaluation (sending it back to beta) or without the community's majority agreement. Also the honors of authors, translators and fellow solvers should be preserved or else someone (a huge crowd) is gonna complain about losing honors for no reason agn.

6. First by difficulty. The performance version of one should be kept and an easy one should too but with its solution that uses the performance one hidden. + If one kata is being translated to more languages than the other, then we can fix based on that. Maybe making description clearer, adding more test cases and changing the rank if necessary and retire the other one with less language versions. SO when it comes to priority, difficulty > no. of language versions

7. First & foremost, CW should develop an algo to search based on the contents & solution of a kata and check its similarity rate. If it's > 20% or some other threshold value, it will display to the user. (OFC not easy as mentioned in 4) Second, the tag ofcourse to make searching relevant katas easier. LAST BUT NOT LEAST, a team of professional moderators to monitor and handle the whole progress from time to time.

That's my humble opinion, and I'm feel free to contribute if the community needs it XD~~😊

[JohanWiltink]

Ad (2): Different input or output encodings should not make kata not duplicates. This is somewhat of a gliding scale, but stringifying inputs or outputs does not make kata different ( and generally does not meaningfully add to any task ). Adding input validation is frowned upon, why would other input handling be different? ( Just use appropriate datatypes in general. )

Other opinions are available, but this is mine.

[imjasonmiller]

1. Do you think duplicate kata are a problem? Is it bad to have similar kata, or is it good?

I do see these as a problem as it leads to a higher amount of maintenance, i.e. resolving suggestions and issues in discourse, updating descriptions, dependencies and so on.

2. When are two kata duplicate? Are kata which present different tasks or problems, which can be solved by the same algorithm or approach, a duplicate?

If the solution is a near copy and paste of another kata of the same rank, I would count it as duplicate. I think it's fine for lower ranked kata to introduce one to certain concepts and then having higher ranked kata build on top of that knowledge.

3. Can/should duplicates be prevented? If so, how?

Absolutely. I'm not sure yet what the kata approval process looks like, but I think I've seen some talk of it on Gitter in which some issues were raised. I'd like the barrier to entry for new kata to be higher.

3. Should the CW community work out some common method or guidelines for handling duplicates, or they should be always approached on a case-by-case basis?

On a case-by-case basis, but I'd like to see some guidelines and a consensus based vote by multiple moderators. Perhaps even weighing in the community for kata that are likely to retired.

4. Should existing duplicates be handled somehow? How? Let them be? Retire/unpublish? Handle betas and leave approved ones? Should authors or existing solutions be considered in any way?

I'm not sure what retiring means in this case, but if we define it as not changing statistics for the author and those that completed it, I think that'd be fine. The kata could then be hidden from view for those that have not completed it.

5. How to de-duplicate kata which overlap only partially and differ in terms of available languages, difficulty, or one of them requires some additional pre- or post-processing? Which one to leave, which to remove?

Some, certainly not all, factors that I would weigh are:

• Number of completions
• Number of translations
• Description quality
• Satisfaction rating
• Ability to translate in an idiomatic way to other languages ― I came across someone's review on some of the existing Rust kata and I try to incorporate those in my own translations.
• Having good test coverage ― this somewhat ties in to the former.

7. What would be your ideas for using existing tools, or what tools would be needed to make handling of duplicate kata easier?

It's not a panacea and unfortunately not easy either, but if possible, a tool that would be able to assist in identifying and linking possibly similar kata based on solutions and descriptions would be great.

[g964]

Some simple thoughts.

• kata B is a duplicate of kata A if
  -- kata B is posterior to kata A
  -- kata B presents in its description the same situation as presented in kata A
  -- B has different input or output but these differences don't significantly demand additional work
  -- code passing A passes B and vice versa with minor variations

• which one to keep
  A except if B has better description and/or better tests and/or more translations.
  In this case it would be fair to tell A's author that A and B are duplicates,
  that A will be suppressed unless he/her betters the description and/or the tests and/or commits to add translations within some time limits.

If a kata is suppressed it must be without any consequences for the author and those who have solved it (but maybe for the one who approved it;-)

If nothing is done to prevent new duplicates the process will never end (is it really necessary that it ends? Attracting translations, inducing issues, requiring maintenance all that seem to be something natural and unavoidable).