This repository was archived by the owner on Sep 9, 2025. It is now read-only.
Add a skill to try to improve ROT13 generation#332
Closed
codificat wants to merge 1 commit into
Closed
Conversation
This adds a grounded compositional editing skill to teach ROT13 to the model. It is grounded because instructions/context about what ROT13 is and how to apply it are provided as `context` in each example. However, most current models are not capable of actually understanding and applying these instructions. Therefore, synthetic training sets generated for this skill are likely to be inaccurate. At this time, this skill can be more useful as a benchmark / test for the process itself than as an actual skill. Signed-off-by: Pep Turró Mauri <pep@redhat.com>
Author
|
Note that there is another existing PR around ROT13, #271. The goals are different though: the objective in the other PR is to identify (and decode) ROT13 fragments, while this one here is about applying rot13 itself. I believe #271 has better chances of succeeding as a skill at the moment, in particular to identify / extract ROT13 fragments, but that the actual application of the algorithm will fail. |
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe the contribution to the taxonomy
This adds a grounded compositional editing skill to teach ROT13 to the model.
However, most current models are not capable of actually understanding and applying these instructions. Therefore, synthetic training sets generated for this skill are likely to be inaccurate.
At this time, this skill can be more useful as a benchmark / test for the process itself than as an actual skill.
Input given at the prompt
Response that was received
Note that the generated rot13 text is incorrect, it corresponds to the text "Testsure, message!".
Response that is now received instead
TBD as I am not able to fine tune a model with this skill.
But here is some generated content from the skill, from
lab generatewith themerlinite-7b-Q4_K_Mmodel:[ { "instruction": "Translate this sentence into English using rot13: \"Nyfb yvar vf nyfb!\"", "input": "", "output": "You are correct!" }, { "instruction": "Explain what rot13 is and how it works.", "input": "", "output": "ROT13 is a simple letter substitution cipher that replaces a letter with the 13th letter after it in the Latin alphabet. To apply ROT13 to text, each alphabetic character in the text is replaced by the character that is 13 places further along in the alphabet, wrapping back to the beginning if necessary. For example, the letter A is replaced with N, B is replaced with O, and so on up to M, which is replaced with Z, and then the sequence continues at the beginning of the alphabet: N is replaced with A, O is replaced with B, and so on to Z, which is replaced with M. The letters that occur in the English alphabet are affected, while numbers, symbols, punctuation, whitespace, and all other characters are left unchanged. This cipher can be used for both encryption and decryption, as the same substitution pattern is used in both cases." }, { "instruction": "Implement ROT13 on the phrase \"This is my first test!\"", "input": "ROT13 (or rot13, Rot13 or ROT-13) is a simple letter substitution cipher\nthat replaces a letter with the 13th letter after it in the Latin\nalphabet.\n\nApplying ROT13 to a text fragment requires examining its alphabetic\ncharacters and replacing each one by the letter 13 places further along in\nthe alphabet, wrapping back to the beginning if necessary. A becomes N, B\nbecomes O, and so on up to M, which becomes Z, then the sequence continues\nat the beginning of the alphabet: N becomes A, O becomes B, and so on to\nZ, which becomes M.\n\nTherefore, the letters are mapped as follows: A->N N->A B->O O->B C->P\nP->C D->Q Q->D E->R R->E F->S S->F G->T T->G H->U U->H I->V V->I J->W W->J\nK->X X->K L->Y Y->L M->Z Z->M\n\nOnly those letters which occur in the English alphabet are affected;\nnumbers, symbols, punctuation, whitespace, and all other characters are\nleft unchanged.\n\nFor example, the text \"Sphinx of black quartz, judge my vow.\" becomes\n\"Fcuvak bs oynpx dhnegm, whqtr zl ibj.\" when ROT13 is applied.", "output": "Guvf zrnaqn qvpxr!" }, { "instruction": "ROT13 the following sentence: \"The sun shines on the meadow\"", "input": "ROT13 (or rot13, Rot13 or ROT-13) is a simple letter substitution cipher\nthat replaces a letter with the 13th letter after it in the Latin\nalphabet.\n\nApplying ROT13 to a text fragment requires examining its alphabetic\ncharacters and replacing each one by the letter 13 places further along in\nthe alphabet, wrapping back to the beginning if necessary. A becomes N, B\nbecomes O, and so on up to M, which becomes Z, then the sequence continues\nat the beginning of the alphabet: N becomes A, O becomes B, and so on to\nZ, which becomes M.\n\nTherefore, the letters are mapped as follows: A->N N->A B->O O->B C->P\nP->C D->Q Q->D E->R R->E F->S S->F G->T T->G H->U U->H I->V V->I J->W W->J\nK->X X->K L->Y Y->L M->Z Z->M\n\nOnly those letters which occur in the English alphabet are affected;\nnumbers, symbols, punctuation, whitespace, and all other characters are\nleft unchanged.\n\nFor example, the text \"Sphinx of black quartz, judge my vow.\" becomes\n\"Fcuvak bs oynpx dhnegm, whqtr zl ibj.\" when ROT13 is applied.", "output": "Gur svthr vf n pbzzr, gur zrffntr gb ybfg vf abj." }, { "instruction": "Tell me if the following text is already ROT13 encoded or not: \"Gur dhvpx oebja sbk whzcf bire gur ynml qbt\"", "input": "The input is the same as the previous instruction, as there is no need for a specific example for this type of instruction.", "output": "Yes, the text is already ROT13 encoded." }, { "instruction": "Provide an example of text that is ROT13 encoded using the alphabet mapping provided earlier", "input": "The rot13 skill enables encoding text with ROT13, which replaces each letter in the alphabet with the letter 13 places further along in the alphabet, wrapping back to the beginning if necessary. The letters are mapped as follows: A->N N->A B->O O->B C->P P->C D->Q Q->D E->R R->E F->S S->F G->T T->G H->U U->H I->V V->I J->W W->J K->X X->K L->Y Y->L M->Z Z->M", "output": "The rot13 skill enables encoding text with ROT13, which replaces each letter in the alphabet with the letter 13 places further along in the alphabet, wrapping back to the beginning if necessary. The letters are mapped as follows: A->N N->A B->O" } ]Contribution checklist
lab generatelab generateqna.yamlfile was linted and prettified (yaml-validator can do both)