GitHub - aalto-speech/llm-morph-tests: A test suite to assess the knowledge of morphology in LLMs

Presented in the CMCL Workshop at ACL 2024

Experiments are launched from the script run-expts.sh

Name		Name	Last commit message	Last commit date
Latest commit History 44 Commits
data		data
expts/random2000		expts/random2000
rnn-results		rnn-results
.gitignore		.gitignore
README.md		README.md
combine_words_with_same_surface_form.py		combine_words_with_same_surface_form.py
common.py		common.py
compare_tokenizers.py		compare_tokenizers.py
eval_classifier.py		eval_classifier.py
evaluate.py		evaluate.py
generate-wordlist-withtags-nouns.sh		generate-wordlist-withtags-nouns.sh
generate-wordlist-withtags-verbs.sh		generate-wordlist-withtags-verbs.sh
generate_prompts.py		generate_prompts.py
inference_gpt.py		inference_gpt.py
inference_transformers.py		inference_transformers.py
preprocess.py		preprocess.py
random_guess_baseline.py		random_guess_baseline.py
run-expts.sh		run-expts.sh
slrm-llama.sh		slrm-llama.sh
slrm-train-fairseq.sh		slrm-train-fairseq.sh
slrm-transformers.sh		slrm-transformers.sh
visualise_results.py		visualise_results.py

Provide feedback