Review of Evaluation Metrics

[hedonistrh]

I have added some evaluation metrics, however, i will add more of them and more info about some of them. This is related with #20

[hedonistrh]

I think, we can discuss our metrics via this summary in the next meeting with all group members. If you have time, may you review it before the meeting? @magdalenafuentes

Ps. I think, we do not need TL;DR for this review. I have simplified it. ⛱

[magdalenafuentes]

Just reviewed this. It's looking better than last week :) I like the order better.

Minor things:

• Define tokens in this context.
• Define what the Folk-folks (nice joke eh ;) ) mean by "resolution"
• I like metrics 4, 5, 9, 11, 13, 14, 15 and 16 because they seem musically informative, though still unclear to me if they're suitable for this music (and how to implement them in some cases). Let's discuss this in the next meeting.
• 6 and 7 seem a bit weird to me, don't get if directly apply here.

I think it could be useful to contrast whatever metric we choose with the output of the model on a "standard" scenario with Western music (i.e. show how the generator matches the distribution in metric 1 in Turkish vs. Pop music) . @sertansenturk 's opinion?

[sertansenturk]

Thanks Magdalena for the review! Sorry, I was sick during the weekend, I will review the write-ups in the evening.

…

On Mon, Feb 18, 2019, 20:30 Magdalena Fuentes ***@***.*** wrote: Just reviewed this. It's looking better than last week :) I like the order better. Minor things: - Define tokens in this context. - Define what the Folk-folks (nice joke eh ;) ) mean by "resolution" - I like metrics 4, 5, 9, 11, 13, 14, 15 and 16 because they seem musically informative, though still unclear to me if they're suitable for this music (and how to implement them in some cases). Let's discuss this in the next meeting. - 6 and 7 seem a bit weird to me, don't get if directly apply here. I think it could be useful to contrast whatever metric we choose with the output of the model on a "standard" scenario with Western music (i.e. show how the generator matches the distribution in metric 1 in Turkish vs. Pop music) . @sertansenturk <https://github.com/sertansenturk> 's opinion? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#25 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEVI4cL748feX-L64rbupk9DyBuy3V2Tks5vOw2BgaJpZM4a6bAz> .

[hedonistrh]

We can use chapter 5 of this Master Thesis to get more evaluation metrics.

[sertansenturk]

I think this list is pretty comprehensive, and you seem to understand the measures.

Obviously there could be more to check (e.g. the Gatech paper), but I think you have done enough at this stage.

I have written some comments. There are very few change requests, they are mostly explanation/clarification & suggestions. Feel free to start a discussion on the comments so we can clear out any doubts/question marks in your mind. Once the minor changes are complete and you are satisfied with the discussion, we can merge the pull request.