How do we Validate changes in SMIRNOFF?

[bannanc]

As was mentioned in issue #62, we need a way to validate or at least check the changes we make in SMIRNOFF.

When we changed R to x to get agreement between OpenEye SMARTS matching and RDKit reading, I created a jupyter notebook (check_different_smirnoffs.ipynb). This notebook reads in all of DrugBank and types it with the current smirnoff99frosst (from its own github repo) and the new one created in that directory. Then it lists and visualizes all of the molecules that were typed differently.

I think the minimum test we should perform is to run this notebook and list molecules with changed parameters verifying manually (for now) that anything changed is changed the way we expect it to be.

In issue #62 @davidlmobley also mentioned:

This also raises a larger question of how we test a change like this. Any thoughts? It seems to me we might ultimately be interested in two levels of tests:
a) Scan a set of molecules and see which molecules it affects and check (manually?) they are those we expect it to affect
b) Repeat our validation work for the affected compounds (e.g. dielectric constants, densities, hydration free energies)

Thoughts?

[davidlmobley]

Yes, that's an excellent start. Totally agree.

I would like to try and get the dielectric constant/density thing working routinely for validation, as I presently want to do this for the hydroxyl hydrogen radius issue and if we can automate it well as part of that, it would be easy to use it for future such changes. Let me know if you have any interest in working on that at all, otherwise it'll stay in the same place it is right now (which is to say that I'll try to get to it for the paper, but I'm not confident I'll end up with the time to make it generally useful as it ought to be).