See conversation at dcf1a36#r2246227041.
nh_compare should check for nh_family as well. But, nh_family is actually always set in the netlink messages. So, we could create a NH_ATTR_FAMILY and set ce_mask to that in rtnl_nh_alloc.
Not sure what the best path forward would be.