Fix IsTransitive to require that the group actually acts on the given domain (strictly speaking, the previous behavior was "as documented", but it made no sense and led to bugs elsewhere)#4907