We should drop the high variety of building workflows and instead just running a sub-set (native platform-wheel pairs only) of publishing CI builds