We don't currently currently build binary wheels for M1, and there are a number of users who have run into compilation issues. I'm not adding cross compiling in #189 since it seems to be a bit of a nightmare (ref: scipy/scipy), but it would be good to revisit in the future. In the short term, maybe at least have some docs?