In other languages & is called a reference and borrowing looks very much like references, so to me & and ref seem like the same thing.

I'd appreciate an example that contrasts the two — why let Point { x: ref ref_to_x, y: _ } is OK, but let Point { x: & ref_to_x, y: _ } isn't?