Optimize `from_bitwise_unary_op`

[Dandandan]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We can optimize the path further where we have non-zero offsets.

Describe the solution you'd like

Describe alternatives you've considered

Additional context