Gracefully apply read_only mode on old master

[R-omk]

It need to wait until all rw requests are completed.

[Gerold103]

It is not possible to wait until all of them are finished. It leads to infinite waiting in case there is a request, which is not going to be finished. Some long polling request. It can be postponed with a timeout though, and an error can be raised + refuse to become read-only. But I don't think it is a good idea. Perhaps it should turn to read-only anyway, but wait a bit if needed.

[R-omk]

Bump this issue since raft/promote is coming.

[R-omk]

I see that there is an on_election trigger, but maybe it fires too late and does not give ability to delay the process at all.

[R-omk]

how to do this:

1. ref.rw_lock = true (same as in bucket_send_xc)
2. wait global ro
3. restore ref.rw_lock = false

if vshard supports such a mechanism, it will be possible to make a graceful version of box,ctl.demote()

for #219 a similar approach can be used

---

related tarantool/tarantool#10795