dht: redirect SEEK to the migration destination instead of leaking ENXIO

[ThalesBarretto]

Fixes: #4771

Problem

During rebalance, a SEEK_DATA/SEEK_HOLE on an fd still open on the source subvolume
can return ENXIO to the application even though the file's data is intact on the
destination.

Every DHT FOP callback redirects an in-flight operation to the migration destination
when it detects the file is migrating. The data-path callbacks receive an iatt and
detect it proactively via IS_DHT_MIGRATION_PHASE2(stbuf). dht_seek_cbk receives no
iatt, so its only migration signal is the errno — and it only redirects on
dht_inode_missing() (ENOENT/ESTALE):

local->op_errno = op_errno;
if ((op_ret == -1) && !dht_inode_missing(op_errno)) {
    gf_msg_debug(this->name, op_errno, "subvolume %s returned -1", prev->name);
    goto out;
}
if ((op_ret == -1) && ((op_errno == ENXIO) || (op_errno == EOVERFLOW)))
    goto out;                       /* unreachable: caught by the handler above */

Near the end of dht_migrate_file the source is truncated to 0
(dht-rebalance.c:2109) before it is unlinked (:2154), while the source's linkto
xattr is already set (:1224). A SEEK_DATA that reaches the 0-byte source returns
ENXIO (the normal POSIX "no data at/after offset"). Since that is neither ENOENT nor
ESTALE, dht_seek_cbk takes the generic goto out and returns it to the client;
dht_rebalance_complete_check (the redirect) is never reached.

The explicit ENXIO/EOVERFLOW check below the generic handler is dead code — it has
been unreachable since it was added alongside the generic handler in the original
SEEK-at-DHT implementation (#3792).

Fix

Exempt ENXIO from the generic handler so the callback falls through to the redirect,
and drop the now-redundant dead check:

     local->op_errno = op_errno;
-    if ((op_ret == -1) && !dht_inode_missing(op_errno)) {
+    if ((op_ret == -1) && !dht_inode_missing(op_errno) && (op_errno != ENXIO)) {
         gf_msg_debug(this->name, op_errno, "subvolume %s returned -1",
                      prev->name);
         goto out;
     }

-    if ((op_ret == -1) && ((op_errno == ENXIO) || (op_errno == EOVERFLOW)))
-        goto out;
-
     if (!op_ret || (local->call_cnt != 1))
         goto out;

A window SEEK_DATA on the source now falls through to dht_rebalance_complete_check,
which reads the source linkto xattr, re-resolves to the destination, and dht_seek2
re-dispatches the SEEK there. For a non-migrating file the check reads ENODATA (no
linkto) and unwinds the original errno unchanged, so legitimate end-of-data ENXIO is
preserved.

Only ENXIO is exempted (the dead check named EOVERFLOW too, but that is not
migration-related). ENXIO from SEEK_DATA/SEEK_HOLE means "offset ≥ EOF", so it is
data-dependent — the truncated 0-byte source raises it while the full destination does
not, which is exactly why redirecting helps. EOVERFLOW means "the seek result does not
fit off_t" (fs/read_write.c ksys_lseek), which is subvolume-independent — the
destination would return the same EOVERFLOW — so redirecting it is pointless; and with a
64-bit off_t it cannot arise from SEEK_DATA/SEEK_HOLE at all.

Cost / trade-off

ENXIO is also the normal end-of-data signal for SEEK_DATA/SEEK_HOLE, so this adds one
syncop_getxattr(linkto) on the source per terminal ENXIO, even for non-migrating files —
the same check already done for ENOENT/ESTALE, but hit far more often (sparse scanners
seek to EOF once per file). A cheaper in-memory gate is not available: SEEK has no iatt,
and during the in-progress window the inode's migration info is not yet set, so
dht_inode_ctx_get_mig_info would not catch it. The cost is accepted because the
alternative is a silent wrong result; reviewers who prefer to bound it may want a volume-level
"rebalance active" guard, which is out of scope for this fix.

Impact

Low severity (narrow trigger) but a silent wrong result, not a benign retry. The GlusterFS
SEEK FOP only carries GF_SEEK_DATA/GF_SEEK_HOLE (gf_seek_what_t), so the exposed
workloads are sparse-extent scanners (cp --sparse, tar -S, qemu-img), which treat
SEEK_DATA=ENXIO as end-of-data and stop — silently producing a short/all-holes copy.
On the same fd during the same window fstat/read return correct data (they redirect),
so SEEK is the only FOP that leaks the source's ENXIO.

Testing

A regression test is included: tests/bugs/distribute/seek-during-rebalance.t. It holds
an fd open on a file whose data is on brick 0, migrates it to brick 1 with an external
remove-brick (so the held fd is not re-resolved), and asserts the held-fd SEEK_DATA
returns the data offset (0) rather than the source's spurious ENXIO. fstat() on the
same fd is a control — it redirects on both builds, keeping the failure isolated to SEEK.

• Unpatched: not ok … 'Got "ENXIO" instead of "0"' (SEEK_DATA and SEEK_HOLE), fstat ok.
• Patched: all pass (SEEK_DATA 0, SEEK_HOLE <size>, fstat ok); deterministic across
  repeated runs.

(The same behavior was also confirmed manually on a 2-node volume, and — to make the
timing deterministic — by freezing the rebalance daemon in the truncate→unlink window
under gdb; the committed .t reproduces it without that, using the fact that the held fd
keeps hitting the emptied source until something re-resolves it.)

Change summary

• xlators/cluster/dht/src/dht-inode-read.c — dht_seek_cbk (1 insertion, 4 deletions)
• tests/bugs/distribute/seek-during-rebalance.{t,py} — regression test (new)