Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

neofs-storage.service gets continuously restarting when internal interface is down #1893

Closed
ikovalev1 opened this issue Oct 13, 2022 · 0 comments · Fixed by #1968
Closed
Assignees
Labels
bug Something isn't working U0 Needs to be resolved immediately
Milestone

Comments

@ikovalev1
Copy link

ikovalev1 commented Oct 13, 2022

Expected Behavior

If a node loses internal network, it should continiue to respond on data network via gRPC/HTTP/S3

Current Behavior

neofs storage gets restarting endlessly every 5 sec (systemd configuration):
systemctl status neofs-storage.service ... Active: activating (auto-restart) (Result: exit-code) since Thu 2022-10-13 12:14:49 UTC; 3s ago

Steps to Reproduce (for bugs)

  1. there are 4 nodes with 2 interfaces (int and data)
  2. create a public container with 1 replication, put an object in the container:
    neofs-cli -r node1:8080 -w wallet.json container create --policy "REP 1 IN Node1 SELECT 1 FROM N1 AS Node1 FILTER Node EQ node1 AS N1" --await --basic-acl public-read-write neofs-cli -r node1.data:8080 -w wallet.json object put --file /home/service/test.txt --no-progress --cid Fk89aSd6YErcmGyf9UMvKx4bAwXbGBJCSvG4Zza7U5nK Enter password > [/home/service/test.txt] Object successfully stored OID: hR4kM3aPD4icZuPPFKmoocCogHep4HfJjMfcTG2spUY CID: Fk89aSd6YErcmGyf9UMvKx4bAwXbGBJCSvG4Zza7U5nK
  3. Verify that you can read the object fron other nodes via data network:
    neofs-cli -r node1.data:8080 -w wallet.json object get --cid Fk89aSd6YErcmGyf9UMvKx4bAwXbGBJCSvG4Zza7U5nK --oid hR4kM3aPD4icZuPPFKmoocCogHep4HfJjMfcTG2spUY --file /tmp/get_object.sample --no-progress
  4. bring down the internal interface on node1
    ifdown int
  5. Try to read the object again:
    oot@buky:/home/service# neofs-cli -r node1.data:8080 -w /etc/neofs/storage/wallet.json object put --file /home/service/test.txt --no-progress --cid Fk89aSd6YErcmGyf9UMvKx4bAwXbGBJCSvG4Zza7U5nK Enter password > create session: can't fetch network info: write request: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 172.26.161.238:8080: connect: connection refused"
  6. Look if neofs-storage is running on node1:
    root@az:/home/service# systemctl status neofs-storage.service Active: activating (auto-restart) (Result: exit-code) since Thu 2022-10-13 12:14:49 UTC; 3s ago

Your Environment

sbercloun 4 VM with 2 interfaces
root@az:/home/service# uname -a
Linux az 5.10.0-18-amd64 #1 SMP Debian 5.10.140-1 (2022-09-02) x86_64 GNU/Linux

root@az:/home/service# neofs-cli --version
NeoFS CLI
Version: v0.32.0-34-g76cfcc24
GoVersion: go1.18.4

root@az:/home/service# neofs-node --version
NeoFS Storage node
Version: v0.32.0-34-g76cfcc24
GoVersion: go1.18.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working U0 Needs to be resolved immediately
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants