asyncdispatch: activeDescriptors, shouldAcceptRequest doesn't work

[timotheecour]

While looking into the asynchttpserver example (eg #16599 + recent PR's), I noticed that activeDescriptors (and hence shouldAcceptRequest) doesn't seem to work.

Example

when true:
  import asynchttpserver, strformat
  import asyncdispatch
  proc main {.async.} =
    const port = 8080
    var server = newAsyncHttpServer()
    proc cb(req: Request) {.async.} =
      await sleepAsync(20000) # if comment this out, will still fail if enough requests come fast enough
      await req.respond(Http200, "Hello World")

    server.listen(Port(port))
    while true:
      let num = activeDescriptors()
      let ok = server.shouldAcceptRequest(assumedDescriptorsPerRequest = 1)
      let maxi = maxDescriptors()
      echo (ok: ok, maxi: maxi, num: num)
      if ok:
        await server.acceptRequest(cb)
        echo "after"
      else:
        echo "before poll"
        poll()
        echo "after poll"

  waitFor main()

ulimit -n 13 # adjust as needed for your OS; small enough to make it easier to reproduce bug
nim r main
# in another tab:
nim r -d:nimAsyncHttpServerEnableTest -d:case3 --lib:lib $timn_D/tests/nim/all/t11653.nim

Current Output

Error: unhandled exception: Too many open files is raised after < 1 second, eg:

(ok: true, maxi: 12, num: 0)
after
(ok: true, maxi: 12, num: 1)
after
(ok: true, maxi: 12, num: 1)
after
(ok: true, maxi: 12, num: 1)
after
(ok: true, maxi: 12, num: 1)
after
(ok: true, maxi: 12, num: 1)
after
(ok: true, maxi: 12, num: 1)
after
(ok: true, maxi: 12, num: 1)
/Users/timothee/git_clone/nim/timn/tests/nim/all/t11653.nim(123) t11653
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/asyncdispatch.nim(1935) waitFor
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/asyncdispatch.nim(1627) poll
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/asyncdispatch.nim(1368) runOnce
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/asyncdispatch.nim(208) processPendingCallbacks
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/asyncmacro.nim(29) acceptRequestNimAsyncContinue
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/asyncmacro.nim(145) acceptRequestIter
/Users/timothee/git_clone/nim/Nim_devel/lib/pure/asyncfutures.nim(372) read
...

Expected Output

works

Additional Information

1.5.1 d2f4f25

Not sure how to fix this.
=> see RFC nim-lang/RFCs#382

notes

Using await sleepAsync(500) doesn't change anything (still crashes the server), the problem is that activeDescriptors, and hence shouldAcceptRequest , is incorrect (under-reports).

note that if you comment out await sleepAsync(20000), it'll still crash but will take more time to crash it, eg this will crash it: for i in {1..2000}; do curl localhost:8080/&; ; done

with:

(ok: true, maxi: 12, num: 3)
after
(ok: true, maxi: 12, num: 4)
after
(ok: true, maxi: 12, num: 5)
after
(ok: true, maxi: 12, num: 5)
after
(ok: true, maxi: 12, num: 6)

and then crashes

using let ok = server.shouldAcceptRequest() instead of let ok = server.shouldAcceptRequest(assumedDescriptorsPerRequest = 1) (ie, using the default 5) doesn't help

links

• The asychttpserver dies when a reverse proxy is used. #18161
• improve shouldAcceptRequest timotheecour/Nim#750

[Araq]

How do you know that assumedDescriptorsPerRequest = 1 is correct? I tested the code as I wrote it and it worked fine.

[timotheecour]

as menetioned in the issue:

using let ok = server.shouldAcceptRequest() instead of let ok = server.shouldAcceptRequest(assumedDescriptorsPerRequest = 1) (ie, using the default 5) doesn't help

I tested the code as I wrote it and it worked fine.

did you test this issue? see in particular the ulimit -n 13 (to make the issue easier to reproduce)

one problem seems that activeDescriptors isn't incremented on each accepted connection.