custom DNS server with aiodns got err: ot Future <Future pending> attached to a different loop

[gainskills]

Long story short

aiohttp doesn't work with custom DNS Server.

The DNS Resolver should works without error

With the error:

/aiohttp-1.py:38: DeprecationWarning: The object should be created from async function
  resolver = AsyncResolver(nameservers=["114.114.114.114"])
/aiohttp-1.py:41: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
  async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(verify_ssl=False, use_dns_cache=False, resolver=resolver)) assession:
Traceback (most recent call last):
  File "/aiohttp-1.py", line 51, in <module>
    asyncio.run(dnstesting())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/aiohttp-1.py", line 42, in dnstesting
    r = await session.get("http://google.com")
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/client.py", line 476, in _request
    timeout=real_timeout
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 522, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 854, in _create_connection
    req, traces, timeout)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 955, in _create_d/aiohttp-1.py:38: DeprecationWarning: The object should be created from async function
  resolver = AsyncResolver(nameservers=["114.114.114.114"])
/aiohttp-1.py:41: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
  async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(verify_ssl=False, use_dns_cache=False, resolver=resolver)) assession:
Traceback (most recent call last):
  File "/aiohttp-1.py", line 51, in <module>
    asyncio.run(dnstesting())
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/aiohttp-1.py", line 42, in dnstesting
    r = await session.get("http://google.com")
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/client.py", line 476, in _request
    timeout=real_timeout
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 522, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 854, in _create_connection
    req, traces, timeout)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 955, in _create_direct_connection
    traces=traces), loop=self._loop)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 788, in _resolve_host
    host, port, family=self._family))
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/resolver.py", line 64, in resolve
    resp = await self._resolver.gethostbyname(host, family)
RuntimeError: Task <Task pending coro=<TCPConnector._resolve_host() running at /pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py:788> cb=[shield.<locals>._done_callback() at /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/tasks.py:776]> got Future <Future pending> attached to a different loop
python3(38437,0x107b185c0) malloc: *** error for object 0x9000000000000000: pointer being freed was not allocated
python3(38437,0x107b185c0) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6irect_connection
    traces=traces), loop=self._loop)
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py", line 788, in _resolve_host
    host, port, family=self._family))
  File "/pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/resolver.py", line 64, in resolve
    resp = await self._resolver.gethostbyname(host, family)
RuntimeError: Task <Task pending coro=<TCPConnector._resolve_host() running at /pyvenvs/aiohttp/venv/lib/python3.7/site-packages/aiohttp/connector.py:788> cb=[shield.<locals>._done_callback() at /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/tasks.py:776]> got Future <Future pending> attached to a different loop
python3(38437,0x107b185c0) malloc: *** error for object 0x9000000000000000: pointer being freed was not allocated
python3(38437,0x107b185c0) malloc: *** set a breakpoint in malloc_error_break to debug
Abort trap: 6

Steps to reproduce

Run the code:

import aiohttp
from aiohttp.resolver import AsyncResolver

resolver = AsyncResolver(nameservers=["114.114.114.114"])

async def dnstesting():
    async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(verify_ssl=False, use_dns_cache=False, resolver=resolver)) as session:
        r = await session.get("http://google.com")
        print(r.header)

asyncio.run(dnstesting())

Your environment

Here are the packages I'm using:
Package Version

---

aiohttp 3.5.4
aiodns 2.0.0b0
async-timeout 3.0.1
attrs 18.2.0
cffi 1.11.5
chardet 3.0.4
idna 2.8
multidict 4.5.2
pip 18.1
pycares 3.0.0b2
pycparser 2.19
setuptools 40.6.3
typing 3.6.6
wheel 0.32.3
yarl 1.3.0

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are #2168 (server often throw "concurrent.futures._base.CancelledError"), and #3143 (Problems using custom DNS with aiohttp).

[asvetlov]

I'm sorry.
Your stack trace points on a problem in aiodns project.
Please report to aiodns bugtracker
aiohttp uses aiodns but the project is not responsible for aiodns failures when we use it properly.

[gainskills]

@asvetlov ,

Thanks for your update. Before I submitting the bug, I checked aiodns with the code:

import asyncio
import aiodns

loop = asyncio.new_event_loop()
loop.close
resolver = aiodns.DNSResolver(loop=loop, nameservers=['114.114.114.114'])
f = resolver.query('www.google.com', 'A')
result = loop.run_until_complete(f)
print(result)

and it works.

[asvetlov]

Looks like you have a memory corruption error.
A simple test from your second message can be processed successfully by the chance.
More complex code from the initial message uses much more moving parts and fails by a memory problem.
I've reported to upstream: aio-libs/aiodns#56

[gainskills]

@asvetlov ,
Thanks a lot for the update.

[saghul]

I took a quick look. With a small change it works fine:

import asyncio
import aiohttp
from aiohttp.resolver import AsyncResolver

async def dnstesting():
    resolver = AsyncResolver(nameservers=["114.114.114.114"])

    async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(verify_ssl=False, use_dns_cache=False, resolver=resolver)) as session:
        r = await session.get("http://google.com")
        print(r.header)

asyncio.run(dnstesting())

/aiohttp-1.py:38: DeprecationWarning: The object should be created from async function resolver = AsyncResolver(nameservers=["114.114.114.114"])

Loos like somehow loops gets mixed. In master that will raise RuntimeError. If you replace the resolver with the ThreadedResolver you'll also get the incorrect loop exception, but not the memory corruption message. I suspect it has to do with destructors. I'll look into it, but that's not the cause of this bug.

[asvetlov]

Aha, I see.
AsyncResolver was created in a global scope, with default event loop.
aiohttp 3.5 raises a warning for this mode. On master the warning is replaced with a strict error.
asyncio.run() creates a new event loop to run, that's why loops get mixed.

It is a programming error and aiohttp misusage.
Closing the issue.

@saghul thanks for investigation.