bind to interface still exposes searches

[aleqx]

I like the bind to interface feature but the searches are still sent through the main NIC, regardless of which NIC qBT is bound to (I did restart).

I'm using Windows 8.1 Pro x64. Used Wireshark to reveal this.

I noticed searches are done by launching a separate Python process, which doesn't look bound to the IP from the preferences. Python can be bound to an interface and I would expect qBT to do that if the option is set in the settings.

I noticed each search engine uses a python script. You can just embed that into a Python wrapper that first binds Python to a specific network interface.

I understand it's extra work but right now the "bind to interface" feature exposes private activity. If a user activates "bind to interface" then they know what they are doing and expect all the comms to go through that NIC, including searches, i would say.

[DoumanAsh]

I suppose it is possible to do, but that would requires some changes in a way how to invoke search engine.. and it is also may be quite tricky to do in python code(cuz it is platform dependent).
I'm not familiar how exactly QBt binds to interface though, may need to look into that.

[aleqx]

Actually it should not be very difficult. I haven't looked at the code myself, but qBT fetches the current IP of the NIC it binds to (if the binding option is enabled), then binds to that IP - it doesn't actually bind to the NIC, which is why it requires restating qBT if the NIC IP changes.

Once the IP is fetched, it can be passed to Python. In Python you can set the source IP for the http request, it can be done with native code or via some available packages. See here for examples:

http://stackoverflow.com/questions/1150332/source-interface-with-python-and-urllib2
https://github.com/mrahma01/py-outgoing-ip

Then you can wrap the existing python code for the search.

[DoumanAsh]

The second one is just getting IP address :)
Anyway it will take some thinking about how to make it in a good and pythonic way.

Less-user friendly HTTPConnection(s) provides a way to specify source address without hacks, but we would need to remove urlib from scripts at all which is a bit bothersome

[aleqx]

Sorry, somehow I pasted the wrong link. The core httplib package is able to bind to a source IP, implemented after a suggestion: https://bugs.python.org/issue3972

It's now available in all Python 2.7.*: https://docs.python.org/2/library/httplib.html#httplib.HTTPConnection

Or if you want to use urllib then see the 1st link in my post above.

[sledgehammer999]

That binding option is actually passed to libtorrent and it only handles the bittorrent connections.
rss/search/program updates/etc uses the default route.

@DoumanAsh I remember that qbt passes the proxy settings to the (python) search code too, so the connections can be setup correctly by the plugins. Maybe the interface info should be passed in a similar way.

[DoumanAsh]

@sledgehammer999
Hmm... as far as i can see search engine takes it from os env "sock_proxy"
nova2.py does not expect to receive settings in its invocation

This env variable is set when proxy settings are being updated
I suppose there should be no problem to do it in a similar way. Though i'm not sure how to handle correctly the case when we have both proxy settings and bind to specific interface.
Actually we may need to check how it works right now... But it would be good to do after my big refactoring of search engine... :)

@sledgehammer999
How do you thin we should handle case with both bind to interface and proxy?(I suppose we should give priority to proxy?)
The easy way would be to set new variable QBT_PY_IP_TO_USE="IP address"
But i'm not really fond of such solution...

There is also problem that any sorta hacks to python sockets would require careful testing on all platforms(unix and mac should not be a problem, but i'm afraid win can cause some headache)

P.s. i think some plugins cannot use SOCKS5 proxy now... i'm at fault :D

[sledgehammer999]

we have both proxy settings and bind to specific interface.

Separate env vars for each of these.

How do you thin we should handle case with both bind to interface and proxy?

Maybe I am missing something, but can we tell it to contact the proxy via X interface?

There is also problem that any sorta hacks to python sockets would require careful testing on all platforms(unix and mac should not be a problem, but i'm afraid win can cause some headache)

I presume that if you use portable python code (do they have a standard lib) it should work ok with Windows if it already works ok with linux+mac.

[DoumanAsh]

Oh... yes... proxy's address is remote while we bind to some local IP address.
I suppose yes, it should not be a problem to do. I just need to think about how to do it properly

[DoumanAsh]

@aleqx
Can you please download nova3 from my repository https://github.com/DoumanAsh/qBittorrent/tree/py_use_interface/src/searchengine/nova3
And replace with it your nova3(make backup of original nova3)

Once you'll have time please set new environment variable QBT_PY_IP_TO_USE=ip adress (valid for some interface)
And check if it works correctly :)

p.s. let me know if you're using py2. I made it for python3 only for now :)

[DoumanAsh]

@aleqx ping in case if you're still interested.

[aleqx]

Oops, I didn't notice the updates until now. Yes, I'm still interested. I downloaded the latest v3.2.x and was hoping it's already implemented but alas ...

I'll get your nova3 version. However, must I have QBT_PY_IP_TO_USE=<static_ip> set as a global env var? That would defeat the purpose a little ... in my case for instance, this IP changes as I connect and disconnect to a VPN.

Why not obtained from the running qBT instance which already got it? Can we query qBT?
Otherwise, why not get it from an interface name (like qBT does) instead of specifying it? (yes, it would be a little OS specific, but it's not much work)

I'll post after I try it.

[DoumanAsh]

It is mostly for test purposes in order to understand if i done it correctly.

If it so then we'll need to think of the best way to deliver it into plugins

[aleqx]

Is it just helpers.py that you changed?

p.s. Also, I'm on Windows (if that affects anything).

[DoumanAsh]

Yes. There was link to my commit with diff.

[aleqx]

I'm hit by the old qBT bug in which only btdigg.org returns results (I never really found out why) ... there were reports about this in the forum too a long while back but nothing was done. It comes back to normal after a random while. This has nothing to do with the current nova3 (the original one does the same). I'll post back after it works correctly ... too busy to use sharkwire

[aleqx]

This doesn't seem to work for me under windows. Have you actually tested it yourself? If yes, did it work for you? If yes, what OS, and how did you confirm that it actually work? Wireshark?

It also looks like some tracker comms is still done via the main nic, instead of the bound nic, according to my Wireshark traces.

The option to bind to interface is obviously there for privacy reasons, but if tracker and search comms are still seen via the main nic and seen by the ISP, then there is not so much point in having the bind to nic option ... ALL comms must go through the bound nic.

[aleqx]

Maybe it's a Python3 issue ... if you have time to patch the Python2 version of nova then I'll give that a try as well

[aleqx]

Ok, so I applied the same patch to nova/helpers.py (for Python2) which I'm now using and I think I discovered why it doesn't work ... my bound IP is a VPN and most of the BT search engines are now using CloudFlare for DNS, which redirects the http/https call to a CAPTCHA before opening the actual bt search website ...

Soon enough using a VPN will be more painful/annoying than without

[DoumanAsh]

Hm... Python should use the one you specified in Environment variable, is it not one your set or?

[aleqx]

See my last post above. It appears to work, the issue is with CloudFlare/VPN.

[DoumanAsh]

Oh... Sorry there were a lot of text and i read by diagonal :D

@sledgehammer999
What do you think about how should we specify IP used by python searches?

[aleqx]

My vote is certainly to have it as an option in the advanced settings, right below the existing "bind to interface" option. Make it a checkbox saying "Use this interface also for searches?"

Unrelated: is qBT at the moment using the bound interface for communicating with the trackers? Or does it use the default interface for trackers? I haven't checked this one ... if it's not using the bound interface then it really should! :)

[aleqx]

Would anyone care to give this some attention to implement it so that search, tracker comms, etc is done through the same interface/IP that is specified as the bound one?

@DoumanAsh you have a typo in your .bind hack (**l -> **k)

[DoumanAsh]

Aww... i kinda forgot about this commit.
Do we still need it?

[aleqx]

Has this been integrated so that searches also use whatever IP/interface is bound to in the qBT settings?

[sledgehammer999]

This issue has been closed and locked for being too old, and thus either most likely resolved in recent versions or no longer applicable.
If you experience the reported problem or similar in the latest version, please open a new issue report with the requested information in the issue template.

A new issue report with relevant updated data gathered from the latest version is preferable to necroing an old report with a comment like "still happens in version x.y.z", even if you think the bug is the same, or suspect of a regression.
Due to the changes made to the qBittorrent code and its dependencies over time, the exact cause of your problem could be totally different than the original one, despite the visible symptoms of the bug being similar.
Thus, providing relevant updated information is crucial to find and fix the root cause of a recurrent problem or regression.

Thank you for your contributions.