Adds support for listening on and connecting to I2P and Onion services securely

[eyedeekay]

This PR adds 2 dendrite-demo main's, each designed expressly to serve a Hidden Service/Overlay network.

The first, dendrite-demo-i2p add self-configuration for use of dendrite as an I2P hidden service(eepsite) and to connect to I2P services(federate) as an I2P client. It further disables the dendrite server from communicating with non-anonymous servers by federation(because I2P does not canonically have the ability to exit, we rely on donors for exit traffic), and enables the use of self-signed TLS certificates(because I2P services are self-authenticating but TLS is still required for other aspects of the system to work reliably). This demo turns the system into an "pseudonymous" homeserver which people can connect to using an I2P-enabled Matrix client(I like cinny and it's what I tested with).

The second, dendrite-demo-tor adds self-configuration for the use of dendrite as an Onion service and to connect to other onion services and non-anonymous web sites using Tor to obfuscate it's physical location and providing, optionally, pseudonymity. It also enables the use of self-signed TLS certificates, for the same reason as with I2P, because onion services aren't typically eligible for TLS certificates. It has also been tested with cinny.

These services are both pseudonymous like myself, not anonymous. I will be meeting members of the element team at the CCC assembly shortly to discuss contributing under my pseudonym.

As none of the other dendrite-demo have unit tests I did not add them to these checkins.

• [*] I have added Go unit tests or Complement integration tests for this PR or I have justified why this PR doesn't need tests

[eyedeekay]

I hereby donate this code to Anoa.

[anoadragon453]

I accept this donation and sign off on this code.

Signed-off-by: Andrew Morgan <andrewm@amorgan.xyz>

[eyedeekay]

Moving this from README_I2P.md to here becasue README_I2P.md is about to me mostly irrelevant.

How to build a Dendrite Homeserver modified to run over I2P or Tor
==================================================================

I2P mode: I2P must be installed first, and the SAMv3 API bridge must
be activated.

1. First, clone the `matrix-org/dendrite` implementation of dendrite into your GOPATH and change branch to the `i2p-demo` checkout.


2. Second, build the binary:

    go build -o bin/dendrite-demo-i2p ./cmd/dendrite-demo-i2p

3. Third, run it.

    go build -o bin/dendrite-demo-i2p ./cmd/dendrite-demo-i2p

Tor mode: Tor must be installed first.

1. First, clone the `matrix-org/dendrite` implementation of dendrite into your GOPATH and change branch to the `i2p-demo` checkout.

2. Second, build the binary:

    go build -o bin/dendrite-demo-tor ./cmd/dendrite-demo-tor

3. Third, run it.

    go build -o bin/dendrite-demo-tor ./cmd/dendrite-demo-tor

[anoadragon453]

(@eyedeekay I can officially confirm that the code donation above checks out 🙂)

[eyedeekay]

I have demo instances up and running but I'm not sure how best to share them, as I have enabled open registration for now and don't think that I should make them that public. Let me know if there's a best way to share them with testers on your side.

[S7evinK]

Sorry for just moving this to contrib, as much as I like this contribution, I don't see us maintain this.
Hope this is OK with you, @eyedeekay

[nihilist001]

Hi, see how I suggest this to be implemented here : element-hq/synapse#7088

Apparently the synapse devs can't do much about it, maybe you guys would take a better shot at it ?

[S7evinK]

As I'm not too familiar with the code, can you take a look at the linter issue -> Error: var samAddr is unused (unused) @eyedeekay. Is that expected?

[eyedeekay]

@nihilist001 it's definitely easier, IMO, to support hidden services well in the Go language than it is in the python languate, however that's simply my opinion based on the methods I am familiar with. This PR applies those methods.

@S7evinK Nope, looks like I forgot to remove the --samaddr flag when I converted the I2P demo into a Tor demo. The checkin I just made should resolve that if that's the case. I'll have a look at these other unit tests too.

[eyedeekay]

I don't see any way to fix the other tests when running on the CI, however they work locally. The unit tests are currently failing because there isn't a Tor or I2P client running on the CI server where the tests can see it. I also don't want to remove them completely, because using them locally is in fact useful. Is there a way to simply disable them in the CI but leave the tests intact for local use?

[S7evinK]

I don't see any way to fix the other tests when running on the CI, however they work locally. The unit tests are currently failing because there isn't a Tor or I2P client running on the CI server where the tests can see it. I also don't want to remove them completely, because using them locally is in fact useful. Is there a way to simply disable them in the CI but leave the tests intact for local use?

You probably can do something like:

if os.Getenv("CI") != "" {
    t.Skip("skipping test, as no TOR/I2P client is available")
  }

in the test.

[eyedeekay]

If we can let them run I think the CI should all pass now.

[anoadragon453]

@eyedeekay heads up you have some CI failures

[eyedeekay]

Yeah for some reason it's still running the failing tests, I'll have to find some other way to keep them from running

[eyedeekay]

Well that was weirder than it should have been, lol. Turns out disabling the unit test just ran the main without any of the instrumentation, so the only way to fix it was to look for the CI in the demo application being tested. Works now though, locally and in CI. I see you fixed the linter errors so I'll leave it here until you have more feedback.

[eyedeekay]

The demand for something like matrix-over-I2P is really growing from my side of things. As a result, I changed one fundamental thing about how this PR works I2P hosted dendrite servers. In the latest checkin, I changed it so that:

• in the case where it attempts to make a server-to-server connection to a hostname which does not end in .i2p
• it will attempt to contact it over Tor instead
• allowing the I2P version to communicate with I2P, Onion, and regular-internet hosts if configured to do so.

I had not intended to do this until later in a future PR, but I think the demand for intra-overlay-network communication will be present fairly early when I deploy the I2P demo of Dendrite and if it's all the same to you, I'd rather do it in this PR.

[eyedeekay]

At some point my onion-on-TLS listener support broke and I didn't notice until I tried running the onion version of this again. It is now fixed in onramp library 0.33.8 and I have also updated this PR to use the new library version.