Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(icos): First phase of the firewall setup feature (DRE-258) #1451

Open
wants to merge 66 commits into
base: master
Choose a base branch
from

Conversation

DFINITYManu
Copy link
Contributor

@DFINITYManu DFINITYManu commented Sep 11, 2024

Per https://dfinity.atlassian.net/browse/DRE-258 .

We have successfully loaded rules of well-formed /boot/config/firewall.json in HostOS and GuestOS.

Details:
We would like to allow the node operator / provider to modify the firewall rules in order to allow incoming traffic to their nodes. This can be used to get the ability to fetch logs and metrics from the nodes in the same DC.

To achieve this, the node provider should add a new config file named firewall.json in the SetupOS config. This file is then propagated to the other parts of the IC-OS stack.

Note that the firewall rules can only be configured during node (re)deployment, in this phase. In phase 2, it will become possible to reconfigure already running nodes. Phase 2 will be more effort than phase 1.

This PR rides atop this other one: #2077

@DFINITYManu DFINITYManu changed the title DRE-252: First phase of the firewall setup feature feat(icos) DRE-252: First phase of the firewall setup feature Sep 11, 2024
@DFINITYManu DFINITYManu changed the title feat(icos) DRE-252: First phase of the firewall setup feature feat(icos): First phase of the firewall setup feature (DRE-258) Sep 11, 2024
@github-actions github-actions bot added the feat label Sep 11, 2024
@DFINITYManu DFINITYManu force-pushed the firewallingsetupos branch 8 times, most recently from 31603d7 to ca53f81 Compare September 12, 2024 15:46
rs/ic_os/config/src/types.rs Outdated Show resolved Hide resolved
ic-os/setupos/data/deployment.json.template Outdated Show resolved Hide resolved
ic-os/components/setupos-scripts/devices.sh Outdated Show resolved Hide resolved
ic-os/components/setupos-scripts/devices.sh Outdated Show resolved Hide resolved
rs/ic_os/config/src/types/firewall.rs Show resolved Hide resolved
rs/ic_os/config/src/types/firewall.rs Show resolved Hide resolved
rs/ic_os/config/src/firewall_json.rs Show resolved Hide resolved
rs/ic_os/config/src/lib.rs Outdated Show resolved Hide resolved
rs/ic_os/utils/Cargo.toml Outdated Show resolved Hide resolved
@DFINITYManu DFINITYManu force-pushed the firewallingsetupos branch 2 times, most recently from 2c44825 to c849b6e Compare September 13, 2024 17:22
@DFINITYManu DFINITYManu force-pushed the firewallingsetupos branch 5 times, most recently from 773291f to f9c86eb Compare September 13, 2024 18:32
@DFINITYManu
Copy link
Contributor Author

DFINITYManu commented Sep 13, 2024

We don't yet do anything with the firewall.json file, but all this work, already, is enormous just to pass a silly config file down the pipe. Horrible. I am so looking forward to your work @andrewbattat enabling the use of the configuration structures making this job enormously easier and deleting so much code from our codebase.

@DFINITYManu DFINITYManu force-pushed the firewallingsetupos branch 2 times, most recently from 9121ce5 to 0ec762c Compare September 13, 2024 19:00
Copy link
Contributor

Vulnerable dependency information
The dependency-check job for the MR has new findings. Please update or remove these dependencies or obtain a commit exception from product security.

The findings are:
[Finding(repository='ic', scanner='BAZEL_RUST', vulnerable_dependency=Dependency(id='https://crates.io/crates/mio', name='mio', version='0.8.10', fix_version_for_vulnerability={'https://rustsec.org/advisories/RUSTSEC-2024-0019': ['>=0.8.11', '<0.7.2']}), vulnerabilities=[Vulnerability(id='https://rustsec.org/advisories/RUSTSEC-2024-0019', name='RUSTSEC-2024-0019', description='Tokens for named pipes may be delivered after deregistration', score=-1, risk_note=' ')], first_level_dependencies=[], projects=[], risk_assessor=[], risk=None, owning_teams=[], patch_responsible=[], due_date=None, score=-1, more_info=None), Finding(repository='ic', scanner='BAZEL_RUST', vulnerable_dependency=Dependency(id='https://crates.io/crates/rsa', name='rsa', version='0.9.2', fix_version_for_vulnerability={}), vulnerabilities=[Vulnerability(id='https://rustsec.org/advisories/RUSTSEC-2023-0071', name='RUSTSEC-2023-0071', description='Marvin Attack: potential key recovery through timing sidechannels', score=5, risk_note=' ')], first_level_dependencies=[], projects=[], risk_assessor=[], risk=None, owning_teams=[], patch_responsible=[], due_date=None, score=5, more_info=None)]

@venkkatesh-sekar
Copy link
Member

venkkatesh-sekar commented Sep 26, 2024

Vulnerable dependency information

This was due to a mixup in testing. Should be resolved now.

In case of absence of the default rules file, an empty text is rendered
(empty text is valid input for the `nft` command).  If the rules file is
explicitly specified, and it does not exist, the subcommand exits with
an error.

When rules are in fact generated, in case the nftables chains they depend
upon are not defined or referenced anywhere, the chains are created prior
to flushing.  This prevents errors of coordination where perhaps the system
rules (which create the chains) have not yet been added to the system.

Both commands default to their respective OSes' config partitions to search
for the file in question (`firewall.json`).  This file is added by the
SetupOS into HostOS, and later propagated by HostOS into GuestOS when
the GuestOS is bootstrapped.

This commit does not yet incorporate the mechanisms that will actually load
the rules into the kernel.  That part is still being developed.
…config`.

Proof from HostOS:

```
Ubuntu 20.04.6 LTS localhost ttyS0

localhost login: root
Password:
root@localhost:~# systemctl status nftables.service | cat
● nftables.service - nftables
     Loaded: loaded (/lib/systemd/system/nftables.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/nftables.service.d
             └─nftables-add-operator-rules.conf
     Active: active (exited) since Mon 2024-09-30 12:55:02 UTC; 4min 2s ago
       Docs: man:nft(8)
             http://wiki.nftables.org
    Process: 277 ExecStart=/usr/sbin/nft -f /etc/nftables.conf (code=exited, status=0/SUCCESS)
    Process: 332 ExecStart=/usr/bin/bash -c set -o pipefail ; /opt/ic/bin/hostos_tool render-firewall-config | nft -f - (code=exited, status=0/SUCCESS)
   Main PID: 332 (code=exited, status=0/SUCCESS)

Sep 30 12:55:01 localhost bash[335]: Firewall config (from default /boot/config/firewall.json): None
Sep 30 12:55:02 localhost systemd[1]: Finished nftables.
Warning: journal has been rotated since unit was started, output may be incomplete.
root@localhost:~#
```

Similar with GuestOS.
An invalid ruleset may cause the system to be in an insecure state post-install.
…g because who knows thought this was a good idea.
Base automatically changed from testingimprovements to master November 11, 2024 16:00
Copy link
Contributor

@aterga aterga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the changes under rs/registry/canister/Cargo.toml

@DFINITYManu
Copy link
Contributor Author

Firewall rules loaded successfully in installed HostOS.

From this config:

[
    {
        "from": "2001:db8:abcd:0013::0/64",
        "to": "HostOS",
        "protocol": "tcp",
        "action": "accept"
    }
]

we get this result (see chain provider_INPUT):

root@host-696e76697274:~# nft list table ip6 filter
table ip6 filter {
	set rate_limit {
		type ipv6_addr
		size 65535
		flags dynamic
	}

	set connection_limit {
		type ipv6_addr
		size 65535
		flags dynamic
	}

	set dfinity_dcs {
		type ipv6_addr
		flags interval
		auto-merge
		elements = { 2001:438:fffd:11c::/64,
			     2001:470:1:c76::/64,
			     2001:920:401a:1706::/64,
			     2001:920:401a:1708::/64,
			     2001:920:401a:1710::/64,
			     2001:1900:2100:2827::/64,
			     2001:4d78:400:10a::/64,
			     2001:4d78:40d::/48,
			     2401:3f00:1000:22::-2401:3f00:1000:24:ffff:ffff:ffff:ffff,
			     2600:c00:2:100::/64,
			     2600:c02:b002:15::/64,
			     2600:c0d:3002:4::/64,
			     2600:2c01:21::/64,
			     2600:3000:1300:1300::/64,
			     2600:3000:6100:200::/64,
			     2600:3004:1200:1200::/56,
			     2600:3006:1400:1500::/64,
			     2602:fb2b:100::/48,
			     2602:fb2b:110::/48,
			     2602:fb2b:120::/48,
			     2602:ffe4:801:16::-2602:ffe4:801:18:ffff:ffff:ffff:ffff,
			     2604:1380:4091:3000::/56,
			     2604:1380:40e1:4700::/56,
			     2604:1380:45e1:a600::/56,
			     2604:1380:4601:6200::/56,
			     2604:1380:4641:6100::/56,
			     2604:3fc0:2001::/48,
			     2604:3fc0:3002::/48,
			     2604:6800:258:1::/64,
			     2604:7e00:30:3::/64,
			     2604:7e00:50::/64,
			     2604:b900:4001:76::/64,
			     2607:f1d0:10:1::/64,
			     2607:f6f0:3004::/48,
			     2607:f758:1220::/64,
			     2607:f758:c300::/64,
			     2607:ff70:3:2::/64,
			     2610:190:6000:1::/64,
			     2610:190:df01:5::/64,
			     2a00:fa0:3::/48,
			     2a00:fc0:5000:300::/64,
			     2a00:fb01:400::/55,
			     2a01:138:900a::/48,
			     2a01:2a8:a13c::-2a01:2a8:a13e:ffff:ffff:ffff:ffff:ffff,
			     2a02:418:3002::/48,
			     2a02:41b:300e::/48,
			     2a02:800:2:2003::/64,
			     2a04:9dc0:0:108::/64,
			     2a0b:21c0:4003:2::/64,
			     2a0b:21c0:4006:100::/56,
			     2a0b:21c0:b002:2::/64,
			     2a0f:cd00:2::/56,
			     fd00:2:1:1::/64 }
	}

	set telemetry_clients {
		type ipv6_addr
		flags interval
		elements = { 2001:4d78:40d::/48,
			     2602:fb2b:100::/48,
			     2602:fb2b:110::/48,
			     2602:fb2b:120::/48,
			     2607:f6f0:3004::/48 }
	}

	set node_providers {
		type ipv6_addr
		flags interval
	}

	chain metrics_proxy {
		ct state new add @rate_limit { ip6 saddr limit rate over 100/minute burst 5 packets } drop comment "Maximum 100 connections per minute"
		ct state new add @connection_limit { ip6 saddr ct count over 6 } drop comment "No more than 6 connections per source at a time"
		accept
	}

	chain provider_INPUT {
		ip6 saddr 2001:db8:abcd:13::/64 ct state new tcp flags syn accept
	}

	chain INPUT {
		type filter hook input priority filter; policy drop;
		iif "lo" accept
		ct state invalid drop
		ct state { established, related } accept
		icmpv6 type destination-unreachable accept
		icmpv6 type packet-too-big accept
		icmpv6 type time-exceeded accept
		icmpv6 type parameter-problem accept
		icmpv6 type echo-request accept
		icmpv6 type echo-reply accept
		icmpv6 type nd-router-advert accept
		icmpv6 type nd-neighbor-solicit accept
		icmpv6 type nd-neighbor-advert accept
		ip6 saddr @dfinity_dcs ct state new tcp dport { 22, 9100, 19100, 19531 } accept
		ip6 saddr @telemetry_clients ct state new tcp dport { 9100, 19100, 19531 } accept
		ip6 saddr @node_providers ct state new tcp dport { 22, 9100, 19531 } accept
		jump provider_INPUT
		tcp dport 42372 goto metrics_proxy
	}

	chain FORWARD {
		type filter hook forward priority filter; policy drop;
	}

	chain OUTPUT {
		type filter hook output priority filter; policy drop;
		oif "lo" accept
		ct state invalid drop
		ct state { established, related } accept
		icmpv6 type destination-unreachable accept
		icmpv6 type packet-too-big accept
		icmpv6 type time-exceeded accept
		icmpv6 type parameter-problem accept
		icmpv6 type echo-request accept
		icmpv6 type echo-reply accept
		icmpv6 type nd-router-solicit accept
		icmpv6 type nd-neighbor-solicit accept
		icmpv6 type nd-neighbor-advert accept
		ip6 daddr ::/0 ct state new tcp dport 53 accept
		ip6 daddr ::/0 ct state new udp dport 53 accept
		ip6 daddr ::/0 ct state new udp dport 123 accept
		ip6 daddr ::/0 ct state new tcp dport { 80, 443, 8080 } accept
		ip6 daddr ::/0 ct state new tcp dport 42372 accept comment "Permit outbound connections to metrics-proxy instances so local metrics-proxy can fetch data from GuestOS 
metrics-proxy."
	}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants