🔍 Deep Dive: Add docker interfaces to firewalld docker zone by arkodg · Pull Request #2548 · moby/libnetwork · GitHub

arkodg · 2020-05-04T20:54:35Z

If firewalld is running, create a new docker zone and
add the docker interfaces to the docker zone to allow
container networking for distros with Firewalld enabled

Fixes: #2496
Addresses docker/for-linux#957

Debug output

[vagrant@centos8 ~]$ uname -a
Linux centos8 4.18.0-80.el8.x86_64 #1 SMP Tue Jun 4 09:19:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[vagrant@centos8 ~]$ sudo firewall-cmd --info-zone docker
docker (active)
  target: ACCEPT
  icmp-block-inversion: no
  interfaces: docker0
  sources: 
  services: 
  ports: 
  protocols: 
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 

[vagrant@centos8 ~]$ sudo docker run -it alpine nslookup www.google.com
  Server:		10.0.2.3
Address:	10.0.2.3:53

Non-authoritative answer:
Name:	www.google.com
Address: 2607:f8b0:4005:80a::2004

Non-authoritative answer:
Name:	www.google.com
Address: 172.217.164.100

Signed-off-by: Arko Dasgupta arko.dasgupta@docker.com

arkodg · 2020-05-04T20:55:14Z

PTAL @yrro @cpuguy83 @thaJeztah

yrro · 2020-05-05T10:16:04Z

IMO, Docker should follow libvirt's example and create its own docker zone, rather than default to putting its interfaces into trusted.

I think you're modifying firewalld's runtime configuration only, so interfaces will be moved back to the default zone when firewalld is reloaded or restarted. libvirt handles this by listening to firewalld's Reloaded signal, and the normal D-Bus NameOwnerChanged signal (to handle firewalld restarts), and re-doing its runtime configuration changes when signalled.

yrro · 2020-05-05T10:19:56Z

It occurs to me that a user might want different interfaces to be created in different zones. A docker zone would be a good default but it would be nice to have a --zone option to docker network create...

arkodg · 2020-05-05T20:42:05Z

@yrro thanks for taking a look !

attempted to create a new docker zone but hitting some issues, following it up with firewalld
libnetwork does handle the reload case -

libnetwork/iptables/firewalld.go

Line 138 in 1ea375d

func OnReloaded(callback func()) {
the notion of zones or just security policy is definitely something that would be nice to have but not sure what the interface would look like yet, and hoping to keep that out of this PR

arkodg · 2020-05-07T00:07:57Z

updated the PR to add docker interfaces to a new docker zone
PTAL @yrro @cpuguy83 @thaJeztah

erig0

First, thanks for adding firewalld integration. This will please users. In general this looks functional. I have various cleanup type comments. But those are non-blocking.

I realize that masquerading and port forwarding is still done via iptables. In the future I think firewalld could be used for those as well. Upstream is currently working on support for forward/output filtering.

iptables/firewalld.go

erig0 · 2020-05-07T18:56:09Z

iptables/firewalld.go

+
+	settings := getDockerZoneSettings()
+	// Permanent
+	if err := connection.sysConfObj.Call(dbusInterface+".config.addZone", 0, dockerZone, settings).Err; err != nil {


Just a note that the addZone() method has been deprecated. The next firewalld feature release introduces addZone2() which is dictionary based and doesn't require all fields be filled. That being said, docker has to support older versions of firewalld so this is the correct one to call. In the future support for addZone2() should be considered.

FWIW, libvirt ships a static zone definition as part of the package. Therefore they avoid the addZone() and reload(). They also allow other things; dhcp, dns, icmp, ipv6-icmp. But they're also filtering (denying) other packets destined to the host.

erig0 · 2020-05-07T19:00:13Z

iptables/firewalld.go

+	// Check if interface is already added to the zone
+	if err := connection.sysObj.Call(dbusInterface+".zone.getInterfaces", 0, dockerZone).Store(&intfs); err != nil {
+		return err
+	}
+	// Return if interface is already part of the zone
+	if contains(intfs, intf) {
+		logrus.Infof("Firewalld: interface %s already part of %s zone, returning", intf, dockerZone)
+		return nil
+	}
+
+	logrus.Debugf("Firewalld: adding %s interface to %s zone", intf, dockerZone)
+	// Runtime
+	if err := connection.sysObj.Call(dbusInterface+".zone.addInterface", 0, dockerZone, intf).Err; err != nil {
+		return err
+	}
+	return nil


You're probably aware, but this is racey. It's possible the interface gets deleted/added by another entity/user between the getInterfaces() and addInterface() calls. It may be simpler to always call addInterface() and handle the error codes appropriately. e.g. ALREADY_ENABLED.

I'd like to keep it this way, since AFAIK I would need to compare the error string to handle the above case, which might change in the future

erig0 · 2020-05-07T19:02:40Z

iptables/firewalld.go

+	if err := connection.sysObj.Call(dbusInterface+".zone.getInterfaces", 0, dockerZone).Store(&intfs); err != nil {
+		return err
+	}
+	// Remove interface if it exists
+	if !contains(intfs, intf) {
+		return fmt.Errorf("Firewalld: unable to find interface %s in %s zone", intf, dockerZone)
+	}
+
+	logrus.Debugf("Firewalld: removing %s interface from %s zone", intf, dockerZone)
+	// Runtime
+	if err := connection.sysObj.Call(dbusInterface+".zone.removeInterface", 0, dockerZone, intf).Err; err != nil {
+		return err
+	}


Similar comment as above. Maybe always call removeInterface() and handle the errors.

same comment as above

If firewalld is running, create a new docker zone and add the docker interfaces to the docker zone to allow container networking for distros with firewalld enabled Fixes: moby#2496 Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>

cpuguy83 · 2020-05-08T15:34:02Z

What happens here if someone already has a pre-configured dockerZone zone?
I see in the code we'll return early in certain places.

arkodg · 2020-05-08T15:59:49Z

If a user manually created a docker zone, libnetwork would just reuse that and insert interfaces into that zone. However exact behavior of networking depends on the zone settings (which libnetwork will not try and override)

thaJeztah · 2020-05-14T19:15:20Z

@arkodg discussing with @cpuguy83 and wondering if we should have an option (configurable on the daemon, and passed to libnetwork) to disable this; wdyt?

arkodg · 2020-05-14T19:21:36Z

So I believe that should happen if the user sets iptables=false

Edit - tested this with iptables=false and no docker zone was created

cpuguy83 · 2020-05-14T21:49:04Z

I think the idea was to not have to set iptables=false but allow users to enable the specific firewalld enhancement (or rather disable it as needed), for example cases where they are already managing this.

I just know that firewall changes Docker does always ends up being problematic.

arkodg · 2020-05-18T20:08:04Z

@cpuguy83 even w/o this change, we have been pushing iptables rules via firewalld
I'm not a fan of creating yet another daemon knob . Although iptables=false is less granular it should be the recommended approach if the user wants to manage their own iptables rules via iptables or firewalld

cpuguy83 · 2020-05-18T20:26:45Z

We've been pushing rules such that firewalld ensures they exist in iptables, but this is changing how firewalld is configured entirely.

arkodg · 2020-05-19T20:43:53Z

@cpuguy83 do you prefer having a firewall-integration flag that can be used for firewalld and ufw, and would it be disabled/enabled by default .

cpuguy83

LGTM

full diff: moby/libnetwork@2e24aed...9e99af2 - moby/libnetwork#2548 Add docker interfaces to firewalld docker zone - fixes docker/for-linux#957 DNS Not Resolving under Network [CentOS8] - fixes moby/libnetwork#2496 Port Forwarding does not work on RHEL 8 with Firewalld running with FirewallBackend=nftables - store.getNetworksFromStore() remove unused error return - moby/libnetwork#2554 Fix 'failed to get network during CreateEndpoint' - fixes/addresses docker/for-linux#888 failed to get network during CreateEndpoint - moby/libnetwork#2558 [master] bridge: disable IPv6 router advertisements - moby/libnetwork#2563 log error instead if disabling IPv6 router advertisement failed - fixes docker/for-linux#1033 Shouldn't be fatal: Unable to disable IPv6 router advertisement: open /proc/sys/net/ipv6/conf/docker0/accept_ra: read-only file system Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

full diff: moby/libnetwork@2e24aed...9e99af2 - moby/libnetwork#2548 Add docker interfaces to firewalld docker zone - fixes docker/for-linux#957 DNS Not Resolving under Network [CentOS8] - fixes moby/libnetwork#2496 Port Forwarding does not work on RHEL 8 with Firewalld running with FirewallBackend=nftables - store.getNetworksFromStore() remove unused error return - moby/libnetwork#2554 Fix 'failed to get network during CreateEndpoint' - fixes/addresses docker/for-linux#888 failed to get network during CreateEndpoint - moby/libnetwork#2558 [master] bridge: disable IPv6 router advertisements - moby/libnetwork#2563 log error instead if disabling IPv6 router advertisement failed - fixes docker/for-linux#1033 Shouldn't be fatal: Unable to disable IPv6 router advertisement: open /proc/sys/net/ipv6/conf/docker0/accept_ra: read-only file system Signed-off-by: Sebastiaan van Stijn <github@gone.nl> Upstream-commit: 219e7e7ddcf5f0314578d2a517fc0832f03622c1 Component: engine

https://build.opensuse.org/request/show/851816 by user mrostecki + dimstar_suse - Add a patch which makes Docker compatible with firewalld with nftables backend. Backport of moby/libnetwork#2548 (boo#1178801, SLE-16460) * boo1178801-0001-Add-docker-interfaces-to-firewalld-docker-zone.patch

idc77 · 2024-05-13T17:35:11Z

Rocky 9

dnf info firewalld

Last metadata expiration check: 1:39:33 ago on Mon 13 May 2024 05:54:03 PM CEST.
Installed Packages
Name         : firewalld
Version      : 1.3.4
Release      : 1.el9
Architecture : noarch
Size         : 2.0 M
Source       : firewalld-1.3.4-1.el9.src.rpm
Repository   : @System
From repo    : baseos
Summary      : A firewall daemon with D-Bus interface providing a dynamic firewall
URL          : http://www.firewalld.org
License      : GPLv2+
Description  : firewalld is a firewall service daemon that provides a dynamic customizable
             : firewall with a D-Bus interface.

dnf info docker-ce
Last metadata expiration check: 1:40:29 ago on Mon 13 May 2024 05:54:03 PM CEST.
Installed Packages
Name         : docker-ce
Epoch        : 3
Version      : 26.1.2
Release      : 1.el9
Architecture : x86_64
Size         : 104 M
Source       : docker-ce-26.1.2-1.el9.src.rpm
Repository   : @System
From repo    : docker-ce-stable
Summary      : The open-source application container engine
URL          : https://www.docker.com
License      : ASL 2.0
Description  : Docker is a product for you to build, ship and run any application as a
             : lightweight container.
             : 
             : Docker containers are both hardware-agnostic and platform-agnostic. This means
             : they can run anywhere, from your laptop to the largest cloud compute instance
             : and everything in between - and they don't require you to use a particular
             : language, framework or packaging system. That makes them great building blocks
             : for deploying and scaling web apps, databases, and backend services without
             : depending on a particular stack or provider.

May 13 19:08:40 my.server.tld systemd[1]: Starting firewalld - dynamic firewall daemon...
May 13 19:08:40 my.server.tld systemd[1]: Started firewalld - dynamic firewall daemon.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER' failed: iptables v1.8.10 (nf_tables): Chain 'DOCKER' d>
                                           Try `iptables -h' or 'iptables --help' for more information.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER' failed: iptables v1.8.10 (nf_tables): >
                                           Try `iptables -h' or 'iptables --help' for more information.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER' failed: iptables v1.8.10 (nf_tables): Chain 'DOCKER' does >
                                           Try `iptables -h' or 'iptables --help' for more information.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D PREROUTING' failed: iptables: Bad rule (does a matching rule exist in that chain?).
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -D OUTPUT' failed: iptables: Bad rule (does a matching rule exist in that chain?).
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -F DOCKER' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t nat -X DOCKER' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION-STAGE-1' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION-STAGE-1' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION-STAGE-2' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION-STAGE-2' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -F DOCKER-ISOLATION' failed: iptables: No chain/target/match by that name.
May 13 19:08:42 my.server.tld firewalld[727]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w10 -t filter -X DOCKER-ISOLATION' failed: iptables: No chain/target/match by that name.

All docker containers listening on 0.0.0.0:PORT are accessible from the outside via servername:PORT

erig0 · 2024-05-13T18:42:55Z

All docker containers listening on 0.0.0.0:PORT are accessible from the outside via servername:PORT

This is expected. This what docker requests.

I recently wrote a blog about this and how to do strict filtering of docker containers.

arkodg force-pushed the add-intf-firewalld-zone branch from 7378ec4 to f4b58e7 Compare May 4, 2020 22:25

arkodg mentioned this pull request May 4, 2020

[feature request] nftables support moby/moby#26824

Closed

arkodg force-pushed the add-intf-firewalld-zone branch from f4b58e7 to cd39849 Compare May 5, 2020 20:26

arkodg changed the title ~~Add docker interfaces to firewalld trusted zone~~ Add docker interfaces to firewalld docker zone May 5, 2020

arkodg mentioned this pull request May 5, 2020

Unable to create new zone using the dbus interface firewalld/firewalld#624

Closed

arkodg force-pushed the add-intf-firewalld-zone branch 2 times, most recently from ed91f60 to d0d4036 Compare May 6, 2020 23:57

erig0 reviewed May 7, 2020

View reviewed changes

arkodg force-pushed the add-intf-firewalld-zone branch 2 times, most recently from 57b7633 to 902a280 Compare May 7, 2020 22:55

Add docker interfaces to firewalld docker zone

7a72092

If firewalld is running, create a new docker zone and add the docker interfaces to the docker zone to allow container networking for distros with firewalld enabled Fixes: moby#2496 Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>

arkodg force-pushed the add-intf-firewalld-zone branch from 902a280 to 7a72092 Compare May 8, 2020 01:37

thaJeztah mentioned this pull request May 8, 2020

[do not merge] vendor libnetwork with firewalld changes moby/moby#40933

Closed

arkodg mentioned this pull request May 8, 2020

Docker bypasses ufw firewall rules docker/for-linux#690

Open

3 tasks

cpuguy83 approved these changes May 21, 2020

View reviewed changes

tiborvass approved these changes May 21, 2020

View reviewed changes

arkodg merged commit 13a4da0 into moby:master May 21, 2020

arkodg deleted the add-intf-firewalld-zone branch May 21, 2020 18:35

erig0 mentioned this pull request Jun 17, 2020

firewalld and Docker chains firewalld/firewalld#503

Closed

thaJeztah mentioned this pull request Jul 8, 2020

vendor: libnetwork 9e99af28df21367340c95a3863e31808d689c92a moby/moby#41189

Merged

erig0 mentioned this pull request Jul 9, 2020

firewalld with nftables and docker firewalld/firewalld#666

Closed

thaJeztah mentioned this pull request Aug 31, 2020

[19.03 backport] Add docker interfaces to firewalld docker zone #2575

Closed

This was referenced Sep 18, 2020

Please provide repo for docker-ce on Fedora 32 docker/for-linux#955

Closed

Docker Installation on Fedora 32 moby/moby#41489

Closed

thaJeztah mentioned this pull request Nov 3, 2020

20.10.0-beta1@Fedora 33: Failed to program NAT chain: ZONE_CONFLICT: 'docker0' already bound to a zone moby/moby#41609

Closed

cyphar mentioned this pull request Feb 16, 2021

firewalld COMMAND_FAILED warnings #2622

Open

erig0 mentioned this pull request Mar 8, 2021

Add a FirewalldZone & fix systemd-nspawn connectivity systemd/systemd#15412

Open

erig0 mentioned this pull request Aug 24, 2021

FirewallD doesn't go well with Docker firewalld/firewalld#461

Closed

polarathene mentioned this pull request May 23, 2023

docs: Remove firewalld zone instructions docker/docs#17389

Merged

Mar	APR	May
	11
2025	2026	2027

Conversation

arkodg commented May 4, 2020 • edited by thaJeztah Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arkodg commented May 4, 2020

Uh oh!

yrro commented May 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yrro commented May 5, 2020

Uh oh!

arkodg commented May 5, 2020

Uh oh!

arkodg commented May 7, 2020

Uh oh!

erig0 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

erig0 May 7, 2020

Choose a reason for hiding this comment

Uh oh!

erig0 May 7, 2020

Choose a reason for hiding this comment

Uh oh!

erig0 May 7, 2020

Choose a reason for hiding this comment

Uh oh!

arkodg May 7, 2020

Choose a reason for hiding this comment

Uh oh!

erig0 May 7, 2020

Choose a reason for hiding this comment

Uh oh!

arkodg May 7, 2020

Choose a reason for hiding this comment

Uh oh!

cpuguy83 commented May 8, 2020

Uh oh!

arkodg commented May 8, 2020

Uh oh!

thaJeztah commented May 14, 2020

Uh oh!

arkodg commented May 14, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpuguy83 commented May 14, 2020

Uh oh!

arkodg commented May 18, 2020

Uh oh!

cpuguy83 commented May 18, 2020

Uh oh!

arkodg commented May 19, 2020

Uh oh!

cpuguy83 left a comment

Choose a reason for hiding this comment

Uh oh!

idc77 commented May 13, 2024

Uh oh!

erig0 commented May 13, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

arkodg commented May 4, 2020 •

edited by thaJeztah

Loading

yrro commented May 5, 2020 •

edited

Loading

arkodg commented May 14, 2020 •

edited

Loading