You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

pve-firewall.adoc 17KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649
  1. [[chapter_pve_firewall]]
  2. ifdef::manvolnum[]
  3. pve-firewall(8)
  4. ===============
  5. :pve-toplevel:
  6. NAME
  7. ----
  8. pve-firewall - PVE Firewall Daemon
  9. SYNOPSIS
  10. --------
  11. include::pve-firewall.8-synopsis.adoc[]
  12. DESCRIPTION
  13. -----------
  14. endif::manvolnum[]
  15. ifndef::manvolnum[]
  16. {pve} Firewall
  17. ==============
  18. :pve-toplevel:
  19. endif::manvolnum[]
  20. ifdef::wiki[]
  21. :title: Firewall
  22. endif::wiki[]
  23. {pve} Firewall provides an easy way to protect your IT
  24. infrastructure. You can setup firewall rules for all hosts
  25. inside a cluster, or define rules for virtual machines and
  26. containers. Features like firewall macros, security groups, IP sets
  27. and aliases help to make that task easier.
  28. While all configuration is stored on the cluster file system, the
  29. `iptables`-based firewall service runs on each cluster node, and thus provides
  30. full isolation between virtual machines. The distributed nature of
  31. this system also provides much higher bandwidth than a central
  32. firewall solution.
  33. The firewall has full support for IPv4 and IPv6. IPv6 support is fully
  34. transparent, and we filter traffic for both protocols by default. So
  35. there is no need to maintain a different set of rules for IPv6.
  36. Zones
  37. -----
  38. The Proxmox VE firewall groups the network into the following logical zones:
  39. Host::
  40. Traffic from/to a cluster node
  41. VM::
  42. Traffic from/to a specific VM
  43. For each zone, you can define firewall rules for incoming and/or
  44. outgoing traffic.
  45. Configuration Files
  46. -------------------
  47. All firewall related configuration is stored on the proxmox cluster
  48. file system. So those files are automatically distributed to all
  49. cluster nodes, and the `pve-firewall` service updates the underlying
  50. `iptables` rules automatically on changes.
  51. You can configure anything using the GUI (i.e. *Datacenter* -> *Firewall*,
  52. or on a *Node* -> *Firewall*), or you can edit the configuration files
  53. directly using your preferred editor.
  54. Firewall configuration files contain sections of key-value
  55. pairs. Lines beginning with a `#` and blank lines are considered
  56. comments. Sections start with a header line containing the section
  57. name enclosed in `[` and `]`.
  58. [[pve_firewall_cluster_wide_setup]]
  59. Cluster Wide Setup
  60. ~~~~~~~~~~~~~~~~~~
  61. The cluster wide firewall configuration is stored at:
  62. /etc/pve/firewall/cluster.fw
  63. The configuration can contain the following sections:
  64. `[OPTIONS]`::
  65. This is used to set cluster wide firewall options.
  66. include::pve-firewall-cluster-opts.adoc[]
  67. `[RULES]`::
  68. This sections contains cluster wide firewall rules for all nodes.
  69. `[IPSET <name>]`::
  70. Cluster wide IP set definitions.
  71. `[GROUP <name>]`::
  72. Cluster wide security group definitions.
  73. `[ALIASES]`::
  74. Cluster wide Alias definitions.
  75. Enabling the Firewall
  76. ^^^^^^^^^^^^^^^^^^^^^
  77. The firewall is completely disabled by default, so you need to
  78. set the enable option here:
  79. ----
  80. [OPTIONS]
  81. # enable firewall (cluster wide setting, default is disabled)
  82. enable: 1
  83. ----
  84. IMPORTANT: If you enable the firewall, traffic to all hosts is blocked by
  85. default. Only exceptions is WebGUI(8006) and ssh(22) from your local
  86. network.
  87. If you want to administrate your {pve} hosts from remote, you
  88. need to create rules to allow traffic from those remote IPs to the web
  89. GUI (port 8006). You may also want to allow ssh (port 22), and maybe
  90. SPICE (port 3128).
  91. TIP: Please open a SSH connection to one of your {PVE} hosts before
  92. enabling the firewall. That way you still have access to the host if
  93. something goes wrong .
  94. To simplify that task, you can instead create an IPSet called
  95. ``management'', and add all remote IPs there. This creates all required
  96. firewall rules to access the GUI from remote.
  97. [[pve_firewall_host_specific_configuration]]
  98. Host Specific Configuration
  99. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  100. Host related configuration is read from:
  101. /etc/pve/nodes/<nodename>/host.fw
  102. This is useful if you want to overwrite rules from `cluster.fw`
  103. config. You can also increase log verbosity, and set netfilter related
  104. options. The configuration can contain the following sections:
  105. `[OPTIONS]`::
  106. This is used to set host related firewall options.
  107. include::pve-firewall-host-opts.adoc[]
  108. `[RULES]`::
  109. This sections contains host specific firewall rules.
  110. [[pve_firewall_vm_container_configuration]]
  111. VM/Container Configuration
  112. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  113. VM firewall configuration is read from:
  114. /etc/pve/firewall/<VMID>.fw
  115. and contains the following data:
  116. `[OPTIONS]`::
  117. This is used to set VM/Container related firewall options.
  118. include::pve-firewall-vm-opts.adoc[]
  119. `[RULES]`::
  120. This sections contains VM/Container firewall rules.
  121. `[IPSET <name>]`::
  122. IP set definitions.
  123. `[ALIASES]`::
  124. IP Alias definitions.
  125. Enabling the Firewall for VMs and Containers
  126. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  127. Each virtual network device has its own firewall enable flag. So you
  128. can selectively enable the firewall for each interface. This is
  129. required in addition to the general firewall `enable` option.
  130. Firewall Rules
  131. --------------
  132. Firewall rules consists of a direction (`IN` or `OUT`) and an
  133. action (`ACCEPT`, `DENY`, `REJECT`). You can also specify a macro
  134. name. Macros contain predefined sets of rules and options. Rules can be
  135. disabled by prefixing them with `|`.
  136. .Firewall rules syntax
  137. ----
  138. [RULES]
  139. DIRECTION ACTION [OPTIONS]
  140. |DIRECTION ACTION [OPTIONS] # disabled rule
  141. DIRECTION MACRO(ACTION) [OPTIONS] # use predefined macro
  142. ----
  143. The following options can be used to refine rule matches.
  144. include::pve-firewall-rules-opts.adoc[]
  145. Here are some examples:
  146. ----
  147. [RULES]
  148. IN SSH(ACCEPT) -i net0
  149. IN SSH(ACCEPT) -i net0 # a comment
  150. IN SSH(ACCEPT) -i net0 -source 192.168.2.192 # only allow SSH from 192.168.2.192
  151. IN SSH(ACCEPT) -i net0 -source 10.0.0.1-10.0.0.10 # accept SSH for IP range
  152. IN SSH(ACCEPT) -i net0 -source 10.0.0.1,10.0.0.2,10.0.0.3 #accept ssh for IP list
  153. IN SSH(ACCEPT) -i net0 -source +mynetgroup # accept ssh for ipset mynetgroup
  154. IN SSH(ACCEPT) -i net0 -source myserveralias #accept ssh for alias myserveralias
  155. |IN SSH(ACCEPT) -i net0 # disabled rule
  156. IN DROP # drop all incoming packages
  157. OUT ACCEPT # accept all outgoing packages
  158. ----
  159. [[pve_firewall_security_groups]]
  160. Security Groups
  161. ---------------
  162. A security group is a collection of rules, defined at cluster level, which
  163. can be used in all VMs' rules. For example you can define a group named
  164. ``webserver'' with rules to open the 'http' and 'https' ports.
  165. ----
  166. # /etc/pve/firewall/cluster.fw
  167. [group webserver]
  168. IN ACCEPT -p tcp -dport 80
  169. IN ACCEPT -p tcp -dport 443
  170. ----
  171. Then, you can add this group to a VM's firewall
  172. ----
  173. # /etc/pve/firewall/<VMID>.fw
  174. [RULES]
  175. GROUP webserver
  176. ----
  177. [[pve_firewall_ip_aliases]]
  178. IP Aliases
  179. ----------
  180. IP Aliases allow you to associate IP addresses of networks with a
  181. name. You can then refer to those names:
  182. * inside IP set definitions
  183. * in `source` and `dest` properties of firewall rules
  184. Standard IP Alias `local_network`
  185. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  186. This alias is automatically defined. Please use the following command
  187. to see assigned values:
  188. ----
  189. # pve-firewall localnet
  190. local hostname: example
  191. local IP address: 192.168.2.100
  192. network auto detect: 192.168.0.0/20
  193. using detected local_network: 192.168.0.0/20
  194. ----
  195. The firewall automatically sets up rules to allow everything needed
  196. for cluster communication (corosync, API, SSH) using this alias.
  197. The user can overwrite these values in the `cluster.fw` alias
  198. section. If you use a single host on a public network, it is better to
  199. explicitly assign the local IP address
  200. ----
  201. # /etc/pve/firewall/cluster.fw
  202. [ALIASES]
  203. local_network 1.2.3.4 # use the single IP address
  204. ----
  205. [[pve_firewall_ip_sets]]
  206. IP Sets
  207. -------
  208. IP sets can be used to define groups of networks and hosts. You can
  209. refer to them with `+name` in the firewall rules' `source` and `dest`
  210. properties.
  211. The following example allows HTTP traffic from the `management` IP
  212. set.
  213. IN HTTP(ACCEPT) -source +management
  214. Standard IP set `management`
  215. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  216. This IP set applies only to host firewalls (not VM firewalls). Those
  217. IPs are allowed to do normal management tasks (PVE GUI, VNC, SPICE,
  218. SSH).
  219. The local cluster network is automatically added to this IP set (alias
  220. `cluster_network`), to enable inter-host cluster
  221. communication. (multicast,ssh,...)
  222. ----
  223. # /etc/pve/firewall/cluster.fw
  224. [IPSET management]
  225. 192.168.2.10
  226. 192.168.2.10/24
  227. ----
  228. Standard IP set `blacklist`
  229. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  230. Traffic from these IPs is dropped by every host's and VM's firewall.
  231. ----
  232. # /etc/pve/firewall/cluster.fw
  233. [IPSET blacklist]
  234. 77.240.159.182
  235. 213.87.123.0/24
  236. ----
  237. [[pve_firewall_ipfilter_section]]
  238. Standard IP set `ipfilter-net*`
  239. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  240. These filters belong to a VM's network interface and are mainly used to prevent
  241. IP spoofing. If such a set exists for an interface then any outgoing traffic
  242. with a source IP not matching its interface's corresponding ipfilter set will
  243. be dropped.
  244. For containers with configured IP addresses these sets, if they exist (or are
  245. activated via the general `IP Filter` option in the VM's firewall's *options*
  246. tab), implicitly contain the associated IP addresses.
  247. For both virtual machines and containers they also implicitly contain the
  248. standard MAC-derived IPv6 link-local address in order to allow the neighbor
  249. discovery protocol to work.
  250. ----
  251. /etc/pve/firewall/<VMID>.fw
  252. [IPSET ipfilter-net0] # only allow specified IPs on net0
  253. 192.168.2.10
  254. ----
  255. Services and Commands
  256. ---------------------
  257. The firewall runs two service daemons on each node:
  258. * pvefw-logger: NFLOG daemon (ulogd replacement).
  259. * pve-firewall: updates iptables rules
  260. There is also a CLI command named `pve-firewall`, which can be used to
  261. start and stop the firewall service:
  262. # pve-firewall start
  263. # pve-firewall stop
  264. To get the status use:
  265. # pve-firewall status
  266. The above command reads and compiles all firewall rules, so you will
  267. see warnings if your firewall configuration contains any errors.
  268. If you want to see the generated iptables rules you can use:
  269. # iptables-save
  270. [[pve_firewall_default_rules]]
  271. Default firewall rules
  272. ----------------------
  273. The following traffic is filtered by the default firewall configuration:
  274. Datacenter incoming/outgoing DROP/REJECT
  275. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  276. If the input or output policy for the firewall is set to DROP or REJECT, the
  277. following traffic is still allowed for all {pve} hosts in the cluster:
  278. * traffic over the loopback interface
  279. * already established connections
  280. * traffic using the IGMP protocol
  281. * TCP traffic from management hosts to port 8006 in order to allow access to
  282. the web interface
  283. * TCP traffic from management hosts to the port range 5900 to 5999 allowing
  284. traffic for the VNC web console
  285. * TCP traffic from management hosts to port 3128 for connections to the SPICE
  286. proxy
  287. * TCP traffic from management hosts to port 22 to allow ssh access
  288. * UDP traffic in the cluster network to port 5404 and 5405 for corosync
  289. * UDP multicast traffic in the cluster network
  290. * ICMP traffic type 3 (Destination Unreachable), 4 (congestion control) or 11
  291. (Time Exceeded)
  292. The following traffic is dropped, but not logged even with logging enabled:
  293. * TCP connections with invalid connection state
  294. * Broadcast, multicast and anycast traffic not related to corosync, i.e., not
  295. coming through port 5404 or 5405
  296. * TCP traffic to port 43
  297. * UDP traffic to ports 135 and 445
  298. * UDP traffic to the port range 137 to 139
  299. * UDP traffic form source port 137 to port range 1024 to 65535
  300. * UDP traffic to port 1900
  301. * TCP traffic to port 135, 139 and 445
  302. * UDP traffic originating from source port 53
  303. The rest of the traffic is dropped or rejected, respectively, and also logged.
  304. This may vary depending on the additional options enabled in
  305. *Firewall* -> *Options*, such as NDP, SMURFS and TCP flag filtering.
  306. [[pve_firewall_iptables_inspect]]
  307. Please inspect the output of the
  308. ----
  309. # iptables-save
  310. ----
  311. system command to see the firewall chains and rules active on your system.
  312. This output is also included in a `System Report`, accessible over a node's
  313. subscription tab in the web GUI, or through the `pvereport` command line tool.
  314. VM/CT incoming/outgoing DROP/REJECT
  315. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  316. This drops or rejects all the traffic to the VMs, with some exceptions for
  317. DHCP, NDP, Router Advertisement, MAC and IP filtering depending on the set
  318. configuration. The same rules for dropping/rejecting packets are inherited
  319. from the datacenter, while the exceptions for accepted incomming/outgoing
  320. traffic of the host do not apply.
  321. Again, you can use xref:pve_firewall_iptables_inspect[iptables-save (see above)]
  322. to inspect all rules and chains applied.
  323. Logging of firewall rules
  324. -------------------------
  325. By default, all logging of traffic filtered by the firewall rules is disabled.
  326. To enable logging, the `loglevel` for incommig and/or outgoing traffic has to be
  327. set in *Firewall* -> *Options*. This can be done for the host as well as for the
  328. VM/CT firewall individually. By this, logging of {PVE}'s standard firewall rules
  329. is enabled and the output can be observed in *Firewall* -> *Log*.
  330. Further, only some dropped or rejected packets are logged for the standard rules
  331. (see xref:pve_firewall_default_rules[default firewall rules]).
  332. `loglevel` does not affect how much of the filtered traffic is logged. It
  333. changes a `LOGID` appended as prefix to the log output for easier filtering and
  334. post-processing.
  335. `loglevel` is one of the following flags:
  336. [[pve_firewall_log_levels]]
  337. [width="25%", options="header"]
  338. |===================
  339. | loglevel | LOGID
  340. | nolog | --
  341. | emerg | 0
  342. | alert | 1
  343. | crit | 2
  344. | err | 3
  345. | warning | 4
  346. | notice | 5
  347. | info | 6
  348. | debug | 7
  349. |===================
  350. A typical firewall log output looks like this:
  351. ----
  352. VMID LOGID CHAIN TIMESTAMP POLICY: PACKET_DETAILS
  353. ----
  354. In case of the host firewall, `VMID` is equal to 0.
  355. Logging of user defined firewall rules
  356. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  357. In order to log packets filtered by user-defined firewall rules, it is possible
  358. to set a log-level parameter for each rule individually.
  359. This allows to log in a fine grained manner and independent of the log-level
  360. defined for the standard rules in *Firewall* -> *Options*.
  361. While the `loglevel` for each individual rule can be defined or changed easily
  362. in the WebUI during creation or modification of the rule, it is possible to set
  363. this also via the corresponding `pvesh` API calls.
  364. Further, the log-level can also be set via the firewall configuration file by
  365. appending a `-log <loglevel>` to the selected rule (see
  366. xref:pve_firewall_log_levels[possible log-levels]).
  367. For example, the following two are ident:
  368. ----
  369. IN REJECT -p icmp -log nolog
  370. IN REJECT -p icmp
  371. ----
  372. whereas
  373. ----
  374. IN REJECT -p icmp -log debug
  375. ----
  376. produces a log output flagged with the `debug` level.
  377. Tips and Tricks
  378. ---------------
  379. How to allow FTP
  380. ~~~~~~~~~~~~~~~~
  381. FTP is an old style protocol which uses port 21 and several other dynamic ports. So you
  382. need a rule to accept port 21. In addition, you need to load the `ip_conntrack_ftp` module.
  383. So please run:
  384. modprobe ip_conntrack_ftp
  385. and add `ip_conntrack_ftp` to `/etc/modules` (so that it works after a reboot).
  386. Suricata IPS integration
  387. ~~~~~~~~~~~~~~~~~~~~~~~~
  388. If you want to use the http://suricata-ids.org/[Suricata IPS]
  389. (Intrusion Prevention System), it's possible.
  390. Packets will be forwarded to the IPS only after the firewall ACCEPTed
  391. them.
  392. Rejected/Dropped firewall packets don't go to the IPS.
  393. Install suricata on proxmox host:
  394. ----
  395. # apt-get install suricata
  396. # modprobe nfnetlink_queue
  397. ----
  398. Don't forget to add `nfnetlink_queue` to `/etc/modules` for next reboot.
  399. Then, enable IPS for a specific VM with:
  400. ----
  401. # /etc/pve/firewall/<VMID>.fw
  402. [OPTIONS]
  403. ips: 1
  404. ips_queues: 0
  405. ----
  406. `ips_queues` will bind a specific cpu queue for this VM.
  407. Available queues are defined in
  408. ----
  409. # /etc/default/suricata
  410. NFQUEUE=0
  411. ----
  412. Notes on IPv6
  413. -------------
  414. The firewall contains a few IPv6 specific options. One thing to note is that
  415. IPv6 does not use the ARP protocol anymore, and instead uses NDP (Neighbor
  416. Discovery Protocol) which works on IP level and thus needs IP addresses to
  417. succeed. For this purpose link-local addresses derived from the interface's MAC
  418. address are used. By default the `NDP` option is enabled on both host and VM
  419. level to allow neighbor discovery (NDP) packets to be sent and received.
  420. Beside neighbor discovery NDP is also used for a couple of other things, like
  421. auto-configuration and advertising routers.
  422. By default VMs are allowed to send out router solicitation messages (to query
  423. for a router), and to receive router advertisement packets. This allows them to
  424. use stateless auto configuration. On the other hand VMs cannot advertise
  425. themselves as routers unless the ``Allow Router Advertisement'' (`radv: 1`) option
  426. is set.
  427. As for the link local addresses required for NDP, there's also an ``IP Filter''
  428. (`ipfilter: 1`) option which can be enabled which has the same effect as adding
  429. an `ipfilter-net*` ipset for each of the VM's network interfaces containing the
  430. corresponding link local addresses. (See the
  431. <<pve_firewall_ipfilter_section,Standard IP set `ipfilter-net*`>> section for details.)
  432. Ports used by {pve}
  433. -------------------
  434. * Web interface: 8006
  435. * VNC Web console: 5900-5999
  436. * SPICE proxy: 3128
  437. * sshd (used for cluster actions): 22
  438. * rpcbind: 111
  439. * corosync multicast (if you run a cluster): 5404, 5405 UDP
  440. ifdef::manvolnum[]
  441. Macro Definitions
  442. -----------------
  443. include::pve-firewall-macros.adoc[]
  444. include::pve-copyright.adoc[]
  445. endif::manvolnum[]