Avoiding MPI All-Zero Listening

  • The port of the background mpirun process started by MPI initialization is not authenticated, which leads to attacks.

    Workaround: After MPI is started, add firewall rules to prevent external networks from connecting to the listening ports of mpirun and orted (internal components of MPI). After MPI ends execution, clear the firewall rules.

    • Add firewall rules.
      1
      2
      3
      4
      5
      6
      7
      8
      ```shell
      iptables -D INPUT -p tcp -j ${Rule name}
      iptables -F ${Rule name}
      iptables -X ${Rule name}
      iptables -t filter -N ${Rule name}
      iptables -I INPUT -p tcp -j ${Rule name}
      iptables -t filter -I ${Rule name} -i ${NIC to be restricted} -p tcp --dport ${Port to be restricted} -j DROP
      ```
      
    • Clearing firewall rules
      1
      2
      3
      4
      ```shell
      iptables -D INPUT -p tcp -j ${Rule name}
      iptables -F ${Rule name}
      iptables -X ${Rule name}
      
  • The default MPI configuration item listens to the 0.0.0.0 address when multiple Python training processes are started.
    Workaround: Locate the etc folder in the MPI configuration path and add the following configuration items to the end of the openmpi-mca-params.conf file:
    1
    2
    btl_tcp_if_include = lo
    btl_tcp_if_exclude = docker0
    

    You are advised to use MPI only in containers and isolate MPIs by network namespaces. This ensures that no all-zero listening exists on the host and other users of the host cannot connect to the unauthenticated OpenMPI ports.