#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Posts
    2
    Rep Power
    0

    Question Need help to Write ipfw rules for FreeBSD host and vnet jail servers


    Hi all,

    I need to use IPFW instead of any other one since I use vnet jails and they don't support something alse. But it's now two weeks I fight against the documentation, the very few tutorials available, and the examples in /etc/rc.firewall, and several months I make my homeworks on networking. I need some help please . I am still also a bit lost with natd, ipfw, routes.

    I have a FreeBSD host with a single hardware nic em0, and 2 full zfs jails. Let's consider only the second one which is the simpliest one. In the host, I built 3 bridges with individual subnets for each 3 jails. I followed the quick start of zjail here. jail02 is declared with vswitch2.

    I would like first just to be able to perform a host google.fr from jail02, or a ping.

    Here are the configurations. For the host :

    Code:
    root@host:/root # cat /etc/rc.conf  | sed -e 's/#.*//;/^\s*$/d' 
    zfs_enable=YES
    hostname="host.mondomaine.com"
    ifconfig_em0="inet 192.168.99.66 netmask 0xffffff00"
    defaultrouter="192.168.99.1"
    sshd_enable="YES"
    pf_enable="NO"
    moused_enable="YES"
    dumpdev="AUTO"
    font8x8="iso-8x8"
    font8x14="iso-8x14"
    font8x16="iso-8x16"
    scrnmap="iso-8859-1_to_cp437"
    keymap="fr.iso.acc"
    ipv6_activate_all_interfaces="YES"
    ipv6_ifconfig_em0_alias0="inet6 xx:xx:xx:xx::99:66 prefixlen 64"
    ipv6_ifconfig_em0_alias1="inet6 fe80::99:66 prefixlen 64"
    ipv6_defaultrouter="fe80:2095:a4ff:fe97:bedb"
    ezjail_enable="NO" 
    openntpd_enable="YES" 
    openntpd_flags="-s"
    inetd_enable="NO" 
    rpc_bind_enable="NO" 
    sendmail_enable="NO" 
    syslogd_enable="YES" 
    syslogd_flags="-s -b 127.0.0.1" 
    gateway_enable="YES"
    cloned_interfaces="bridge0 bridge1 bridge2" 
    ifconfig_bridge0_name="vswitch0" 
    ifconfig_bridge1_name="vswitch1" 
    ifconfig_bridge2_name="vswitch2" 
    ifconfig_vswitch0="inet 10.5.100.254 netmask 255.255.255.0 up" 
    ifconfig_vswitch1="inet 10.6.100.254 netmask 255.255.255.0 up" 
    ifconfig_vswitch2="inet 10.7.100.254 netmask 255.255.255.0 up"
    gateway_enable="YES"
    firewall_enable="YES"
    firewall_type="OPEN"
    natd_enable="YES"
    natd_interface="em0"
    Code:
    root@host:/root # ifconfig | sed -e 's/#.*//;/^\s*$/d'
    em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
    	ether 02:5a:4b:3c:2d:1e
    	inet 192.168.99.66 netmask 0xffffff00 broadcast 192.168.99.255
    	inet6 fe80::5a:4bff:fe3c:2d1e%em0 prefixlen 64 scopeid 0x1 
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	media: Ethernet autoselect (1000baseT <full-duplex>)
    	status: active
    ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    	inet6 ::1 prefixlen 128 
    	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 
    	inet 127.0.0.1 netmask 0xff000000 
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    vswitch0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	ether 02:93:99:a5:4c:00
    	inet 10.5.100.254 netmask 0xffffff00 broadcast 10.5.100.255
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    	member: vnet0-2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
    	        ifmaxaddr 0 port 8 priority 128 path cost 2000
    	member: vnet0-1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
    	        ifmaxaddr 0 port 7 priority 128 path cost 2000
    vswitch1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	ether 02:93:99:a5:4c:01
    	inet 10.6.100.254 netmask 0xffffff00 broadcast 10.6.100.255
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    vswitch2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	ether 02:93:99:a5:4c:02
    	inet 10.7.100.254 netmask 0xffffff00 broadcast 10.7.100.255
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    	maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    	root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    vnet0-1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	description: Attached to Jail: jail01
    	options=8<VLAN_MTU>
    	ether 02:1a:17:00:07:0a
    	inet6 fe80::1a:17ff:fe00:70a%vnet0-1 prefixlen 64 scopeid 0x7 
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    	status: active
    vnet0-2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	description: Attached to Jail: jail02
    	options=8<VLAN_MTU>
    	ether 02:16:53:00:08:0a
    	inet6 fe80::16:53ff:fe00:80a%vnet0-2 prefixlen 64 scopeid 0x8 
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    	status: active
    Code:
    root@host:/root # zjail list all
    +---------------------------------------------------------------------------------------------------------------------------+
    | Id   Name                Hostname                            IPv4/6          Quota   Free   Size  Clone  Boot  BP  Status |
    +---------------------------------------------------------------------------------------------------------------------------+
    |  1   jail01              jail01.local                       192.168.99.11       6G  1.02G  1008K    yes    on  99    Up   |
    |  2   jail02              jail02.local                       10.7.100.2          6G  1.02G   692K    yes    on  99    Up   |
    | --   base                -                                  -                 none  1.02G   369M     no     -   -   down  |
    +---------------------------------------------------------------------------------------------------------------------------+
    For jail02 :

    Code:
    root@jail02:/root # ifconfig 
    lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    	inet6 ::1 prefixlen 128 
    	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 
    	inet 127.0.0.1 netmask 0xff000000 
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    vnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    	options=8<VLAN_MTU>
    	ether 02:16:53:00:09:0b
    	inet 10.7.100.2 netmask 0xffffff00 broadcast 10.7.100.255
    	inet6 fe80::16:53ff:fe00:90b%vnet0 prefixlen 64 scopeid 0x2 
    	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    	media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    	status: active
    Code:
    root@jail02:/root # cat /etc/rc.conf | sed -e 's/#.*//;/^\s*$/d'
    defaultrouter="10.7.100.254"
    pf_enable="NO"
    rpcbind_enable="NO"
    cron_flags="$cron_flags -J 15"
    syslogd_flags="-ss"
    sendmail_enable="NO"
    sendmail_submit_enable="NO"
    sendmail_outbound_enable="NO"
    sendmail_msp_queue_enable="NO"
    Code:
    root@host:/root # zjail show jail02
                       id : 2
                     name : jail02
                       ip : 10.7.100.2
                 hostname : jail02.local
                 hostuuid : eb7faa4c-b22b-11e2-b9a5-025a4b3c2d1e
                    quota : 6G
                available : 1.02G
                     used : 704K
                     boot : on
                    notes : none
               mountpoint : /zjails/jds/local/jail02
            devfs_ruleset : devfsrules_jail
               exec_start : /bin/sh /etc/rc
                exec_stop : /bin/sh /etc/rc.shutdown
             mount_enable : YES
             devfs_enable : YES
            procfs_enable : YES
           fdescfs_enable : YES
                    flags : -l -U root
                   cpuset : 0,1,2
                      fib : 0
              compression : off
            compressratio : 1.00x
                    dedup : off
                    clone : yes
                 seclevel : 2
                 priority : 99
             last_started : 01.05.2013@08:55:39
            last_shutdown : -
             defaultroute : 0.0.0.0
                  vnet_if : vnet0
                  netmask : 24
                 vnet0_sw : vswitch2
                 vnet1_sw : -
                 vnet2_sw : -
                     vnet : on
                     jzfs : on
    At the end, here is what I want to do :
    Code:
    Web --- Router ---- em0 Host vswitch0 ------------- zjail00 (Mongrel2 server)
                                |              |------- zjail01 (Brubeck web python framework)
                                |              |------- zjail02 (Mongo DB)
                                |
                                |vswitch1 ------------- zjail10 (Mongrel2 server)
                                               |------- zjail11 (Special Application)
                                               |------- zjail12 (Mongo DB)
    But if you can help just to help me have the jail see the web, that would be a great help for me.
  2. #2
  3. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Posts
    2
    Rep Power
    0
    For a reason I cannot yet explain, the
    Code:
    route add default xx.xx.xx.xx
    command in /etc/rc.conf is ignored.

    If I perform as a workaround one of the two following commands, it works :
    Code:
    route add default 10.7.100.254
    Code:
    service routing start

IMN logo majestic logo threadwatch logo seochat tools logo