paint-brush
Container Networking Guide: Part Iby@atulthosar
New Story

Container Networking Guide: Part I

by Atul Thosar7mJanuary 15th, 2025
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

This is the first part of the series Container Networking. I will cover Virtual Ethernet devices in this blog post.
featured image - Container Networking Guide: Part I
Atul Thosar HackerNoon profile picture

This is the first part of the series Container Networking. I will cover Virtual Ethernet devices in this blog post.


Generally, any machine has loopback and ethernet network interfaces. You can check the available interfaces using

ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 02:fd:4d:34:55:76 brd ff:ff:ff:ff:ff:ff

enp0s3 is an interface of type ethernet. You can communicate with machines outside the VM using this interface.


Let’s create a Virtual Ethernet device. veth man page says

The veth devices are Virtual Ethernet devices.  They can act as
tunnels between network namespaces to create a bridge to a
physical network device in another namespace, but can also be
used as standalone network devices.

We will see namespaces and bridges later, but let us see how we can create veth interface(s) and their usage.

Creating veth Pair

sudo ip link add vethX type veth peer name vethY


ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 02:fd:4d:34:55:76 brd ff:ff:ff:ff:ff:ff
3: vethY@vethX: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 5a:bc:4d:7e:76:b1 brd ff:ff:ff:ff:ff:ff
4: vethX@vethY: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 5e:4b:13:90:7d:63 brd ff:ff:ff:ff:ff:ff


By default, they are down. You need to make them up.

sudo ip link set vethX up
sudo ip link set vethY up


ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 02:fd:4d:34:55:76 brd ff:ff:ff:ff:ff:ff
3: vethY@vethX: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 5a:bc:4d:7e:76:b1 brd ff:ff:ff:ff:ff:ff
4: vethX@vethY: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 5e:4b:13:90:7d:63 brd ff:ff:ff:ff:ff:ff


The veth pair is very special. Packets transmitted on one device in the pair are immediately received on the other device.

Traffic on Veth Pair

To send the packet using veth, we will use the Scapy tool. It is a Python-based interactive packet manipulation program and library.


The code to send a single packet looks like

cat <<EOF >> onepkt.py
#! /usr/bin/env python3

import sys
from scapy.all import *


if __name__ == '__main__':
    usage_string = """Usage:
  onepkt.py <src-mac> <dst-mac> <iface> <msg>
    where <msg> is unique identifier for this message"""

    # total arguments
    if (len(sys.argv) != 5):
        sys.exit("Incorrect usage - num args.\n"+usage_string)

    src_mac = sys.argv[1]
    dst_mac = sys.argv[2]
    iface = sys.argv[3]
    msg = sys.argv[4]

    src_ip = "1.1.1.1"
    dst_ip = "2.2.2.2"
    sport = 1111
    dport = 2222


    pkt = Ether(dst=dst_mac, src=src_mac) / IP (src=src_ip, dst=dst_ip) / TCP(sport=sport, dport=dport) / msg

    pkt.show()

    sendp(pkt, iface=iface)

EOF

Code Credits: https://github.com/eric-keller/npp-linux-01-intro/blob/main/demo3/onepkt.py


In one terminal window, start the tshark on vethY

sudo tshark -T fields -e eth -i vethY
Running as user "root" and group "root". This could be dangerous.
Capturing on 'vethY'
 ** (tshark:9349) 10:26:21.105768 [Main MESSAGE] -- Capture started.
 ** (tshark:9349) 10:26:21.105897 [Main MESSAGE] -- File: "/tmp/wireshark_vethY4R8LY2.pcapng"


Now, run the following Python code to send the single packet to vethX device.

sudo python3 ./onepkt.py 22:11:11:11:11:11 22:22:22:22:22:22 vethX 123
###[ Ethernet ]###
  dst       = 22:22:22:22:22:22
  src       = 22:11:11:11:11:11
  type      = IPv4
###[ IP ]###
     version   = 4
     ihl       = None
     tos       = 0x0
     len       = None
     id        = 1
     flags     =
     frag      = 0
     ttl       = 64
     proto     = tcp
     chksum    = None
     src       = 1.1.1.1
     dst       = 2.2.2.2
     \options   \ 
###[ TCP ]###     
        sport     = 1111
        dport     = 2222
        seq       = 0   
        ack       = 0
        dataofs   = None
        reserved  = 0
        flags     = S
        window    = 8192
        chksum    = None
        urgptr    = 0
        options   = []
###[ Raw ]### 
           load      = '123'

.
Sent 1 packets.


On the tshark window, you will see the packet has been received.

sudo tshark -T fields -e eth -i vethY
Running as user "root" and group "root". This could be dangerous.
Capturing on 'vethY'
 ** (tshark:9349) 10:26:21.105768 [Main MESSAGE] -- Capture started.
 ** (tshark:9349) 10:26:21.105897 [Main MESSAGE] -- File: "/tmp/wireshark_vethY4R8LY2.pcapng"

Ethernet II, Src: 22:11:11:11:11:11 (22:11:11:11:11:11), Dst: 22:22:22:22:22:22 (22:22:22:22:22:22)

The veth pairs play a critical role in establishing Container Networking. We will see it in the next part of the blog.

More on Veth

If you have many veth pairs, then given a veth device how can you identify the peer? ethtool utility comes to the rescue. ethtool tells us the peer’s index.

ethtool -S vethX | grep peer
     peer_ifindex: 3

peer_ifindex: 3 indicates an index of peer devices. The index is shown in ip a command.


ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 02:fd:4d:34:55:76 brd ff:ff:ff:ff:ff:ff
3: vethY@vethX: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 5a:bc:4d:7e:76:b1 brd ff:ff:ff:ff:ff:ff
4: vethX@vethY: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 5e:4b:13:90:7d:63 brd ff:ff:ff:ff:ff:ff


Although device name vethX@vethY tells the peer name, but when devices are created by container software (like docker, podman, etc.), the device names are not that straightforward.


You can get detailed, prettier information about a veth device using

ip -d -j -p link show vethX
[ {
        "ifindex": 4,
        "link": "vethY",
        "ifname": "vethX",
        "flags": [ "BROADCAST","MULTICAST","UP","LOWER_UP" ],
        "mtu": 1500,
        "qdisc": "noqueue",
        "operstate": "UP",
        "linkmode": "DEFAULT",
        "group": "default",
        "txqlen": 1000,
        "link_type": "ether",
        "address": "5e:4b:13:90:7d:63",
        "broadcast": "ff:ff:ff:ff:ff:ff",
        "promiscuity": 0,
        "min_mtu": 68,
        "max_mtu": 65535,
        "linkinfo": {
            "info_kind": "veth"
        },
        "inet6_addr_gen_mode": "eui64",
        "num_tx_queues": 2,
        "num_rx_queues": 2,
        "gso_max_size": 65536,
        "gso_max_segs": 65535
    } ]


With this, we come to the end of this blog post. Next in the series is Network Namespaces and Bridges