Saturday, September 24, 2011

How to use Linux containers (lxc) under Debian Squeeze

In this article, you will learn what is virtualization, why use it and how to use one kind of virtualization, Linux containers (aka lxc) under Debian Squeeze.

What is virtualization ?

Virtualization can be defined as a way to run a virtual machine on a real machine.
A taxonomy of virtual machines has be defined in the book Virtual Machines: Versatile Platforms for Systems and Processes by Jim Smith and Ravi Nair. The first types are Process VMs and System VMs. Process VMs support an ABI (user instructions and system calls). Systems VMs support a complete ISA (instruction set architecture) both user and system instructions.
Then sub-types are defined if they use the same ISA as the host system:
  1. Process VM using the same ISA. Example: WINE.
  2. Process VM using a different ISA. Example: Java Virtual Machines.
  3. System VM using the same ISA. Example: KVM.
  4. System VM using a different ISA. Example: QEMU for other architecture.
In this article we focus on System VMs using the same ISA. This category can be divided as follow:
  1. Type1 virtual machines: hypervisor runs directly on the hardware. Examples: VMware ESXi, Citrix XenServer or Microsoft Hyper-V hypervisor.
  2. Type2 virtual machines: hypervisor runs on another operating system, such as Linux. Examples: VirtualBox.
On the type 2 VMs, there is a special sub-category: Containers. These VMs run under the same kernel. Examples: LXC, OpenVZ, Solaris zones. Containers are very efficient in term of performance because you have no need to virtualize devices and the host system drivers do the jobs for the containers with a very low overhead.
We will concentrate in this article on a container implementation distributed in the standard Linux kernel: lxc. Containers are a very light virtualization technology that is implemented on a given kernel here for lxc under the Linux kernel. So it means that you will be able to run only Linux systems that are compatible with the hosting kernel in your containers. You will not be able to run Windows for example in a container.
A container works like a chroot on steroids by adding a complete virtual system to a chroot and bringing tools to handle the virtual resources.

Why virtualization ?

Virtualization can be useful in multiple scenarios. Mainly you want isolation when you use virtualization. Isolation can be needed for security reasons, flexibility reasons, incompatibility reasons, test reasons, licence reasons…
Here in our example, imagine you need to run a Debian sid system under your stable Debian Squeeze system to be able to run tests on this in development Linux distribution without dedicating a physical machine to these tests.

lxc step by step under Debian Squeeze

The following commands to setup lxc are done under Debian Squeeze but they could be adapted to run on any Linux distribution.
Under Debian Squeeze every needed parts are present. No need to recompile your kernel or download anything extra.

lxc setup

The command line tools to control lxc containers are in the lxc package:
# apt-get install lxc
Then you need to create a special filesystem for the tools to work. Create a /cgroup directory and then add the following to /etc/fstab to have it mounted automatically on each boot:
cgroup        /cgroup        cgroup        defaults    0    0
and mount it manually:
# mount /cgroup
The cgroups filesystem allows to manage control groups which provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour. This is the base of the container implementation used by lxc. For the curious you will find more informations about cgroup in the kernel documentation:

Network setup

To setup the networing, you will need bridge tools:
# apt-get install bridge-utils
First you need to configure a bridge to be used between the VMs and the host system. Change /etc/network/interfaces to declare your bridge interface br0 which includes your ethernet card eth0 like this:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
#allow-hotplug eth0
#iface eth0 inet dhcp

# Setup bridge
auto br0
iface br0 inet dhcp
   bridge_ports eth0
   bridge_fd 0
Adapt to your local network setup.

Creating your first Debian sid container

One way to install a Debian distribution on disk is to use debootstrap.
# apt-get install debootstrap
With this tool, you will install a minimal set of packages to be able to access your container via ssh:
# debootstrap --verbose --variant=minbase --arch=i386 --include ifupdown,locales,libui-dialog-perl,\
dialog,dhcp3-client,netbase,net-tools,iproute,openssh-server \
sid /var/lib/lxc/sid
I: Base system installed successfully.
To define your sid container, create a config file under /etc/lxc/sid.conf to set where is the root directory location, various settings on devices and how to create the network device:
lxc.tty = 6
lxc.pts = 1024
lxc.rootfs = /var/lib/lxc/sid
lxc.cgroup.devices.deny = a
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rwm
# network = veth = up = br0 = eth0 = 00:FF:12:34:56:78
Configure networking on your container in /var/lib/lxc/sid/etc/network/interfaces:
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
Then execute the following commands to disable not needed services:
# rootfs=/var/lib/lxc/sid
# chroot $rootfs /usr/sbin/update-rc.d -f umountfs remove
# chroot $rootfs /usr/sbin/update-rc.d -f remove
# chroot $rootfs /usr/sbin/update-rc.d -f remove
Simplify /etc/inittab because you don’t need virtual console and other stuff:
# /etc/inittab: init(8) configuration.
# $Id: inittab,v 1.91 2002/01/25 13:35:21 miquels Exp $

# The default runlevel.

# Boot-time system configuration/initialization script.
# This is run first except when booting in emergency (-b) mode.

# What to do in single-user mode.

# /etc/init.d executes the S and K scripts upon change
# of runlevel.
# Runlevel 0 is halt.
# Runlevel 1 is single-user.
# Runlevels 2-5 are multi-user.
# Runlevel 6 is reboot.

l0:0:wait:/etc/init.d/rc 0
l1:1:wait:/etc/init.d/rc 1
l2:2:wait:/etc/init.d/rc 2
l3:3:wait:/etc/init.d/rc 3
l4:4:wait:/etc/init.d/rc 4
l5:5:wait:/etc/init.d/rc 5
l6:6:wait:/etc/init.d/rc 6
# Normally not reached, but fallthrough in case of emergency.

# What to do when CTRL-ALT-DEL is pressed.
ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now

# /sbin/getty invocations for the runlevels.
# The "id" field MUST be the same as the last
# characters of the device (after "tty").
# Format:
#  :::

1:2345:respawn:/sbin/getty 38400 console
Add a password for the root user to be able to do something on your system:
# rootfs=/var/lib/lxc/sid
# chroot $rootfs passwd
So now you are ready to launch your container:
# lxc-start -n sid -f /etc/lxc/sid.conf
INIT: version 2.88 booting
Using makefile-style concurrent boot in runlevel S.
Activating swap...done.
Mount point '/dev/console' does not exist. Skipping mount. ... (warning).
Mount point '/dev/ptmx' does not exist. Skipping mount. ... (warning).
Activating lvm and md swap...done.
Checking file systems...fsck from util-linux 2.19.1
Mounting local filesystems...done.
Activating swapfile swap...done.
Cleaning up temporary files....
Cleaning up ifupdown....
Setting up networking....
Configuring network interfaces...Internet Systems Consortium DHCP Client 4.1.1-P1
Copyright 2004-2010 Internet Systems Consortium.
All rights reserved.
For info, please visit

Listening on LPF/eth0/00:ff:12:34:56:78
Sending on   LPF/eth0/00:ff:12:34:56:78
Sending on   Socket/fallback
DHCPREQUEST on eth0 to port 67
bound to -- renewal in 1671 seconds.
Cleaning up temporary files....
Setting kernel variables ...done.
INIT: Entering runlevel: 2
Using makefile-style concurrent boot in runlevel 2.
Starting OpenBSD Secure Shell server: sshd.

Debian GNU/Linux wheezy/sid debian console

debian login:
If you login on the root account, you can now do whatever you want in your container like in a normal Linux system under Debian sid.
If you want to stop your container do the following in another terminal:
# lxc-stop -n sid

Integration into the boot sequence

To have your containers started at boot, you have to edit /etc/default/lxc the folllowing way:
# Comment out to run the lxc init script

# Directory containing the container configurations

# Start /etc/lxc/example.conf, /etc/lxc/autostart.conf, etc.
Then the lxc init script will take care to start and stop containers when needed. You can do it manually if you want:
# /etc/init.d/lxc start
Starting LXC containers: sid.
# /etc/init.d/lxc info
Info on LXC containers: lxc.
'sid' is RUNNING
# /etc/init.d/lxc stop
Stopping LXC containers: sid.
# /etc/init.d/lxc info
Info on LXC containers: lxc.
'sid' is STOPPED
In other articles we will present advanced usages of lxc. Stay tuned !

No comments:

Post a Comment