Provisioning secure cloud instances locally

December 12, 2012

Hello,

I decided to deploy Ray Cloud Browser on a local virtual machine.

This guide is for installing and using libvirt on Fedora 17.

The guest instance in this tutorial is OpenBSD 5.2 as it is very lightweight and secure.

To use NAT with a bridge that has no interface, virsh net-start must be run as root.

Connecting to the host (Fedora 17)

First we connect to ls31 (Fedora 17). ls31 is the host on which we will run domain 0. In our case, ls31 has 64 VCPU (AMD Opteron(TM) Processor 6272) and 128 GiB of memory.

$ ssh -lboiseb01 192.168.3.31 -X

Then, we need to install virtualization packages.

$ sudo su

Installing packages

# yum install -y @virtualization

We will put disks in /kvm/img and cdroms in /kvm/iso.

Installing the guest (OpenBSD 5.2)

# mkdir -p /kvm/{iso,img}

Then we install the virtual machine with 1024 MiB of memory and a 8 GiB disk and graphics.
We'll remove the graphics once ssh is working.

# virt-install --name test-1 --ram 1024 --disk path=/kvm/img/test-1.img,size=8 --cdrom /kvm/iso/OpenBSD-5.2.iso --graphics vnc

Now we can check that it is running

# virsh list
Id Name State
----------------------------------
1 test-1 running

Now, we start the viewer. 1 is the domain number, domain 0 is the host.
We can do this as a normal user.

$ vncviewer :0

Inside the guest, we basically just install OpenBSD 5.2. I prefer to use only 1 mount point for all the file system.

Installing vim, the editor

Once this is over, reboot the guest, and install vim with this (in OpenBSD 5.2):

# export PKG_PATH=http://ftp.openbsd.org/pub/OpenBSD/5.2/packages/`machine -a`/
# pkg_add vim

Setup keys for the guest

Now, we create a key pair on the client (can be the host).

$ ssh-keygen -t rsa -f openbsd-vm.pem -P ""

Then, we add the content of openbsd-vm.pem.pub to ~/.ssh/authorized_keys in the guest.

vim ~/.ssh/authorized_keys

Finally, we change OpenBSD passwords to something silly and only use the key to connect to the guest with (this will work after adding NAT with iptables):

ssh -i openbsd-vm.pem seb@192.168.3.31 -p 23422

To stop a virtual machine, the best way is to use halt or shutdown, otherwise:

virsh destroy 1

To start a virtual machine, use

virsh start test-1

Network

By default, the guest can use the network of the host (domain 0) and a bridge is configured for the other way too. But the bridge only works on the host.

Enable port forwarding on the host (domain 0)

# sysctl -w net.ipv4.ip_forward=1

To forward a specific port of the host to a guest:

If the virtual machine runs with a virsh network, than its ipv4 address is NAT'ed by default.

Route frames from 192.168.3.31:23422 to 192.168.122.234:22

# iptables -t nat -A PREROUTING -p tcp -d 192.168.3.31 --dport 23422 -j DNAT --to 192.168.122.234:22

Redirect local trafic too so that we can connect to the guest from 192.168.3.31 too

# iptables -t nat -A OUTPUT -p tcp -d 127.0.0.1 --dport 23422 -j DNAT --to 192.168.122.234:22
# iptables -t nat -A OUTPUT -p tcp -d 192.168.3.31 --dport 23422 -j DNAT --to 192.168.122.234:22

Accept payload to 192.168.122.234:22

# iptables -A FORWARD -p tcp -d 192.168.122.234 --dport 22 -j ACCEPT

Voilà, we just configured a OpenBSD 5.2 guest with libvirt, and configured routes so that 192.168.3.31:23422 goes to 192.168.122.234:22

To connect to the guest instance:

$ ssh -i openbsd-vm.pem seb@192.168.3.31 -p 23422