My Home Test Lab – Part 2

Written by Tao Yang

This is the part 2 of my 2-part series on how my home test lab is configured.

Part 1 can be found here.

In this second part, I’m going to talk about my previous experience with vyatta virtual router appliance and how I replaced vyatta with CentOS.

Disclaimer:

The content of this article is purely based on my personal experience and opinions. I have absolutely no intentions to criticise vyatta. To be honest, I still think it’s a great product, however, it just does not suit my needs in my lab environment.

About Vyatta:

Stefan Stranger has written a great article on Vyatta Virtual Router on Hyper-V back in 2008. The version Stefan used in his article was 4.0 and when I am writing this article, the latest version is 6.4. It can be downloaded here: http://www.vyatta.org/downloads

Vyatta is extremely light-weight. In my previous environment, I only needed to assign 256MB of RAM to each Vyatta instance. It is also very easy to setup. for me to configure an instance from scratch, would take me no more than 10 minutes.

Below is a list the setup commands I had to run to configure a Vyatta from scratch (based on my previous lab configuration):

configure
set system host-name vyatta
set interfaces ethernet eth0 address 192.168.1.254/24
set interfaces ethernet eth1 address 192.168.2.254/24
set interfaces ethernet eth2 address 192.168.3.254/24
set interfaces ethernet eth3 address 192.168.6.254/24
set service ssh
set service telnet
set system name-server 192.168.2.10
set system name-server 192.168.4.10

set system gateway-address 192.168.1.1

set protocols static route 0.0.0.0/0 next-hop 192.168.1.1
set protocols static route 192.168.1.0/24 next-hop 192.168.1.254
set protocols static route 192.168.2.0/24 next-hop 192.168.2.254
set protocols static route 192.168.3.0/24 next-hop 192.168.3.254
set protocols static route 192.168.4.0/24 next-hop 192.168.1.253
set protocols static route 192.168.5.0/24 next-hop 192.168.1.253
set protocols static route 192.168.6.0/24 next-hop 192.168.6.254
set protocols static route 192.168.7.0/24 next-hop 192.168.1.252
set protocols static route 192.168.8.0/24 next-hop 192.168.1.252
set protocols static route 192.168.9.0/24 next-hop 192.168.1.252

set service snmp community public
set service dhcp-relay server 192.168.4.10
set service dhcp-relay interface eth0
set service dhcp-relay interface eth1
set service dhcp-relay interface eth2
set service dhcp-relay interface eth3

set system login user vyatta authentication plaintext-password password1234

commit
save

So why am I moving away from Vyatta? the short answer is: Vyatta does not officially support Hyper-V:

image

Read between the lines, Hyper-V is not supported.

What does it mean in my lab Hyper-V 2008 R2 hosts in the past (Based on version 6.1 that I have implemented)?

  • After I reboot the Vyatta VM, all configurations were lost. It needed to be reconfigured again from scratch (that’s why I became so good at its commands). To work around this issue, I created a VM snapshot after it’s fully configured and I had to revert it back to the snapshot after every reboot.
  • It does not support Hyper-V Synthetic NICs. This means I’m stuck with legacy NICs for Vyatta. Legacy NICs means 100mb/s instead of 10GB/s and I can only assign maximum 4 NICs to the Vyatta instance. This is why I’ve only got 4 virtual switches configured for each Hyper-V host in my lab.
  • Vyatta is a cut down version of Linux, I could not install Linux Integration Service for Hyper-V. Otherwise, I might have already fixed the above mentioned 2 issues.
  • Besides myself, two of my colleagues also tried to “Install” Vyatta 6.4 on the VHD (as oppose to run the Live CD mode). All 3 of us had the same issue: it runs OK, but the network latency caused by Vyatta is unacceptable. The ping latency from subnet A to subnet B in the same Hyper-V host gets to 2-4 seconds (2000-4000ms).

My colleague Matt McGowan spent sometime over a weekend and tried all versions of Vyatta after version 6 on a Hyper-V 2012 server, according to him, none of them could even boot up without disconnect all the legacy NICs first. This has become the last straw for me to give up on Vyatta. I had to find a better solution for my Hyper-V environment.

At that time, I was seriously thinking about buying a layer 3 managed switch (HP 1910-16G), which costs $400 AUD in my local computer shop.  In the end, I’m glad I didn’t. After spoken to my good friend Zheng Han who is a RHCE and VCP, he advised me to take a look at CentOS.

So long story short, I’m hopeless when comes to Linux / Unix. I haven’t really done much with it ever since I graduated from uni. After a week or so playing with the latest version (CentOS 6.3) and learning how to use “vi”, with Zheng’s help, I got it working.

You’ve probably already seen the network diagram for my lab from Part 1. Here’s a logical diagram for the 3 Hyper-V hosts and the CentOS router in each host:

Hyper-V Logical View

The Visio diagram of the above view can be downloaded here.

Now, I’ll use HyperVRT01 (the router on HyperV01) as an example and go through the steps of setting it up.

The following software is required:

  1. CentOS 6.3 (CentOS-6.3-x86_64-bin-DVD1.iso) –can be downloaded from a mirror site near you.
  2.  Linux Integration Service For Hyper-V 3.4 (LinuxICv34.iso) – download from Microsoft.

Other Network information:

I have 2 domain controllers in my lab domain:

DC01: 192.168.4.10

DC02: 192.168.2.10

DC01 is also configured as a DHCP server serving multiple scopes in my lab. You’ll see the IP address of these 2 machines a lot in below steps.

*Note: if you are going to use CentOS and you are like me, a Linux noob, before you start, please make sure you get familiar with “vi” editor because it is heavily used.

Now, let’s start…

1. Create a new virtual machine in Hyper-V with the following settings:

  • CPU: 1 virtual processor
  • Memory: 512MB static
  • Hard disk: 10GB (after install, I checked and only used 3GB)
  • Assign 4 network adapters (note, DO NOT use legacy network adapters):
  • Assign the 4 network adapters to virtual switches IN ORDER:
    • #1: 192.168.1.0
    • #2: 192.168.7.0
    • #3: 192.168.8.0
    • #4: 192.168.9.0

image

2. Mount the CentOS-6.3-x86_64-bin-DVD1.iso to the VM

3. Power on the VM to start installing CentOS

  • Choose “Install system with basic video driver”

image

*Note: if you choose the first option to use the GUI based install wizard, you’ll need to assign minimum 1GB of memory to the VM. GUI based install won’t run on 512MB RAM.

  • the rest of the install process is pretty much fool proof. I won’t waste my time going through the entire CentOS 6 install here.

Assume now CentOS is installed.

4. Install Linux Integration Service for Hyper-V:

  1. Mount LinuxICv34.ISO to the guest OS (HyperVRT01)
  2. Use the following command to install:
mount /dev/cdrom /media
cd /media/RHEL63
./install.sh
reboot

5. Disable Firewall

service iptables stop
chkconfig iptables off

6. Configure DNS

echo “nameserver 192.168.4.10” >/etc/resolv.conf
echo “nameserver 192.168.2.10” >>/etc/resolv.conf

7. Network settings

vi /etc/sysconfig/network then insert

HOSTNAME=”hypervrt01.corp.tyang.org”
NETWORKING=yes
NETWORKING_IPV6=no
GATEWAYDEV=eth0
GATEWAY=192.168.1.1

image

8. Set IP Address:

  • Start up all NICs:
ifconfig eth0 up
ifconfig eth1 up
ifconfig eth2 up
ifconfig eth3 up
  • eth0

vi /etc/sysconfig/network-scripts/ifcfg-eth0 then insert

DEVICE=eth0
IPADDR=192.168.1.252
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
BROADCAST=192.168.1.255
DNS1=192.168.4.10
DNS2=192.168.2.10
ONBOOT=yes
NAME=External
TYPE=”Ethernet”
IPV6INIT=no

image

*Hint: Configure the first NIC eth0, then you can use Putty to connect via SSH. once in Putty, you can copy & paste commands.

  • eth1

vi /etc/sysconfig/network-scripts/ifcfg-eth1 then insert

DEVICE=eth1
IPADDR=192.168.7.254
NETMASK=255.255.255.0
BROADCAST=192.168.7.255
GATEWAY=192.168.1.252
DNS1=192.168.4.10
DNS2=192.168.2.10
ONBOOT=yes
NAME=VLAN7
TYPE=”Ethernet”
IPV6INIT=no
  • eth2

vi /etc/sysconfig/network-scripts/ifcfg-eth2 then insert

DEVICE=eth2
IPADDR=192.168.8.254
NETMASK=255.255.255.0
BROADCAST=192.168.8.255
GATEWAY=192.168.1.252
DNS1=192.168.4.10
DNS2=192.168.2.10
ONBOOT=yes
NAME=VLAN8
TYPE=”Ethernet”
IPV6INIT=no
  • eth3

vi /etc/sysconfig/network-scripts/ifcfg-eth3 then insert

DEVICE=eth3
IPADDR=192.168.9.254
NETMASK=255.255.255.0
BROADCAST=192.168.9.255
GATEWAY=192.168.1.252
DNS1=192.168.4.10
DNS2=192.168.2.10
ONBOOT=yes
NAME=VLAN9
TYPE=”Ethernet”
IPV6INIT=no
  • Restart network service
service network restart

Make sure all NICs start up OK:

image

9. OS Update

yum update

10. Enable IP forwarding (Routing)

  • Check if routing is enabled:
cat /proc/sys/net/ipv4/ip_forward

0 = disabled

1 = enabled

  • To enable routing:

vi /etc/sysctl.conf Then Edit:

net.ipv4.ip_forward = 1

SNAGHTMLf993100

11. Configure Route:

vi /etc/sysconfig/network-scripts/route-eth0 and insert:

192.168.2.0/24 via 192.168.1.254 dev eth0
192.168.3.0/24 via 192.168.1.254 dev eth0
192.168.4.0/24 via 192.168.1.253 dev eth0
192.168.5.0/24 via 192.168.1.253 dev eth0
192.168.6.0/24 via 192.168.1.254 dev eth0

image

*Note: above list represent all subnets in the other 2 Hyper-V servers in my lab.

12. Restart Network Service again

service network restart

13. Configure DHCP Relay

  • Install DHCP
yum install dhcp
  • Configure DHCP Relay service (dhcrelay)

vi /etc/sysconfig/dhcrelay and Modify:

INTERFACES=”eth0 eth1 eth2 eth3″
DHCPSERVERS=”192.168.4.10″

SNAGHTMLf9fb07b

  • Start DHCP Relay service (dhcrelay)
chkconfig dhcrelay on
service dhcrelay start

14. Configure SNMP (Optional)

  • Install SNMP:
yum install net-snmp-utils
yum install net-snmp
  • Backup SNMP Config File

mv /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.org

  • Create new config file :

vi /etc/snmp/snmpd.conf and insert:

rocommunity  public
syslocation  “TYANG.ORG Head Office”
syscontact  your@email.com
  • Start SNMP service
service snmpd start
chkconfig snmpd on

15. Install Webmin (Optional)

  • Install the GPG key
rpm –import http://www.webmin.com/jcameron-key.asc
  • Add webmin repository
    vi /etc/yum.repos.d/webmin.repo and add:
[Webmin]
name=Webmin Distribution Neutral
#baseurl=http://download.webmin.com/download/yum
mirrorlist=http://download.webmin.com/download/yum/mirrorlist
enabled=1

image

  • update the repos:
yum update
  • Install webmin
yum install webmin
  • access webmin page:

http://<ipaddress>:10000

This is it. the router is now setup!

After setup, check system status via webmin:

image

As you can see, after everything is configured, it only uses 136MB of memory and 3GB of disk space.

Routing and Gateways:

SNAGHTMLfafef30

NIC configurations:

image

Additional, I made sure a windows client OS could obtain an IP address from the DHCP server (which is located on another Hyper-V host). and I was able to ping / trace route other VMs in other Hyper-V servers (I have already demonstrated it in part 1).

This concludes this 2-part series. Before I go, I have to reiterate, there is nothing wrong with the Vyatta product, it’s just it does not integrate with Hyper-V too well. Unlike Vyatta, CentOS 6 is a fully supported guest OS in Hyper-V (With Linux Integration Service for Hyper-V 3.4) and CentOS 5 and 6 are also supported in SCOM 2012 SP1 beta! Having said that, Vyatta was running perfectly fine in my VMware workstation previously. If Vyatta adds support to Hyper-V in the future, I would definitely consider it again.

Lastly, please feel free to get in touch with me if you believe there are anything inaccurate in this series or you need more information in regards to the CentOS router setup.

10 comments on “My Home Test Lab – Part 2

  1. Hi,

    Try the latest Pacifica build from here http://ftp.het.net/iso/vyatta/pacifica/images/. I’ve just downloaded it and as I have seen so far, the boot issue has been fixed!!

  2. some other alternatives

    clearos (customised router/gateway distro based on CentOS) http://www.clearfoundation.com/Software/overview.html

    pfsense (same but freeBSD based..no hyperv support)
    http://www.pfsense.org/

  3. hello,

    I monitor Vyatta on Debian 6 OS through SCOM 2012, is there a way to monitor the backup live tunnel when become active ( ex when vyatta failover to backup line )

    Please advise.

    Rgds,
    Bashar

  4. Hi,

    How’s the performance of your Server 2012 hyper-V running 64gb?

    I’m planning to build two nodes with this kind of hardware.

    Thanks!

    • For the 64GB machine, I found the bottle neck is the disk I/O. Most likely you’ll start suffering from disk I/O before memory becomes an issue. Currently my memory utilisation is only 80% and I have 19 VM’s running on it. I’m a bit hesitated to add more VM’s on it because the disk performance is poor – given I have 2 SSDs and and 3 traditional SATA HDDs on it. so my recommendation is, if you are going to use all those memory, make sure you chuck some SSDs in.

  5. Hi Tao,

    Thanks for your response.

    I’ll be using this for about 5 to 6 virtual machines.

    I’m going to put 3x500gb SSD on raid 5.

    I think it can handle the job.

    You’ve got 19VM, that’s a lot for a test lab.

  6. Hi,

    That’s an excellent post! thanks for sharing!

    Quick question:

    Have you every configured Trunk mode on vvyatta interface? It looks like Vyatta community is facing issue with passing multiple VLAN IDs to the Vyatta Interface.

    Thanks!
    D

    • no, I haven’t tried it. and when I upgraded my hosts to Windows 2012 R2 couple of months ago, I tried the latest vyatta and seemed it didn’t support 2012 R2. I haven’t tried it again since then.

  7. esxi works perfect with vlan tags(vifs) . however 2012 still has issues with 6.5 and 6.6 vyatta

Leave a Reply