Wednesday, June 17, 2009

Howto install a xVM Ops Center proxy in a local zone.

xVM Ops Center 2.1 has a well designed and flexible architecture with its satellite controller, proxies and agents. But the zoning support is quite poor. It is only supported to put the satellite controller in a local zone, the proxy needs to be installed in a global zone.

If Solaris container are your preferred virtualization technology, like in my company, it could be that you don't like it, if you have to deploy a proxy in the global zone along with many local zones. So the only solution is perhaps, to put the proxy on an exclusive server hardware. If you consider HA, multiple datacenters, costs and the very low resource utilization of the proxy services itself, you will not love this solution, too.

AFAIK, the proxy of 2.1 can't run in a local zone, because the shipped JET needs a local NFS-server and the Solaris NFS-Server can't run in a local zone. But luckily, the latest version of JET (4.7) now supports remote NFS-servers.

This Howto outlines how to install the xVM Ops Center proxy in a local zone, how to upgrade to JET 4.7 and how to use a remote NFS-server. This setup works quite fine in our environment, so far.

Please consider, that this Howto should only be a proof of concept, Ops Center is not indented for editing scripts by your own. Of course you will get into trouble if you open support requests at SUN for your modified Ops Center setup. If you want support for a zoned proxy, open a feature request at SUN. If enough people request this feature, it will get a higher priority and I guess it will be implemented soon.


HOWTO


Consider for reading

10.156.64.20 is the IP of the remote NFS-Server.

10.156.64.42 is the IP of the XVMOC Proxy.

10.156.64.41 is the IP of the XVMOC Enterprise Controller.


Install Zone


Some devices are needed in the zone.

The ISC-dhcpd needs the NIC cloning device, for e1000g, e.g. /dev/e1000g. Some JET-scripts need also the devices for lofiadm (/dev/lofictl, /dev/lofi/*, /dev/rlofi/*).

For DHCP also exlusive NICs are needed, you need for each (v)lan a NIC, e.g. e1000g2 for the „ILO“-lan and e1000g1 for the OC/JET/NFS-lan.

So create a zone config:

bash-3.00# zonecfg -z proxy info
zonename: proxy
zonepath: /zones/proxy

brand: native
autoboot: false
bootargs:
pool:
limitpriv:
scheduling-class:
ip-type: exclusive
net:
address not specified
physical: e1000g2
defrouter not specified
net:
address not specified
physical: e1000g1
defrouter not specified
device
match: /dev/e1000g
device
match: /dev/lofictl
device
match: /dev/lofi/*
device
match: /dev/rlofi/*


Install and boot it:


zoneadm -z proxy install
zoneadm -z proxy boot

Configure it:

zlogin -C proxy

Config for e1000g2

Use DHCP: No
Host name: proxy
IP address: 10.156.64.42
System part of a subnet: Yes
Netmask: 255.255.240.0
Enable IPv6: No
Default Route: Specify one
Router IP Address: 10.156.64.1

After configuring with „zlogin -C“, setup also the second interface for the „ILO“-lan.

Persitent or e.g

ifconfig e1000g1 10.156.16.42/20 up

Install TFTP-Server

Packages should be already installed, copy the SMF manifest from the global zone, to the proxy zone and import it:

svccfg import tftp-udp6.xml

Install Proxy

You need the XVMOC 2.1 installer. I don't know why, but you have to create some directories before executing the installer:

mkdir -p /opt/sun/n1gc/pkgs/var/sadm/install
mkdir -p /var/opt/sun/xvm/osp/web/pub/
mkdir -p /var/opt/sun/xvm/osp/ssh/

bash-3.00# ./install -p
Sun Microsystems, Inc. Binary Code License Agreement


xVM Ops Center Proxy Server Installer (version 2.1.0.900 on SunOS)


1. Install Expect. [Completed]
2. Install IPMI tool. [Completed]
3. Install Agent components. [Completed]
4. Install application packages. [Completed]
5. Install Core Channel components. [Completed]
6. Install Proxy Server components. [Completed]
7. Install UCE Http proxy. [Completed]
8. Install OS provisioning components. [Completed]
9. Initialize (but do not start) services. [Completed]

xVM Ops Center Proxy Server installation is complete.
xVM Ops Center Proxy Server is now ready to be configured.



Now upgrading to JET 4.7 is necessary

pkgrm JetFLASH
pkgrm SUNWjet

Download Jet 4.7 and install only the SUNWjet and the JetFLASH package. Ignore share_nfs errors.

pkgadd -d jet.pkg

Migrate to remote NFS-Shares

On NFS-Server create following directories:

mkdir /opt/SUNWjet
mkdir /var/js
mkdir -p /var/opt/sun/xvm/osp/share/allstart

On NFS-Server, share this directories, the proxy zone needs write access, the future JET-clients only read access.

share -F nfs -o ro,anon=0,rw=proxy -d "Allstart Share" /var/opt/sun/xvm/osp/share/allstart
share -F nfs -o ro,anon=0,rw=proxy -d "Allstart Share" /var/js
share -F nfs -o ro,anon=0,rw=proxy -d "JET Framework" /opt/SUNWjet

On Proxy Zone:

  1. Mount this shares somewhere.

  2. Move the local content to the mounted shares

  3. Delete the local content, e.g. rm /opt/SUNWjet/*

  4. Unmount the NFS-shares

  5. Remount the NFS-Shares on the now emtpy mountpoints (/var/opt/sun/xvm/osp/share/allstart, /var/js, /opt/SUNWjet)
    mount -F nfs nfsserver:/opt/SUNWjet /opt/SUNWjet
    mount -F nfs nfsserver:/var/js /var/js
    mount -F nfs nfsserver:/var/opt/sun/xvm/osp/share/allstart \
    /var/opt/sun/xvm/osp/share/allstart



Configure and start xvmoc proxy

See also: XVMOC-WIKI http://wikis.sun.com/display/xvmOC2dot1/Configuring+an+Enterprise+Controller+for+Updates+%28Optional%29

/opt/SUNWxvmoc/bin/proxyadm configure -s 10.156.64.41 -u root -p /var/tmp/xVM/mypasswd -a 10.156.64.42



Edit file: /opt/SUNWjet/etc/jumpstart.conf

JS_CFG_SVR=10.156.64.20
JS_CLIENT_BOOT="remote"
JS_PKG_DIR="10.156.64.20:/var/js/pkg"
JS_PATCH_DIR="10.156.64.20:/var/js/patch"



For some reason the „add_install_client“ from the Solaris Update6 DVD checks if the JET-filesystems are local ones, this breaks the installation, so we have to fix it. The Fix manipulates the return value. This procedure is necessary after every „OS-Image import“.

File: /var/js//Solaris_10/Tools/add_install_client

under the line „df -l ${IMAGE_PATH} > /dev/null 2>&1“ insert:

echo "====FIX====="

so it looks like:

df -l ${IMAGE_PATH} > /dev/null 2>&1
echo "====FIX====="
if [ $? -ne 0 ] ; then


Finish

Now create a OS-Profile for Solaris 10 with this extra custom JET-parameters:

base_config_media=10.156.64.20
base_config_client_boot=remote



You should be able to deploy this OS-Profile, now.

0 comments:

Post a Comment