ESS Cluster Functional Description
# Please enter
the desired hostname for this machine.
# Please enter
the desired IP address for bge0.
# Please enter
the netmask for bge0.
# Please enter
the default router for bge0.
Please verify your configuration information:
hostname:
ip:
netmask:
router:
Are these correct? (y/n)
-------cut from here------
#!/sbin/sh
# /etc/rc2.d/S68net-tune
PATH=/usr/bin:/usr/sbin
echo "Implementing Solaris
ndd Tuning Changes "
# bge-Interfaces
# Force bge0
to 100fdx autoneg off
ndd -set /dev/bge0
adv_1000fdx_cap 0
ndd -set /dev/bge0
adv_1000hdx_cap 0
ndd -set /dev/bge0
adv_100fdx_cap 1
ndd -set /dev/bge0
adv_100hdx_cap 0
ndd -set /dev/bge0
adv_10fdx_cap 0
ndd -set /dev/bge0
adv_10hdx_cap 0
ndd -set /dev/bge0
adv_autoneg_cap 0
-------end script-------
# chmod 755 /etc/rc2.d/S68net_tune
#/etc/init.d/sshd stop
#/etc/init.d/sshd start
#cd /mnt
#chmod 777 *.sh
# ./user_config.sh
Enter password for user
ssmon.
New Password:
Re-enter new Password:
passwd: password successfully
changed for ssmon
Enter password for user
ssadmin.
New Password:
Re-enter new Password:
passwd: password successfully
changed for ssadmin
Enter password for user
ssconfig.
New Password:
Re-enter new Password:
passwd: password successfully
changed for ssconfig
Enter password for user
essadmin.
New Password:
Re-enter new Password:
passwd: password successfully
changed for essadmin
Enter password for user.
New Password:
Re-enter new Password:
passwd: password successfully
changed for user
#reboot --
-r
# exec bash
# export DISPLAY=<local_IP_address>:0.0
# /usr/openwin/bin/xhost +
#ssconsole
IMPORTANT:
[root@intracerR
CAM_linux]# ./RunMe.bin -c
Initializing
Wizard........
Launching
InstallShield Wizard........
-------------------------------------------------------------------------------
Sun StorageTek(TM)
Common Array Manager 6.2
The InstallShield Wizard
will install Sun StorageTek(TM)
Common Array Manager
on your computer.
To continue, choose
Next.
Sun
StorageTek(TM) Common Array Manager 6.2
Sun
Microsystems, Inc.
http://www.sun.com
Press 1 for Next, 3
to Cancel or 5 to Redisplay [1]
-------------------------------------------------------------------------------
Sun StorageTek(TM)
Common Array Manager 6.2
Please read the following
license agreement carefully.
Sun StorageTek(TM)
Common Array Manager
Copyright 2008 Sun
Microsystems, Inc. All rights reserved. Sun Microsystems,
Inc. has intellectual
property rights relating to technology embodied in the
product that is described
in this document. In particular, and without
limitation, these intellectual
property rights may include one or more of the
U.S. patents listed
at http://www.sun.com/patents and one or
more additional
patents or pending
patent applications in the U.S. and in other countries. U.S.
Government Rights -
Commercial software. Government users are subject to the
Sun Microsystems, Inc.
standard license agreement and applicable provisions of
the FAR and its supplements.
Use is subject to license terms. This distribution
may include materials
developed by third parties. Portions may be derived from
Berkeley BSD systems,
licensed from U. of CA. Sun, Sun Microsystems, the Sun
logo, Java, Solaris
and Sun StorageTek Common Array Manager are trademarks or
registered trademarks
of Sun Microsystems, Inc. in the U.S. and other
countries. All SPARC
trademarks are used under license and are trademarks or
registered trademarks
of SPARC International, Inc. in the U.S. and other
countries.
Please choose from
the following options:
[ ] 1
- I accept the terms of the license agreement.
[X] 2
- I do not accept the terms of the license agreement.
To select an item enter
its number, or 0 when you are finished: [0] 1
[X] 1
- I accept the terms of the license agreement.
[ ] 2
- I do not accept the terms of the license agreement.
To select an item enter
its number, or 0 when you are finished: [0]
Press 1 for Next, 2
for Previous, 3 to Cancel or 5 to Redisplay [1]
-------------------------------------------------------------------------------
Sun StorageTek(TM)
Common Array Manager 6.2
Choose the installation
type that best suits your needs.
[X] 1
- Typical
The
program will be installed with the suggested configuration.
Recommended
for most users.
[ ] 2
- Custom
The
program will be installed with the features you choose.
Recommended
for advanced users.
Select the number corresponding
to the type of install you would like: [0]
Press 1 for Next, 2
for Previous, 3 to Cancel or 5 to Redisplay [1]
-------------------------------------------------------------------------------
Checking current system
...
|-----------|-----------|-----------|------------|
0% 25% 50% 75% 100%
||||||||||||||||||||||||||||||||||||||||||||||||||
-------------------------------------------------------------------------------
Sun StorageTek(TM)
Common Array Manager 6.2
Software To Be Installed:
Full Install
* Browser
User Interface (BUI)
* Local and
Remote CLI
* Array Firmware
Press 1 for Next, 2
for Previous, 3 to Cancel or 5 to Redisplay [1] Preparing
for installation ...
Pre Uninstall Old Action
...
Removing old features
...
-------------------------------------------------------------------------------
Sun StorageTek(TM)
Common Array Manager 6.2
Installing Sun StorageTek(TM)
Common Array Manager 6.2. Please wait...
|-----------|-----------|-----------|------------|
0% 25% 50% 75% 100%
||||||||||||||||||||||||||||||||||||||||||||||||||
Installing Java 2 Standard
Edition
-------------------------------------------------------------------------------
Sun StorageTek(TM)
Common Array Manager 6.2
|-----------|-----------|-----------|------------|
0% 25% 50% 75% 100%
||||||||||||||||||||||||||||||||||||||||||||||||||
-------------------------------------------------------------------------------
Sun StorageTek(TM)
Host Software Installation Summary
View results:
Info:
Installation success.
The following have
been installed: Browser User Interface (BUI), Local and
Remote CLI, and Array
Firmware.
To access the Browser
User Interface point a browser at:
https://installation_host:6789
The logs may be found
in /var/opt/cam/
Press 3 to Finish or
5 to Redisplay [3]
vxinstall
The following prompt
appears:
Are you sure that
you want to reinstall [y,n,q,?] (default: n)
Type y if you want to
reinstall the Volume Manager.
Some licenses are
already installed. Do you wish to review them [y,n,q] (default:
y)
IMPORTANT:
Do you want to use
enclosure based name for all disks? [y,n,q,?] (default:
n)
Do you want to setup
a system wide default disk group? [y,n,q,?] (default:
y)
Enter a default
disk group name of rootdg. The installation of Veritas Volume Manager
is now successfully completed.
# for file
in llttab gabtab llthosts; do cp /etc/$file.server1 /etc/$file;
done
#/etc/init.d/gab stop
#/etc/init.d/llt.rc stop
#/etc/init.d/llt.rc start
#/etc/init.d/gab start
gabconfig –a
If you see a line
with membership “01”, then both nodes are talking.
If you see the message as membership “;1” or “0;” then
the node that has the “;” (semi-colon) is misconfigured. Verify
your configurations in the /etc/llttab, /etc/llthosts,
and /etc/gabtab files.
root@less4 # hastop –all
Then, copy the new
types.cf from the /mnt directory to /etc/VRTSvcs/conf/config.
root@less4 # hacf –verify /etc/VRTSvcs/conf/config/
root@less4 # hacf –generate /etc/VRTSvcs/conf/config/
Then execute hastart on
both the nodes.
root@less4 # hastatus
-sum
-- SYSTEM STATE
-- System State Frozen
A less3 RUNNING 0
A less4 RUNNING 0
-- GROUP STATE
-- Group System Prob AutoDisabled State
B LAPP less3 Y N OFFLINE
B LAPP less4 Y N OFFLINE
haconf -makerw
hares -add <APP_RES> Application lesssg
hares -modify <APP_RES> Enabled 1
hares -modify <APP_RES> PidFiles <PSMON_PID_PATH> FOR EG /users/ess/psmon.pid
hares -modify <APP_RES> StartProgram FOR EG /users/ess/start_serv.sh
hares -modify <APP_RES> StopProgram FOR EG
"/users/ess/serv forcestop"
hares -modify <APP_RES> User <USERNAME> For EG essadmin
hares -link <APP_RES> LESS-VIP
hares -link <APP_RES> lessmount
hares -disp <APP_RES>
The output for this
command will display all the required attributes.
hares -dep <APP_RES>
This command displays
the dependencies of the application.
hares -online <APP_RES> -sys <NODE_NAME>
haconf -dump makero
Also, perform a cat
of main.cf file.
A sample output is shown below.
Application
T5220-Application (
User = root
StartProgram = "/less/LESS/ess/start_serv.sh"
StopProgram = "/less/LESS/ess/serv forcestop"
PidFiles = { "/less/LESS/ess/psmon.pid" }
)
// resource
dependency tree
//
//
group T5220-SG
//
{
//
Application T5220-Application
//
{
//
IP T5220-IP
//
{
//
NIC T5220-NIC
//
}
//
Mount T5220-mount
//
{
//
Volume T5220-Vol
//
{
//
DiskGroup T5220-DG
//
}
//
}
//
}
//
}
hagrp -offline <RG_name> -sys <Node1>
hagrp -online <RG_name> -sys <Node2>
root@less4 # hastatus
-sum
-- SYSTEM STATE
-- System State Frozen
A less3 RUNNING 0
A less4 RUNNING 0
-- GROUP STATE
-- Group System Probed AutoDisabled State
B LAPP less3 Y N ONLINE
B LAPP less4 Y N OFFLINE
NotifierMngr LAPP-Notif-Mgr
(
SnmpdTrapPort = 162
SnmpConsoles = { "<SNMP_IP_Address>" = Information }
)
root@less3 # hacf –verify /etc/VRTSvcs/conf/config/
root@less3 # hacf –generate /etc/VRTSvcs/conf/config/
Then, restart both
the nodes by executing the following command:
root@less3 # hastart
root@less4 # hastart
For more details
on the installation of Veritas Volume Manager and Veritas Cluster configuration,
refer to the Veritas Documentation.
Tuning the VxFS File System for Better Performance
$ /opt/VRTS/bin/vxtunefs
-o read_pref_io=131072,read_unit_io=131072,write_pref_io=131072,write_unit_io=131072 /shared_db
IMPORTANT:
$ haconf -makerw
$hagrp -add less-ha
$hagrp –modify
less-ha SystemList <Node1> 0<Node2> 1
$hagrp –modify
less-ha NumRetries 1
Where, Node1 and Node2 are
the hostnames of the active and passive nodes.
$ hares -add less-apps-dg
DiskGroup less-ha
$ hares -modify less-apps-dg
DiskGroup apps_dg
$ hares -modify less-apps-dg
Enabled 1
$ hares -add less-apps-vol
Volume less-ha
$ hares -modify less-apps-vol
DiskGroup apps_dg
$ hares -modify less-apps-vol
Volume apps_vol
$ hares -modify less-apps-vol
Enabled 1
$ hares -add less-apps-mnt
Mount less-ha
$ hares -modify less-apps-mnt
MountPoint /shared_apps
$ hares -modify less-apps-mnt
BlockDevice /dev/vx/dsk/apps_dg/apps_vol
$ hares -modify less-apps-mnt
FSType vxfs
$ hares -modify less-apps-mnt
FsckOpt %-y
$ hares -modify less-apps-mnt
MountOpt largefiles
$ hares -modify less-apps-mnt
Enabled 1
$ hares -add less-app
Application less-ha
$ hares -modify less-app
User lessadmin
$ hares -modify less-app
StartProgram "/<installed dir>/ess/serv
start"
$ hares -modify less-app
StopProgram "/<installed dir>/ess/serv
forcestop"
$ hares -modify less-app
PidFiles "<installed dir>/ess/psmon.pid"
$ hares -modify less-app
Enabled 1
$ hares -add less-nic
NIC less-ha
$ hares -modify less-nic
Device eth0
$ hares -modify less-nic
Enabled 1
$ hares -add less-ip
IP less-ha
$ hares -modify less-ip
Device eth0
IMPORTANT:
$ hares -modify less-ip
Enabled 1
$ hares -link less-app
less-apps-mnt
$ hares -link less-apps-mnt
less-apps-vol
$ hares -link less-apps-vol
less-apps-dg
$ hares -link less-app
less-ip
$ hares -link less-ip
less-nic
$ haconf -dump -makero
$ hagrp -online less-ha
-sys <Node1>
$ vxdctl initdmp
$ vxdctl enable
To see the status
of the new disk, use the command:
$ vxdisk -o alldgs list
DEVICE
TYPE
DISK GROUP
STATUS
disk_0
auto:none
- -
online invalid
disk_1
auto:none
- -
online invalid
disk_2
auto:none
- -
online invalid
disk_3
auto:none
- -
online invalid
emc_clariion0_28 auto
- -
error
emc_clariion0_29 auto
-
- error
$ /etc/vx/bin/vxdisksetup
-i emc_clariion0_28
$ /etc/vx/bin/vxdisksetup
-i emc_clariion0_29
To see the status
of the new disk, use the command:
$ vxdisk -o alldgs list
DEVICE
TYPE
DISK GROUP
STATUS
disk_0
auto:none
- -
online invalid
disk_1
auto:none
- -
online invalid
disk_2
auto:none
- -
online invalid
disk_3
auto:none
- -
online invalid
emc_clariion0_28 auto:cdsdisk
- -
online
emc_clariion0_29 auto:cdsdisk
- -
online
$ vxdg init apps_dg apps_dg01=emc_clariion0_28
IMPORTANT:
$ vxassist -g apps_dg
make apps_vol 299g
$ mkfs -F vxfs -o bsize=4096,largefiles /dev/vx/rdsk/apps_dg/apps_vol
On Linux:
$ mkfs -t vxfs -o bsize=4096,largefiles /dev/vx/rdsk/apps_dg/apps_vol
For
better performance, use a 4Kb block size and enable support for
large files (more than 1 TB).
$ mount -F vxfs -o largefiles /dev/vx/dsk/apps_dg/apps_vol /shared_apps
On Linux
$ mount -t vxfs -o largefiles /dev/vx/dsk/apps_dg/apps_vol /shared_apps
#!/bin/sh
## script
to monitor the status of Veritas... if both nodes are offline, force
to start
## the
number one (1) node.
## put
this in crontab as:
##
0,15,30,45 * * * * /export/home/scripts/keep_vcs_active.sh
2>>/var/adm/messages
VCS=' [VCS] == '
CHECK_VCS=`hastatus
-sum|grep -c ONLINE`
DATE=`date
"+%m/%d/%Y %T"`
echo $DATE $VCS
"Checking for Veritas started..." >> /var/adm/messages
if [ ${CHECK_VCS} -eq
0 ]
then
DATE=`date
"+%m/%d/%Y %T"`
echo $DATE $VCS
"Both nodes are offline... Making first node active" >> /var/adm/messages
hagrp -clear LAPP
>> /var/adm/messages
hagrp -online LAPP
-sys less3
DATE=`date
"+%m/%d/%Y %T"`
echo $DATE $VCS
"First node activated. " >> /var/adm/messages
else
DATE=`date
"+%m/%d/%Y %T"`
echo $DATE $VCS
"Veritas is running normally." >> /var/adm/messages
fi
# chmod 755 /export/home/scripts/keep_vcs_active.sh
# crontab –l
> /tmp/CRON
0,15,30,45 * * * * /export/home/scripts/keep_vcs_active.sh
2>>/var/adm/messages
# crontab /tmp/CRON
On executing the
script, the status of the Veritas cluster can be verified every
15 minutes.
#shutdown –g0 –y –i6
# vxdisksetup
-i c1t1d0 format=sliced
# vxdg -g
rootdg adddisk rootmirror=c1t1d0s2
# vxrootmir
-v rootmirror
The output should
be similar to the following:
! vxassist -g bootdg
mirror rootvol layout=contig,diskalign rootmirror
! vxbootsetup -g bootdg
-v rootmirror
! vxmksdpart -f -v -g
rootdg rootmirror-01 0 0x2 0x200
! vxpartadd /dev/rdsk/c1t1d0s2
0 0x2 0x200 20352 62918208
! /usr/sbin/installboot /usr/platform/SUNW,Netra-T5220/lib/fs/ufs/bootblk /dev/rdsk/c1t1d0s0
! vxeeprom devalias
vx-rootmirror /dev/dsk/c1t1d0s0
IMPORTANT:
# vxmirror
rootdisk rootmirror
The output should
be as follows:
! vxassist -g defaultdg
mirror swapvol rootmirror
! vxassist -g defaultdg
mirror export rootmirror
IMPORTANT:
Testing Veritas Cluster
root@LESS1 # hastatus
attempting to connect....connected
group resource
system message
-------- --------------------
-------------------- --------------------
JPTRFLGN-LESS1 RUNNING
JPTRFLGN-LESS2 RUNNING
LAPP
JPTRFLGN-LESS1 OFFLINE
LAPP
JPTRFLGN-LESS2 ONLINE
-----------------------------------------------------------------------
LAPP
JPTRFLGN-LESS1 OFFLINE
LAPP
JPTRFLGN-LESS2 ONLINE
LAPP-app-ess
JPTRFLGN-LESS1 OFFLINE
LAPP-app-ess
JPTRFLGN-LESS2 ONLINE
LAPP-vmdg-lessdg
JPTRFLGN-LESS1 OFFLINE
-----------------------------------------------------------------------
LAPP-vmdg-lessdg
JPTRFLGN-LESS2 ONLINE
LAPP-ip-vip_ext
JPTRFLGN-LESS1 OFFLINE
LAPP-ip-vip_ext
JPTRFLGN-LESS2 ONLINE
LAPP-mnt-less
JPTRFLGN-LESS1 OFFLINE
LAPP-mnt-less
JPTRFLGN-LESS2 ONLINE
-----------------------------------------------------------------------
LAPP-mnt-less-bk
JPTRFLGN-LESS1 OFFLINE
LAPP-mnt-less-bk
JPTRFLGN-LESS2 ONLINE
LAPP-nic-bge0
JPTRFLGN-LESS1 ONLINE
LAPP-nic-bge0
JPTRFLGN-LESS2 ONLINE
LAPP-app-ess
JPTRFLGN-LESS1 OFFLINE
-----------------------------------------------------------------------
LAPP-app-ess
JPTRFLGN-LESS2 ONLINE
LAPP-vmdg-lessdg
JPTRFLGN-LESS1 OFFLINE
LAPP-vmdg-lessdg
JPTRFLGN-LESS2 ONLINE
LAPP-ip-vip_ext
JPTRFLGN-LESS1 OFFLINE
LAPP-ip-vip_ext
JPTRFLGN-LESS2 ONLINE
-----------------------------------------------------------------------
LAPP-mnt-less
JPTRFLGN-LESS1 OFFLINE
LAPP-mnt-less
JPTRFLGN-LESS2 ONLINE
LAPP-mnt-less-bk
JPTRFLGN-LESS1 OFFLINE
LAPP-mnt-less-bk
JPTRFLGN-LESS2 ONLINE
LAPP-nic-bge0
JPTRFLGN-LESS1 ONLINE
-----------------------------------------------------------------------
LAPP-nic-bge0
JPTRFLGN-LESS2 ONLINE
IMPORTANT:
root@LESS1 # hastatus –sum
-- SYSTEM STATE
-- System State Frozen
A JPTRFLGN-LESS1 RUNNING 0
A JPTRFLGN-LESS2 RUNNING 0
-- GROUP STATE
-- Group System Probed AutoDisabled State
B LAPP JPTRFLGN-LESS1 Y N OFFLINE
B LAPP JPTRFLGN-LESS2 Y N ONLINE
ESS Cluster Failure Handling