Veritas
volume manager
Veritas
volume manager
Veritas volume manager requires that each disk be part of a diskgroup. Furthermore, there is a special diskgroup called "rootdg" which must exist for VxVM to function. The administrator can create additional diskgroups, but the first diskgroup must be named rootdg.
Veritas recommends that the rootdg diskgroup contain the encapsulated operating system disk and its mirror. In order to preserve the data on the operating system disk, the vxinstall script encapsulates the root disk. This preserves the operating system while at the same time creating the private and public disk regions that Veritas requires. However, the process of encapsulation fundamentally alters the structure of the root disk.
Furthermore, the root disk is typically the only disk on the system that is encapsulated. Typically, every other disk is initialized by VxVM, and then user data is placed onto it. The administrator soon comes to understand the nature of initialized disks, but does not get sufficient exposure to an encapsulated disk. This lack of exposure makes it difficult to maintain the critical root disk in the event of a disk failure.
We will demonstrate alternative approaches to managing the rootdg diskgroup:
With this approach, both Solstice DiskSuite and Veritas volume manager are installed on a single host. DiskSuite is used to mirror the operating system, and Veritas volume manager is used to manage all the application data. This approach uses the strengths of each product, and requires that the administrative staff understands both products.
Note that the use of a single-slice rootdg is considered an unsupported configuration by both Veritas and Sun. The "upgrade-start" and "vxunroot" scripts will not work if you use the single-slice rootdg. However, the fact that the operating system is not under control of VxVM obviates the need for "upgrade-start" and "vxunroot" scripts.
So long as you understand what you are doing, the single-slice rootdg solution solves more problems than it creates.
We recommend that you install the operating system so that there are three unused slices of roughly 10MB in size. Two of these slices will be used for DiskSuite state database replicas, and the final slice will be used to the single-slice rootdg diskgroup.
# vxconfigd -m disable # vxdctl init # vxdg init rootdg # vxdctl add disk c0t0d0s7 type=simple # vxdisk -f init c0t0d0s7 type=simple # vxdg adddisk c0t0d0s7 # vxdctl add disk c0t1d0s7 type=simple # vxdisk -f init c0t1d0s7 type=simple # vxdg adddisk c0t1d0s7 # vxdctl enable # rm /etc/vx/reconfig.d/state.d/install-db
In the preceding sequence, VxVM was told to treat slice seven of each disk as a "simple disk" (as opposed to a sliced disk). Consequently, VxVM placed both the private and public region on slice seven. We then added both those "disks" (c0t0d0s7 and c0t1d0s7) into the rootdg diskgroup. This should be clearer from the output of "vxprint -ht":
# vxprint -ht
Disk group: rootdg
DG NAME NCONFIG NLOG MINORS GROUP-ID
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
V NAME RVG KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
dg rootdg default default 0 1036007852.1025.pegasus
dm c0t0d0s7 c0t0d0s7 simple 1024 22495 -
dm c0t1d0s7 c0t1d0s7 simple 1024 22495 -
Originally presented in the August 2000 Sun Blueprint article Toward a Reference Configuration for VxVM Managed Boot Disks, the following procedure results in a root disk and mirror that have been initialized rather than encapsulated. This configuration is easier to maintain in the event of a disk failure.
Use vxinstall to encapsulate the root disk c0t0d0. Do not initialize any other disks. Do not mirror the operating system. Once the root disk c0t0d0 has been encapsulated and the system reboots, the output of "vxprint -ht" is as follows:
# vxprint -ht Disk group: rootdg DG NAME NCONFIG NLOG MINORS GROUP-ID DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK V NAME RVG KSTATE STATE LENGTH READPOL PREFPLEX UTYPE PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE DC NAME PARENTVOL LOGVOL SP NAME SNAPVOL DCO dg rootdg default default 0 1036078650.1025.pegasus dm rootdisk c0t0d0s2 sliced 3359 17690400 - v rootvol - ENABLED ACTIVE 13423200 ROUND - root pl rootvol-01 rootvol ENABLED ACTIVE 13423200 CONCAT - RW sd rootdisk-B0 rootvol-01 rootdisk 17690399 1 0 c0t0d0 ENA sd rootdisk-02 rootvol-01 rootdisk 0 13423199 1 c0t0d0 ENA v swapvol - ENABLED ACTIVE 2100000 ROUND - swap pl swapvol-01 swapvol ENABLED ACTIVE 2100000 CONCAT - RW sd rootdisk-01 swapvol-01 rootdisk 13423199 2100000 0 c0t0d0 ENA v var - ENABLED ACTIVE 2100000 ROUND - fsgen pl var-01 var ENABLED ACTIVE 2100000 CONCAT - RW sd rootdisk-03 var-01 rootdisk 15523199 2100000 0 c0t0d0 ENA
Initialize the root mirror disk c0t1d0:
# /usr/lib/vxvm/bin/vxdisksetup -i c0t1d0
# vxdg adddisk rootmirror=c0t1d0
Manually attach the mirror plexes to the root volumes. The "vxrootmir" command mirrors the rootvol volume. We then manually mirror the swapvol and var volumes in the order that we desire:
# /usr/lib/vxvm/bin/vxrootmir rootmirror
# vxassist mirror swapvol alloc=rootmiror
# vxassist mirror var alloc=rootmiror
The mirror process may take a while to complete. One can monitor its progress via the "vxtask list" command.
Now that the root disk is mirrored, disassociate the original plexes on c0t0d0, remove them, and remove the encapsulated rootdisk from the rootdg diskgroup:
# vxplex dis rootvol-01 swapvol-01 var-01
# vxedit -r rm rootvol-01 swapvol-01 var-01
# vxdg rmdisk rootdisk
At this point, the operating system is based just on the initialized disk c0t1d0:
# vxprint -ht Disk group: rootdg DG NAME NCONFIG NLOG MINORS GROUP-ID DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK V NAME RVG KSTATE STATE LENGTH READPOL PREFPLEX UTYPE PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE DC NAME PARENTVOL LOGVOL SP NAME SNAPVOL DCO dg rootdg default default 0 1036078650.1025.pegasus dm rootmirror c0t1d0s2 sliced 6463 17683680 - v rootvol - ENABLED ACTIVE 13423200 ROUND - root pl rootvol-02 rootvol ENABLED ACTIVE 13423200 CONCAT - RW sd rootmirror-01 rootvol-02 rootmirror 0 13423200 0 c0t1d0 ENA v swapvol - ENABLED ACTIVE 2100000 ROUND - swap pl swapvol-02 swapvol ENABLED ACTIVE 2100000 CONCAT - RW sd rootmirror-02 swapvol-02 rootmirror 13423200 2100000 0 c0t1d0 ENA v var - ENABLED ACTIVE 2100000 ROUND - fsgen pl var-02 var ENABLED ACTIVE 2100000 CONCAT - RW sd rootmirror-03 var-02 rootmirror 15523200 2100000 0 c0t1d0 ENA
Initialize (as opposed to encapsulate) c0t0d0:
# /usr/lib/vxvm/bin/vxdisksetup -i c0t0d0
# vxdg adddisk rootdisk=c0t0d0
Now, mirror the root volumes from c0t1d0 to the freshly initialized rootdisk c0t0d0:
# /usr/lib/vxvm/bin/vxrootmir rootdisk
# vxassist mirror swapvol alloc=rootdisk
# vxassist mirror var alloc=rootdisk
Once the mirror process is complete, the output of "vxprint -ht" is as follows. Note that there is no longer a "rootdisk-BO" subdisk, which is used by VxVM to protect the boot block:
# vxprint -ht Disk group: rootdg DG NAME NCONFIG NLOG MINORS GROUP-ID DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK V NAME RVG KSTATE STATE LENGTH READPOL PREFPLEX UTYPE PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE DC NAME PARENTVOL LOGVOL SP NAME SNAPVOL DCO dg rootdg default default 0 1036078650.1025.pegasus dm rootdisk c0t0d0s2 sliced 6463 17683680 - dm rootmirror c0t1d0s2 sliced 6463 17683680 - v rootvol - ENABLED ACTIVE 13423200 ROUND - root pl rootvol-01 rootvol ENABLED ACTIVE 13423200 CONCAT - RW sd rootdisk-01 rootvol-01 rootdisk 0 13423200 0 c0t0d0 ENA pl rootvol-02 rootvol ENABLED ACTIVE 13423200 CONCAT - RW sd rootmirror-01 rootvol-02 rootmirror 0 13423200 0 c0t1d0 ENA v swapvol - ENABLED ACTIVE 2100000 ROUND - swap pl swapvol-01 swapvol ENABLED ACTIVE 2100000 CONCAT - RW sd rootdisk-02 swapvol-01 rootdisk 13423200 2100000 0 c0t0d0 ENA pl swapvol-02 swapvol ENABLED ACTIVE 2100000 CONCAT - RW sd rootmirror-02 swapvol-02 rootmirror 13423200 2100000 0 c0t1d0 ENA v var - ENABLED ACTIVE 2100000 ROUND - fsgen pl var-01 var ENABLED ACTIVE 2100000 CONCAT - RW sd rootdisk-03 var-01 rootdisk 15523200 2100000 0 c0t0d0 ENA pl var-02 var ENABLED ACTIVE 2100000 CONCAT - RW sd rootmirror-03 var-02 rootmirror 15523200 2100000 0 c0t1d0 ENA
Create disk slices corresponding to the Veritas volume manager subdisks. The syntax of the vxmksdpart command is:
vxmksdpart [-f] [-g diskgroup] subdisk sliceno
[tag flags]
According to the fmthard manual page, tag and flag numbers are defined as follows:
|
|
Since the vxrootmir command creates the slices for the root (/) partition, there is no need to run vxmksdpart for this slice. However, one needs to create slices for swap, var, and any other slices on the rootdisk and rootmirror disks.
First, create the slices for the swap slice on both disks:
# /usr/lib/vxvm/bin/vxmksdpart rootdisk-02 1
0x03 0x01
# /usr/lib/vxvm/bin/vxmksdpart rootmirror-02 1 0x03
0x01
Create the slices for the var partition on both disks. Since slices three and four are normally used by Veritas for private and public regions, we will use slice five for the var partition:
# /usr/lib/vxvm/bin/vxmksdpart rootdisk-03 5
0x07 0x00
# /usr/lib/vxvm/bin/vxmksdpart rootmirror-03 5 0x07
0x00
To assist in recovery, update the /etc/vfstab and /etc/vfstab.prevm files to reflect the slice selections made with vxmksdpart. Furthermore, capture the current disk partition information for both disks:
#
/usr/lib/vxvm/bin/vxprtvtoc -f /tmp/v1
/dev/rdsk/c0t0d0s2
# /usr/lib/vxvm/bin/vxprtvtoc -f /tmp/v2
/dev/rdsk/c0t1d0s2
# cat /tmp/v1 /tmp/v2 > /etc/vtoc
Specify the primary swap partition as the system dump device:
# dumpadm -d /dev/dsk/c0t0d0s1
Dump content: kernel pages
Dump device: /dev/dsk/c0t0d0s1 (swap)
Savecore directory: /var/crash/pegasus
Savecore enabled: yes
If the primary boot disk should fail, make it easy to boot from the mirror. Some sites choose to alter the OBP "boot-device" variable; in this case, we choose to simply define the device aliases "vx-rootdisk" and "vx-rootmirror". In the event that the primary boot device ("disk" or "vx-rootdisk") should fail, the administrator simply needs to type "boot vx-rootmirror" at the OBP prompt.
Determine the device path to the boot devices for both the primary and mirror:
# ls -l /dev/dsk/c0t0d0s0 /dev/dsk/c0t1d0s0 lrwxrwxrwx 1 root root 41 Oct 17 11:48 /dev/dsk/c0t0d0s0 -> ../.. /devices/pci@1f,4000/scsi@3/sd@0,0:a lrwxrwxrwx 1 root root 41 Oct 17 11:48 /dev/dsk/c0t1d0s0 -> ../.. /devices/pci@1f,4000/scsi@3/sd@1,0:a
Use the device paths to define the vx-rootdisk and vx-rootmirror device aliases (note that we use the label "disk" instead of "sd" in the device alias path):
# eeprom "nvramrc=devalias vx-rootdisk /pci@1f,4000/scsi@3/disk@0,0 devalias vx-rootmirror /pci@1f,4000/scsi@3/disk@1,0" # eeprom "use-nvramrc?=true"
Test the process of booting from either vx-rootdisk or vx-rootmirror.