Installing Vinum on FreeBSD 5.4:

Step 1: Adding a new drive (or if a new drive was installed, but have not yet edited the partition). Otherwise skip over to Step 2

According to FreeBSD Handbook:

· Run sysinstall and enter the Configure menu. Within the FreeBSD Configuration Menu, scroll down and select the Fdisk option.

root@home-gw2# /stand/sysinstall

Few things I found important:

1. Make sure to write changes (W) for each partition you have created.

2. Usually standard would do. Unless of course situation requires partitions to remain untouched. It would make little sense to install another boot-mgr.

Step 2: Label new disk for vinum type file system

For vinum to utilize a partition we’re about to create, it needs to be labeled as vinum in order for it to have any use.

Unless vi is your thing, I’d prefer bash for simplicity. The following assumes you’ve got bash shell installed, and it replaces default editing environment to PICO (which can be installed along with pine from port)

home-gw2# bash

root@home-gw2# export EDITOR="pico";

root@home-gw2# fdisk -BI ad1

******* Working on device /dev/ad1 *******

root@home-gw2# disklabel -B -w -r ad1s1 auto

root@home-gw2# disklabel -e ad1s1

For example, I have

# /dev/ad2s1:

8 partitions:

# size offset fstype [fsize bsize bps/cpg]

a: 78165281 16 vinum 0 0

c: 78165297 0 unused 0 0 # "raw" part, don't edit

Above is a rather simple example. In most cases, user creates multiple partitions labels for a disk using sysinstall. Sysinstall optimize in such a way to prevent holes. As such, you are now confronting a situation where you must create more than just one device name. Your probably encounter “the size is too big!” with sysinstall or “partition a: partition extends past end of unit” error with disklabel or bsdlabel command line utilities. Take the following for an example:

root@home-gw2# disklabel -e /dev/ad0s1

# /dev/ad0s1:

8 partitions:

# size offset fstype [fsize bsize bps/cpg]

a: 524288 0 4.2BSD 2048 16384 32776

c: 12656385 0 unused 0 0 # "raw" part, don't edit

d: 2097152 2307984 4.2BSD 2048 16384 28552

e: 524288 1783696 4.2BSD 2048 16384 32776

There is a hole between 1783696 and 524288. 1259408 blocks (or 1259508/2048 = 614.94MB) is wasted because of sysinstall bug. We don’t want to waste this space, so lets add this entry

f: 1259406 524289 4.2BSD 2048 16384 13184

Now, don’t forget about rest of the space:

g: 8251249 4405136 4.2BSD 2048 16384 28552

We are now ready to create the new file system

root@home-gw2# newfs /dev/ad0s1f

/dev/ad0s1f: 614.9MB (1259404 sectors) block size 16384, fragment size 2048

using 4 cylinder groups of 153.75MB, 9840 blks, 19712 inodes.

super-block backups (for fsck -b #) at:

160, 315040, 629920, 944800

root@home-gw2# newfs /dev/ad0s1g

/dev/ad0s1g: 4028.9MB (8251248 sectors) block size 16384, fragment size 2048

using 22 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.

super-block backups (for fsck -b #) at:

160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976, 3387328, 3763680, 4140032, 4516384, 4892736, 5269088,

5645440, 6021792, 6398144, 6774496, 7150848, 7527200, 7903552

But wait! Don’t forget to rename fstype to vinum!

Step 3: Creating a concatenated Vinum volume

If disk size varies and redundancy is not needed but we need the maximum storage space instead, concatenated disk volume is the most common configuration for this scenario.

Concatenated volumes are rather easy. Just put the following in /etc/fstab

drive d1 device /dev/ad0s1f

drive d2 device /dev/ad1s1a

drive d3 device /dev/ad2s1a

drive d4 device /dev/ad0s1g

volume usr2

plex org concat

sd length 0 drive d1

sd length 0 drive d2

sd length 0 drive d3

If you plan to use latest 5.4-RELEASE (as time of writing), you must use gvinum to create the volume.

home-gw2# bash

root@home-gw2# export EDITOR='pico'

root@home-gw2# gvinum

gvinum -> create /etc/gvinum.cfg

Then paste our configuration:

# Vinum configuration of home-gw2.hw-dc.aoloser.com, saved at Sat Mar 25 18:15:42 2006

# Current configuration:

drive d1 device /dev/ad0s1f

drive d2 device /dev/ad1s1a

drive d3 device /dev/ad2s1a

drive d4 device /dev/ad0s1g

volume usr2

plex org concat

sd length 0 drive d1

sd length 0 drive d2

sd length 0 drive d3

sd length 0 drive d4

After quitting and changes written with your favorite editor, the result would be:

4 drives:

D d4 State: up /dev/ad0s1g A: 0/4028 MB (0%)

D d3 State: up /dev/ad2s1a A: 0/38166 MB (0%)

D d2 State: up /dev/ad1s1a A: 0/16123 MB (0%)

D d1 State: up /dev/ad0s1f A: 0/614 MB (0%)

1 volume:

V usr2 State: up Plexes: 1 Size: 57 GB

1 plex:

P usr2.p0 C State: up Subdisks: 4 Size: 57 GB

4 subdisks:

S usr2.p0.s3 State: up D: d4 Size: 4028 MB

S usr2.p0.s2 State: up D: d3 Size: 37 GB

S usr2.p0.s1 State: up D: d2 Size: 15 GB

S usr2.p0.s0 State: up D: d1 Size: 614 MB

vinum -> saveconfig

gvinum -> quit

root@home-gw2# newfs /dev/gvinum/usr2

/dev/gvinum/usr2: 58934.0MB (120696876 sectors) block size 16384, fragment size 2048

using 321 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.

super-block backups (for fsck -b #) at:

160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976, 3387328, 3763680, 4140032, 4516384, 4892736, 5269088,

5645440, 6021792, 6398144, 6774496, 7150848, 7527200, 7903552, 8279904, 8656256, 9032608, 9408960, 9785312, 10161664, 10538016,

10914368, 11290720, 11667072, 12043424, 12419776, 12796128, 13172480, 13548832, 13925184, 14301536, 14677888, 15054240, 15430592,

15806944, 16183296, 16559648, 16936000, 17312352, 17688704, 18065056, 18441408, 18817760, 19194112, 19570464, 19946816, 20323168,

20699520, 21075872, 21452224, 21828576, 22204928, 22581280, 22957632, 23333984, 23710336, 24086688, 24463040, 24839392, 25215744,

25592096, 25968448, 26344800, 26721152, 27097504, 27473856, 27850208, 28226560, 28602912, 28979264, 29355616, 29731968, 30108320,

30484672, 30861024, 31237376, 31613728, 31990080, 32366432, 32742784, 33119136, 33495488, 33871840, 34248192, 34624544, 35000896,

35377248, 35753600, 36129952, 36506304, 36882656, 37259008, 37635360, 38011712, 38388064, 38764416, 39140768, 39517120, 39893472,

40269824, 40646176, 41022528, 41398880, 41775232, 42151584, 42527936, 42904288, 43280640, 43656992, 44033344, 44409696, 44786048,

45162400, 45538752, 45915104, 46291456, 46667808, 47044160, 47420512, 47796864, 48173216, 48549568, 48925920, 49302272, 49678624,

50054976, 50431328, 50807680, 51184032, 51560384, 51936736, 52313088, 52689440, 53065792, 53442144, 53818496, 54194848, 54571200,

54947552, 55323904, 55700256, 56076608, 56452960, 56829312, 57205664, 57582016, 57958368, 58334720, 58711072, 59087424, 59463776,

59840128, 60216480, 60592832, 60969184, 61345536, 61721888, 62098240, 62474592, 62850944, 63227296, 63603648, 63980000, 64356352,

64732704, 65109056, 65485408, 65861760, 66238112, 66614464, 66990816, 67367168, 67743520, 68119872, 68496224, 68872576, 69248928,

69625280, 70001632, 70377984, 70754336, 71130688, 71507040, 71883392, 72259744, 72636096, 73012448, 73388800, 73765152, 74141504,

74517856, 74894208, 75270560, 75646912, 76023264, 76399616, 76775968, 77152320, 77528672, 77905024, 78281376, 78657728, 79034080,

79410432, 79786784, 80163136, 80539488, 80915840, 81292192, 81668544, 82044896, 82421248, 82797600, 83173952, 83550304, 83926656,

84303008, 84679360, 85055712, 85432064, 85808416, 86184768, 86561120, 86937472, 87313824, 87690176, 88066528, 88442880, 88819232,

89195584, 89571936, 89948288, 90324640, 90700992, 91077344, 91453696, 91830048, 92206400, 92582752, 92959104, 93335456, 93711808,

94088160, 94464512, 94840864, 95217216, 95593568, 95969920, 96346272, 96722624, 97098976, 97475328, 97851680, 98228032, 98604384,

98980736, 99357088, 99733440, 100109792, 100486144, 100862496, 101238848, 101615200, 101991552, 102367904, 102744256, 103120608,

103496960, 103873312, 104249664, 104626016, 105002368, 105378720, 105755072, 106131424, 106507776, 106884128, 107260480,

107636832, 108013184, 108389536, 108765888, 109142240, 109518592, 109894944, 110271296, 110647648, 111024000, 111400352,

111776704, 112153056, 112529408, 112905760, 113282112, 113658464, 114034816, 114411168, 114787520, 115163872, 115540224,

115916576, 116292928, 116669280, 117045632, 117421984, 117798336, 118174688, 118551040, 118927392, 119303744, 119680096,

120056448, 120432800

root@home-gw2# cd /

root@home-gw2# mkdir usr2

root@home-gw2# mount /dev/gvinum/usr2 /usr2

root@home-gw2# df

Filesystem 1K-blocks Used Avail Capacity Mounted on

/dev/ad0s1a 253678 53486 179898 23% /

devfs 1 1 0 100% /dev

/dev/ad0s1e 253678 20 233364 0% /tmp

/dev/da0s1d 8101176 2329538 5123544 31% /usr

/dev/ad0s1d 1012974 28906 903032 3% /var

/dev/gvinum/usr2 58448032 4 53772186 0% /usr2

Make sure file system table (etc/fstab) mount our newly created volume as system starts:

/dev/gvinum/usr2 /usr2 ufs rw 2 2

Step 4: Remove and recreate a new vinum configuration:

Shown below are the steps to remove a vinum configuration and 
then create it (both steps on 5.4-RELEASE-p6) once the vinum.ko was 
unloaded and geom_vinum.ko loaded :
 
resetconfig steps:
 
gvinum rm stripe.p0.s0
gvinum rm stripe.p0.s1
gvinum rm d1
gvinum rm d2
gvinum rm stripe.p0
(error displayed on above command, ignored)
gvinum rm stripe
(error displayed on above command, ignored)
gvinum saveconfig
reboot
 
You can also try kldunload geom_vinum.ko to reload the kernel module for gvinum.  But for me (since it’s a new system) I’d rather reboot. 
 

home-gw2# apropos kld

kld(4) - dynamic kernel linker facility

kldconfig(8) - display or modify the kernel module search path

kldfind(2) - returns the fileid of a kld file

kldfirstmod(2) - return first module id from the kld file specified

kldload(2) - load KLD files into the kernel

kldload(8) - load a file into the kernel

kldnext(2) - return the fileid of the next kld file

kldstat(2) - get status of kld file

kldstat(8) - display status of dynamic kernel linker

kldsym(2) - look up address by symbol name in a KLD

kldunload(2) - unload kld files

kldunload(8) - unload a file from the kernel

kldxref(8) - generate hints for the kernel loader

home-gw2# kldstat

Id Refs Address Size Name

1 4 0xc0400000 3fb528 kernel

2 1 0xc07fc000 10a78 geom_vinum.ko

3 1 0xc27fc000 2000 dragon_saver.ko

 
create steps:
 
gvinum create gvinum.cfg
 
# drive d1 device /dev/da1d
# drive d2 device /dev/da2d
# volume stripe
# plex org striped 4096k
# (ive done my own testing, dont talk to me about 4096k)
# sd drive d1
# sd drive d2 
# (normally use "size 0" for subdisks but gvinum didnt like that)
 
gvinum saveconfig
newfs -U /dev/gvinum/stripe
fsck -t ufs /dev/gvinum/stripe
(above is a preference, i like to fsck new partitions)
 
other changes:
 
- inside /boot/loader.conf add following line
 
geom_vinum_load="YES"
 
- may not be needed: inside /etc/rc.d/vinum alter the following variable
 
start_cmd="gvinum start"
 
- add entry in /etc/fstab
 
/var/log/vinum_history is no longer updated, guess that's a gvinum issue 
and not sparc64 specific though. next round of testing is to create a 
mirror, break it abnormally and attempt to attach a new device to 
establish the mirror again.