Ubuntu 8.04.1 Server LTSにDRBDを入れてほげる
すべて VirtualBox 上の Ubuntu Server で実行しています．
セットアップ
Ubuntu 8.04.1 Server のインストール
- 日本語
- キー切り替えは無効
- パーティション
| 基本 | 4.0GB | / | ext3 | システム |
| 論理 | 4.2GB | /opt | ext3 | DRBD用 |
| swap | 400MB | swap | swap |
- ソフトウェアは何も入れない
- ホストインターフェイスを使う
ホスト*1上で実行．ユーザ名は rust とします．
% sudo apt-get install bridge-utils % sudo vim /etc/network/interfaces ----add---- auto br0 iface br0 inet dhcp bridge_ports eth0 ----------- % sudo /etc/init.d/networking restart % sudo VBoxAddIf vbox0 rust br0 # Primary 用 % sudo VBoxAddIf vbox1 rust br0 # Secondary 用 % VBoxManage modifyvm "Ubuntu 8.04.1 Server Primary" -hostifdev1 vbox0 % VBoxManage modifyvm "Ubuntu 8.04.1 Server Secondary" -hostifdev1 vbox1
これで2つの仮想マシンがお互いに通信できるようになります．
DRBD のインストール
以後は Primary/Secondary 共に実行していきます．
外部からアクセスするために，openssh-server をインストール
$ sudo apt-get install openssh-server
後はホスト上から操作します．
DRBD のインストール
$ sudo apt-get install build-essential binutils cpp gcc autoconf automake1.9 libtool \ autotools-dev g++ make flex
Kernel sourceのダウンロードとビルドなど
DRBD は 8.2.7 を．
$ sudo -i # apt-get install libncurses5-dev linux-source-2.6.24 # cd /usr/src # tar jxvf linux-source-2.6.24.tar.bz2 # cd linux-source-2.6.24 # make mrproper # cp /boot/config-2.6.24-19-server /usr/src/linux-source-2.6.24/.config # make menuconfig # make prepare # make # cd ../ # wget http://oss.linbit.com/drbd/8.2/drbd-8.2.7.tar.gz # tar zxvf drbd-8.2.7.tar.gz # cd drbd-8.2.7.tar.gz # make KDIR=/usr/src/linux-source-2.6.24 # make install # mv /lib/modules/2.6.24.6/kernel/drivers/block/drbd.ko \ /lib/modules/2.6.24-19-server/kernel/drivers/block/ # modprobe drbd # echo 'drbd' >> /etc/modules # update-rc.d drbd defaults Adding system startup for /etc/init.d/drbd ... /etc/rc0.d/K20drbd -> ../init.d/drbd /etc/rc1.d/K20drbd -> ../init.d/drbd /etc/rc6.d/K20drbd -> ../init.d/drbd /etc/rc2.d/S20drbd -> ../init.d/drbd /etc/rc3.d/S20drbd -> ../init.d/drbd /etc/rc4.d/S20drbd -> ../init.d/drbd /etc/rc5.d/S20drbd -> ../init.d/drbd # lsmod | grep drbd drbd 213000 0 cn 9632 1 drbd
設定と実行と
次は設定．サンプルとここを参考にしました．(設定ファイルはこちらhttp://gist.github.com/60611)
# cp /etc/drbd.conf /etc/drbd.conf.org # /etc/init.d/drbd restart Restarting all DRBD resources/dev/drbd0: Failure: (114) Lower device is already claimed. This usually means it is mounted. Command '/sbin/drbdsetup /dev/drbd0 disk /dev/sda6 /dev/sda6 internal --set-defaults --create-device --on-io-error=detach' terminated with exit code 10 .
などとエラーになる．なので unmount して fstab からも外す．
# umount /opt # vim /etc/fstab # /etc/init.d/drbd restart Restarting all DRBD resources/dev/drbd0: Failure: (119) No valid meta-data signature found. ==> Use 'drbdadm create-md res' to initialize meta-data area. <== Command '/sbin/drbdsetup /dev/drbd0 disk /dev/sda6 /dev/sda6 internal --set-defaults --create-device --on-io-error=detach' terminated with exit code 10 .
またしてもエラー．どうも初期化しないといけないようなので，初期化する．res はリソース名なので，この場合は r0 を指定．
# drbdadm create-md r0 md_offset 4170145792 al_offset 4170113024 bm_offset 4169981952 Found ext3 filesystem which uses 4072412 kB current configuration leaves usable 4072248 kB Device size would be truncated, which would corrupt data and result in 'access beyond end of device' errors. You need to either * use external meta data (recommended) * shrink that filesystem first * zero out the device (destroy the filesystem) Operation refused. Command 'drbdmeta /dev/drbd0 v08 /dev/sda6 internal create-md' terminated with exit code 40 drbdadm create-md r0: exited with code 40
またしてもまたしてもエラー．今度は容量制限をオーバーしたようなので，仕方なくリサイズして再起動．
# fdisk /dev/sda The number of cylinders for this disk is set to 1044. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Command (m for help): p Disk /dev/sda: 8589 MB, 8589934592 bytes 255 heads, 63 sectors/track, 1044 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x00048f37 Device Boot Start End Blocks Id System /dev/sda1 * 1 486 3903763+ 83 Linux /dev/sda2 487 1044 4482135 5 Extended /dev/sda5 994 1044 409626 82 Linux swap / Solaris /dev/sda6 487 993 4072446 83 Linux Partition table entries are not in disk order Command (m for help): d Partition number (1-6): 6 Command (m for help): n Command action l logical (5 or over) p primary partition (1-4) l First cylinder (487-1044, default 487): Using default value 487 Last cylinder or +size or +sizeM or +sizeK (487-993, default 993): +2000M Command (m for help): p Disk /dev/sda: 8589 MB, 8589934592 bytes 255 heads, 63 sectors/track, 1044 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x00048f37 Device Boot Start End Blocks Id System /dev/sda1 * 1 486 3903763+ 83 Linux /dev/sda2 487 1044 4482135 5 Extended /dev/sda5 994 1044 409626 82 Linux swap / Solaris /dev/sda6 487 730 1959898+ 83 Linux Partition table entries are not in disk order Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot. Syncing disks. # reboot
再起動したら初期化．
# drbdadm create-md r0 md_offset 2006929408 al_offset 2006896640 bm_offset 2006835200 Found some data ==> This might destroy existing data! <== Do you want to proceed? [need to type 'yes' to confirm] yes Writing meta data... initialising activity log NOT initialized bitmap New drbd meta data block sucessfully created.
今度はうまくいったようだ．そして起動．
# /etc/init.d/drbd restart
Restarting all DRBD resources.
# cat /proc/drbd
version: 8.0.11 (api:86/proto:86)
GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43
0: cs:Connected st:Secondary/Secondary ds:Inconsistent/Inconsistent C r---
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0
resync: used:0/31 hits:0 misses:0 starving:0 dirty:0 changed:0
act_log: used:0/257 hits:0 misses:0 starving:0 dirty:0 changed:0Inconsistent なのでどちらも Secondary として動いている．同期をとるために，Primary の方で下記を実行．
# drbdadm -- --overwrite-data-of-peer primary all /etc/init.d/drbd status drbd driver loaded OK; device status: version: 8.0.11 (api:86/proto:86) GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43 m:res cs st ds p mounted fstype 0:r0 SyncSource Primary/Secondary UpToDate/Inconsistent C ... sync'ed: 2.3% (1917560/1959800)K
同期が始まりました．これは「初期フル同期」というらしいです．Secondary ではこんな感じ．
# /etc/init.d/drbd status drbd driver loaded OK; device status: version: 8.0.11 (api:86/proto:86) GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43 m:res cs st ds p mounted fstype 0:r0 SyncTarget Secondary/Primary Inconsistent/UpToDate C ... sync'ed: 4.4% (1878456/1959800)K
しばらくして同期が完了すると，
on Primary # /etc/init.d/drbd status drbd driver loaded OK; device status: version: 8.0.11 (api:86/proto:86) GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43 m:res cs st ds p mounted fstype 0:r0 Connected Primary/Secondary UpToDate/UpToDate C on Secondary # /etc/init.d/drbd status drbd driver loaded OK; device status: version: 8.0.11 (api:86/proto:86) GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43 m:res cs st ds p mounted fstype 0:r0 Connected Secondary/Primary UpToDate/UpToDate C
となります．
format と mount
次は format します．なんとなく ext3 で．そして mount も．
# mkfs.ext3 /dev/drbd0 mke2fs 1.40.8 (13-Mar-2008) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 122880 inodes, 489950 blocks 24497 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=503316480 15 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Writing inode tables: done Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 26 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. # mount /dev/drbd0 /opt # ls /opt/ lost+found # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 3.7G 2.9G 650M 82% / varrun 189M 44K 189M 1% /var/run varlock 189M 4.0K 189M 1% /var/lock udev 189M 52K 189M 1% /dev devshm 189M 0 189M 0% /dev/shm /dev/drbd0 1.9G 35M 1.8G 2% /opt
実験など
ここにある実験を同じようにやってみます．
1. Primary でファイルをコピー後に，強制的に Secondary に降格させる．
# cp /etc/drbd.conf /opt # ls /opt/ drbd.conf lost+found # umount /opt # drbdadm secondary r0 # /etc/init.d/drbd status drbd driver loaded OK; device status: version: 8.0.11 (api:86/proto:86) GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43 m:res cs st ds p mounted fstype 0:r0 Connected Secondary/Secondary UpToDate/UpToDate C
2. 次にSecondary を Primary に昇格させて，mount する．
# /etc/init.d/drbd status drbd driver loaded OK; device status: version: 8.0.11 (api:86/proto:86) GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43 m:res cs st ds p mounted fstype 0:r0 Connected Secondary/Secondary UpToDate/UpToDate C # drbdadm primary r0 # /etc/init.d/drbd status drbd driver loaded OK; device status: version: 8.0.11 (api:86/proto:86) GIT-hash: b3fe2bdfd3b9f7c2f923186883eb9e2a0d3a5b1b build by phil@mescal, 2008-02-12 11:56:43 m:res cs st ds p mounted fstype 0:r0 Connected Primary/Secondary UpToDate/UpToDate C # mount /dev/drbd0 /opt # ls /opt/ drbd.conf lost+found
よし，レプリケーション成功．さらに元に戻す．
3. unmount して Primary(旧Secondary)を Secondary に降格する．
# umount /opt # drbdadm secondary r0
4. Secondary(旧Primary)を Primary に昇格する．
# drbdadm primary r0 # mount /dev/drbd0 /opt # ls /opt/ drbd.conf lost+found
ひとまず実験は終わり．次は Heatbeat と組み合わせてほげるかな．
*1:VirtualBox を動かしている Ubuntu
