測試 ZFS copies 在單顆磁碟下的復原能力.

感謝 Kuli 學長提供良好的測試方法 - 使用 memory disk 測試. 測試指令如下 :

1. 準備階段
1.1. 建立 zpool test 及測試 vol 1 及 2

# mdconfig -a -t swap -s 66m
# zpool create test /dev/md0
# zfs create -o copies=1 test/1
# zfs create -o copies=2 test/2

1.2. 編輯測試檔案

# vi /test/1/test1.txt

輸入 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

# vi /test/2/test2.txt

輸入 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

1.3. 匯出 zpool test 並製作 memory disk dump file.

# zpool export test
# dd if=/dev/md0 of=/root/md0

2. 複製 dump file 並使用 hexedit 搜尋連續的 A 及 B 編輯檔案.

# cp -a /root/md0 /root/md0-edit
# hexedit md0

2.1. 搜尋有 36 個 A 的區塊將 A 改成 C. 進行 3. 觀察 ZFS 的反應.

2.2. 搜尋有 36 個 B 的區塊將 B 改成 D, 但只修改其中一組. 進行 3. 觀察 ZFS 的反應.

2.3. 搜尋有 36 個 B 的區塊, 其中一組將 B 改成 D, 另一組將 B 改成 E. 進行 3. 觀察 ZFS 的反應.

3. 將檔案寫回 memory disk

# dd if=/root/md0-edit of=/dev/md0 bs=1m
# zpool import -d /dev test
# zfs scrub test

4. 觀察結果:
4.1. 進行 2.1. 後, vol 1 出現無法修復的錯誤, 並拒絕使用該檔案.

4.2. 進行 2.2. 後, vol 2 發現錯誤並直接修正錯誤(不需 zfs scrub test), 且匯入後可正常使用該檔案. 此時將 zpool test 匯出後並製作 dump file. 可以觀察到手動變更的 D 已經被修改回 B.

4.3. 進行 2.3. 後, vol 2 出現無法修復的錯誤, 並拒絕使用該檔案.

5. 不過這是可修正的軟體錯誤, 如果發生硬體錯誤會是如何處理. 目前尚未想到好方法測試. 不過可偵測錯誤並顯示出來或許對系統管理就蠻有用的. 至少可以提早發現更換硬碟.

實做安裝FreeBSD 9.0R root on ZFS

基本上本文幾乎和參考資料相近, 差異在部份分割採用 copies=2. 由於實體機器上並沒有光碟機, 所以使用 USB 外接光碟機啟動 LiveCD.
0. 啟動安裝環境
0.1.  Boot 選單直接選 1. Boot
0.2. 中途若遇到找不到光碟機, 請下 cd9660:/dev/cd0 ro .
0.3. 出現安裝選項時, 請選 LiveCD.

1. 硬體
1.1. 系統硬碟 16GB SLC SSD. 快取硬碟 MLC 80GB SSD. 資料硬碟 1TB * 3.
1.2. 預計在系統上啟用 copies=2.

2. 建立 GPT 分割及 root on ZFS

gpart show ada0
gpart destroy ada0
gpart create -s GPT ada0
gpart add -b 34 -s 128 -t freebsd-boot ada0
gpart add -t freebsd-zfs -l disk0 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0

gnop create -S 4096 /dev/gpt/disk0
zpool create -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache zroot /dev/gpt/disk0.nop
zpool export zroot
gnop destroy /dev/gpt/disk0.nop
zpool import -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache zroot

zpool set bootfs=zroot zroot
zfs set checksum=fletcher4 zroot

3. 建立子目錄

zfs create zroot/usr
zfs create zroot/usr/home
zfs create zroot/var

zfs create -o compression=on   -o exec=on  -o setuid=off zroot/tmp
zfs create -o compression=lzjb -o setuid=off zroot/usr/ports
zfs create -o compression=off  -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o compression=off  -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash

zfs create -o exec=off -o setuid=off -o copies=2 zroot/var/db
zfs create -o exec=off -o setuid=off zroot/var/empty
zfs create -o exec=off -o setuid=off zroot/var/run

zfs create -o compression=lzjb -o exec=on  -o setuid=off -o copies=2 zroot/var/db/pkg
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off -o copies=2 zroot/var/mail
zfs create -o compression=lzjb -o exec=on  -o setuid=off zroot/var/tmp

4. 建立 swap

zfs create -V 4G zroot/swap
zfs set org.freebsd:swap=on zroot/swap
zfs set checksum=off zroot/swap

5. 處理暫存目錄及home目錄

chmod 1777 /mnt/tmp
cd /mnt; ln -s usr/home home
chmod 1777 /mnt/var/tmp

6. 複製系統檔案

sh
cd /usr/freebsd-dist
export DESTDIR=/mnt
for file in base.txz lib32.txz kernel.txz doc.txz ports.txz src.txz; do(cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done
cp /var/tmp/zpool.cache /mnt/boot/zfs/zpool.cache

7. 處理開機設定

echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf
echo 'zfs_load="YES"' >> /mnt/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf
touch /mnt/etc/fstab

8. /tmp 改用 TMPFS
8.1. 重新開機後進 single user mode

mount -u /
swapon -a
zfs mount -a
zfs rename zroot/tmp zroot/tmpold
mkdir -p /tmp
chmod 1777 /tmp

8.2. 在 /etc/fstab 加上

# Device        Mountpoint      FStype  Options         Dump    Pass#
tmpfs           /tmp            tmpfs   rw              0       0

參考資料 :
1. ROOT ON ZFS FREEBSD 9 (NON LEGACY MOUNTPOINT – 4K OPTIMIZED)

ZFS zpool 匯出(export)及匯入(import)

為啥會出現這個問題, 並不是因為換機器. 而是之前 BIOS 都是使用 SATA mode, 導致有兩個 port 的硬碟跑 Ultra DMA 133. 因為不熟 ZFS, 不敢亂玩(其實是怕資料沒了).
今天就大膽一點直接上了, 重開機後發現 BIOS 只抓到四個硬碟, 心裡涼了一半. 不過在 Unix* 環境似乎還可以期待一下系統自己辨識硬碟. 進了 single user mode 看了一下 dmesg 及 /dev, 發現硬碟全數找到了. 只是硬碟代號有部份異動, 剛好動到的都是 ZFS 的硬碟.
立馬回頭改回原設定, 一樣也是進 single user mode 下

zpool export POOL_NAME

再做一次轉換為 AHCI, 換好了以後再進 single user mode. 此時用 zpool import 去偵測發現只找到一個 STO_NAME. 一時原因無法理解, 本來想要就放棄了. 仔細比較磁碟代號, 發現有重複的磁碟代號出現(adX之前是 POOL1 現在變成是 POOL2). 趕緊先執行

zpool import POOL1

重新開機後再用 zpool import 掃描, 果真出現 POOL2. 再執行

zpool import POOL2

收工.

參考資料 :
1. Migrating ZFS Storage Pools Chapter 4. Managing ZFS Storage Pools

ZFS儲存設備更換硬碟

只能說硬碟不能放在太熱的環境(硬碟工作溫度攝氏 45 ~ 58 度), 否則企業級硬碟還是一樣會出問題.這個硬碟倒不是完全讀不到, 只是三不五時就會出現 smartd 的警告訊息.

SMART error (CurrentPendingSector) detected on host: XXXXXX.daemonland.org

This email was generated by the smartd daemon running on:

   host name: XXXXXX.daemonland.org
  DNS domain: daemonland.org
  NIS domain: 

The following warning/error was logged by the smartd daemon:

Device: /dev/ad10, 3 Currently unreadable (pending) sectors


For details see host's SYSLOG.

You can also use the smartctl utility for further investigation.
No additional email messages about this problem will be sent.

只好將備用的 HITACHI 2TB 硬碟上線, 換 WD 硬碟下來維修. 順便打開系統的 AHCI.


1.更換 ZFS RAIDZ1 硬碟
因為沒熱插拔可用所以就重新開機進 single user mode. 然後依照步驟進行:

# zpool offline POOL_NAME DISKID

關機後, 更換故障硬碟.

# zpool online POOL_NAME DISKID
# zpool replace POOL_NAME DISKID

經過漫長的 resilvering (14小時, 可以執行 zpool status POOL_NAME 檢查進度), 終於完成更換硬碟. 沒有機會測試在多使用者模式下, 系統效能會是什麼情景.

2. 設定 AHCI
很簡單只要在 /boot/loader.conf 加上

ataahci_load="YES"

很訝異重新開機後磁碟機代號竟然沒有改變(沒有從 adX 變成 adaX).

參考資料 :
1. ZFS: how to replace a dead disk?
2. [Solved] ZFS cannot replace failed drive

使用國高研發的 Clonezilla 複製 SSD

之前不慎買到美光的倒數計時 SSD(5000 小時), 上頭已經裝好了 Ubuntu. 基於懶人性格, 就不想要再安裝一次. 此時想到 clonezilla-live. 趕緊採購 Intel SSD 更換.

剛開始使用一般模式都讀到 99% 就直接吐錯誤訊息, 只好改用專家模式硬著頭皮上, 沒想到還是不行. 但是看到有個 RAW 複製的選項後面接著效率差的警語, 心想效率差總比沒有好. 沒想到竟然成功複製(複製速度 1.1GB/min), 還好 SSD 只是從 60GB 擴展複製到 120GB, 所以耐心等了不少時間終於完成複製 ext4 檔案系統. (圖片稍候再附上)

接下來是要將儲存設備的暫存 SSD(ufs) 從 160GB 換到 180GB, 也是再做一次, 不過這次不用做複製 GRUB.一切搞定.

使用 rsnapshot 備份及比較設定檔

1. 安裝 rsnapshot
在 Ubuntu 下直接用 apt-get install rsnapshot.
在 FreeBSD 下請到 /usr/ports/sysutils/rsnapshot 下 make install clean. (在我的 FreeBSD 實體及虛擬環境中並沒有實際安裝測試, 以下僅用 Ubuntu 作為範例.)

2. 調整設定檔 /etc/rsnapshot.conf

# diff /etc/rsnapshot.conf /etc/rsnapshot.conf.20120322
27c27
< snapshot_root /.rsnapshot/
---
> snapshot_root /var/cache/rsnapshot/
97c97
< retain                hourly  8
---
> retain                hourly  6
100c100
< retain                monthly 6
---
> #retain       monthly 3
124c124
< logfile       /var/log/rsnapshot.log
---
> #logfile      /var/log/rsnapshot.log
230c230
< #backup       /home/          localhost/
---
> backup        /home/          localhost/
232,235c232
< backup        /usr/local/etc/ localhost/
< backup        /var/log/       localhost/
< backup        /var/mail/      localhost/
< backup        /root           localhost/
---
> backup        /usr/local/     localhost/

註 : 因為 /home 已將大部分資料放上 NFS, 已經有 ZFS snapshot. 就不浪費時間多做 snapshot.

3. 設定定期執行

0 */3 * * * /usr/bin/rsnapshot hourly
0 23 * * * /usr/bin/rsnapshot daily
15 23 * * 1 /usr/bin/rsnapshot weekly
30 23 1 * * /usr/bin/rsnapshot monthly
0 0 * * 1 /PATH/TO/REPORT/PROG/report_CHK_ETC_diff.sh

4. CHK_ETC_diff.sh

#!/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        CHKPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSKEYWORD="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSBASE="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 check_path snapshot_keyword snapshot_base_path"
        echo "Example: $0 /etc Weekly /SNAPSHOT/System"
        exit 1
else
        echo "Collect difference between $CHKPATH and $SSBASE/${SSKEYWORD}*$CHKPATH."
fi

date +"%Y/%m/%d %H:%M:%S"
LAST_DIR="/"
CHKPATH=`echo $CHKPATH | sed -e 's/\///'`

for CONF in `find $LAST_DIR$CHKPATH -type f -name '*'`
do
        if [ `ls $SSBASE | grep ${SSKEYWORD} |wc -l` -lt 1 ]; then
                echo "$SSBASE/${SSKEYWORD}* does not existed!"
                exit 1
        fi
        ls -d $SSBASE/${SSKEYWORD}* | while read DIR
        do
                if [ $LAST_DIR = "/" ]; then
                        HOST=""
                else
                        HOST="/localhost"
                fi
                if [ -f ${LAST_DIR}${HOST}${CONF} ]&&[ -f ${DIR}/localhost${CONF} ]; then
                        if [ $LAST_DIR = "/" ]; then
                                LAST_DIR=""
                        fi
                        DIFFC=`diff ${LAST_DIR}${HOST}${CONF} ${DIR}/localhost${CONF} | wc -l`
                        if [ $DIFFC -gt 0 ]; then
                                echo "[[[ Difference between ${LAST_DIR}${HOST}${CONF} and ${DIR}/localhost${CONF} ]]]"
                                diff ${LAST_DIR}${HOST}${CONF} ${DIR}/localhost${CONF}
                                echo ""
                        fi
                        DIFFC=0
                else
                        if [ ! -f ${LAST_DIR}${HOST}${CONF} ]; then
                                echo "File ${LAST_DIR}${HOST}${CONF} does not existed."
                        fi
                fi

                LAST_DIR=$DIR
        done
done
echo ""
date +"%Y/%m/%d %H:%M:%S"

5. report_CHK_ETC_diff.sh

#!/bin/bash

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:${PATH}

CHKPROG="/PATH/TO/PROG/CHK_ETC_diff.sh /etc daily /.rsnapshot"
REPORTPATH="/PATH/TO/STORE/REPORT/report_CHK_ETC_diff"
REPORTKEYWORD="etcdiff"
REPORTTITLE="Check Difference of ETC Report"
REMAILADDR="ACCOUNT@DOMAIN.NAME"

mkdir -p $REPORTPATH/`date +"%Y"`
mv $REPORTPATH/${REPORTKEYWORD}_*.log $REPORTPATH/`date +"%Y"`
$CHKPROG > $REPORTPATH/${REPORTKEYWORD}_`hostname`_`date +"%Y%m%d"`.log
cat $REPORTPATH/${REPORTKEYWORD}_`hostname`_`date +"%Y%m%d"`.log | mail -s "$REPORTTITLE" $REMAILADDR

參考資料:
淺談linux下的快照備份軟體—rsnapshot

應用 UFS snapshot 進行比對 /etc 及 /usr/local/etc 下的檔案內容

1. 這隻程式運作的前提是已經完成 UFS snapshot 並且已經正常掛載. 沒有掛載 snapshot 只會看到出現很多找不到檔案的訊息.

2. 程式碼

#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        CHKPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSKEYWORD="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSBASE="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 check_path snapshot_keyword snapshot_base_path"
        echo "Example: $0 /etc Weekly /SNAPSHOT/System"
        exit 1
else
        echo "Collect difference between $CHKPATH and $SSBASE/${SSKEYWORD}*$CHKPATH."
fi

date +"%Y/%m/%d %H:%M:%S"
LAST_DIR="/"
CHKPATH=`echo $CHKPATH | sed -e 's/\///'`

for CONF in `find $LAST_DIR$CHKPATH -type f -name '*'`
do
        if [ `ls $SSBASE | grep ${SSKEYWORD} |wc -l` -lt 1 ]; then
                echo "$SSBASE/${SSKEYWORD}* does not existed!"
                exit 1
        fi
        ls -d $SSBASE/${SSKEYWORD}* | while read DIR
        do
                if [ -f ${LAST_DIR}${CONF} ]&&[ -f ${DIR}${CONF} ]; then
                        if [ $LAST_DIR = "/" ]; then
                                LAST_DIR=""
                        fi
                        DIFFC=`diff ${LAST_DIR}${CONF} ${DIR}${CONF} | wc -l`
                        if [ $DIFFC -gt 0 ]; then
                                echo "[[[ Difference between ${LAST_DIR}${CONF} and ${DIR}${CONF} ]]]"
                                diff ${LAST_DIR}${CONF} ${DIR}${CONF}
                                echo ""
                        fi
                        DIFFC=0
                else
                        if [ ! -f ${LAST_DIR}${CONF} ]; then
                                echo "File ${LAST_DIR}${CONF} does not existed."
                        fi
                fi

                LAST_DIR=$DIR
        done
done
echo ""
date +"%Y/%m/%d %H:%M:%S"

3. 使用範例

/PATH/TO/CHK_ETC_diff.sh /PATH/TO/SRC SNAPSHOT_KEYWORD /PATH/TO/SNAPSHOT_ROOT

在 FreeBSD 8 對 UFS filesystem 做 snapshot

1. 在 crontab 定期做 snapshot

### UFS snapshot
0    1      *       *       *       root    /usr/local/sbin/snapshot make -g7 /SRCPATH:Daily
0      2      *       *       1       root    /usr/local/sbin/snapshot make -g4 /SRCPATH:Weekly

2. 掛載 UFS snapshot 的 shell script

#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        SSSRCPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SnapshotName="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        MountPath="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 Snapshot_Src_Path SnapshotName MountPath"
        echo "Example: $0 /:Weekly.0 /mnt"
        exit 1
else
        echo "Try to mount $SSSRCPATH:$SnapshotName to $MountPath now."
fi

MC=`df -h |grep $MountPath |wc -l`

if [ -d $SSSRCPATH ]; then
        if [ -f $SSSRCPATH/.snap/$SnapshotName ]; then
                if [ -d $MountPath ]; then
                        if [ $MC -gt 0 ]; then
                                echo "MountPath $MountPath had been mounted!"
                        else
                                snapshot mount $SSSRCPATH:$SnapshotName $MountPath
                                df -h |grep $MountPath
                                echo "MountPath $MountPath was mounted."
                        fi
                else
                        echo "MountPath $MountPath does not existed!"
                fi
        else
                echo "Snapshot $SSSRCPATH/.snap/$SnapshotName does not existed!"
        fi
else
        echo "Snapshot source path $SSSRCPATH does not existed!"
fi

3. 掛載範例

# mkdir -p /SNAPSHOT/Mount/Path/Weekly0

/PATH/TO/mount_UFS_snapshot.sh /SRCPATH Weekly.0 /SNAPSHOT/Mount/Path/Weekly0

4. 卸載 UFS snapshot 的 shell script

#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        MountPath="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 MountPath"
        echo "Example: $0 /mnt"
        exit 1
else
        echo "Try to umount $MountPath now."
fi

MC=`df -h |grep $MountPath |wc -l`

if [ -d $MountPath ]; then
        if [ $MC -gt 0 ]; then
                snapshot umount $MountPath
                df -h |grep $MountPath
                echo "MountPath $MountPath had been umounted!"
        else
                echo "MountPath $MountPath had not been mounted!"
        fi
else
        echo "MountPath $MountPath does not existed!"
fi

5. 卸載範例

/PATH/TO/umount_UFS_snapshot.sh /SNAPSHOT/Mount/Path/Weekly0

6. UFS 做 snapshot 速度還蠻慢的. (相較於 ZFS 的快速完成 snapshot)

參考資料
FreeBSD UFS/ZFS Snapshot Management Environment

在 FreeBSD 及 Ubuntu 啟動 MLS SSD 支援 TRIM

目前手邊的機器每一台都至少裝了一顆 MLC SSD. 不論是當系統碟或當暫存區使用, 都有需要啟動 TRIM. 剛好 FreeBSD 8.2 及 Ubuntu 11.10 的 kernel 都有支援 TRIM. 所以就參考文件設定.

[[[ FreeBSD 8.2 ]]]
1. Reboot and login in Single User Mode
2. tunefs -p /dev/adXXsXX
3. tunefs -t enable /dev/adXXsXX
4. Reboot the machine
5. Check the setting

HHHHH(XXXXXX)/root# tunefs -p /dev/adXsXX
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         enabled
tunefs: maximum blocks per file in a cylinder group: (-e)  2048
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

[[[ Ubuntu 11.10 ]]]
1. Edit /etc/fstab and add "discard,noatime,nodiratime,".

root@HOSTNAME:~# diff /etc/fstab.bak /etc/fstab
10c10
< UUID=dXXXdXXX-5899-XXXX-XXXd-XXXXcXXX8XXX /               ext4    errors=remount-ro 0       1
---
> UUID=dXXXdXXX-5899-XXXX-XXXd-XXXXcXXX8XXX /               ext4    discard,noatime,nodiratime,errors=remount-ro 0       1

2. Edit /etc/rc.local and add following statements before 'exit 0'.

### For tweak SSD
echo noop > /sys/block/sda/queue/scheduler
echo 1 > /sys/block/sda/queue/iosched/fifo_batch

3. Reboot the machine

4. 參考這篇做測試. HOWTO: Check If TRIM On Ext4 Is Enabled And Working On Ubuntu And Other Distributions


參考資料 Reference:
TRIM support on UFS2 FreeBSD 9
Racklin's 阿土伯程式大觀園: 為 Thinkpad T61 Debian Linux 換上 SSD
HOWTO: Configure Ext4 to Enable TRIM Support for SSDs on Ubuntu and Other Distributions
HOWTO: Check If TRIM On Ext4 Is Enabled And Working On Ubuntu And Other Distributions
How to Tweak Your SSD in Ubuntu for Better Performance

在 FreeBSD, PC-BSD, Ubuntu, Mint, CentOS, Fedora, OpenSUSE, Debian 上使用 tmpfs 掛載 /tmp

首先說明一下, 以下是給懶人專用, RAM Disk 大小都是預設最大到 50% 記憶體. 如果有需要調整空間使用上限或在大型伺服器上實做, 請務必研究一下附件還有 tmpfs. 另外 NetBSD 5, OpenSolaris 及 Indiana 就不用傷腦筋了, 系統預設已經使用 tmpfs(在 NetBSD 使用 mfs) 掛載 /tmp.

由於現代的機器的記憶體基本上都是 GB 等級. 讓記憶體閒著也是一種浪費. 而更重要的原因是不想讓讀寫 /tmp 的動作大量存取 MLC SSD（雖然已經啟動了 TRIM）. 再加上 guest VM 常常讀寫, 多跑幾台對 VM host 還是有效能上的影響. 就這樣看著大家的文件跟著做. 也順手將手邊的測試用 guest VM 加上這個簡單的設定. 以下就是實際上在各平台上的設定方式:

[[[ FreeBSD 8, PC-BSD 8 & 9 beta ]]]
1. Edit /etc/fstab

tmpfs  /tmp  tmpfs  rw 0 0

2. Reboot the machine

[[[ Ubuntu 11.X 12.04 LTS, Mint 11 ]]]
1. Edit /etc/init.d/ramtmp.sh

#!/bin/sh
# RamDisk tmp
PATH=/sbin:/bin:/usr/bin:/usr/sbin

mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp
mount --bind /dev/shm/tmp /tmp

2. chmod 755 /etc/init.d/ramtmp.sh
3. cd /etc/rcS.d; ln -s /etc/init.d/ramtmp.sh SXXramtmp
XX : 必須在所有需要 /tmp 目錄的服務之前. 請注意英文字母排序.
簡單的作法就是在 /etc/rcS.d 下 grep tmp ./* 看有哪些檔案需要 /tmp 目錄.
4. Reboot the machine

[[[ CentOS 6, Fedora 15, ScientificLinux 5.6 ]]]
1. Edit /etc/rc.d/init.d/ramtmp

#!/bin/bash
# chkconfig: 35 80 70
# description: RAMDisk4Tmp
#
# History:
### 2012/02/25 RamDisk tmp

PATH=/sbin:/bin:/usr/bin:/usr/sbin

mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp
mount --bind /dev/shm/tmp /tmp

2. chmod 755 /etc/rc.d/init.d/ramtmp
3. chkconfig --add ramtmp
4. Reboot the machine

[[[ OpenSUSE 11.4 ]]]
1. Edit /etc/fstab

tmpfs  /tmp  tmpfs  defaults 0 0

2. Reboot the machine

[[[ Smoothwall express 3.0 sp2 ]]]
1. Edit /etc/fstab

none  /tmp  tmpfs  defaults 0 0

2. Reboot the machine

[[[ ArchLinux 3.0 ]]]
1. Edit /etc/fstab

tmpfs  /tmp  tmpfs  nodev,nosuid 0 0

2. Reboot the machine

[[[ Debian 6 ]]]
1. Edit /etc/fstab

tmpfs /tmp tmpfs defaults,nodev,nosuid 0 0

2. Reboot the machine

[[[ Windows 7 Ultimate, Windows server 2003, Windows XP ]]]
請安裝 Dataram_RAMDisk_V3.5.130R22, 詳細步驟請看參考資料.

參考資料 Reference:
FreeBSD : 【FreeBSD】用 tmpmfs 或 tmpfs 建立 Ramdisk
Ubuntu : Firefox & Chrome Cache on RAM Drive -Fedora / Ubuntu
                 在 Ubuntu 將 Firefox 網頁快取檔，設到 Ramdisk 的方法
CentOS : [筆記]Linux Tmpfs 系統加速實作
OpenSUSE : SDB:SSD performance
Debian : HOWTO: Using tmpfs for /tmp
Windows : [桌面系統]Windows 7 (32bit/64bit) 使用RamDisk 外加ReadyBoost 增加效能

在 FreeBSD 使用 portmaster 更新已安裝的 ports - 程式碼修改

程式碼:

#!/usr/local/bin/bash

PKGLISTPATH="/PATH/TO/STORE/PKGLIST"

if [ -n "$1" ]; then
        keyword="$1"
        shift
else
        echo "Usage: $0 keyword"
        echo "Example: $0 lib"
        exit 1
fi

echo $keyword

if [ -f $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt ]
then
        echo "Had $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt"
else
        echo "Generating $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt"
        portversion -l '<' > $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt
fi

for i in $(cat $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt | grep $keyword | awk '{ print $1 }'); do portmaster -D $i; done

備份系統設定且紀錄差異

為了玩玩在 shell script 使用函式功能, 寫了一隻簡單的程式碼紀錄系統設定的差異. 不過增加項目還是需要修改程式碼. 暫時先能用再來看如何修改.
程式碼:

#!/usr/local/bin/bash

BasePath="/PATH/TO/BACKUP/DIRECTORY"
PATH=$PATH

BackupConfig(){
if [ -z $1 ]
# Checks if any params.
then
 echo "No parameters passed to function."
 return 0
else
 ProgName=$1
fi

if [ $2 ]
then
 BackupFolder=$2
fi

case $ProgName in
postfix) CMD="postconf -n"
  ;;
sysctl)  CMD="sysctl -a"
  ;;
pkglist) CMD="ls /var/db/pkg"
  ;;
*)  echo "Do not support this program $ProgName"
                ;;
esac

mkdir -p $BasePath/$BackupFolder
rm -f $BasePath/$BackupFolder/${ProgName}_old.txt

echo "Check if had OLD data : "
if [ -f $BasePath/$BackupFolder/${ProgName}_now.txt ]; then
 echo "   Had OLD data! Rename it to ${ProgName}_old.txt"
 mv $BasePath/$BackupFolder/${ProgName}_now.txt $BasePath/$BackupFolder/${ProgName}_old.txt
else
 echo "   Do not have OLD data."
fi

echo "Backup configuration of ${ProgName} now."
$CMD > $BasePath/$BackupFolder/${ProgName}_now.txt

echo "Generate diff log : "
if [ -f $BasePath/$BackupFolder/${ProgName}_old.txt ]; then
 echo "   Generate diff log between ${ProgName}_new.txt and ${ProgName}_old.txt."
 diff $BasePath/$BackupFolder/${ProgName}_now.txt $BasePath/$BackupFolder/${ProgName}_old.txt > $BasePath/$BackupFolder/diff-`date +"%Y%m%d"`.txt
 find  $BasePath/$BackupFolder/ -type f -size 0 | xargs rm -f
else
 echo "   Do not have OLD data."
fi

}


### Backup postfix setting
BackupConfig postfix postfix_config_backup

### Backup sysctl setting
BackupConfig sysctl sysctl_config_backup

### Backup /var/db/pkg
BackupConfig pkglist installed-pkglist

Shell script 搭配 ipfw table 阻擋入侵 sshd 攻擊 authlog_banip.sh 修正版

1. 目前已將之前只有阻擋 30 天改為先阻擋 30 天, 若是 30 ~ 90 天前仍有多筆紀錄就再加鎖一段時間直到符合 Good Boy 條件.

2. 程式碼如下:
For SSHD

#!/usr/local/bin/bash

LOGFILE="/PATH/TO/auth.log"
PROCFOLDER="/PATH/TO/auth_sshd_banip"
TOTALBANLOGFILE="/PATH/TO/auth_sshd_banip.log"
TOTALBANLOGFILESORTED="/PATH/TO/auth_sshd_banip_sorted.log"
TOTALBANLOGFILEFROMPROCFOLDERSORTED="/PATH/TO/auth_sshd_banip_from_procfolder_sorted.log"
TOTALRESCUELIST="/PATH/TO/auth_sshd_rescue_list.log"
CRONJOBDATE="crontjobdate.log"
TODAYLOGFM=`date +"%b %e"`
DefaultCheckPeriod="30"
MVFilePeriod="90"

TODAY=`date +"%Y%m%d"`
OLDDAY=`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`
OLDYEAR=`date -v-${DefaultCheckPeriod}d +"%Y"`

fwcmd="/sbin/ipfw"

echo "Prepare folder and file"
mkdir -p $PROCFOLDER
touch $TOTALBANLOGFILE
touch $TOTALRESCUELIST
touch $PROCFOLDER/$CRONJOBDATE
touch $TOTALBANLOGFILEFROMPROCFOLDERSORTED

SSHDErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Invalid user" | awk '{ print $10 }' | wc -l`
SSHDErrorCounter2=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Did not receive identification string from" | awk '{ print $12 }' | wc -l`
SSHDErrorCounter3=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "not allowed because none of user" | awk '{ print $9 }' | wc -l`

if [ $SSHDErrorCounter1 -ne 0 ]||[ $SSHDErrorCounter2 -ne 0 ]||[ $SSHDErrorCounter3 -ne 0 ]||[ -f $PROCFOLDER/sshdpre-$TODAY.log ]; then
        echo "Write pre-process file"
        touch $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Invalid user" | awk '{ print $10 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Did not receive identification string from" | awk '{ print $12 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "not allowed because none of user" | awk '{ print $9 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cp -a $PROCFOLDER/sshdpre-$TODAY.log /tmp
        cat /tmp/sshdpre-$TODAY.log | sort -n | sort -u > $PROCFOLDER/sshdpre-$TODAY.log
        rm -f /tmp/sshdpre-$TODAY.log

        echo "Collect new ban IP"
        for ip in $(awk '{ print }' $PROCFOLDER/sshdpre-$TODAY.log)
        do
                if [ `grep $ip $PROCFOLDER/sshd-*.log | wc -l` -gt 0 ]; then
                        echo $ip "Old bad boy"
                else
                        echo $ip
                        
                        touch $PROCFOLDER/sshd-$TODAY.log
                        echo $ip >> $PROCFOLDER/sshd-$TODAY.log
                        echo $ip >> $TOTALBANLOGFILE
                fi
        done

        echo "Ban bad boy"
        if [ -f $PROCFOLDER/sshd-$TODAY.log ]; then
                for banip in $(awk '{ print }' $PROCFOLDER/sshd-$TODAY.log)
                do
                        #Ban IP
                        echo "Check $banip"

                        if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                                echo "Had ban bad boy($banip)."
                        else
                                echo "Ban bad boy($banip) now."
                                ${fwcmd} table 3 add $banip/32
                                ${fwcmd} table 4 add $banip/32
                        fi
                done
        else
                echo "No bad boy"
        fi

        echo "Rescue good boy"
        if [ -f $PROCFOLDER/sshd-$OLDDAY.log ]; then
                for rescueip in $(awk '{ print }' $PROCFOLDER/sshd-$OLDDAY.log)
                do
                        #Rescue IP
                        echo "Check $rescueip"
                        
                        BadBoyDCPCounter=`find $PROCFOLDER -d 1 -ctime +${DefaultCheckPeriod} -type f -name 'sshd*.log' -exec grep $rescueip {} \; | wc -l`
                        BadBoyMVFCounter=`find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'sshd*.log' -exec grep $rescueip {} \; | wc -l`

                        if [ $BadBoyDCPCounter -lt 1 ]; then
                                if [ $BadBoyMVFCounter -le 2 ]; then
                                        echo "Rescue $rescueip"
                                        ${fwcmd} table 3 delete $rescueip/32
                                        ${fwcmd} table 4 delete $rescueip/32
                                
                                        grep -v $rescueip $TOTALBANLOGFILE > /tmp/stillbanip-$TODAY
                                        cp -f /tmp/stillbanip-$TODAY $TOTALBANLOGFILE
                                        rm -f /tmp/stillbanip-$TODAY
                                        echo $rescueip >> $TOTALRESCUELIST
                                        echo "Rescued"
                                else
                                        echo "Not bad boy in ${DefaultCheckPeriod} days, but had bad boy record in ${MVFilePeriod} days."
                                fi
                        else
                                echo "Still bad boy in ${DefaultCheckPeriod} days"
                        fi
                done
                
                #Move to old folder
                echo "Move old file to storage folder"
                mkdir -p $PROCFOLDER/$OLDYEAR
                mv $PROCFOLDER/sshd*-$OLDDAY.log $PROCFOLDER/$OLDYEAR
        else
                echo "No candidate file"
        fi

        cat $TOTALBANLOGFILE | sort -n | sort -u > $TOTALBANLOGFILESORTED
        
else
        echo "No bad boy"
fi

### Move old files to backup folder
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'sshd*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;
cat $PROCFOLDER/sshd-*.log | sort -n | sort -u > $TOTALBANLOGFILEFROMPROCFOLDERSORTED

### Additional rescure goodboy
for rescueip in `diff $TOTALBANLOGFILEFROMPROCFOLDERSORTED $TOTALBANLOGFILESORTED | grep -v a | awk '{ print $2 }' | sort -r -n`
do
        echo "Additional rescure goodboy from diff $TOTALBANLOGFILEFROMPROCFOLDERSORTED and $TOTALBANLOGFILESORTED"     
        echo "Rescue $rescueip"
        ${fwcmd} table 3 delete $rescueip/32
        ${fwcmd} table 4 delete $rescueip/32

        grep -v $rescueip $TOTALBANLOGFILE > /tmp/stillbanip-$TODAY
        cp -f /tmp/stillbanip-$TODAY $TOTALBANLOGFILE
        rm -f /tmp/stillbanip-$TODAY
        echo $rescueip >> $TOTALRESCUELIST
        echo "Rescued"
done

cat $TOTALBANLOGFILE | sort -n | sort -u > $TOTALBANLOGFILESORTED

echo $TODAY > $PROCFOLDER/$CRONJOBDATE

### For backup and administration
/PATH/TO/BACKUP_SCRIPT

For MAIL

#!/usr/local/bin/bash

PATH=$PATH

LOGFILE="/PATH/TO/maillog"
DOVECOTERRORLOGFILE="/PATH/TO/dovecot/error.log"
DOVECOTINFOLOGFILE="/PATH/TO/dovecot/info.log"
PROCFOLDER="/PATH/TO/mail_banip"
DAILYLOG="mail_banip"`date +"%Y%m%d"`".log"
TMPFILE="/tmp/mailban-"`date +"%s"`
TODAYLOGFM=`date +"%b %e"`
fwcmd="/sbin/ipfw"

DefaultCheckPeriod="30"
MVFilePeriod="30"

OLDDAILYLOG="mail_banip"`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`".log"
OLDYEAR=`date -v-${MVFilePeriod}d +"%Y"`

mkdir -p $PROCFOLDER

MAILErrorString1="NOQUEUE: reject"
MAILErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep postfix | grep "$MAILErrorString1" | wc -l`
MAILErrorString2="auth-worker(default): Error: pam"
MAILErrorCounter2=`cat $DOVECOTERRORLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString2" | awk 'BEGIN { FS = ": " } ; { print $3 }' | awk 'BEGIN { FS = "," } ; { print $2 }' | sort -n | sort -u | sed -Ee 's/)//' | wc -l`
MAILErrorString3="pop3-login: Info: Aborted login (auth failed"
MAILErrorCounter3=`cat $DOVECOTINFOLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString3" | awk 'BEGIN { FS = ", " } ; { print $4 }' | awk 'BEGIN { FS = "=" } ; { print $2 }' | sort -n | sort -u | wc -l`

if [ $MAILErrorCounter1 -ne 0 ]||[ $MAILErrorCounter2 -ne 0 ]||[ $MAILErrorCounter3 -ne 0 ]||[ -f $PROCFOLDER/$DAILYLOG ]; then
        if [ ! -f $PROCFOLDER/$DAILYLOG ]; then
                touch $PROCFOLDER/$DAILYLOG
        fi

        echo "Collect Bad boy"
        touch $TMPFILE
        cat $LOGFILE | grep "$TODAYLOGFM" | grep postfix | grep "NOQUEUE: reject" | awk 'BEGIN { FS = "[" } ; { print $3 }' | awk 'BEGIN { FS = "]" } ; { print $1 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $DOVECOTERRORLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString2" | awk 'BEGIN { FS = ": " } ; { print $3 }' | awk 'BEGIN { FS = "," } ; { print $2 }' | sort -n | sort -u | sed -Ee 's/)//' >> $PROCFOLDER/$DAILYLOG
        cat $DOVECOTINFOLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString3" | awk 'BEGIN { FS = ", " } ; { print $4 }' | awk 'BEGIN { FS = "=" } ; { print $2 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $PROCFOLDER/$DAILYLOG | sort -n | sort -u | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" > $TMPFILE
        cp -f $TMPFILE $PROCFOLDER/$DAILYLOG

        echo "Ban Bad Boy"
        for banip in $(awk '{ print }' $PROCFOLDER/$DAILYLOG)
        do
                if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                        echo "Had ban bad boy($banip)."
                else
                        echo "Ban bad boy($banip) now."
                        ${fwcmd} table 3 add $banip/32
                        ${fwcmd} table 4 add $banip/32
                fi
        done
        
        rm -f $TMPFILE
fi

if [ -f $PROCFOLDER/$OLDDAILYLOG ]; then
        for rescueip in $(awk '{ print }' $PROCFOLDER/$OLDDAILYLOG)
        do
                echo "Rescue $rescueip"
                ${fwcmd} table 3 delete $rescueip/32
                ${fwcmd} table 4 delete $rescueip/32
        done
else
        echo "No candidate file"
fi

#Move to old folder
echo "Move old file to storage folder"
mkdir -p $PROCFOLDER/$OLDYEAR
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'mail_banip*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;

For HTTPD

#!/usr/local/bin/bash

PATH=$PATH

LOGFILE="/PATH/TO/httpd-access.log"
PROCFOLDER="/PATH/TO/httpd_banip"
DAILYLOG="httpd_banip"`date +"%Y%m%d"`".log"
TMPFILE="/tmp/httpdban-"`date +"%s"`
TODAYLOGFM=`date +"%d/%b/%Y"`
fwcmd="/sbin/ipfw"

DefaultCheckPeriod="30"
MVFilePeriod="30"

OLDDAILYLOG="httpd_banip"`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`".log"
OLDYEAR=`date -v-${MVFilePeriod}d +"%Y"`

mkdir -p $PROCFOLDER

ErrorString1="xmlrpc.php"
ErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep "$ErrorString1" | grep 404 | awk 'BEGIN { FS = " " } ; { print $1 }' | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" | wc -l`

if [ $ErrorCounter1 -ne 0 ]||[ -f $PROCFOLDER/$DAILYLOG ]; then
        if [ ! -f $PROCFOLDER/$DAILYLOG ]; then
                touch $PROCFOLDER/$DAILYLOG
        fi

        echo "Collect Bad boy"
        touch $TMPFILE
        cat $LOGFILE | grep "$TODAYLOGFM" | grep "$ErrorString1" | grep 404 | awk 'BEGIN { FS = " " } ; { print $1 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $PROCFOLDER/$DAILYLOG | sort -n | sort -u | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" > $TMPFILE
        cp -f $TMPFILE $PROCFOLDER/$DAILYLOG
        
        echo "Ban Bad Boy"
        for banip in $(awk '{ print }' $PROCFOLDER/$DAILYLOG)
        do
                if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                        echo "Had ban bad boy($banip)."
                else
                        echo "Ban bad boy($banip) now."
                        ${fwcmd} table 3 add $banip/32
                        ${fwcmd} table 4 add $banip/32
                fi
        done
        
        rm -f $TMPFILE
fi

if [ -f $PROCFOLDER/$OLDDAILYLOG ]; then
        for rescueip in $(awk '{ print }' $PROCFOLDER/$OLDDAILYLOG)
        do
                echo "Rescue $rescueip"
                ${fwcmd} table 3 delete $rescueip/32
                ${fwcmd} table 4 delete $rescueip/32
        done
else
        echo "No candidate file"
fi

#Move to old folder
echo "Move old file to storage folder"
mkdir -p $PROCFOLDER/$OLDYEAR
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'httpd_banip*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;

3. 以上程式碼為自由軟體, 請自行取用. 強烈建議服用前先看清楚再服用, 避免發生意外.

如何掛載 FreeBSD 光碟影像檔

Script 如下 :

mail(xxx)# more mount_FreeBSDDVD.sh
#!/bin/sh

mkdir -p /FreeBSD-RELEASEDVD
mdconfig -a -t vnode -f /PATH/TO/FreeBSD-IMAGE.iso -u 1
mount -t cd9660 /dev/md1 /FreeBSD-RELEASEDVD

mail(xxx)# more umount_FreeBSDDVD.sh
#!/bin/sh

umount /FreeBSD-RELEASEDVD
mdconfig -d -u 1

在 FreeBSD 使用 portmaster 更新已安裝的 ports

在 bash 下完整跑完所有已安裝的 ports

for i in $(ls /var/db/pkg/); do portmaster -D $i; done
或
for i in `ls /var/db/pkg/`; do portmaster -D $i; done

在 bash 下針對需要更新版本的 ports

pkg_version -v > /PATH/TO/pkg_version_`date +"%Y%m%d"`.txt
grep "needs updating" /PATH/TO/pkg_version_`date +"%Y%m%d"`.txt > /PATH/TO/pkg_version_needs-updating_`date +"%Y%m%d"`.txt
for i in $(awk '{ print $1 }' /PATH/TO/pkg_version_needs-updating_`date +"%Y%m%d"`.txt); do portmaster -D $i; done

較簡易版

portversion -l '<' > /root/pkg_list/pkgversion_`date +"%Y%m%d"`.txt
for i in $(awk '{ print $1 }' /root/pkg_list/pkgversion_`date +"%Y%m%d"`.txt); do portmaster -D $i; done

也可以將 portmaster -D 改用 pkg_upgrade 取代.

How to create ZFS storage in FreeBSD 7.4

1. Edit /etc/rc.conf. Add
zfs_enable="YES"

2. Shutdown machine and plug all hard drives.

3. Create storage pool
#   zpool create PoolName raidz ad1 ad2 ad3 ...

4. Create volume
# zfs create PoolName/VolumeName

5. Set mount point
# zfs set mountpoint=/Path/To/Mount/Point PoolName/VolumeName

6. Setup daily check
# echo 'daily_status_zfs_enable="YES"' >> /etc/periodic.conf

7. Setup volume compression
# zfs set compression=gzip PoolName/VolumeName

8. Check ZFS pool status
# zpool status

9. Check ZFS command history
# zpool history

10. Check ZFS mounting point
# zfs get mountpoint

11. Setup volume quota
# zfs set quota=QuotaSize PoolName/VolumeName

12. Show volume setting
# zfs get all PoolName/VolumeName

13. Show ZFS I/O status
# zfs iostat 1

使用 crontab 管理 ZFS snapshot

近期終於將之前延遲已久的 ZFS storage 計畫實現. 已試過當 raidz 硬碟發生問題時, 使用 zfs replace 更換硬碟. 當然就會想用 snapshot 這個重要的功能. 所以就動手寫了簡單的 shell script 搭配 crontab 做自動化管理.

1. 簡單指令

觀察 zpool 清單, 狀態
   # zpool list
   # zpool status

   觀察 ZFS volume 清單
   # zfs list

   觀察 snapshot 清單 (/usr/ports/sysutils/freebsd-snapshot)
   # snapshot list

2. shell script

#!/usr/local/bin/bash

PATH=$PATH

LOGFILE="/var/log/snapshotfs.log"
TODAY=`date +"%Y%m%d"`
CHKPARA=0

if [ -n "$1" ]; then
 fstype="$1"
 shift
else
 CHKPARA=1
fi


if [ -n "$1" ]; then
 sstype="$1"
 shift
else
 CHKPARA=1
fi


if [ -n "$1" ]; then
 maxnum="$1"
 shift
else
 CHKPARA=1
fi


if [ -n "$1" ]; then
 volpath="$1"
 shift
else
 CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
 echo "Usage: $0 fstype sstype maxnum /filesystem|volume_path"
 echo "Example: $0 zfs [H|D|W|M|Q|Y] 7 /databackup/backup"
 exit 1
else
 touch $LOGFILE
 echo `date +"%Y/%m/%d %H:%M:%S"` >> $LOGFILE
 echo "$0 $fstype $sstype $maxnum $volpath" >> $LOGFILE
 echo "Check parameters for taking a snapshot of $volpath($fstype)." >> $LOGFILE
fi

if [ -d $volpath ]; then
 if [ $fstype = 'zfs' ]; then
  if [ -d $volpath/.zfs/snapshot ]; then
   echo "Check path : OK" >> $LOGFILE
  else
   echo "Check path : Failed, snapshot folder does not exist." >> $LOGFILE
   exit 1
  fi
 elif [ $fstype = 'ufs' ]; then
  if [ -d $volpath/.snap ]; then
   echo "Check path : OK" >> $LOGFILE
  else
   echo "Check path : Failed, snapshot folder does not exist." >> $LOGFILE
   exit 1
  fi
 else
  echo "Unsupported file system $fstype" >> $LOGFILE
  exit 1
 fi
else
 echo "Check path : Failed, filesystem|volume_path does not exist." >> $LOGFILE
 exit 1
fi

if [ $maxnum -gt 0 ]; then
 echo "Check maxnum : OK" >> $LOGFILE
else
 echo "Check maxnum : Failed, must input positive integer." >> $LOGFILE
        exit 1
fi

if [ $fstype = 'zfs' ]; then
 echo "ZFS" >> $LOGFILE

 case $sstype in
 H)  echo "Hourly Snapshot" >> $LOGFILE
   TODAY=`date +"%Y%m%d%H"`
   ;;
 D)  echo "Daily Snapshot" >> $LOGFILE
   ;;
 W)  echo "Weekly Snapshot" >> $LOGFILE
   ;;
 M)  echo "Monthly Snapshot" >> $LOGFILE
   ;;
 Q)  echo "Quaterly Snapshot" >> $LOGFILE
   ;;
 Y)  echo "Yearly Snapshot" >> $LOGFILE
   ;;
 *)  echo "sstype must be H | D | W | M | Q | Y " >> $LOGFILE
   exit 1
 esac

 #Snapshot
 if [ -d $volpath/.zfs/snapshot/$sstype$TODAY ]; then
  echo "Had snapshot($volpath $sstype$TODAY)." >> $LOGFILE
 else
  echo "Take a snapshot of $volpath." >> $LOGFILE
  zfs snapshot `zfs list -H -o name ${volpath}`@${sstype}${TODAY}
 fi

 #Remove more old snapshot

 KEEPSSNUM=$maxnum
 REALSSNUM=`ls $volpath/.zfs/snapshot/ | grep $sstype | wc -l`
 RMSSNUM=`expr $REALSSNUM - $KEEPSSNUM`
 RMSSID=""

 if [ $RMSSNUM -gt 0 ]; then
  for RMSSID in `ls $volpath/.zfs/snapshot/ | grep $sstype | sort -n | head -n $RMSSNUM | sed -e 's/\///'`
  do
   if [ -d $volpath/.zfs/snapshot/$RMSSID ]; then
    echo "Remove old snapshot($volpath $RMSSID)." >> $LOGFILE
    zfs destroy `zfs list -H -o name ${volpath}`@$RMSSID
   else
    echo "Old snapshot($volpath $RMSSID) does not exist." >> $LOGFILE
   fi
  done
 fi

elif [ $fstype = 'ufs' ]; then
 echo "UFS" >> $LOGFILE

 case $sstype in
 H) echo "Hourly Snapshot" >> $LOGFILE
  sskey='Hourly'
  ;;
 D) echo "Daily Snapshot" >> $LOGFILE
  sskey='Daily'
  ;;
 W) echo "Weekly Snapshot" >> $LOGFILE
  sskey='Weekly'
  ;;
 M) echo "Monthly Snapshot" >> $LOGFILE
  sskey='Monthly'
  ;;
 Q) echo "Quaterly Snapshot" >> $LOGFILE
  sskey='Quaterly'
  ;;
 Y) echo "Yearly Snapshot" >> $LOGFILE
  sskey='Yearly'
  ;;
 *) echo "sstype must be H | D | W | M | Q | Y " >> $LOGFILE
  exit 1
 esac
 
 #Snapshot
 echo "Take a snapshot UFS ${volpath}" >> $LOGFILE
 /usr/local/sbin/snapshot make -g${maxnum} ${volpath}:${sskey}
else
 echo "Unsupported file system $fstype" >> $LOGFILE
fi

3. crontab 排程自動化

### ZFS snapshot
### Hourly
45      *       *       *       *       root    /PATH/TO/zfs_snapshot_hourly.sh
### 3 Hourly
50      */3     *       *       *       root    /PATH/TO/zfs_snapshot_3hourly.sh
### Daily
0       23      *       *       *       root    /PATH/TO/zfs_snapshot_daily.sh
### Weekly
20      23      *       *       1       root    /PATH/TO/zfs_snapshot_weekly.sh
### Monthly
40      23      1       *       *       root    /PATH/TO/zfs_snapshot_monthly.sh
### Quaterly
53      23      1       1,4,7,10        *       *       root    /PATH/TO/zfs_snapshot_quaterly.sh
### Yearly
55      23      31      12      *       root    /PATH/TO/zfs_snapshot_yearly.sh


#!/usr/local/bin/bash

PATH=$PATH

### ZFS snapshot               SSTYPE SSCOUNT
### H for hourly or 3hourly ex: H 8
### D for daily             ex: D 7
### W for weekly            ex: W 4
### M for monthly           ex: M 6
### Q for quaterly          ex: Q 4
### Y for yearly            ex: Y 3

/root/root_scripts/snapshotfs.sh zfs SSTYPE SSCOUNT /zfs-volumn/path1
/root/root_scripts/snapshotfs.sh zfs SSTYPE SSCOUNT /zfs-volumn/path2
/root/root_scripts/snapshotfs.sh zfs SSTYPE SSCOUNT /zfs-volumn/path3