Wednesday, March 28, 2012

使用 rsnapshot 備份及比較設定檔

1. 安裝 rsnapshot
在 Ubuntu 下直接用 apt-get install rsnapshot.
在 FreeBSD 下請到 /usr/ports/sysutils/rsnapshot 下 make install clean. (在我的 FreeBSD 實體及虛擬環境中並沒有實際安裝測試, 以下僅用 Ubuntu 作為範例.)

2. 調整設定檔 /etc/rsnapshot.conf
# diff /etc/rsnapshot.conf /etc/rsnapshot.conf.20120322
27c27
< snapshot_root /.rsnapshot/
---
> snapshot_root /var/cache/rsnapshot/
97c97
< retain                hourly  8
---
> retain                hourly  6
100c100
< retain                monthly 6
---
> #retain       monthly 3
124c124
< logfile       /var/log/rsnapshot.log
---
> #logfile      /var/log/rsnapshot.log
230c230
< #backup       /home/          localhost/
---
> backup        /home/          localhost/
232,235c232
< backup        /usr/local/etc/ localhost/
< backup        /var/log/       localhost/
< backup        /var/mail/      localhost/
< backup        /root           localhost/
---
> backup        /usr/local/     localhost/
註 : 因為 /home 已將大部分資料放上 NFS, 已經有 ZFS snapshot. 就不浪費時間多做 snapshot.

3. 設定定期執行
0 */3 * * * /usr/bin/rsnapshot hourly
0 23 * * * /usr/bin/rsnapshot daily
15 23 * * 1 /usr/bin/rsnapshot weekly
30 23 1 * * /usr/bin/rsnapshot monthly
0 0 * * 1 /PATH/TO/REPORT/PROG/report_CHK_ETC_diff.sh

4. CHK_ETC_diff.sh
#!/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        CHKPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSKEYWORD="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSBASE="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 check_path snapshot_keyword snapshot_base_path"
        echo "Example: $0 /etc Weekly /SNAPSHOT/System"
        exit 1
else
        echo "Collect difference between $CHKPATH and $SSBASE/${SSKEYWORD}*$CHKPATH."
fi

date +"%Y/%m/%d %H:%M:%S"
LAST_DIR="/"
CHKPATH=`echo $CHKPATH | sed -e 's/\///'`

for CONF in `find $LAST_DIR$CHKPATH -type f -name '*'`
do
        if [ `ls $SSBASE | grep ${SSKEYWORD} |wc -l` -lt 1 ]; then
                echo "$SSBASE/${SSKEYWORD}* does not existed!"
                exit 1
        fi
        ls -d $SSBASE/${SSKEYWORD}* | while read DIR
        do
                if [ $LAST_DIR = "/" ]; then
                        HOST=""
                else
                        HOST="/localhost"
                fi
                if [ -f ${LAST_DIR}${HOST}${CONF} ]&&[ -f ${DIR}/localhost${CONF} ]; then
                        if [ $LAST_DIR = "/" ]; then
                                LAST_DIR=""
                        fi
                        DIFFC=`diff ${LAST_DIR}${HOST}${CONF} ${DIR}/localhost${CONF} | wc -l`
                        if [ $DIFFC -gt 0 ]; then
                                echo "[[[ Difference between ${LAST_DIR}${HOST}${CONF} and ${DIR}/localhost${CONF} ]]]"
                                diff ${LAST_DIR}${HOST}${CONF} ${DIR}/localhost${CONF}
                                echo ""
                        fi
                        DIFFC=0
                else
                        if [ ! -f ${LAST_DIR}${HOST}${CONF} ]; then
                                echo "File ${LAST_DIR}${HOST}${CONF} does not existed."
                        fi
                fi

                LAST_DIR=$DIR
        done
done
echo ""
date +"%Y/%m/%d %H:%M:%S"

5. report_CHK_ETC_diff.sh
#!/bin/bash

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:${PATH}

CHKPROG="/PATH/TO/PROG/CHK_ETC_diff.sh /etc daily /.rsnapshot"
REPORTPATH="/PATH/TO/STORE/REPORT/report_CHK_ETC_diff"
REPORTKEYWORD="etcdiff"
REPORTTITLE="Check Difference of ETC Report"
REMAILADDR="ACCOUNT@DOMAIN.NAME"

mkdir -p $REPORTPATH/`date +"%Y"`
mv $REPORTPATH/${REPORTKEYWORD}_*.log $REPORTPATH/`date +"%Y"`
$CHKPROG > $REPORTPATH/${REPORTKEYWORD}_`hostname`_`date +"%Y%m%d"`.log
cat $REPORTPATH/${REPORTKEYWORD}_`hostname`_`date +"%Y%m%d"`.log | mail -s "$REPORTTITLE" $REMAILADDR

參考資料:
淺談linux下的快照備份軟體—rsnapshot

Wednesday, March 21, 2012

應用 UFS snapshot 進行比對 /etc 及 /usr/local/etc 下的檔案內容

1. 這隻程式運作的前提是已經完成 UFS snapshot 並且已經正常掛載. 沒有掛載 snapshot 只會看到出現很多找不到檔案的訊息.

2. 程式碼
#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        CHKPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSKEYWORD="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSBASE="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 check_path snapshot_keyword snapshot_base_path"
        echo "Example: $0 /etc Weekly /SNAPSHOT/System"
        exit 1
else
        echo "Collect difference between $CHKPATH and $SSBASE/${SSKEYWORD}*$CHKPATH."
fi

date +"%Y/%m/%d %H:%M:%S"
LAST_DIR="/"
CHKPATH=`echo $CHKPATH | sed -e 's/\///'`

for CONF in `find $LAST_DIR$CHKPATH -type f -name '*'`
do
        if [ `ls $SSBASE | grep ${SSKEYWORD} |wc -l` -lt 1 ]; then
                echo "$SSBASE/${SSKEYWORD}* does not existed!"
                exit 1
        fi
        ls -d $SSBASE/${SSKEYWORD}* | while read DIR
        do
                if [ -f ${LAST_DIR}${CONF} ]&&[ -f ${DIR}${CONF} ]; then
                        if [ $LAST_DIR = "/" ]; then
                                LAST_DIR=""
                        fi
                        DIFFC=`diff ${LAST_DIR}${CONF} ${DIR}${CONF} | wc -l`
                        if [ $DIFFC -gt 0 ]; then
                                echo "[[[ Difference between ${LAST_DIR}${CONF} and ${DIR}${CONF} ]]]"
                                diff ${LAST_DIR}${CONF} ${DIR}${CONF}
                                echo ""
                        fi
                        DIFFC=0
                else
                        if [ ! -f ${LAST_DIR}${CONF} ]; then
                                echo "File ${LAST_DIR}${CONF} does not existed."
                        fi
                fi

                LAST_DIR=$DIR
        done
done
echo ""
date +"%Y/%m/%d %H:%M:%S"

3. 使用範例
/PATH/TO/CHK_ETC_diff.sh /PATH/TO/SRC SNAPSHOT_KEYWORD /PATH/TO/SNAPSHOT_ROOT

Sunday, March 18, 2012

在 FreeBSD 8 對 UFS filesystem 做 snapshot

1. 在 crontab 定期做 snapshot
### UFS snapshot
0    1      *       *       *       root    /usr/local/sbin/snapshot make -g7 /SRCPATH:Daily
0      2      *       *       1       root    /usr/local/sbin/snapshot make -g4 /SRCPATH:Weekly

2. 掛載 UFS snapshot 的 shell script
#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        SSSRCPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SnapshotName="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        MountPath="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 Snapshot_Src_Path SnapshotName MountPath"
        echo "Example: $0 /:Weekly.0 /mnt"
        exit 1
else
        echo "Try to mount $SSSRCPATH:$SnapshotName to $MountPath now."
fi

MC=`df -h |grep $MountPath |wc -l`

if [ -d $SSSRCPATH ]; then
        if [ -f $SSSRCPATH/.snap/$SnapshotName ]; then
                if [ -d $MountPath ]; then
                        if [ $MC -gt 0 ]; then
                                echo "MountPath $MountPath had been mounted!"
                        else
                                snapshot mount $SSSRCPATH:$SnapshotName $MountPath
                                df -h |grep $MountPath
                                echo "MountPath $MountPath was mounted."
                        fi
                else
                        echo "MountPath $MountPath does not existed!"
                fi
        else
                echo "Snapshot $SSSRCPATH/.snap/$SnapshotName does not existed!"
        fi
else
        echo "Snapshot source path $SSSRCPATH does not existed!"
fi

3. 掛載範例
# mkdir -p /SNAPSHOT/Mount/Path/Weekly0

/PATH/TO/mount_UFS_snapshot.sh /SRCPATH Weekly.0 /SNAPSHOT/Mount/Path/Weekly0

4. 卸載 UFS snapshot 的 shell script
#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        MountPath="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 MountPath"
        echo "Example: $0 /mnt"
        exit 1
else
        echo "Try to umount $MountPath now."
fi

MC=`df -h |grep $MountPath |wc -l`

if [ -d $MountPath ]; then
        if [ $MC -gt 0 ]; then
                snapshot umount $MountPath
                df -h |grep $MountPath
                echo "MountPath $MountPath had been umounted!"
        else
                echo "MountPath $MountPath had not been mounted!"
        fi
else
        echo "MountPath $MountPath does not existed!"
fi

5. 卸載範例
/PATH/TO/umount_UFS_snapshot.sh /SNAPSHOT/Mount/Path/Weekly0

6. UFS 做 snapshot 速度還蠻慢的. (相較於 ZFS 的快速完成 snapshot)

參考資料
FreeBSD UFS/ZFS Snapshot Management Environment

Saturday, March 03, 2012

在 FreeBSD 及 Ubuntu 啟動 MLS SSD 支援 TRIM

目前手邊的機器每一台都至少裝了一顆 MLC SSD. 不論是當系統碟或當暫存區使用, 都有需要啟動 TRIM. 剛好 FreeBSD 8.2 及 Ubuntu 11.10 的 kernel 都有支援 TRIM. 所以就參考文件設定.

[[[ FreeBSD 8.2 ]]]
1. Reboot and login in Single User Mode
2. tunefs -p /dev/adXXsXX
3. tunefs -t enable /dev/adXXsXX
4. Reboot the machine
5. Check the setting
HHHHH(XXXXXX)/root# tunefs -p /dev/adXsXX
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         enabled
tunefs: maximum blocks per file in a cylinder group: (-e)  2048
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

[[[ Ubuntu 11.10 ]]]
1. Edit /etc/fstab and add "discard,noatime,nodiratime,".
root@HOSTNAME:~# diff /etc/fstab.bak /etc/fstab
10c10
< UUID=dXXXdXXX-5899-XXXX-XXXd-XXXXcXXX8XXX /               ext4    errors=remount-ro 0       1
---
> UUID=dXXXdXXX-5899-XXXX-XXXd-XXXXcXXX8XXX /               ext4    discard,noatime,nodiratime,errors=remount-ro 0       1

2. Edit /etc/rc.local and add following statements before 'exit 0'.
### For tweak SSD
echo noop > /sys/block/sda/queue/scheduler
echo 1 > /sys/block/sda/queue/iosched/fifo_batch

3. Reboot the machine

4. 參考這篇做測試. HOWTO: Check If TRIM On Ext4 Is Enabled And Working On Ubuntu And Other Distributions


參考資料 Reference:
TRIM support on UFS2 FreeBSD 9
Racklin's 阿土伯程式大觀園: 為 Thinkpad T61 Debian Linux 換上 SSD
HOWTO: Configure Ext4 to Enable TRIM Support for SSDs on Ubuntu and Other Distributions
HOWTO: Check If TRIM On Ext4 Is Enabled And Working On Ubuntu And Other Distributions
How to Tweak Your SSD in Ubuntu for Better Performance

在 FreeBSD, PC-BSD, Ubuntu, Mint, CentOS, Fedora, OpenSUSE, Debian 上使用 tmpfs 掛載 /tmp

首先說明一下, 以下是給懶人專用, RAM Disk 大小都是預設最大到 50% 記憶體. 如果有需要調整空間使用上限或在大型伺服器上實做, 請務必研究一下附件還有 tmpfs. 另外 NetBSD 5, OpenSolaris 及 Indiana 就不用傷腦筋了, 系統預設已經使用 tmpfs(在 NetBSD 使用 mfs) 掛載 /tmp.

由於現代的機器的記憶體基本上都是 GB 等級. 讓記憶體閒著也是一種浪費. 而更重要的原因是不想讓讀寫 /tmp 的動作大量存取 MLC SSD(雖然已經啟動了 TRIM). 再加上 guest VM 常常讀寫, 多跑幾台對 VM host 還是有效能上的影響. 就這樣看著大家的文件跟著做. 也順手將手邊的測試用 guest VM 加上這個簡單的設定. 以下就是實際上在各平台上的設定方式:

[[[ FreeBSD 8, PC-BSD 8 & 9 beta ]]]
1. Edit /etc/fstab
tmpfs  /tmp  tmpfs  rw 0 0

2. Reboot the machine

[[[ Ubuntu 11.X 12.04 LTS, Mint 11 ]]]
1. Edit /etc/init.d/ramtmp.sh
#!/bin/sh
# RamDisk tmp
PATH=/sbin:/bin:/usr/bin:/usr/sbin

mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp
mount --bind /dev/shm/tmp /tmp

2. chmod 755 /etc/init.d/ramtmp.sh
3. cd /etc/rcS.d; ln -s /etc/init.d/ramtmp.sh SXXramtmp
XX : 必須在所有需要 /tmp 目錄的服務之前. 請注意英文字母排序.
簡單的作法就是在 /etc/rcS.d 下 grep tmp ./* 看有哪些檔案需要 /tmp 目錄.
4. Reboot the machine

[[[ CentOS 6, Fedora 15, ScientificLinux 5.6 ]]]
1. Edit /etc/rc.d/init.d/ramtmp
#!/bin/bash
# chkconfig: 35 80 70
# description: RAMDisk4Tmp
#
# History:
### 2012/02/25 RamDisk tmp

PATH=/sbin:/bin:/usr/bin:/usr/sbin

mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp
mount --bind /dev/shm/tmp /tmp

2. chmod 755 /etc/rc.d/init.d/ramtmp
3. chkconfig --add ramtmp
4. Reboot the machine

[[[ OpenSUSE 11.4 ]]]
1. Edit /etc/fstab
tmpfs  /tmp  tmpfs  defaults 0 0

2. Reboot the machine

[[[ Smoothwall express 3.0 sp2 ]]]
1. Edit /etc/fstab
none  /tmp  tmpfs  defaults 0 0

2. Reboot the machine

[[[ ArchLinux 3.0 ]]]
1. Edit /etc/fstab
tmpfs  /tmp  tmpfs  nodev,nosuid 0 0

2. Reboot the machine

[[[ Debian 6 ]]]
1. Edit /etc/fstab
tmpfs /tmp tmpfs defaults,nodev,nosuid 0 0

2. Reboot the machine

[[[ Windows 7 Ultimate, Windows server 2003, Windows XP ]]]
請安裝 Dataram_RAMDisk_V3.5.130R22, 詳細步驟請看參考資料.

參考資料 Reference:
FreeBSD : 【FreeBSD】用 tmpmfs 或 tmpfs 建立 Ramdisk
Ubuntu : Firefox & Chrome Cache on RAM Drive -Fedora / Ubuntu
                 在 Ubuntu 將 Firefox 網頁快取檔,設到 Ramdisk 的方法
CentOS : [筆記]Linux Tmpfs 系統加速實作
OpenSUSE : SDB:SSD performance
Debian : HOWTO: Using tmpfs for /tmp
Windows : [桌面系統]Windows 7 (32bit/64bit) 使用RamDisk 外加ReadyBoost 增加效能

Saturday, February 18, 2012

在 FreeBSD 使用 portmaster 更新已安裝的 ports - 程式碼修改

程式碼:
#!/usr/local/bin/bash

PKGLISTPATH="/PATH/TO/STORE/PKGLIST"

if [ -n "$1" ]; then
        keyword="$1"
        shift
else
        echo "Usage: $0 keyword"
        echo "Example: $0 lib"
        exit 1
fi

echo $keyword

if [ -f $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt ]
then
        echo "Had $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt"
else
        echo "Generating $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt"
        portversion -l '<' > $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt
fi

for i in $(cat $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt | grep $keyword | awk '{ print $1 }'); do portmaster -D $i; done

備份系統設定且紀錄差異

為了玩玩在 shell script 使用函式功能, 寫了一隻簡單的程式碼紀錄系統設定的差異. 不過增加項目還是需要修改程式碼. 暫時先能用再來看如何修改.
程式碼:
#!/usr/local/bin/bash

BasePath="/PATH/TO/BACKUP/DIRECTORY"
PATH=$PATH

BackupConfig(){
if [ -z $1 ]
# Checks if any params.
then
 echo "No parameters passed to function."
 return 0
else
 ProgName=$1
fi

if [ $2 ]
then
 BackupFolder=$2
fi

case $ProgName in
postfix) CMD="postconf -n"
  ;;
sysctl)  CMD="sysctl -a"
  ;;
pkglist) CMD="ls /var/db/pkg"
  ;;
*)  echo "Do not support this program $ProgName"
                ;;
esac

mkdir -p $BasePath/$BackupFolder
rm -f $BasePath/$BackupFolder/${ProgName}_old.txt

echo "Check if had OLD data : "
if [ -f $BasePath/$BackupFolder/${ProgName}_now.txt ]; then
 echo "   Had OLD data! Rename it to ${ProgName}_old.txt"
 mv $BasePath/$BackupFolder/${ProgName}_now.txt $BasePath/$BackupFolder/${ProgName}_old.txt
else
 echo "   Do not have OLD data."
fi

echo "Backup configuration of ${ProgName} now."
$CMD > $BasePath/$BackupFolder/${ProgName}_now.txt

echo "Generate diff log : "
if [ -f $BasePath/$BackupFolder/${ProgName}_old.txt ]; then
 echo "   Generate diff log between ${ProgName}_new.txt and ${ProgName}_old.txt."
 diff $BasePath/$BackupFolder/${ProgName}_now.txt $BasePath/$BackupFolder/${ProgName}_old.txt > $BasePath/$BackupFolder/diff-`date +"%Y%m%d"`.txt
 find  $BasePath/$BackupFolder/ -type f -size 0 | xargs rm -f
else
 echo "   Do not have OLD data."
fi

}


### Backup postfix setting
BackupConfig postfix postfix_config_backup

### Backup sysctl setting
BackupConfig sysctl sysctl_config_backup

### Backup /var/db/pkg
BackupConfig pkglist installed-pkglist

Monday, February 13, 2012

自動化抓衛星雲圖(全球及氣象局) 更新版

1.抓取全球衛星雲圖(Hemisphere)程式碼:

#!/usr/local/bin/bash

### http://www.opentopia.com/sunlightmaprect.html
### http://www.opentopia.com/sunlightmaphemi.html
### http://static.die.net/earth/mercator/1600.jpg
### http://static.die.net/earth/hemisphere/1600.jpg

DATESTR=`date "+%Y%m%d%H%M"`
LASTMONTH=`date -v -1m +"%Y%m"`
THISMONTH=`date "+%Y%m"`
LASTYEAR=`date -v -1y +"%Y"`
THISYEAR=`date "+%Y"`
WEBROOTDIR="/PATH/to/Web/Root/Directory"
BASEDIR="$WEBROOTDIR/world_sunlight_map"
TEMPDIR="/tmp"
WGETLOC="/usr/local/bin/wget"
USERAGENT='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"'
CONVERTLOC="/usr/local/bin/convert"

#下載全球衛星雲圖
###$WGETLOC -q -O $WEBROOTDIR/world_sunlight_map_ori.jpg http://www.opentopia.com/images/data/sunlight/world_sunlight_map_rectangular.jpg
###$WGETLOC -q -O $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg http://www.opentopia.com/images/data/sunlight/world_sunlight_map_hemisphere.jpg
$WGETLOC -q ${USERAGENT} -O $WEBROOTDIR/world_sunlight_map_ori.jpg http://static.die.net/earth/mercator/1600.jpg
$WGETLOC -q ${USERAGENT} -O $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg http://static.die.net/earth/hemisphere/1600.jpg
/bin/rm -f $WEBROOTDIR/world_sunlight_map_rectangular.jpg
/bin/rm -f $WEBROOTDIR/world_sunlight_map_hemisphere.jpg
/bin/mv $WEBROOTDIR/world_sunlight_map_ori.jpg $WEBROOTDIR/world_sunlight_map_rectangular.jpg
/bin/mv $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg $WEBROOTDIR/world_sunlight_map_hemisphere.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_rectangular.jpg ${BASEDIR}/world_sunlight_map_rectangular_${DATESTR}.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_hemisphere.jpg ${BASEDIR}/world_sunlight_map_hemisphere_${DATESTR}.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_rectangular.jpg ${TEMPDIR}/original.jpg

#切割、合併圖片, 讓東亞在地圖中間
$CONVERTLOC ${TEMPDIR}/original.jpg -crop 600x887+0+0 ${TEMPDIR}/left.jpg
$CONVERTLOC ${TEMPDIR}/original.jpg -crop 1600x887+600+0 ${TEMPDIR}/right.jpg
$CONVERTLOC ${TEMPDIR}/right.jpg ${TEMPDIR}/left.jpg +append ${BASEDIR}/TW/world_sunlight_map.jpg

/bin/cp ${BASEDIR}/TW/world_sunlight_map.jpg ${BASEDIR}/TW/world_sunlight_map/world_sunlight_map_${DATESTR}.jpg
/bin/rm -f ${TEMPDIR}/original.jpg ${TEMPDIR}/left.jpg ${TEMPDIR}/right.jpg
2. 抓取中央氣象局 V7 東亞雲圖程式碼:

#!/usr/local/bin/bash

PATH=$PATH

### http://www.cwb.gov.tw/V7/observe/satellite/Sat_H_EA.htm?type=1#

WEBROOTDIR="/PATH/to/Web/Root/Directory"
WGETLOC="/usr/local/bin/wget"

###中央氣象局衛星雲圖
CWBWEBBASEURL="http://www.cwb.gov.tw/V7/observe/satellite/Data"
CWBBASEDIR="$WEBROOTDIR/CWB"
CWBDATE=`date +"%Y-%m-%d-%H-"`
CWBDATE1="${CWBDATE}00"
CWBDATE2="${CWBDATE}30"
NOWMINSEC=`date +"%H%M"`
USERAGENT='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"'

###中央氣象局衛星雲圖
# MTSAT 紅外線雲圖      HS1P
# MTSAT 紅外線 色調強化 HS1Q
# MTSAT 紅色線 黑白     HS1O
# MTSAT 可見光          HSAO

for MapType in HS1P HS1Q HS1O HSAO
do
        if [ -f ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg ]; then
                echo "${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg was existed."
        else
                if [ ${MapType} == 'HSAO' ]; then
                        if [ $NOWMINSEC -gt 0600 ]&&[ $NOWMINSEC -le 1930 ]; then
                                $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE1}.jpg
                        fi
                else
                        $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE1}.jpg
                fi
        fi

        if [ -f ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg ]; then
                echo "${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg was existed."
        else
                if [ ${MapType} == 'HSAO' ]; then
                        if [ $NOWMINSEC -gt 0600 ]&&[ $NOWMINSEC -le 1930 ]; then
                                $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE1}.jpg
                        fi
                else
                        $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE2}.jpg
                fi
        fi

        sleep 2
done

# Remove empty file
find  ${CWBBASEDIR}/ -type f -size 0 | xargs rm -f
舊版程式碼:
#!/usr/local/bin/bash

### http://www.opentopia.com/sunlightmaprect.html
### http://www.opentopia.com/sunlightmaphemi.html
### http://www.cwb.gov.tw/V7/observe/satellite/Sat_H_EA.htm?type=1#

DATESTR=`date "+%Y%m%d%H%M"`
LASTMONTH=`date -v -1m +"%Y%m"`
THISMONTH=`date "+%Y%m"`
LASTYEAR=`date -v -1y +"%Y"`
THISYEAR=`date "+%Y"`
WEBROOTDIR="/PATH/to/Web/Root/Directory"
BASEDIR="$WEBROOTDIR/world_sunlight_map"
TEMPDIR="/tmp"
WGETLOC="/usr/local/bin/wget"
CONVERTLOC="/usr/local/bin/convert"

###中央氣象局衛星雲圖
CWBWEBBASEURL="http://www.cwb.gov.tw/V7/observe/satellite/Data"
CWBBASEDIR="$WEBROOTDIR/CWB"
CWBDATE=`date +"%Y-%m-%d-%H-"`
CWBDATE1="${CWBDATE}00"
CWBDATE2="${CWBDATE}30"
USERAGENT='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"'

#下載全球衛星雲圖
$WGETLOC -q -O $WEBROOTDIR/world_sunlight_map_ori.jpg http://www.opentopia.com/images/data/sunlight/world_sunlight_map_rectangular.jpg
$WGETLOC -q -O $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg http://www.opentopia.com/images/data/sunlight/world_sunlight_map_hemisphere.jpg
/bin/rm -f $WEBROOTDIR/world_sunlight_map_rectangular.jpg
/bin/rm -f $WEBROOTDIR/world_sunlight_map_hemisphere.jpg
/bin/mv $WEBROOTDIR/world_sunlight_map_ori.jpg $WEBROOTDIR/world_sunlight_map_rectangular.jpg
/bin/mv $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg $WEBROOTDIR/world_sunlight_map_hemisphere.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_rectangular.jpg ${BASEDIR}/world_sunlight_map_rectangular_${DATESTR}.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_hemisphere.jpg ${BASEDIR}/world_sunlight_map_hemisphere_${DATESTR}.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_rectangular.jpg ${TEMPDIR}/original.jpg

#切割、合併圖片, 讓東亞在地圖中間
$CONVERTLOC ${TEMPDIR}/original.jpg -crop 600x887+0+0 ${TEMPDIR}/left.jpg
$CONVERTLOC ${TEMPDIR}/original.jpg -crop 1600x887+600+0 ${TEMPDIR}/right.jpg
$CONVERTLOC ${TEMPDIR}/right.jpg ${TEMPDIR}/left.jpg +append ${BASEDIR}/TW/world_sunlight_map.jpg

/bin/cp ${BASEDIR}/TW/world_sunlight_map.jpg ${BASEDIR}/TW/world_sunlight_map/world_sunlight_map_${DATESTR}.jpg
/bin/rm -f ${TEMPDIR}/original.jpg ${TEMPDIR}/left.jpg ${TEMPDIR}/right.jpg

sleep 5

###中央氣象局衛星雲圖
# MTSAT 紅外線雲圖      HS1P
# MTSAT 紅外線 色調強化 HS1Q
# MTSAT 紅色線 黑白     HS1O
# MTSAT 可見光          HSAO

for MapType in HS1P HS1Q HS1O HSAO
do
 if [ -f ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg ]; then
  echo "${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg was existed."
 else
  $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE1}.jpg
 fi

 if [ -f ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg ]; then
  echo "${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg was existed."
 else
  $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE2}.jpg
 fi

 sleep 5
done

# Remove empty file
find  ${CWBBASEDIR}/ -type f -size 0 | xargs rm -f

Monday, January 30, 2012

Shell script 搭配 ipfw table 阻擋入侵 sshd 攻擊 authlog_banip.sh 修正版

1. 目前已將之前只有阻擋 30 天改為先阻擋 30 天, 若是 30 ~ 90 天前仍有多筆紀錄就再加鎖一段時間直到符合 Good Boy 條件.

2. 程式碼如下:
For SSHD

#!/usr/local/bin/bash

LOGFILE="/PATH/TO/auth.log"
PROCFOLDER="/PATH/TO/auth_sshd_banip"
TOTALBANLOGFILE="/PATH/TO/auth_sshd_banip.log"
TOTALBANLOGFILESORTED="/PATH/TO/auth_sshd_banip_sorted.log"
TOTALBANLOGFILEFROMPROCFOLDERSORTED="/PATH/TO/auth_sshd_banip_from_procfolder_sorted.log"
TOTALRESCUELIST="/PATH/TO/auth_sshd_rescue_list.log"
CRONJOBDATE="crontjobdate.log"
TODAYLOGFM=`date +"%b %e"`
DefaultCheckPeriod="30"
MVFilePeriod="90"

TODAY=`date +"%Y%m%d"`
OLDDAY=`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`
OLDYEAR=`date -v-${DefaultCheckPeriod}d +"%Y"`

fwcmd="/sbin/ipfw"

echo "Prepare folder and file"
mkdir -p $PROCFOLDER
touch $TOTALBANLOGFILE
touch $TOTALRESCUELIST
touch $PROCFOLDER/$CRONJOBDATE
touch $TOTALBANLOGFILEFROMPROCFOLDERSORTED

SSHDErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Invalid user" | awk '{ print $10 }' | wc -l`
SSHDErrorCounter2=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Did not receive identification string from" | awk '{ print $12 }' | wc -l`
SSHDErrorCounter3=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "not allowed because none of user" | awk '{ print $9 }' | wc -l`

if [ $SSHDErrorCounter1 -ne 0 ]||[ $SSHDErrorCounter2 -ne 0 ]||[ $SSHDErrorCounter3 -ne 0 ]||[ -f $PROCFOLDER/sshdpre-$TODAY.log ]; then
        echo "Write pre-process file"
        touch $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Invalid user" | awk '{ print $10 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Did not receive identification string from" | awk '{ print $12 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "not allowed because none of user" | awk '{ print $9 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cp -a $PROCFOLDER/sshdpre-$TODAY.log /tmp
        cat /tmp/sshdpre-$TODAY.log | sort -n | sort -u > $PROCFOLDER/sshdpre-$TODAY.log
        rm -f /tmp/sshdpre-$TODAY.log

        echo "Collect new ban IP"
        for ip in $(awk '{ print }' $PROCFOLDER/sshdpre-$TODAY.log)
        do
                if [ `grep $ip $PROCFOLDER/sshd-*.log | wc -l` -gt 0 ]; then
                        echo $ip "Old bad boy"
                else
                        echo $ip
                        
                        touch $PROCFOLDER/sshd-$TODAY.log
                        echo $ip >> $PROCFOLDER/sshd-$TODAY.log
                        echo $ip >> $TOTALBANLOGFILE
                fi
        done

        echo "Ban bad boy"
        if [ -f $PROCFOLDER/sshd-$TODAY.log ]; then
                for banip in $(awk '{ print }' $PROCFOLDER/sshd-$TODAY.log)
                do
                        #Ban IP
                        echo "Check $banip"

                        if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                                echo "Had ban bad boy($banip)."
                        else
                                echo "Ban bad boy($banip) now."
                                ${fwcmd} table 3 add $banip/32
                                ${fwcmd} table 4 add $banip/32
                        fi
                done
        else
                echo "No bad boy"
        fi

        echo "Rescue good boy"
        if [ -f $PROCFOLDER/sshd-$OLDDAY.log ]; then
                for rescueip in $(awk '{ print }' $PROCFOLDER/sshd-$OLDDAY.log)
                do
                        #Rescue IP
                        echo "Check $rescueip"
                        
                        BadBoyDCPCounter=`find $PROCFOLDER -d 1 -ctime +${DefaultCheckPeriod} -type f -name 'sshd*.log' -exec grep $rescueip {} \; | wc -l`
                        BadBoyMVFCounter=`find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'sshd*.log' -exec grep $rescueip {} \; | wc -l`

                        if [ $BadBoyDCPCounter -lt 1 ]; then
                                if [ $BadBoyMVFCounter -le 2 ]; then
                                        echo "Rescue $rescueip"
                                        ${fwcmd} table 3 delete $rescueip/32
                                        ${fwcmd} table 4 delete $rescueip/32
                                
                                        grep -v $rescueip $TOTALBANLOGFILE > /tmp/stillbanip-$TODAY
                                        cp -f /tmp/stillbanip-$TODAY $TOTALBANLOGFILE
                                        rm -f /tmp/stillbanip-$TODAY
                                        echo $rescueip >> $TOTALRESCUELIST
                                        echo "Rescued"
                                else
                                        echo "Not bad boy in ${DefaultCheckPeriod} days, but had bad boy record in ${MVFilePeriod} days."
                                fi
                        else
                                echo "Still bad boy in ${DefaultCheckPeriod} days"
                        fi
                done
                
                #Move to old folder
                echo "Move old file to storage folder"
                mkdir -p $PROCFOLDER/$OLDYEAR
                mv $PROCFOLDER/sshd*-$OLDDAY.log $PROCFOLDER/$OLDYEAR
        else
                echo "No candidate file"
        fi

        cat $TOTALBANLOGFILE | sort -n | sort -u > $TOTALBANLOGFILESORTED
        
else
        echo "No bad boy"
fi

### Move old files to backup folder
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'sshd*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;
cat $PROCFOLDER/sshd-*.log | sort -n | sort -u > $TOTALBANLOGFILEFROMPROCFOLDERSORTED

### Additional rescure goodboy
for rescueip in `diff $TOTALBANLOGFILEFROMPROCFOLDERSORTED $TOTALBANLOGFILESORTED | grep -v a | awk '{ print $2 }' | sort -r -n`
do
        echo "Additional rescure goodboy from diff $TOTALBANLOGFILEFROMPROCFOLDERSORTED and $TOTALBANLOGFILESORTED"     
        echo "Rescue $rescueip"
        ${fwcmd} table 3 delete $rescueip/32
        ${fwcmd} table 4 delete $rescueip/32

        grep -v $rescueip $TOTALBANLOGFILE > /tmp/stillbanip-$TODAY
        cp -f /tmp/stillbanip-$TODAY $TOTALBANLOGFILE
        rm -f /tmp/stillbanip-$TODAY
        echo $rescueip >> $TOTALRESCUELIST
        echo "Rescued"
done

cat $TOTALBANLOGFILE | sort -n | sort -u > $TOTALBANLOGFILESORTED

echo $TODAY > $PROCFOLDER/$CRONJOBDATE

### For backup and administration
/PATH/TO/BACKUP_SCRIPT

For MAIL

#!/usr/local/bin/bash

PATH=$PATH

LOGFILE="/PATH/TO/maillog"
DOVECOTERRORLOGFILE="/PATH/TO/dovecot/error.log"
DOVECOTINFOLOGFILE="/PATH/TO/dovecot/info.log"
PROCFOLDER="/PATH/TO/mail_banip"
DAILYLOG="mail_banip"`date +"%Y%m%d"`".log"
TMPFILE="/tmp/mailban-"`date +"%s"`
TODAYLOGFM=`date +"%b %e"`
fwcmd="/sbin/ipfw"

DefaultCheckPeriod="30"
MVFilePeriod="30"

OLDDAILYLOG="mail_banip"`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`".log"
OLDYEAR=`date -v-${MVFilePeriod}d +"%Y"`

mkdir -p $PROCFOLDER

MAILErrorString1="NOQUEUE: reject"
MAILErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep postfix | grep "$MAILErrorString1" | wc -l`
MAILErrorString2="auth-worker(default): Error: pam"
MAILErrorCounter2=`cat $DOVECOTERRORLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString2" | awk 'BEGIN { FS = ": " } ; { print $3 }' | awk 'BEGIN { FS = "," } ; { print $2 }' | sort -n | sort -u | sed -Ee 's/)//' | wc -l`
MAILErrorString3="pop3-login: Info: Aborted login (auth failed"
MAILErrorCounter3=`cat $DOVECOTINFOLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString3" | awk 'BEGIN { FS = ", " } ; { print $4 }' | awk 'BEGIN { FS = "=" } ; { print $2 }' | sort -n | sort -u | wc -l`

if [ $MAILErrorCounter1 -ne 0 ]||[ $MAILErrorCounter2 -ne 0 ]||[ $MAILErrorCounter3 -ne 0 ]||[ -f $PROCFOLDER/$DAILYLOG ]; then
        if [ ! -f $PROCFOLDER/$DAILYLOG ]; then
                touch $PROCFOLDER/$DAILYLOG
        fi

        echo "Collect Bad boy"
        touch $TMPFILE
        cat $LOGFILE | grep "$TODAYLOGFM" | grep postfix | grep "NOQUEUE: reject" | awk 'BEGIN { FS = "[" } ; { print $3 }' | awk 'BEGIN { FS = "]" } ; { print $1 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $DOVECOTERRORLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString2" | awk 'BEGIN { FS = ": " } ; { print $3 }' | awk 'BEGIN { FS = "," } ; { print $2 }' | sort -n | sort -u | sed -Ee 's/)//' >> $PROCFOLDER/$DAILYLOG
        cat $DOVECOTINFOLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString3" | awk 'BEGIN { FS = ", " } ; { print $4 }' | awk 'BEGIN { FS = "=" } ; { print $2 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $PROCFOLDER/$DAILYLOG | sort -n | sort -u | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" > $TMPFILE
        cp -f $TMPFILE $PROCFOLDER/$DAILYLOG

        echo "Ban Bad Boy"
        for banip in $(awk '{ print }' $PROCFOLDER/$DAILYLOG)
        do
                if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                        echo "Had ban bad boy($banip)."
                else
                        echo "Ban bad boy($banip) now."
                        ${fwcmd} table 3 add $banip/32
                        ${fwcmd} table 4 add $banip/32
                fi
        done
        
        rm -f $TMPFILE
fi

if [ -f $PROCFOLDER/$OLDDAILYLOG ]; then
        for rescueip in $(awk '{ print }' $PROCFOLDER/$OLDDAILYLOG)
        do
                echo "Rescue $rescueip"
                ${fwcmd} table 3 delete $rescueip/32
                ${fwcmd} table 4 delete $rescueip/32
        done
else
        echo "No candidate file"
fi

#Move to old folder
echo "Move old file to storage folder"
mkdir -p $PROCFOLDER/$OLDYEAR
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'mail_banip*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;

For HTTPD

#!/usr/local/bin/bash

PATH=$PATH

LOGFILE="/PATH/TO/httpd-access.log"
PROCFOLDER="/PATH/TO/httpd_banip"
DAILYLOG="httpd_banip"`date +"%Y%m%d"`".log"
TMPFILE="/tmp/httpdban-"`date +"%s"`
TODAYLOGFM=`date +"%d/%b/%Y"`
fwcmd="/sbin/ipfw"

DefaultCheckPeriod="30"
MVFilePeriod="30"

OLDDAILYLOG="httpd_banip"`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`".log"
OLDYEAR=`date -v-${MVFilePeriod}d +"%Y"`

mkdir -p $PROCFOLDER

ErrorString1="xmlrpc.php"
ErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep "$ErrorString1" | grep 404 | awk 'BEGIN { FS = " " } ; { print $1 }' | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" | wc -l`

if [ $ErrorCounter1 -ne 0 ]||[ -f $PROCFOLDER/$DAILYLOG ]; then
        if [ ! -f $PROCFOLDER/$DAILYLOG ]; then
                touch $PROCFOLDER/$DAILYLOG
        fi

        echo "Collect Bad boy"
        touch $TMPFILE
        cat $LOGFILE | grep "$TODAYLOGFM" | grep "$ErrorString1" | grep 404 | awk 'BEGIN { FS = " " } ; { print $1 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $PROCFOLDER/$DAILYLOG | sort -n | sort -u | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" > $TMPFILE
        cp -f $TMPFILE $PROCFOLDER/$DAILYLOG
        
        echo "Ban Bad Boy"
        for banip in $(awk '{ print }' $PROCFOLDER/$DAILYLOG)
        do
                if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                        echo "Had ban bad boy($banip)."
                else
                        echo "Ban bad boy($banip) now."
                        ${fwcmd} table 3 add $banip/32
                        ${fwcmd} table 4 add $banip/32
                fi
        done
        
        rm -f $TMPFILE
fi

if [ -f $PROCFOLDER/$OLDDAILYLOG ]; then
        for rescueip in $(awk '{ print }' $PROCFOLDER/$OLDDAILYLOG)
        do
                echo "Rescue $rescueip"
                ${fwcmd} table 3 delete $rescueip/32
                ${fwcmd} table 4 delete $rescueip/32
        done
else
        echo "No candidate file"
fi

#Move to old folder
echo "Move old file to storage folder"
mkdir -p $PROCFOLDER/$OLDYEAR
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'httpd_banip*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;

3. 以上程式碼為自由軟體, 請自行取用. 強烈建議服用前先看清楚再服用, 避免發生意外.

Wednesday, October 19, 2011

向真正的 Hacker 致敬

準備考試真的是太忙碌, 一直沒有注意新聞消息. 相信學過 C 語言的同好們應該都知道 Dennis MacAlistair Ritchie . 在 C 及 UNIX 上的貢獻. 也影響到資訊科技的發展. 很可惜在 2011/10/12 世界失去了這位真正的 Hacker


向您至上最高的敬意!  R.I.P. 

Friday, September 30, 2011

如何掛載 FreeBSD 光碟影像檔

Script 如下 :

mail(xxx)# more mount_FreeBSDDVD.sh
#!/bin/sh

mkdir -p /FreeBSD-RELEASEDVD
mdconfig -a -t vnode -f /PATH/TO/FreeBSD-IMAGE.iso -u 1
mount -t cd9660 /dev/md1 /FreeBSD-RELEASEDVD
mail(xxx)# more umount_FreeBSDDVD.sh
#!/bin/sh

umount /FreeBSD-RELEASEDVD
mdconfig -d -u 1

關於今年初安裝的虛擬化伺服器硬體規格

當時先採購一台近來做 ESXi 及 OracleVM 安裝測試. 測試完成後, 再採購七台組虛擬化環境. 儲存設備則是採購一台 NexentaStor CE 及組一台 NexentaStor Community.


INTEL Corei7 950 3.0GHz/3.2GHz
Supermicro X8SAX
WD 1002FAEX 1TB SATA HDD(黑標)
創見 DDR3-1333 ECC 4GB
ASUS EAH5450 SILENT/DI/512MD2(LP) PCI-E
海韻 430W 電源供應器


硬體相容性參考文件 :
HCL for ESXi whitebox
Deploy Linux Faster: Oracle Validated Configurations
HCL for Oracle Solaris
HCL for OpenSolaris

Thursday, September 29, 2011

在 ESXi 安裝 Oracle VM 與 NexentaStor 測試環境

以下是今年安裝了一組 Intel Core i7 虛擬化伺服器, 先安裝 ESXi 4.1 後再安裝 OracleVM 及 NexentaStor 測試環境. 當然在 ESXi 上的 OracleVM 就不能再拿來安裝 guest VM. 測試完成後, 就將該批虛擬化伺服器部份安裝 OracleVM server, 而 NexentaStor 則是採購另一台實體機器(~16TB). OracleVM manager 則是繼續躲在 ESXi 上當 guest VM.

Architecture of OracleVM and NexentaStor
IP list

1_Install_Windows_VM
2_Install_OracleVM_manager
3_Install_Storage_NexentaStor
4_Install_OracleVM_server
5_Add_disk_space_in_OracleVM_and_ESXi
6_Snapshot_folder_in_NexentaStor
7_Auto_snapshot_in_NexentaStor
8_Function_list_of_NexentaStor
9_SingleNode_Server_Pool_of_OracleVM

Folder of collection of OracleVm and NexentaStor 

為了避免忘記, 將文件放上來紀錄一下.

註解 : 新版的 OracleVM server 在移除預設的 local storage 時會順便移除 /etc/ocfs2/cluster.conf. 所以要記得將檔案補回來.

Sunday, September 18, 2011

上傳 ISO 影像檔到 ESXi 伺服器

1. Login ESXi server as administrative account.
2. Select VM server name then click summary page.
3. Right click on datastore then choose ‘Browse Datastore’
4. Click ‘Upload files to this datastore’  
5. Waiting for file transmission.