Jengyic's 正義部落

Saturday, February 18, 2012

在 FreeBSD 使用 portmaster 更新已安裝的 ports - 程式碼修改

程式碼:

#!/usr/local/bin/bash

PKGLISTPATH="/PATH/TO/STORE/PKGLIST"

if [ -n "$1" ]; then
        keyword="$1"
        shift
else
        echo "Usage: $0 keyword"
        echo "Example: $0 lib"
        exit 1
fi

echo $keyword

if [ -f $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt ]
then
        echo "Had $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt"
else
        echo "Generating $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt"
        portversion -l '<' > $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt
fi

for i in $(cat $PKGLISTPATH/pkgversion_`date +"%Y%m%d"`.txt | grep $keyword | awk '{ print $1 }'); do portmaster -D $i; done

備份系統設定且紀錄差異

為了玩玩在 shell script 使用函式功能, 寫了一隻簡單的程式碼紀錄系統設定的差異. 不過增加項目還是需要修改程式碼. 暫時先能用再來看如何修改.
程式碼:

#!/usr/local/bin/bash

BasePath="/PATH/TO/BACKUP/DIRECTORY"
PATH=$PATH

BackupConfig(){
if [ -z $1 ]
# Checks if any params.
then
 echo "No parameters passed to function."
 return 0
else
 ProgName=$1
fi

if [ $2 ]
then
 BackupFolder=$2
fi

case $ProgName in
postfix) CMD="postconf -n"
  ;;
sysctl)  CMD="sysctl -a"
  ;;
pkglist) CMD="ls /var/db/pkg"
  ;;
*)  echo "Do not support this program $ProgName"
                ;;
esac

mkdir -p $BasePath/$BackupFolder
rm -f $BasePath/$BackupFolder/${ProgName}_old.txt

echo "Check if had OLD data : "
if [ -f $BasePath/$BackupFolder/${ProgName}_now.txt ]; then
 echo "   Had OLD data! Rename it to ${ProgName}_old.txt"
 mv $BasePath/$BackupFolder/${ProgName}_now.txt $BasePath/$BackupFolder/${ProgName}_old.txt
else
 echo "   Do not have OLD data."
fi

echo "Backup configuration of ${ProgName} now."
$CMD > $BasePath/$BackupFolder/${ProgName}_now.txt

echo "Generate diff log : "
if [ -f $BasePath/$BackupFolder/${ProgName}_old.txt ]; then
 echo "   Generate diff log between ${ProgName}_new.txt and ${ProgName}_old.txt."
 diff $BasePath/$BackupFolder/${ProgName}_now.txt $BasePath/$BackupFolder/${ProgName}_old.txt > $BasePath/$BackupFolder/diff-`date +"%Y%m%d"`.txt
 find  $BasePath/$BackupFolder/ -type f -size 0 | xargs rm -f
else
 echo "   Do not have OLD data."
fi

}


### Backup postfix setting
BackupConfig postfix postfix_config_backup

### Backup sysctl setting
BackupConfig sysctl sysctl_config_backup

### Backup /var/db/pkg
BackupConfig pkglist installed-pkglist

Monday, February 13, 2012

自動化抓衛星雲圖(全球及氣象局) 更新版

1.抓取全球衛星雲圖(Hemisphere)程式碼:


#!/usr/local/bin/bash

### http://www.opentopia.com/sunlightmaprect.html
### http://www.opentopia.com/sunlightmaphemi.html
### http://static.die.net/earth/mercator/1600.jpg
### http://static.die.net/earth/hemisphere/1600.jpg

DATESTR=`date "+%Y%m%d%H%M"`
LASTMONTH=`date -v -1m +"%Y%m"`
THISMONTH=`date "+%Y%m"`
LASTYEAR=`date -v -1y +"%Y"`
THISYEAR=`date "+%Y"`
WEBROOTDIR="/PATH/to/Web/Root/Directory"
BASEDIR="$WEBROOTDIR/world_sunlight_map"
TEMPDIR="/tmp"
WGETLOC="/usr/local/bin/wget"
USERAGENT='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"'
CONVERTLOC="/usr/local/bin/convert"

#下載全球衛星雲圖
###$WGETLOC -q -O $WEBROOTDIR/world_sunlight_map_ori.jpg http://www.opentopia.com/images/data/sunlight/world_sunlight_map_rectangular.jpg
###$WGETLOC -q -O $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg http://www.opentopia.com/images/data/sunlight/world_sunlight_map_hemisphere.jpg
$WGETLOC -q ${USERAGENT} -O $WEBROOTDIR/world_sunlight_map_ori.jpg http://static.die.net/earth/mercator/1600.jpg
$WGETLOC -q ${USERAGENT} -O $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg http://static.die.net/earth/hemisphere/1600.jpg
/bin/rm -f $WEBROOTDIR/world_sunlight_map_rectangular.jpg
/bin/rm -f $WEBROOTDIR/world_sunlight_map_hemisphere.jpg
/bin/mv $WEBROOTDIR/world_sunlight_map_ori.jpg $WEBROOTDIR/world_sunlight_map_rectangular.jpg
/bin/mv $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg $WEBROOTDIR/world_sunlight_map_hemisphere.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_rectangular.jpg ${BASEDIR}/world_sunlight_map_rectangular_${DATESTR}.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_hemisphere.jpg ${BASEDIR}/world_sunlight_map_hemisphere_${DATESTR}.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_rectangular.jpg ${TEMPDIR}/original.jpg

#切割、合併圖片, 讓東亞在地圖中間
$CONVERTLOC ${TEMPDIR}/original.jpg -crop 600x887+0+0 ${TEMPDIR}/left.jpg
$CONVERTLOC ${TEMPDIR}/original.jpg -crop 1600x887+600+0 ${TEMPDIR}/right.jpg
$CONVERTLOC ${TEMPDIR}/right.jpg ${TEMPDIR}/left.jpg +append ${BASEDIR}/TW/world_sunlight_map.jpg

/bin/cp ${BASEDIR}/TW/world_sunlight_map.jpg ${BASEDIR}/TW/world_sunlight_map/world_sunlight_map_${DATESTR}.jpg
/bin/rm -f ${TEMPDIR}/original.jpg ${TEMPDIR}/left.jpg ${TEMPDIR}/right.jpg

2. 抓取中央氣象局 V7 東亞雲圖程式碼:


#!/usr/local/bin/bash

PATH=$PATH

### http://www.cwb.gov.tw/V7/observe/satellite/Sat_H_EA.htm?type=1#

WEBROOTDIR="/PATH/to/Web/Root/Directory"
WGETLOC="/usr/local/bin/wget"

###中央氣象局衛星雲圖
CWBWEBBASEURL="http://www.cwb.gov.tw/V7/observe/satellite/Data"
CWBBASEDIR="$WEBROOTDIR/CWB"
CWBDATE=`date +"%Y-%m-%d-%H-"`
CWBDATE1="${CWBDATE}00"
CWBDATE2="${CWBDATE}30"
NOWMINSEC=`date +"%H%M"`
USERAGENT='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"'

###中央氣象局衛星雲圖
# MTSAT 紅外線雲圖      HS1P
# MTSAT 紅外線 色調強化 HS1Q
# MTSAT 紅色線 黑白     HS1O
# MTSAT 可見光          HSAO

for MapType in HS1P HS1Q HS1O HSAO
do
        if [ -f ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg ]; then
                echo "${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg was existed."
        else
                if [ ${MapType} == 'HSAO' ]; then
                        if [ $NOWMINSEC -gt 0600 ]&&[ $NOWMINSEC -le 1930 ]; then
                                $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE1}.jpg
                        fi
                else
                        $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE1}.jpg
                fi
        fi

        if [ -f ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg ]; then
                echo "${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg was existed."
        else
                if [ ${MapType} == 'HSAO' ]; then
                        if [ $NOWMINSEC -gt 0600 ]&&[ $NOWMINSEC -le 1930 ]; then
                                $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE1}.jpg
                        fi
                else
                        $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE2}.jpg
                fi
        fi

        sleep 2
done

# Remove empty file
find  ${CWBBASEDIR}/ -type f -size 0 | xargs rm -f

舊版程式碼:

#!/usr/local/bin/bash

### http://www.opentopia.com/sunlightmaprect.html
### http://www.opentopia.com/sunlightmaphemi.html
### http://www.cwb.gov.tw/V7/observe/satellite/Sat_H_EA.htm?type=1#

DATESTR=`date "+%Y%m%d%H%M"`
LASTMONTH=`date -v -1m +"%Y%m"`
THISMONTH=`date "+%Y%m"`
LASTYEAR=`date -v -1y +"%Y"`
THISYEAR=`date "+%Y"`
WEBROOTDIR="/PATH/to/Web/Root/Directory"
BASEDIR="$WEBROOTDIR/world_sunlight_map"
TEMPDIR="/tmp"
WGETLOC="/usr/local/bin/wget"
CONVERTLOC="/usr/local/bin/convert"

###中央氣象局衛星雲圖
CWBWEBBASEURL="http://www.cwb.gov.tw/V7/observe/satellite/Data"
CWBBASEDIR="$WEBROOTDIR/CWB"
CWBDATE=`date +"%Y-%m-%d-%H-"`
CWBDATE1="${CWBDATE}00"
CWBDATE2="${CWBDATE}30"
USERAGENT='--user-agent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"'

#下載全球衛星雲圖
$WGETLOC -q -O $WEBROOTDIR/world_sunlight_map_ori.jpg http://www.opentopia.com/images/data/sunlight/world_sunlight_map_rectangular.jpg
$WGETLOC -q -O $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg http://www.opentopia.com/images/data/sunlight/world_sunlight_map_hemisphere.jpg
/bin/rm -f $WEBROOTDIR/world_sunlight_map_rectangular.jpg
/bin/rm -f $WEBROOTDIR/world_sunlight_map_hemisphere.jpg
/bin/mv $WEBROOTDIR/world_sunlight_map_ori.jpg $WEBROOTDIR/world_sunlight_map_rectangular.jpg
/bin/mv $WEBROOTDIR/world_sunlight_map_hemisphere_ori.jpg $WEBROOTDIR/world_sunlight_map_hemisphere.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_rectangular.jpg ${BASEDIR}/world_sunlight_map_rectangular_${DATESTR}.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_hemisphere.jpg ${BASEDIR}/world_sunlight_map_hemisphere_${DATESTR}.jpg
/bin/cp $WEBROOTDIR/world_sunlight_map_rectangular.jpg ${TEMPDIR}/original.jpg

#切割、合併圖片, 讓東亞在地圖中間
$CONVERTLOC ${TEMPDIR}/original.jpg -crop 600x887+0+0 ${TEMPDIR}/left.jpg
$CONVERTLOC ${TEMPDIR}/original.jpg -crop 1600x887+600+0 ${TEMPDIR}/right.jpg
$CONVERTLOC ${TEMPDIR}/right.jpg ${TEMPDIR}/left.jpg +append ${BASEDIR}/TW/world_sunlight_map.jpg

/bin/cp ${BASEDIR}/TW/world_sunlight_map.jpg ${BASEDIR}/TW/world_sunlight_map/world_sunlight_map_${DATESTR}.jpg
/bin/rm -f ${TEMPDIR}/original.jpg ${TEMPDIR}/left.jpg ${TEMPDIR}/right.jpg

sleep 5

###中央氣象局衛星雲圖
# MTSAT 紅外線雲圖      HS1P
# MTSAT 紅外線 色調強化 HS1Q
# MTSAT 紅色線 黑白     HS1O
# MTSAT 可見光          HSAO

for MapType in HS1P HS1Q HS1O HSAO
do
 if [ -f ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg ]; then
  echo "${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg was existed."
 else
  $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE1}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE1}.jpg
 fi

 if [ -f ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg ]; then
  echo "${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg was existed."
 else
  $WGETLOC -q ${USERAGENT} -O ${CWBBASEDIR}/${MapType}/${MapType}-${CWBDATE2}.jpg $CWBWEBBASEURL/${MapType}/${MapType}-${CWBDATE2}.jpg
 fi

 sleep 5
done

# Remove empty file
find  ${CWBBASEDIR}/ -type f -size 0 | xargs rm -f

Monday, January 30, 2012

Shell script 搭配 ipfw table 阻擋入侵 sshd 攻擊 authlog_banip.sh 修正版

1. 目前已將之前只有阻擋 30 天改為先阻擋 30 天, 若是 30 ~ 90 天前仍有多筆紀錄就再加鎖一段時間直到符合 Good Boy 條件.

2. 程式碼如下:
For SSHD


#!/usr/local/bin/bash

LOGFILE="/PATH/TO/auth.log"
PROCFOLDER="/PATH/TO/auth_sshd_banip"
TOTALBANLOGFILE="/PATH/TO/auth_sshd_banip.log"
TOTALBANLOGFILESORTED="/PATH/TO/auth_sshd_banip_sorted.log"
TOTALBANLOGFILEFROMPROCFOLDERSORTED="/PATH/TO/auth_sshd_banip_from_procfolder_sorted.log"
TOTALRESCUELIST="/PATH/TO/auth_sshd_rescue_list.log"
CRONJOBDATE="crontjobdate.log"
TODAYLOGFM=`date +"%b %e"`
DefaultCheckPeriod="30"
MVFilePeriod="90"

TODAY=`date +"%Y%m%d"`
OLDDAY=`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`
OLDYEAR=`date -v-${DefaultCheckPeriod}d +"%Y"`

fwcmd="/sbin/ipfw"

echo "Prepare folder and file"
mkdir -p $PROCFOLDER
touch $TOTALBANLOGFILE
touch $TOTALRESCUELIST
touch $PROCFOLDER/$CRONJOBDATE
touch $TOTALBANLOGFILEFROMPROCFOLDERSORTED

SSHDErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Invalid user" | awk '{ print $10 }' | wc -l`
SSHDErrorCounter2=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Did not receive identification string from" | awk '{ print $12 }' | wc -l`
SSHDErrorCounter3=`cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "not allowed because none of user" | awk '{ print $9 }' | wc -l`

if [ $SSHDErrorCounter1 -ne 0 ]||[ $SSHDErrorCounter2 -ne 0 ]||[ $SSHDErrorCounter3 -ne 0 ]||[ -f $PROCFOLDER/sshdpre-$TODAY.log ]; then
        echo "Write pre-process file"
        touch $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Invalid user" | awk '{ print $10 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "Did not receive identification string from" | awk '{ print $12 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cat $LOGFILE | grep "$TODAYLOGFM" | grep sshd | grep "not allowed because none of user" | awk '{ print $9 }' | sort -n | sort -u >> $PROCFOLDER/sshdpre-$TODAY.log
        cp -a $PROCFOLDER/sshdpre-$TODAY.log /tmp
        cat /tmp/sshdpre-$TODAY.log | sort -n | sort -u > $PROCFOLDER/sshdpre-$TODAY.log
        rm -f /tmp/sshdpre-$TODAY.log

        echo "Collect new ban IP"
        for ip in $(awk '{ print }' $PROCFOLDER/sshdpre-$TODAY.log)
        do
                if [ `grep $ip $PROCFOLDER/sshd-*.log | wc -l` -gt 0 ]; then
                        echo $ip "Old bad boy"
                else
                        echo $ip
                        
                        touch $PROCFOLDER/sshd-$TODAY.log
                        echo $ip >> $PROCFOLDER/sshd-$TODAY.log
                        echo $ip >> $TOTALBANLOGFILE
                fi
        done

        echo "Ban bad boy"
        if [ -f $PROCFOLDER/sshd-$TODAY.log ]; then
                for banip in $(awk '{ print }' $PROCFOLDER/sshd-$TODAY.log)
                do
                        #Ban IP
                        echo "Check $banip"

                        if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                                echo "Had ban bad boy($banip)."
                        else
                                echo "Ban bad boy($banip) now."
                                ${fwcmd} table 3 add $banip/32
                                ${fwcmd} table 4 add $banip/32
                        fi
                done
        else
                echo "No bad boy"
        fi

        echo "Rescue good boy"
        if [ -f $PROCFOLDER/sshd-$OLDDAY.log ]; then
                for rescueip in $(awk '{ print }' $PROCFOLDER/sshd-$OLDDAY.log)
                do
                        #Rescue IP
                        echo "Check $rescueip"
                        
                        BadBoyDCPCounter=`find $PROCFOLDER -d 1 -ctime +${DefaultCheckPeriod} -type f -name 'sshd*.log' -exec grep $rescueip {} \; | wc -l`
                        BadBoyMVFCounter=`find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'sshd*.log' -exec grep $rescueip {} \; | wc -l`

                        if [ $BadBoyDCPCounter -lt 1 ]; then
                                if [ $BadBoyMVFCounter -le 2 ]; then
                                        echo "Rescue $rescueip"
                                        ${fwcmd} table 3 delete $rescueip/32
                                        ${fwcmd} table 4 delete $rescueip/32
                                
                                        grep -v $rescueip $TOTALBANLOGFILE > /tmp/stillbanip-$TODAY
                                        cp -f /tmp/stillbanip-$TODAY $TOTALBANLOGFILE
                                        rm -f /tmp/stillbanip-$TODAY
                                        echo $rescueip >> $TOTALRESCUELIST
                                        echo "Rescued"
                                else
                                        echo "Not bad boy in ${DefaultCheckPeriod} days, but had bad boy record in ${MVFilePeriod} days."
                                fi
                        else
                                echo "Still bad boy in ${DefaultCheckPeriod} days"
                        fi
                done
                
                #Move to old folder
                echo "Move old file to storage folder"
                mkdir -p $PROCFOLDER/$OLDYEAR
                mv $PROCFOLDER/sshd*-$OLDDAY.log $PROCFOLDER/$OLDYEAR
        else
                echo "No candidate file"
        fi

        cat $TOTALBANLOGFILE | sort -n | sort -u > $TOTALBANLOGFILESORTED
        
else
        echo "No bad boy"
fi

### Move old files to backup folder
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'sshd*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;
cat $PROCFOLDER/sshd-*.log | sort -n | sort -u > $TOTALBANLOGFILEFROMPROCFOLDERSORTED

### Additional rescure goodboy
for rescueip in `diff $TOTALBANLOGFILEFROMPROCFOLDERSORTED $TOTALBANLOGFILESORTED | grep -v a | awk '{ print $2 }' | sort -r -n`
do
        echo "Additional rescure goodboy from diff $TOTALBANLOGFILEFROMPROCFOLDERSORTED and $TOTALBANLOGFILESORTED"     
        echo "Rescue $rescueip"
        ${fwcmd} table 3 delete $rescueip/32
        ${fwcmd} table 4 delete $rescueip/32

        grep -v $rescueip $TOTALBANLOGFILE > /tmp/stillbanip-$TODAY
        cp -f /tmp/stillbanip-$TODAY $TOTALBANLOGFILE
        rm -f /tmp/stillbanip-$TODAY
        echo $rescueip >> $TOTALRESCUELIST
        echo "Rescued"
done

cat $TOTALBANLOGFILE | sort -n | sort -u > $TOTALBANLOGFILESORTED

echo $TODAY > $PROCFOLDER/$CRONJOBDATE

### For backup and administration
/PATH/TO/BACKUP_SCRIPT

For MAIL


#!/usr/local/bin/bash

PATH=$PATH

LOGFILE="/PATH/TO/maillog"
DOVECOTERRORLOGFILE="/PATH/TO/dovecot/error.log"
DOVECOTINFOLOGFILE="/PATH/TO/dovecot/info.log"
PROCFOLDER="/PATH/TO/mail_banip"
DAILYLOG="mail_banip"`date +"%Y%m%d"`".log"
TMPFILE="/tmp/mailban-"`date +"%s"`
TODAYLOGFM=`date +"%b %e"`
fwcmd="/sbin/ipfw"

DefaultCheckPeriod="30"
MVFilePeriod="30"

OLDDAILYLOG="mail_banip"`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`".log"
OLDYEAR=`date -v-${MVFilePeriod}d +"%Y"`

mkdir -p $PROCFOLDER

MAILErrorString1="NOQUEUE: reject"
MAILErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep postfix | grep "$MAILErrorString1" | wc -l`
MAILErrorString2="auth-worker(default): Error: pam"
MAILErrorCounter2=`cat $DOVECOTERRORLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString2" | awk 'BEGIN { FS = ": " } ; { print $3 }' | awk 'BEGIN { FS = "," } ; { print $2 }' | sort -n | sort -u | sed -Ee 's/)//' | wc -l`
MAILErrorString3="pop3-login: Info: Aborted login (auth failed"
MAILErrorCounter3=`cat $DOVECOTINFOLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString3" | awk 'BEGIN { FS = ", " } ; { print $4 }' | awk 'BEGIN { FS = "=" } ; { print $2 }' | sort -n | sort -u | wc -l`

if [ $MAILErrorCounter1 -ne 0 ]||[ $MAILErrorCounter2 -ne 0 ]||[ $MAILErrorCounter3 -ne 0 ]||[ -f $PROCFOLDER/$DAILYLOG ]; then
        if [ ! -f $PROCFOLDER/$DAILYLOG ]; then
                touch $PROCFOLDER/$DAILYLOG
        fi

        echo "Collect Bad boy"
        touch $TMPFILE
        cat $LOGFILE | grep "$TODAYLOGFM" | grep postfix | grep "NOQUEUE: reject" | awk 'BEGIN { FS = "[" } ; { print $3 }' | awk 'BEGIN { FS = "]" } ; { print $1 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $DOVECOTERRORLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString2" | awk 'BEGIN { FS = ": " } ; { print $3 }' | awk 'BEGIN { FS = "," } ; { print $2 }' | sort -n | sort -u | sed -Ee 's/)//' >> $PROCFOLDER/$DAILYLOG
        cat $DOVECOTINFOLOGFILE | grep "$TODAYLOGFM" | grep "$MAILErrorString3" | awk 'BEGIN { FS = ", " } ; { print $4 }' | awk 'BEGIN { FS = "=" } ; { print $2 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $PROCFOLDER/$DAILYLOG | sort -n | sort -u | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" > $TMPFILE
        cp -f $TMPFILE $PROCFOLDER/$DAILYLOG

        echo "Ban Bad Boy"
        for banip in $(awk '{ print }' $PROCFOLDER/$DAILYLOG)
        do
                if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                        echo "Had ban bad boy($banip)."
                else
                        echo "Ban bad boy($banip) now."
                        ${fwcmd} table 3 add $banip/32
                        ${fwcmd} table 4 add $banip/32
                fi
        done
        
        rm -f $TMPFILE
fi

if [ -f $PROCFOLDER/$OLDDAILYLOG ]; then
        for rescueip in $(awk '{ print }' $PROCFOLDER/$OLDDAILYLOG)
        do
                echo "Rescue $rescueip"
                ${fwcmd} table 3 delete $rescueip/32
                ${fwcmd} table 4 delete $rescueip/32
        done
else
        echo "No candidate file"
fi

#Move to old folder
echo "Move old file to storage folder"
mkdir -p $PROCFOLDER/$OLDYEAR
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'mail_banip*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;

For HTTPD


#!/usr/local/bin/bash

PATH=$PATH

LOGFILE="/PATH/TO/httpd-access.log"
PROCFOLDER="/PATH/TO/httpd_banip"
DAILYLOG="httpd_banip"`date +"%Y%m%d"`".log"
TMPFILE="/tmp/httpdban-"`date +"%s"`
TODAYLOGFM=`date +"%d/%b/%Y"`
fwcmd="/sbin/ipfw"

DefaultCheckPeriod="30"
MVFilePeriod="30"

OLDDAILYLOG="httpd_banip"`date -v-${DefaultCheckPeriod}d +"%Y%m%d"`".log"
OLDYEAR=`date -v-${MVFilePeriod}d +"%Y"`

mkdir -p $PROCFOLDER

ErrorString1="xmlrpc.php"
ErrorCounter1=`cat $LOGFILE | grep "$TODAYLOGFM" | grep "$ErrorString1" | grep 404 | awk 'BEGIN { FS = " " } ; { print $1 }' | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" | wc -l`

if [ $ErrorCounter1 -ne 0 ]||[ -f $PROCFOLDER/$DAILYLOG ]; then
        if [ ! -f $PROCFOLDER/$DAILYLOG ]; then
                touch $PROCFOLDER/$DAILYLOG
        fi

        echo "Collect Bad boy"
        touch $TMPFILE
        cat $LOGFILE | grep "$TODAYLOGFM" | grep "$ErrorString1" | grep 404 | awk 'BEGIN { FS = " " } ; { print $1 }' | sort -n | sort -u >> $PROCFOLDER/$DAILYLOG
        cat $PROCFOLDER/$DAILYLOG | sort -n | sort -u | grep -v "192.168.58" | grep -v "192.168.68" | grep -v "192.168.8" | grep -v "60.248.16.64" | grep -v "60.248.16.65" | grep -v "60.248.16.66" > $TMPFILE
        cp -f $TMPFILE $PROCFOLDER/$DAILYLOG
        
        echo "Ban Bad Boy"
        for banip in $(awk '{ print }' $PROCFOLDER/$DAILYLOG)
        do
                if [ `ipfw table 3 list | grep $banip | wc -l` -gt 0 ]; then
                        echo "Had ban bad boy($banip)."
                else
                        echo "Ban bad boy($banip) now."
                        ${fwcmd} table 3 add $banip/32
                        ${fwcmd} table 4 add $banip/32
                fi
        done
        
        rm -f $TMPFILE
fi

if [ -f $PROCFOLDER/$OLDDAILYLOG ]; then
        for rescueip in $(awk '{ print }' $PROCFOLDER/$OLDDAILYLOG)
        do
                echo "Rescue $rescueip"
                ${fwcmd} table 3 delete $rescueip/32
                ${fwcmd} table 4 delete $rescueip/32
        done
else
        echo "No candidate file"
fi

#Move to old folder
echo "Move old file to storage folder"
mkdir -p $PROCFOLDER/$OLDYEAR
find $PROCFOLDER -d 1 -ctime +${MVFilePeriod} -type f -name 'httpd_banip*.log' -exec mv {} $PROCFOLDER/$OLDYEAR \;

3. 以上程式碼為自由軟體, 請自行取用. 強烈建議服用前先看清楚再服用, 避免發生意外.

Wednesday, October 19, 2011

向真正的 Hacker 致敬

準備考試真的是太忙碌, 一直沒有注意新聞消息. 相信學過 C 語言的同好們應該都知道 Dennis MacAlistair Ritchie . 在 C 及 UNIX 上的貢獻. 也影響到資訊科技的發展. 很可惜在 2011/10/12 世界失去了這位真正的 Hacker.

向您至上最高的敬意! R.I.P.

Friday, September 30, 2011

如何掛載 FreeBSD 光碟影像檔

Script 如下 :

mail(xxx)# more mount_FreeBSDDVD.sh
#!/bin/sh

mkdir -p /FreeBSD-RELEASEDVD
mdconfig -a -t vnode -f /PATH/TO/FreeBSD-IMAGE.iso -u 1
mount -t cd9660 /dev/md1 /FreeBSD-RELEASEDVD

mail(xxx)# more umount_FreeBSDDVD.sh
#!/bin/sh

umount /FreeBSD-RELEASEDVD
mdconfig -d -u 1

關於今年初安裝的虛擬化伺服器硬體規格

當時先採購一台近來做 ESXi 及 OracleVM 安裝測試. 測試完成後, 再採購七台組虛擬化環境. 儲存設備則是採購一台 NexentaStor CE 及組一台 NexentaStor Community.

INTEL Corei7 950 3.0GHz/3.2GHz

Supermicro X8SAX

WD 1002FAEX 1TB SATA HDD(黑標)

創見 DDR3-1333 ECC 4GB

ASUS EAH5450 SILENT/DI/512MD2(LP) PCI-E

海韻 430W 電源供應器

硬體相容性參考文件 :
HCL for ESXi whitebox
Deploy Linux Faster: Oracle Validated Configurations
HCL for Oracle Solaris
HCL for OpenSolaris

Thursday, September 29, 2011

在 ESXi 安裝 Oracle VM 與 NexentaStor 測試環境

以下是今年安裝了一組 Intel Core i7 虛擬化伺服器, 先安裝 ESXi 4.1 後再安裝 OracleVM 及 NexentaStor 測試環境. 當然在 ESXi 上的 OracleVM 就不能再拿來安裝 guest VM. 測試完成後, 就將該批虛擬化伺服器部份安裝 OracleVM server, 而 NexentaStor 則是採購另一台實體機器(~16TB). OracleVM manager 則是繼續躲在 ESXi 上當 guest VM.

Architecture of OracleVM and NexentaStor
IP list

1_Install_Windows_VM
2_Install_OracleVM_manager
3_Install_Storage_NexentaStor
4_Install_OracleVM_server
5_Add_disk_space_in_OracleVM_and_ESXi
6_Snapshot_folder_in_NexentaStor
7_Auto_snapshot_in_NexentaStor
8_Function_list_of_NexentaStor
9_SingleNode_Server_Pool_of_OracleVM

Folder of collection of OracleVm and NexentaStor

為了避免忘記, 將文件放上來紀錄一下.

註解 : 新版的 OracleVM server 在移除預設的 local storage 時會順便移除 /etc/ocfs2/cluster.conf. 所以要記得將檔案補回來.

Sunday, September 18, 2011

上傳 ISO 影像檔到 ESXi 伺服器

1. Login ESXi server as administrative account.

2. Select VM server name then click summary page.

3. Right click on datastore then choose ‘Browse Datastore’

4. Click ‘Upload files to this datastore’

5. Waiting for file transmission.