Tuesday, September 25, 2012

Perl範例 : 支援多執行緒

這是很久以前在 NCU ISL 當助理時寫的程式碼, 最近流行考古就找到這隻程式. 話說這年頭大家都玩 CUDA, OpenCL, 應該不會有人對這感興趣. 不過還是將程式碼放上來作為紀錄參考用.

程式碼(runcosmicgmrion.pl)
#!/usr/bin/perl

use threads;
use threads::shared;
use Thread;
use Switch;
use Time::localtime;

# Define floder name of data and get rootpath
my $datafloder : shared;
$datafloder = "lv2data_1";      # Data floder to run
my $rootpath : shared;
$rootpath = shift;

# Define job and nodename as shared variable
print "Begin time : ",ctime(),"\nInit job and cpu : ";
my @job : shared;
my $lastjob : shared;
my @cpu : shared;

# Get jobflodername
@job = `ls $rootpath/$datafloder`;
chop(@job);
$lastjob = $#job;

# Define nodename and cpuid(begin from 0)
@cpu = qw (Logon ve1 ve2 ve3 ve4 ve5);
#@cpu = qw (ve1 ve3 ve5);
#@cpu = qw (ve1);

my $i;

$threadnum = 0;
print "done ! \n";

while($lastjob >=0){
    my $thread0, $thread1, $thread2, $thread3, $thread4, $thread5;
    for($i = 0; $i <= $#cpu;$i++){
        if($lastjob >= 0){
            my $jobpath = $rootpath . "/" . $datafloder . "/" . $job[$lastjob];
            switch ($i) {
                case "0" { $thread0 = threads->new(\&exgmrion, $jobpath, 0); }
                case "1" { $thread1 = threads->new(\&exgmrion, $jobpath, 1); }
                case "2" { $thread2 = threads->new(\&exgmrion, $jobpath, 2); }
                case "3" { $thread3 = threads->new(\&exgmrion, $jobpath, 3); }
                case "4" { $thread4 = threads->new(\&exgmrion, $jobpath, 4); }
                case "5" { $thread5 = threads->new(\&exgmrion, $jobpath, 5); }
                else     { print "Error : can't assign job"; }
            }
            $lastjob--;
        }else{
            last;
        }
    }

    my $thnum = $i - 1;
    switch ($thnum) {
                case "0" { $_->join foreach ($thread0); }
                case "1" { $_->join foreach ($thread0, $thread1); }
                case "2" { $_->join foreach ($thread0, $thread1, $thread2); }
                case "3" { $_->join foreach ($thread0, $thread1, $thread2, $thread3); }
                case "4" { $_->join foreach ($thread0, $thread1, $thread2, $thread3, $thread4); }
                case "5" { $_->join foreach ($thread0, $thread1, $thread2, $thread3, $thread4, $thread5); }
                else     { print "Error : can't run !"; }
    }

}

print "Finished time : ",ctime(),"\n";

sub exgmrion: locked method {
    my ($expath, $cpuid) = @_;
    my $excpu = $cpu[$cpuid];
    print "Excution cosmic_gmrion in $expath on $excpu \n";
    system("rsh $excpu threadperl/t02_do/exgmrion.pl $expath $excpu");
}

Java範例 : 物件導向觀念 : 繼承, 多載, 覆載

範例程式碼中包含以下觀念 :
繼承 : RacingCar 繼承自 Car.
多載 : setCar()
覆載 : showCar()
類別變數 : carsum
類別函式 : showCarSum()

程式碼
class Car
{
 protected int id;
 protected double gasoline;
 private double MAXGAS = 2500;
 private static int carsum = 0;

 public Car()
 {
  id = 0;
  gasoline = 0.0;
  carsum++;
  this.manufactCar();
 }

 public Car(int id)
 {
  this();
  this.setCar(id);
 }

 public Car(int id, double gasoline)
 {
  this();
  this.setCar(id, gasoline);
 }

 public Car(double gasoline, int id)
 {
  this();
  this.setCar(gasoline, id);
 }

 private void prepareParts()
 {
  System.out.println("備料");
 }

 private void manufactCar()
 {
  this.prepareParts();
  System.out.println("製造一輛車");
 }

 public int getID()
 {
  return this.id;
 }

 public double getGas()
 {
  return this.gasoline;
 }

 public boolean setCar(int id)
 {
  if( id <= 0 ){
   System.out.println("!!! 車號 (" + id + ") 不可小於或等於 0. !!!");
   return false;
  } else {
   this.id = id;
   System.out.println("設定車號 " + this.id);
   return true;
  }
 }

 public boolean setCar(double gasoline)
 {
  if( gasoline < 0 || gasoline > this.MAXGAS ){
   System.out.println("!!! 油量 (" + gasoline + ") 不可小於 0 或大於 " + this.MAXGAS + ". !!!");
   return false;
  } else {
   this.gasoline = gasoline;
   System.out.println("車號 " + this.id + " 設定油量 " + this.gasoline);
   return true;
  }
 }

 public boolean setCar(int id, double gasoline)
 {
  return this.setCar(id) && this.setCar(gasoline);
 }

 public boolean setCar(double gasoline, int id)
 {
  return this.setCar(id) && this.setCar(gasoline);
 }

 public void showCar()
 {
  if( this.id == 0 ){
   System.out.print("車號 尚未設定 ");
  } else {
   System.out.print("車號 " + this.id + " ");
  }

  if( this.gasoline <= 0 ){
   System.out.print("尚未設定油量\n");
  } else {
   System.out.print("油量 " + this.gasoline + "\n");
  }
 }

 public static void showCarSum()
 {
  System.out.println("###### 總共生產 " + carsum + " 輛車. ######");
 }
}

class RacingCar extends Car
{
 private int turbo = 1;
 private int speed;
 private static int carsum = 0;

 public RacingCar()
 {
  speed = 300;
  carsum++;
  System.out.println("生產一輛賽車");
 }

 public RacingCar(int id)
 {
  super(id);
  speed = 300;
  carsum++;
  System.out.println("生產一輛賽車(編號 " + this.id + " 預設速度 " + speed + ")");
 }

 public RacingCar(int id, double gasoline)
 {
  super(id, gasoline);
  speed = 300;
  carsum++;
  System.out.println("生產一輛賽車(編號 " + this.id + " 油量 " + this.gasoline + " 預設速度 " + speed + ")");
 }

 public RacingCar(double gasoline, int id)
 {
  super(gasoline, id);
  speed = 300;
  carsum++;
  System.out.println("生產一輛賽車(編號 " + this.id + " 油量 " + this.gasoline + " 預設速度 " + speed + ")");
 }

 public RacingCar(int id, double gasoline, int speed)
 {
  super(id, gasoline);
  this.setSpeed(speed);
  carsum++;
  System.out.println("生產一輛賽車(編號 " + this.id + " 油量 " + this.gasoline + " 速度 " + this.speed + ")");
 }

 public RacingCar(double gasoline, int id, int speed)
 {
  super(gasoline, id);
  this.setSpeed(speed);
  carsum++;
  System.out.println("生產一輛賽車(編號 " + this.id + " 油量 " + this.gasoline + " 速度 " + this.speed + ")");
 }

 public void setSpeed(int speed)
 {
  int oldspeed = this.speed;
  if( oldspeed == speed ){
   System.out.println("賽車速度維持 " + this.speed);
  } else {
   this.speed = speed;
   System.out.println("將賽車速度由 " + oldspeed + " 調整為 " + this.speed);
  }
 }

 public void setTurbo(int turbo)
 {
  int oldturbo = this.turbo;
  if( oldturbo == turbo ){
   System.out.println("渦輪加速級數維持 " + this.turbo);
  } else {
   this.turbo = turbo;
   System.out.println("將渦輪加速級數由 " + oldturbo + " 調整為 " + this.turbo);
  }
 }

 public void showCar()
 {
  System.out.print("賽車");
  super.showCar();
  System.out.print(" 渦輪加速級數 " + this.turbo + " ");

  if( this.speed <= 0 ){
   System.out.print("尚未設定速度\n");
  } else {
   System.out.print("速度 " + this.speed + "\n");
  }
 }

 public static void showCarSum()
 {
  System.out.println("###### 總共生產 " + carsum + " 輛賽車. ######");
 }
}

class C11P271
{
 public static void main(String[] args)
 {
  String querystr = "[[[查詢車輛資訊]]]";
  String setupstr = "[[[設定車輛資訊]]]";

  System.out.println("\n [[[ GM 汽車生產陣列 ]]]");
  Car[] GM = new Car[12];
  int BASEID = 6600;
  for(int i = 0; i < GM.length; i++){
   if( i < 6 ){
    GM[i] = new Car(i+BASEID, 1688.88888);
   } else if( i >= GM.length - 2 ){
    //GM[i] = new Car(i+6600, 0.0); //OK
    GM[i] = new Car(i+BASEID);
    //GM[i].setCar(i+6600);
   } else {
    GM[i] = new Car(i+BASEID, 1666.666);
   }
  }
  Car.showCarSum();

  System.out.println("\n [[[ GM ]]] " + setupstr);
  GM[2].setCar(1888.88);
  GM[0].setCar(1888.88);
  GM[8].setCar(1888.88);
  System.out.println("\n [[[ GM ]]] " + querystr);
  for(int i = 0; i < GM.length; i++){
   GM[i].showCar();
  }

  Car.showCarSum();

  System.out.println("\n\n打造第一台賽車");
  RacingCar rc1 = new RacingCar();
  rc1.setSpeed(600);
  rc1.setCar(666, 2500.0);
  rc1.showCar();
  Car.showCarSum();
  RacingCar.showCarSum();

  System.out.println("\n [[[ LOTUS 賽車生產陣列 ]]]");
  RacingCar[] LOTUS = new RacingCar[6];
  BASEID = 8800;
  for(int i = 0; i < LOTUS.length; i++){
   if( i/3 == 0 ){
    ////LOTUS[i] = new RacingCar();
    ////LOTUS[i].setCar(i+BASEID, 2000.0);
    //LOTUS[i] = new RacingCar(i+BASEID, 2000.0);
    //LOTUS[i].setSpeed(500);
    LOTUS[i] = new RacingCar(i+BASEID, 2000.0, 500);
   } else {
    ////LOTUS[i] = new RacingCar();
    ////LOTUS[i].setCar(i+BASEID, 2500.0);
    //LOTUS[i] = new RacingCar(2500.0, i+BASEID);
    //LOTUS[i].setSpeed(660);
    LOTUS[i] = new RacingCar(2500.0, i+BASEID, 600);
   }
   System.out.print("調整渦輪加速級數 : ");
   LOTUS[i].setTurbo(i%3*2+1+i/3);
  }

  Car.showCarSum();
  RacingCar.showCarSum();

  System.out.println("\n [[[ LOTUS ]]] " + querystr);
  for(int i = 0; i < LOTUS.length; i++){
   LOTUS[i].showCar();
  }

  Car.showCarSum();
  RacingCar.showCarSum();

  System.out.println("\n [[[ Car type RacingCar ]]] ");
  Car dcar;
  dcar = new RacingCar();
  dcar.setCar(777, 1800.0);
  //dcar.setSpeed(360);
  System.out.println("\n [[[ Car type RacingCar ]]] " + querystr);
  dcar.showCar();

  Car.showCarSum();
  RacingCar.showCarSum();

  System.out.println("\n [[[ Car type RacingCar ]]] ");
  Car dcar1;
  dcar1 = new RacingCar(778, 1900.0, 660);
  System.out.println("\n [[[ Car type RacingCar ]]] " + querystr);
  dcar1.showCar();

  Car.showCarSum();
  RacingCar.showCarSum();

  System.out.println("\n [[[ MIT 房車/賽車生產陣列 ]]]");
  Car[] MIT = new Car[6];
  BASEID = 3600;
  for(int i = 0; i < LOTUS.length; i++){
   if( i/3 == 0 ){
    MIT[i] = new Car(i+BASEID, 1200.0);
   } else {
    MIT[i] = new RacingCar(i+BASEID, 2200.0, 500);
    // Compile error : P.267 Car type var. can not access sub-class function
    //System.out.print("調整渦輪加速級數 : ");
    //MIT[i].setTurbo(i%3*2+1+i/3);
   }
  }

  Car.showCarSum();
  RacingCar.showCarSum();

  System.out.println("\n [[[ MIT ]]] " + querystr);
  for(int i = 0; i < MIT.length; i++){
   MIT[i].showCar();
  }

  Car.showCarSum();
  RacingCar.showCarSum();
 }
}


執行結果
 [[[ GM 汽車生產陣列 ]]]
備料
製造一輛車
設定車號 6600
車號 6600 設定油量 1688.88888
備料
製造一輛車
設定車號 6601
車號 6601 設定油量 1688.88888
備料
製造一輛車
設定車號 6602
車號 6602 設定油量 1688.88888
備料
製造一輛車
設定車號 6603
車號 6603 設定油量 1688.88888
備料
製造一輛車
設定車號 6604
車號 6604 設定油量 1688.88888
備料
製造一輛車
設定車號 6605
車號 6605 設定油量 1688.88888
備料
製造一輛車
設定車號 6606
車號 6606 設定油量 1666.666
備料
製造一輛車
設定車號 6607
車號 6607 設定油量 1666.666
備料
製造一輛車
設定車號 6608
車號 6608 設定油量 1666.666
備料
製造一輛車
設定車號 6609
車號 6609 設定油量 1666.666
備料
製造一輛車
設定車號 6610
備料
製造一輛車
設定車號 6611
###### 總共生產 12 輛車. ######
 [[[ GM ]]] [[[設定車輛資訊]]]
車號 6602 設定油量 1888.88
車號 6600 設定油量 1888.88
車號 6608 設定油量 1888.88

 [[[ GM ]]] [[[查詢車輛資訊]]]
車號 6600 油量 1888.88
車號 6601 油量 1688.88888
車號 6602 油量 1888.88
車號 6603 油量 1688.88888
車號 6604 油量 1688.88888
車號 6605 油量 1688.88888
車號 6606 油量 1666.666
車號 6607 油量 1666.666
車號 6608 油量 1888.88
車號 6609 油量 1666.666
車號 6610 尚未設定油量
車號 6611 尚未設定油量
###### 總共生產 12 輛車. ######


打造第一台賽車
備料
製造一輛車
生產一輛賽車
將賽車速度由 300 調整為 600
設定車號 666
車號 666 設定油量 2500.0
賽車車號 666 油量 2500.0
 渦輪加速級數 1 速度 600
###### 總共生產 13 輛車. ######
###### 總共生產 1 輛賽車. ######

 [[[ LOTUS 賽車生產陣列 ]]]
備料
製造一輛車
設定車號 8800
車號 8800 設定油量 2000.0
將賽車速度由 0 調整為 500
生產一輛賽車(編號 8800 油量 2000.0 速度 500)
調整渦輪加速級數 : 渦輪加速級數維持 1
備料
製造一輛車
設定車號 8801
車號 8801 設定油量 2000.0
將賽車速度由 0 調整為 500
生產一輛賽車(編號 8801 油量 2000.0 速度 500)
調整渦輪加速級數 : 將渦輪加速級數由 1 調整為 3
備料
製造一輛車
設定車號 8802
車號 8802 設定油量 2000.0
將賽車速度由 0 調整為 500
生產一輛賽車(編號 8802 油量 2000.0 速度 500)
調整渦輪加速級數 : 將渦輪加速級數由 1 調整為 5
備料
製造一輛車
設定車號 8803
車號 8803 設定油量 2500.0
將賽車速度由 0 調整為 600
生產一輛賽車(編號 8803 油量 2500.0 速度 600)
調整渦輪加速級數 : 將渦輪加速級數由 1 調整為 2
備料
製造一輛車
設定車號 8804
車號 8804 設定油量 2500.0
將賽車速度由 0 調整為 600
生產一輛賽車(編號 8804 油量 2500.0 速度 600)
調整渦輪加速級數 : 將渦輪加速級數由 1 調整為 4
備料
製造一輛車
設定車號 8805
車號 8805 設定油量 2500.0
將賽車速度由 0 調整為 600
生產一輛賽車(編號 8805 油量 2500.0 速度 600)
調整渦輪加速級數 : 將渦輪加速級數由 1 調整為 6
###### 總共生產 19 輛車. ######
###### 總共生產 7 輛賽車. ######

 [[[ LOTUS ]]] [[[查詢車輛資訊]]]
賽車車號 8800 油量 2000.0
 渦輪加速級數 1 速度 500
賽車車號 8801 油量 2000.0
 渦輪加速級數 3 速度 500
賽車車號 8802 油量 2000.0
 渦輪加速級數 5 速度 500
賽車車號 8803 油量 2500.0
 渦輪加速級數 2 速度 600
賽車車號 8804 油量 2500.0
 渦輪加速級數 4 速度 600
賽車車號 8805 油量 2500.0
 渦輪加速級數 6 速度 600
###### 總共生產 19 輛車. ######
###### 總共生產 7 輛賽車. ######

 [[[ Car type RacingCar ]]] 
備料
製造一輛車
生產一輛賽車
設定車號 777
車號 777 設定油量 1800.0

 [[[ Car type RacingCar ]]] [[[查詢車輛資訊]]]
賽車車號 777 油量 1800.0
 渦輪加速級數 1 速度 300
###### 總共生產 20 輛車. ######
###### 總共生產 8 輛賽車. ######

 [[[ Car type RacingCar ]]] 
備料
製造一輛車
設定車號 778
車號 778 設定油量 1900.0
將賽車速度由 0 調整為 660
生產一輛賽車(編號 778 油量 1900.0 速度 660)

 [[[ Car type RacingCar ]]] [[[查詢車輛資訊]]]
賽車車號 778 油量 1900.0
 渦輪加速級數 1 速度 660
###### 總共生產 21 輛車. ######
###### 總共生產 9 輛賽車. ######

 [[[ MIT 房車/賽車生產陣列 ]]]
備料
製造一輛車
設定車號 3600
車號 3600 設定油量 1200.0
備料
製造一輛車
設定車號 3601
車號 3601 設定油量 1200.0
備料
製造一輛車
設定車號 3602
車號 3602 設定油量 1200.0
備料
製造一輛車
設定車號 3603
車號 3603 設定油量 2200.0
將賽車速度由 0 調整為 500
生產一輛賽車(編號 3603 油量 2200.0 速度 500)
備料
製造一輛車
設定車號 3604
車號 3604 設定油量 2200.0
將賽車速度由 0 調整為 500
生產一輛賽車(編號 3604 油量 2200.0 速度 500)
備料
製造一輛車
設定車號 3605
車號 3605 設定油量 2200.0
將賽車速度由 0 調整為 500
生產一輛賽車(編號 3605 油量 2200.0 速度 500)
###### 總共生產 27 輛車. ######
###### 總共生產 12 輛賽車. ######

 [[[ MIT ]]] [[[查詢車輛資訊]]]
車號 3600 油量 1200.0
車號 3601 油量 1200.0
車號 3602 油量 1200.0
賽車車號 3603 油量 2200.0
 渦輪加速級數 1 速度 500
賽車車號 3604 油量 2200.0
 渦輪加速級數 1 速度 500
賽車車號 3605 油量 2200.0
 渦輪加速級數 1 速度 500
###### 總共生產 27 輛車. ######
###### 總共生產 12 輛賽車. ######

PHP範例 : 處理輸入字串

對於輸入的字串必須抱持一個不信任的觀點, 所以使用之前事先清理一下就是必要的. 以下是個人慣用的清理函式, 最後一行是之前慣用的處理方法. ( 輸出字串可能是被拿來執行命令列指令 )
程式碼(輸入字串是 IP 位址)
function proc_str($instr){
  $pattern = '([^0-9\.])';
  $replacement = '';
  $instr = preg_replace($pattern, $replacement, $instr);
  return ereg_replace("&#", "&#", htmlspecialchars(str_replace("'", "\\'", stripslashes($instr))));
}

程式碼
function proc_str($instr){
  $pattern = '([^0-9a-zA-Z\-\_])';
  $replacement = '';
  $instr = preg_replace($pattern, $replacement, $instr);
  return ereg_replace("&#", "&#", htmlspecialchars(str_replace("'", "\\'", stripslashes($instr))));
}

Friday, September 21, 2012

C範例程式 : 判斷 IP 是否在給定的 subnet

為了搭配自己的 shell script 寫了這隻小程式, 沒有做一些多餘的錯誤檢查(因為 shell script 餵過去的資料已經處理過).只支援 IPv4 及一種 subnet 表示法. (只用了 stdio.h stdlib.h string.h )
程式碼
#include 
#include 
#include 

int main(int argc, char **argv)
{
 int uip[4], dst[4], nm[4], i = 0, netmask;
 char *delim = ".";
 char *dstdelim = "/";
 char * pch;
 char * dstip;
 char * mask;
 int j = 128;

 /* Check number of argc */
 if(argc != 3){
  printf("Usage : ipinsubnet UIP DSTIP/NETMASK\n");
  printf("Example : ipinsubnet 192.168.0.1 192.168.0.0/24\n");
  return 1;
 }

 /* For uip */
 i = 0;
 pch = strtok(argv[1], delim);
 while (pch != NULL){
  uip[i] = atoi(pch);
  pch = strtok(NULL, delim);
  i++;
 } 

 /* For dst IP and netmask */
 i = 0;
 /* Get dst IP */
 dstip = strtok(argv[2], dstdelim);

 /* Get netmask */
 mask = strtok(NULL, dstdelim);
 netmask = atoi(mask);

        pch = strtok(dstip, delim);
 while (pch != NULL){
  dst[i] = atoi(pch);
  pch = strtok(NULL, delim);
  i++;
 }

 /* Generate netmask */
 i = 0;
 while((netmask - 8) >= 0){
  nm[i] = 255;
  netmask-=8;
  i++;
 }

 if( netmask > 0 ){
  nm[i] = 0;
  for(; netmask > 0; netmask--){ nm[i] = nm[i] | j; j >>= 1; }
  i++;
 }
 for(;i < 4; i++){ nm[i] = 0; }

 /* Compare uip and dst with netmask */
 for(i = 0; i < 4; i++){
  if( (uip[i] & nm[i]) != (dst[i] & nm[i]) ){ printf("F\n"); return 0; }
 }
 printf("T\n");

 return 0;
}

使用範例
$ ./ipinsubnet
Usage : ipinsubnet UIP DSTIP/NETMASK
Example : ipinsubnet 192.168.0.1 192.168.0.0/24

$ ./ipinsubnet 192.168.68.1 192.168.78.0/16
T
$ ./ipinsubnet 192.168.68.1 192.168.78.0/20
T
$ ./ipinsubnet 192.168.68.1 192.168.78.0/21
F
$ ./ipinsubnet 192.168.68.1 192.168.78.0/22
F

Java範例程式 : 氣泡排序學生成績

以下採用三種不同的氣泡排序將學生成績由高至低排好.
方法一 內層迴圈所指位置與外層迴圈比大小(都是由左至右比較)
程式碼
import java.io.*;

class C7P155
{
 public static void main(String[] args) throws IOException
 {
  int student_num = 0, maxgr = 0, mingr = 0, maxid = 0, minid = 0, sum = 0, ccount = 0, scount = 0;
  boolean status = false;
  BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

  System.out.println("學生成績分析程式 ");
  do{
   System.out.print("請輸入學生人數 ");
   try{
    student_num = Integer.parseInt(br.readLine());
    status = false;
   } catch ( NegativeArraySizeException e ){
    System.out.println("請輸入正整數的學生人數.");
    status = true;
   } catch ( NumberFormatException e ){
    System.out.println("請輸入學生人數.");
    status = true;
   }
  } while (status);

  int grade[] = new int[student_num];

  System.out.println("\n開始輸入學生成績");

  for(int i = 0; i < student_num; i++){
   status = false;
   do{
    System.out.print("學生 " + (i+1) + " 成績 ");
    try{
     grade[i] = Integer.parseInt(br.readLine());
     sum += grade[i];
     if( i == 0 ){
      maxgr = mingr = grade[i];
     }else{
      if(grade[i] > maxgr){
       maxgr = grade[i]; maxid = i;
      } else if(grade[i] < mingr){
       mingr = grade[i]; minid = i;
      }
     }
     status = false;
    } catch ( NumberFormatException e ){
     System.out.println("請輸入正整數的成績.");
     status = true;
    }
   } while (status);
  }

  System.out.println("\n開始輸出學生成績");
  for(int i = 0; i < grade.length; i++){
   System.out.println("學生 " + (i+1) + " 成績 " + grade[i]);
  }
  System.out.println("\n最高最低成績");
  System.out.println("最高成績 第 " + (maxid+1) + " 號學生成績" + grade[maxid]);
  System.out.println("最低成績 第 " + (minid+1) + " 號學生成績" + grade[minid]);

  System.out.println("\n平均成績 " + (float)sum/(float)grade.length);

  System.out.println("\n成績排行榜");
  // Generate student id array
  int stid[] = new int[grade.length];
  for(int i = 0; i < grade.length; i++){ stid[i] = i; }

  // Sort by grade via bubble sort
  System.out.print("成績排序中 ");
  for(int i = 0; i < (grade.length - 1); i++){
   for(int j = i+1; j < grade.length; j++){
    if(grade[j] > grade[i]){
     // swap
     int tmpgr = grade[j];
     int tmpid = stid[j];
     grade[j] = grade[i];
     stid[j] = stid[i];
     grade[i] = tmpgr;
     stid[i] = tmpid;
     scount++;
     System.out.print("s");
    } else {
     System.out.print(".");
    }
    ccount++;
   }
  }
  System.out.println("\nSwap / Total comparison : " + scount + " / " + ccount);

  System.out.println("\n成績排序結果");
  int rank = 1;
  for(int i = 0; i < grade.length; i++){
   if( i != 0 && grade[i] != grade[i-1] ){ rank++; }
   System.out.println("名次 " + (i+1) + " 第 " + (stid[i]+1) + " 號學生成績 " + grade[i]);
  }
 }
}

物件導向版
import java.io.*;

class Student
{
 int sid;
 int grade;

 public void setGrade(int sid, int grade)
 {
  this.sid = sid;
  this.grade = grade;
 }

 public int getGrade()
 {
  return this.grade;
 }

 public int getSid()
 {
  return this.sid;
 }
}

class C8P158
{
 public static void main(String[] args) throws IOException
 {
  int student_num = 0, maxgr = 0, mingr = 0, maxid = 0, minid = 0, sum = 0, ccount = 0, scount = 0;
  boolean status = false;
  BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

  System.out.println("學生成績分析程式 ");
  do{
   System.out.print("請輸入學生人數 ");
   try{
    student_num = Integer.parseInt(br.readLine());
    status = false;
   } catch ( NegativeArraySizeException e ){
    System.out.println("請輸入正整數的學生人數.");
    status = true;
   } catch ( NumberFormatException e ){
    System.out.println("請輸入學生人數.");
    status = true;
   }
  } while (status);

  Student grade[] = new Student[student_num];

  System.out.println("\n開始輸入學生成績");

  for(int i = 0; i < student_num; i++){
   status = false;
   int tmpgrade = 0;
   do{
    System.out.print("學生 " + (i+1) + " 成績 ");
    try{
     tmpgrade = Integer.parseInt(br.readLine());
     grade[i] = new Student();
     grade[i].setGrade( i + 1, tmpgrade );
     sum += grade[i].getGrade();
     if( i == 0 ){
      maxgr = mingr = grade[i].getGrade();
     }else{
      if(grade[i].getGrade() > maxgr){
       maxgr = grade[i].getGrade(); maxid = i;
      } else if(grade[i].getGrade() < mingr){
       mingr = grade[i].getGrade(); minid = i;
      }
     }
     status = false;
    } catch ( NumberFormatException e ){
     System.out.println("請輸入正整數的成績.");
     status = true;
    }
   } while (status);
  }

  System.out.println("\n開始輸出學生成績");
  for(int i = 0; i < grade.length; i++){
   System.out.println("學生 " + (grade[i].getSid()) + " 成績 " + grade[i].getGrade());
  }
  System.out.println("\n最高最低成績");
  System.out.println("最高成績 第 " + grade[maxid].getSid() + " 號學生成績" + grade[maxid].getGrade());
  System.out.println("最低成績 第 " + grade[minid].getSid() + " 號學生成績" + grade[minid].getGrade());

  System.out.println("\n平均成績 " + (float)sum/(float)grade.length);

  System.out.println("\n成績排行榜");
  // Sort by grade via bubble sort
  System.out.print("成績排序中 ");
  for(int i = 0; i < (grade.length - 1); i++){
   for(int j = i+1; j < grade.length; j++){
    if(grade[j].getGrade() > grade[i].getGrade()){
     // swap
     Student tmpgr = grade[j];
     grade[j] = grade[i];
     grade[i] = tmpgr;
     scount++;
     System.out.print("s");
    } else {
     System.out.print(".");
    }
    ccount++;
   }
  }
  System.out.println("\nSwap / Total comparison : " + scount + " / " + ccount);

  System.out.println("\n成績排序結果");
  int rank = 1;
  for(int i = 0; i < grade.length; i++){
   if( i != 0 && grade[i].getGrade() != grade[i-1].getGrade() ){ rank++; }
   System.out.println("名次 " + (rank) + " 第 " + grade[i].getSid() + " 號學生成績 " + grade[i].getGrade());
  }
 }
}
方法二 外層迴圈由右至左, 為限制內層迴圈比較的右邊界. 內層迴圈迴圈由左至右, 相近位置兩兩比較大小.
程式碼
import java.io.*;

class C7P155_1
{
 public static void main(String[] args) throws IOException
 {
  int student_num = 0, maxgr = 0, mingr = 0, maxid = 0, minid = 0, sum = 0, scount = 0, ccount = 0;
  boolean status = false;
  BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

  System.out.println("學生成績分析程式 ");
  do{
   System.out.print("請輸入學生人數 ");
   try{
    student_num = Integer.parseInt(br.readLine());
    status = false;
   } catch ( NegativeArraySizeException e ){
    System.out.println("請輸入正整數的學生人數.");
    status = true;
   } catch ( NumberFormatException e ){
    System.out.println("請輸入學生人數.");
    status = true;
   }
  } while (status);

  int grade[] = new int[student_num];

  System.out.println("\n開始輸入學生成績");

  for(int i = 0; i < student_num; i++){
   status = false;
   do{
    System.out.print("學生 " + (i+1) + " 成績 ");
    try{
     grade[i] = Integer.parseInt(br.readLine());
     sum += grade[i];
     if( i == 0 ){
      maxgr = mingr = grade[i];
     }else{
      if(grade[i] > maxgr){
       maxgr = grade[i]; maxid = i;
      } else if(grade[i] < mingr){
       mingr = grade[i]; minid = i;
      }
     }
     status = false;
    } catch ( NumberFormatException e ){
     System.out.println("請輸入正整數的成績.");
     status = true;
    }
   } while (status);
  }

  System.out.println("\n開始輸出學生成績");
  for(int i = 0; i < grade.length; i++){
   System.out.println("學生 " + (i+1) + " 成績 " + grade[i]);
  }
  System.out.println("\n最高最低成績");
  System.out.println("最高成績 第 " + (maxid+1) + " 號學生成績" + grade[maxid]);
  System.out.println("最低成績 第 " + (minid+1) + " 號學生成績" + grade[minid]);

  System.out.println("\n平均成績 " + (float)sum/(float)grade.length);

  System.out.println("\n成績排行榜");
  // Generate student id array
  int stid[] = new int[grade.length];
  for(int i = 0; i < grade.length; i++){ stid[i] = i; }

  // Sort by grade via bubble sort
  System.out.print("成績排序中 ");
  for(int i = grade.length - 1; i > 0; i--){
   for(int j = 0; j < i; j++){
    if(grade[j+1] > grade[j]){
     // swap
     int tmpgr = grade[j+1];
     int tmpid = stid[j+1];
     grade[j+1] = grade[j];
     stid[j+1] = stid[j];
     grade[j] = tmpgr;
     stid[j] = tmpid;
     scount++;
     System.out.print("s");
    } else {
     System.out.print(".");
    }
    ccount++;
   }
  }
  System.out.println("\nSwap / Total comparison : " + scount + " / " + ccount);

  System.out.println("\n成績排序結果");
  int rank = 1;
  for(int i = 0; i < grade.length; i++){
   if( i != 0 && grade[i] != grade[i-1] ){ rank++; }
   System.out.println("名次 " + (rank) + " 第 " + (stid[i]+1) + " 號學生成績 " + grade[i]);
  }
 }
}

物件導向版
import java.io.*;

class Student
{
 int sid;
 int grade;

 public void setGrade(int sid, int grade)
 {
  this.sid = sid;
  this.grade = grade;
 }

 public int getGrade()
 {
  return this.grade;
 }

 public int getSid()
 {
  return this.sid;
 }
}

class C8P158_1
{
 public static void main(String[] args) throws IOException
 {
  int student_num = 0, maxgr = 0, mingr = 0, maxid = 0, minid = 0, sum = 0, scount = 0, ccount = 0;
  boolean status = false;
  BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

  System.out.println("學生成績分析程式 ");
  do{
   System.out.print("請輸入學生人數 ");
   try{
    student_num = Integer.parseInt(br.readLine());
    status = false;
   } catch ( NegativeArraySizeException e ){
    System.out.println("請輸入正整數的學生人數.");
    status = true;
   } catch ( NumberFormatException e ){
    System.out.println("請輸入學生人數.");
    status = true;
   }
  } while (status);

  Student grade[] = new Student[student_num];

  System.out.println("\n開始輸入學生成績");

  for(int i = 0; i < student_num; i++){
   status = false;
   do{
    System.out.print("學生 " + (i+1) + " 成績 ");
    try{
     grade[i] = new Student();
     grade[i].setGrade( i + 1, Integer.parseInt(br.readLine()) );
     sum += grade[i].getGrade();
     if( i == 0 ){
      maxgr = mingr = grade[i].getGrade();
     }else{
      if(grade[i].getGrade() > maxgr){
       maxgr = grade[i].getGrade(); maxid = i;
      } else if(grade[i].getGrade() < mingr){
       mingr = grade[i].getGrade(); minid = i;
      }
     }
     status = false;
    } catch ( NumberFormatException e ){
     System.out.println("請輸入正整數的成績.");
     status = true;
    }
   } while (status);
  }

  System.out.println("\n開始輸出學生成績");
  for(int i = 0; i < grade.length; i++){
   System.out.println("學生 " + grade[i].getSid() + " 成績 " + grade[i].getGrade());
  }
  System.out.println("\n最高最低成績");
  System.out.println("最高成績 第 " + grade[maxid].getSid() + " 號學生成績" + grade[maxid].getGrade());
  System.out.println("最低成績 第 " + grade[minid].getSid() + " 號學生成績" + grade[minid].getGrade());

  System.out.println("\n平均成績 " + (float)sum/(float)grade.length);

  System.out.println("\n成績排行榜");

  // Sort by grade via bubble sort
  System.out.print("成績排序中 ");
  for(int i = grade.length - 1; i > 0; i--){
   for(int j = 0; j < i; j++){
    if(grade[j+1].getGrade() > grade[j].getGrade()){
     // swap
     Student tmpgr = grade[j+1];
     grade[j+1] = grade[j];
     grade[j] = tmpgr;
     scount++;
     System.out.print("s");
    } else {
     System.out.print(".");
    }
    ccount++;
   }
  }
  System.out.println("\nSwap / Total comparison : " + scount + " / " + ccount);

  System.out.println("\n成績排序結果");
  int rank = 1;
  for(int i = 0; i < grade.length; i++){
   if( i != 0 && grade[i].getGrade() != grade[i-1].getGrade() ){ rank++; }
   System.out.println("名次 " + (rank) + " 第 " + grade[i].getSid() + " 號學生成績 " + grade[i].getGrade());
  }
 }
}
方法三 方法二的進化版. 將最後交換位置設為 sentinel, 讓比較次數減少.
程式碼
import java.io.*;

class C7P155_2
{
 public static void main(String[] args) throws IOException
 {
  int student_num = 0, maxgr = 0, mingr = 0, maxid = 0, minid = 0, sum = 0, scount = 0, ccount = 0;
  boolean status = false;
  BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

  System.out.println("學生成績分析程式 ");
  do{
   System.out.print("請輸入學生人數 ");
   try{
    student_num = Integer.parseInt(br.readLine());
    status = false;
   } catch ( NegativeArraySizeException e ){
    System.out.println("請輸入正整數的學生人數.");
    status = true;
   } catch ( NumberFormatException e ){
    System.out.println("請輸入學生人數.");
    status = true;
   }
  } while (status);

  int grade[] = new int[student_num];

  System.out.println("\n開始輸入學生成績");

  for(int i = 0; i < student_num; i++){
   status = false;
   do{
    System.out.print("學生 " + (i+1) + " 成績 ");
    try{
     grade[i] = Integer.parseInt(br.readLine());
     sum += grade[i];
     if( i == 0 ){
      maxgr = mingr = grade[i];
     }else{
      if(grade[i] > maxgr){
       maxgr = grade[i]; maxid = i;
      } else if(grade[i] < mingr){
       mingr = grade[i]; minid = i;
      }
     }
     status = false;
    } catch ( NumberFormatException e ){
     System.out.println("請輸入正整數的成績.");
     status = true;
    }
   } while (status);
  }

  System.out.println("\n開始輸出學生成績");
  for(int i = 0; i < grade.length; i++){
   System.out.println("學生 " + (i+1) + " 成績 " + grade[i]);
  }
  System.out.println("\n最高最低成績");
  System.out.println("最高成績 第 " + (maxid+1) + " 號學生成績" + grade[maxid]);
  System.out.println("最低成績 第 " + (minid+1) + " 號學生成績" + grade[minid]);

  System.out.println("\n平均成績 " + (float)sum/(float)grade.length);

  System.out.println("\n成績排行榜");
  // Generate student id array
  int stid[] = new int[grade.length];
  for(int i = 0; i < grade.length; i++){ stid[i] = i; }

  // Sort by grade via bubble sort
  System.out.print("成績排序中 ");
  for(int i = grade.length - 1, lastj = grade.length - 1; i > 0; i--){
   // For debug
   // System.out.print("(" + i + ")[" + lastj + "]");
   boolean swapstatus = false;
   for(int j = 0; j < i; j++){
    if(grade[j+1] > grade[j]){
     // swap
     int tmpgr = grade[j+1];
     int tmpid = stid[j+1];
     grade[j+1] = grade[j];
     stid[j+1] = stid[j];
     grade[j] = tmpgr;
     stid[j] = tmpid;
     lastj = j;
     swapstatus = true;
     scount++;
     System.out.print("s");
    } else {
     System.out.print(".");
    }
    ccount++;
   }
   if(swapstatus){ i = lastj + 1; } else { break;}
  }
  System.out.println("\nSwap / Total comparison : " + scount + " / " + ccount);

  System.out.println("\n成績排序結果");
  int rank = 1;
  for(int i = 0; i < grade.length; i++){
   if( i != 0 && grade[i] != grade[i-1] ){ rank++; }
   System.out.println("名次 " + (rank) + " 第 " + (stid[i]+1) + " 號學生成績 " + grade[i]);
  }
 }
}

物件導向版
import java.io.*;

class Student
{
 int sid;
 int grade;

 public void setGrade(int sid, int grade)
 {
  this.sid = sid;
  this.grade = grade;
 }

 public int getGrade()
 {
  return this.grade;
 }

 public int getSid()
 {
  return this.sid;
 }
}

class C8P158_2
{
 public static void main(String[] args) throws IOException
 {
  int student_num = 0, maxgr = 0, mingr = 0, maxid = 0, minid = 0, sum = 0, scount = 0, ccount = 0;
  boolean status = false;
  BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

  System.out.println("學生成績分析程式 ");
  do{
   System.out.print("請輸入學生人數 ");
   try{
    student_num = Integer.parseInt(br.readLine());
    status = false;
   } catch ( NegativeArraySizeException e ){
    System.out.println("請輸入正整數的學生人數.");
    status = true;
   } catch ( NumberFormatException e ){
    System.out.println("請輸入學生人數.");
    status = true;
   }
  } while (status);

  Student grade[] = new Student[student_num];

  System.out.println("\n開始輸入學生成績");

  for(int i = 0; i < student_num; i++){
   status = false;
   do{
    System.out.print("學生 " + (i+1) + " 成績 ");
    try{
     grade[i] = new Student();
     grade[i].setGrade( i + 1, Integer.parseInt(br.readLine()) );
     sum += grade[i].getGrade();
     if( i == 0 ){
      maxgr = mingr = grade[i].getGrade();
     }else{
      if(grade[i].getGrade() > maxgr){
       maxgr = grade[i].getGrade(); maxid = i;
      } else if(grade[i].getGrade() < mingr){
       mingr = grade[i].getGrade(); minid = i;
      }
     }
     status = false;
    } catch ( NumberFormatException e ){
     System.out.println("請輸入正整數的成績.");
     status = true;
    }
   } while (status);
  }

  System.out.println("\n開始輸出學生成績");
  for(int i = 0; i < grade.length; i++){
   System.out.println("學生 " + grade[i].getSid() + " 成績 " + grade[i].getGrade());
  }
  System.out.println("\n最高最低成績");
  System.out.println("最高成績 第 " + grade[maxid].getSid() + " 號學生成績" + grade[maxid].getGrade());
  System.out.println("最低成績 第 " + grade[minid].getSid() + " 號學生成績" + grade[minid].getGrade());

  System.out.println("\n平均成績 " + (float)sum/(float)grade.length);

  System.out.println("\n成績排行榜");

  // Sort by grade via bubble sort
  System.out.print("成績排序中 ");
  for(int i = grade.length - 1, lastj = grade.length - 1; i > 0; i--){
   // For debug
   // System.out.print("(" + i + ")[" + lastj + "]");
   boolean swapstatus = false;
   for(int j = 0; j < i; j++){
    if(grade[j+1].getGrade() > grade[j].getGrade()){
     // swap
     Student tmpgr = grade[j+1];
     grade[j+1] = grade[j];
     grade[j] = tmpgr;
     lastj = j;
     swapstatus = true;
     scount++;
     System.out.print("s");
    } else {
     System.out.print(".");
    }
    ccount++;
   }
   if(swapstatus){ i = lastj + 1; } else { break;}
  }
  System.out.println("\nSwap / Total comparison : " + scount + " / " + ccount);

  System.out.println("\n成績排序結果");
  int rank = 1;
  for(int i = 0; i < grade.length; i++){
   if( i != 0 && grade[i].getGrade() != grade[i-1].getGrade() ){ rank++; }
   System.out.println("名次 " + (rank) + " 第 " + grade[i].getSid() + " 號學生成績 " + grade[i].getGrade());
  }
 }
}

測試範例 : 七位學生的成績
範例一 : 1 2 3 4 5 6 7
反轉表 
1 2 3 4 5 6 7
0 1 2 3 4 5 6

方法一 : 21/21.
方法二 : 21/21.
方法三 : 21/21.

範例二 : 7 6 5 4 3 2 1
反轉表 
1 2 3 4 5 6 7
0 0 0 0 0 0 0

方法一 : 0/21.
方法二 : 0/21.
方法三 : 0/6.

範例三 : 5 7 4 1 6 3 2
反轉表 
1 2 3 4 5 6 7
0 1 1 0 0 3 1

方法一 : 6/21.
方法二 : 6/21.
方法三 : 6/14.

範例四 : 1 6 7 4 5 2 3
反轉表 
1 2 3 4 5 6 7
0 1 2 1 2 1 2

方法一 : 9/21.
方法二 : 9/21.
方法三 : 9/15.

Saturday, September 15, 2012

自家用 ipfw log 統計

雖然有 security/ipfwcount 好用, 但是總想可以自由的分析 ipfw.log. 所以就寫了以下的 stupid script. 哈哈 ... 但是它可以用.

#!/usr/local/bin/bash
##
##  FreeBSD ipfw log analyzer
##

PATH="/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin:$PATH"

TS=`date +"%s"`
TMPFILE="/tmp/ana_ipfw-$TS"
TMPFILEsort="/tmp/ana_ipfw_sort-$TS"
LC_ALL=C
LANG=C

### Default options
datelimit=no
portlimit=no
iplimit=no
verbose=no
sorted=no
onlydata=no
datestr=""
logfile="/var/log/ipfw.log"

### Help
usage () {
 echo "Usage: $0 [-p] [-i] [-v] [-s] [-o] [-d date_limit] [filename]"
 echo "   -p  port analysis"
 echo "   -i  IP address analysis"
 echo "   -d  date format"
 echo "   +DDAY   from DDAY to today"
 echo "   +-DDAY  DDAY days ago"
 echo "   YYYYMMDD-YYYYMMDD  from first YYYYMMDD to last YYYYMMDD"
 echo "   total   all data records in the log file"
 echo "   -s  sorted by count"
 echo "   -o  only data and disable verbose"
 echo "   -v  enable verbose messages"
 echo "   filename default : /var/log/ipfw.log"
}
if [ $# -eq 0 ]; then
 usage
 exit 1
fi

### Get arguments
args=`getopt dpivso $*`

if [ $? != 0 ]; then
 echo "$0:ERROR"
 exit 2
fi

set -- $args

for arg
do
 case "$arg" in
  -d )  datelimit=yes; shift ;;
  -p )  portlimit=yes; shift ;;
  -i )  iplimit=yes; shift ;;
  -v )  verbose=yes; shift ;;
  -s )  sorted=yes; shift ;;
  -o )  onlydata=yes; shift ;;
  -- )  datestr=$2; shift; shift ;;
 esac
done

if [ $datelimit == "no" ]; then
 if [ ! -z $datestr ]&&[ -f $datestr ]; then
  logfile=$datestr
 fi
else
 if [ ! -z $datestr ]&&[ ! -z $1 ]; then
  logfile=$1
 fi
fi

if [ ! -f $logfile ]; then
 echo "Error : Log file $logfile does not existed."
 exit 1
fi

if [ $onlydata == "yes" ]; then
 verbose=no
fi

### For debug
if [ $verbose == "yes" ]; then
 echo "Parameters"
 echo $args
 echo "Date $datelimit($datestr) Port $portlimit IP $iplimit Verbose $verbose Sort $sorted Logfile $logfile"
 echo ""
fi

### Generate date limitation
if [ $datelimit == "no" ]; then
 dtmp="0"
 datearray=($dtmp)
else
 if [ ! -z `echo $datestr | awk 'BEGIN { FS = "+" } ; { print $2 }'` ]; then
  if [ ! -z `echo $datestr | awk 'BEGIN { FS = "+" } ; { print $2 }' | awk 'BEGIN { FS = "-" } ; { print $2 }'` ]; then
   datearray=(`echo $datestr | awk 'BEGIN { FS = "+" } ; { print $2 }' | awk 'BEGIN { FS = "-" } ; { print $2 }'`)
  else
   i=`echo $datestr | awk 'BEGIN { FS = "+" } ; { print $2 }'`
   dtmp=""
   for((; i >= 0; i--))
   do
    dtmp="$dtmp $i"
   done
   datearray=($dtmp)
  fi
 elif [ "$datestr" == "total" ]; then
  datestr="total"
 elif [ -z $datestr ]; then
  dtmp="0"
  datearray=($dtmp)
 else
  if [[ $datestr != *[!0-9\-]* ]]; then
   bdate=`echo $datestr | awk 'BEGIN { FS = "-" } ; { print $1 }'`
   edate=`echo $datestr | awk 'BEGIN { FS = "-" } ; { print $2 }'`
  else
   bdate=`date +"%Y%m%d"`
   edate=`date +"%Y%m%d"`
  fi

  todayts=`date -j "+%s"`
  bdatets=`date -j -f "%Y%m%d" "$bdate" "+%s"`
  edatets=`date -j -f "%Y%m%d" "$edate" "+%s"`

  if [ $bdatets -gt $todayts ]; then
   echo "Error : $bdate is future. I can not analyze it."
   exit 1
  else
   ib=`expr \( $todayts - $bdatets \) / 86400`
  fi
  if [ $edatets -gt $todayts ]; then
   if [ $verbose == "yes" ]; then
    echo "Warning : $edate is future. But I can replace it with today("`date +"%Y%m%d"`")."
    echo ""
   fi
   ie=0
  else
   ie=`expr \( $todayts - $edatets \) / 86400`
  fi

  dtmp=""
  for((i = $ib; i >= $ie; i--))
  do
   dtmp="$dtmp $i"
  done
  datearray=($dtmp)
 fi
fi

### Create tmp file
if [ $verbose == "yes" ]; then
 echo "Create $TMPFILE"
 echo ""
fi
touch $TMPFILE

### Get log
if [ $verbose == "yes" ]; then
 echo "Get log from $logfile"
fi
if [ $datelimit == "no" ]; then
 if [ $verbose == "yes" ]; then
  date +"%b %e"
 fi
 keyfilter=`date +"%b %e"`
 grep "$keyfilter" $logfile >> $TMPFILE
elif [ "$datestr" == "total" ]; then
 cat $logfile > $TMPFILE
else
 for dnum in ${datearray[@]}
 do
  if [ $verbose == "yes" ]; then
   date -v-${dnum}d +"%b %e"
  fi
  keyfilter=`date -v-${dnum}d +"%b %e"`
  grep "$keyfilter" $logfile >> $TMPFILE
 done
fi
if [ $verbose == "yes" ]; then
 echo ""
fi

if [ $verbose == "yes" ]; then
 echo "Count lines in $TMPFILE"
 wc -l $TMPFILE
 echo ""
fi

### Port Analysis
if [ $portlimit == "yes" ]; then
 if [ $sorted == "yes" ]; then
  rm -f $TMPFILEsort
  touch $TMPFILEsort
 fi

 if [ $onlydata == "no" ]; then
  echo "[[[ Port Analysis ]]]"
  printf "%12s\t%12s\n" "Port Number" "Count"
  printf "%12s\t%12s\n" "------------" "--------"
 fi
 for portnum in `cat $TMPFILE | awk 'BEGIN { FS = " " } ; { if( $11=="\-\>" ){ split($12, p, ","); print p[1] } else { print $11 } }' | grep ":" | awk 'BEGIN { FS = ":" } ; { print $2 }' | sort -u -n`
 do
  if [ $sorted == "yes" ]; then
   printf "%12s\t%12s\n" "$portnum" `cat $TMPFILE | awk 'BEGIN { FS = " " } ; { if( $11=="\-\>" ){ split($12, p, ","); print p[1] } else { print $11 } }' | grep ":" | awk 'BEGIN { FS = ":" } ; { print $2 }' | grep $portnum | wc -l` >> $TMPFILEsort
  else
   printf "%12s\t%12s\n" "$portnum" `cat $TMPFILE | awk 'BEGIN { FS = " " } ; { if( $11=="\-\>" ){ split($12, p, ","); print p[1] } else { print $11 } }' | grep ":" | awk 'BEGIN { FS = ":" } ; { print $2 }' | grep $portnum | wc -l`
  fi
 done
 if [ $sorted == "yes" ]; then
  cat $TMPFILEsort | awk 'BEGIN { FS = " " } ; { printf ( "%12s\t%12s\n", $1, $2) }' | sort -rgk2 > /tmp/ttt-$TS
  cat /tmp/ttt-$TS > $TMPFILEsort
  rm -f /tmp/ttt-$TS
  cat $TMPFILEsort
 fi
fi

### Client IP Analysis
if [ $iplimit == "yes" ]; then
 if [ $sorted == "yes" ]; then
  rm -f $TMPFILEsort
  touch $TMPFILEsort
 fi

 if [ $onlydata == "no" ]; then
  echo "[[[ Client IP Analysis ]]]"
  printf "%16s\t%12s\n" "IP Address" "Count"
  printf "%16s\t%12s\n" "----------------" "--------"
 fi
 for ipaddr in `cat $TMPFILE | awk 'BEGIN { FS = " " } ; { print $10 }' | grep ":" | awk 'BEGIN { FS = ":" } ; { print $1 }' | sort -u -n`
 do
  if [ $sorted == "yes" ]; then
   printf "%16s\t%12s\n" "$ipaddr" `cat $TMPFILE | awk 'BEGIN { FS = " " } ; { print $10 }' | grep ":" | awk 'BEGIN { FS = ":" } ; { print $1 }' | grep $ipaddr | wc -l` >> $TMPFILEsort
  else
   printf "%16s\t%12s\n" "$ipaddr" `cat $TMPFILE | awk 'BEGIN { FS = " " } ; { print $10 }' | grep ":" | awk 'BEGIN { FS = ":" } ; { print $1 }' | grep $ipaddr | wc -l`
  fi
 done
 if [ $sorted == "yes" ]; then
  cat $TMPFILEsort | awk 'BEGIN { FS = " " } ; { printf ( "%16s\t%12s\n", $1, $2) }' | sort -rgk2 > /tmp/ttt-$TS
  cat /tmp/ttt-$TS > $TMPFILEsort
  rm -f /tmp/ttt-$TS
  cat $TMPFILEsort
 fi
fi

### Remove tmp file
if [ $verbose == "yes" ]; then
 echo ""
 echo "Remove $TMPFILE"
fi
rm -f $TMPFILE
if [ $sorted == "yes" ]; then
 if [ $verbose == "yes" ]; then
  echo "Remove $TMPFILEsort"
 fi
 rm -f $TMPFILEsort
fi


使用方法如下 :
Usage: /Path/to/ana_ipfw.sh [-p] [-i] [-v] [-s] [-o] [-d date_limit]  [filename]
   -p           port analysis
   -i           IP address analysis
   -d           date format
                        +DDAY   from DDAY to today
                        +-DDAY  DDAY days ago
                        YYYYMMDD-YYYYMMDD  from first YYYYMMDD to last YYYYMMDD
                        total   all data records in the log file
   -s           sorted by count
   -o           only data and disable verbose
   -v           enable verbose messages
   filename     default : /var/log/ipfw.log


分析範例如下 :
今天的 port 分析
# ./ana_ipfw.sh -p
[[[ Port Analysis ]]]
 Port Number           Count
------------        --------
          21               4
          22              14
          25              13
          80              30
         111               2
         199               2
         389               2
         443               8
         873               2
        2049               2
        3000              36
        3306              11
        5800               2
        5900               5
        6000               2
        6001               2


今天的 port 分析(按照數量排序)
# ./ana_ipfw.sh -ps
[[[ Port Analysis ]]]
 Port Number           Count
------------        --------
        3000              36
          80              30
          22              14
          25              13
        3306              12
         443               8
        5900               5
          21               4
        6001               2
        6000               2
        5800               2
        2049               2
         873               2
         389               2
         199               2
         111               2


最近三天的 port 分析
# ./ana_ipfw.sh -p -d +3
[[[ Port Analysis ]]]
 Port Number           Count
------------        --------
          21               7
          22              20
          25              41
          80              72
         111               2
         199               2
         389               2
         443              14
         873               2
        2049               2
        3000              72
        3306              57
        5800               2
        5900              12
        5901               3
        6000               7
        6001               2


最近三天的 port 分析(按照數量排序)
# ./ana_ipfw.sh -p -d +3 -s
[[[ Port Analysis ]]]
 Port Number           Count
------------        --------
        3000              72
          80              72
        3306              57
          25              41
          22              20
         443              14
        5900              12
        6000               7
          21               7
        5901               3
        6001               2
        5800               2
        2049               2
         873               2
         389               2
         199               2
         111               2


最後開進工程模式(用來修 script 用, 應該還有 bug.).
# ./ana_ipfw.sh -pv
Parameters
-p -v --
Date no() Port yes IP no Verbose yes Sort no Logfile /var/log/ipfw.log

Create /tmp/ana_ipfw-1347716436

Get log from /var/log/ipfw.log
Sep 15

Count lines in /tmp/ana_ipfw-1347716436
     495 /tmp/ana_ipfw-1347716436

[[[ Port Analysis ]]]
 Port Number           Count
------------        --------
          21               4
          22              14
          25              13
          80              30
         111               2
         199               2
         389               2
         443               8
         873               2
        2049               2
        3000              36
        3306              12
        5800               2
        5900               5
        6000               2
        6001               2

Remove /tmp/ana_ipfw-1347716436

Tuesday, September 04, 2012

看來社交工程看上我了.

還好我的英文很爛. 下次強烈建議真的該認真一點發中文的給我. 拜託!

Return-Path: 
X-Original-To: ZZZZZZ@XXXXXX.daemonland.org
Delivered-To: ZZZZZZ@XXXXXX.daemonland.org
Received: from mail.daemonland.org (localhost.daemonland.org [127.0.0.1])
 by mail.daemonland.org (Postfix) with ESMTP id BEA8030A8EE8
 for ; Tue, 28 Aug 2012 03:08:44 +0800 (CST)
Authentication-Results: mail.daemonland.org; sender-id=neutral header.from=no-reply@googlemail.com; spf=neutral smtp.mfrom=no-reply@googlemail.com
X-Quarantine-ID: 
X-Virus-Scanned: amavisd-new at daemonland.org
X-Spam-Flag: YES
X-Spam-Score: 28.786
X-Spam-Level: ****************************
X-Spam-Status: Yes, score=28.786 tagged_above=2 required=6.2
 tests=[BAYES_50=0.8, DKIM_ADSP_CUSTOM_MED=0.001,
 FORGED_MUA_OUTLOOK=1.927, FREEMAIL_FROM=0.001, FREEMAIL_REPLYTO=2.398,
 FROM_MISSPACED=2.292, FROM_MISSP_DKIM=0.001,
 FROM_MISSP_EH_MATCH=1.801, FROM_MISSP_FREEMAIL=1.757,
 FROM_MISSP_MSFT=1.324, FROM_MISSP_REPLYTO=1.297,
 FROM_MISSP_TO_UNDISC=0.643, FROM_MISSP_USER=2.153, FSL_UA=0.001,
 FSL_XM_419=0.908, MSOE_MID_WRONG_CASE=2.584, NML_ADSP_CUSTOM_MED=0.9,
 NSL_RCVD_FROM_USER=3.05, RCVD_IN_BRBL_LASTEXT=1.449, RCVD_IN_PSBL=2.7,
 SPF_NEUTRAL=0.779, T_FREEMAIL_DOC_PDF=0.01,
 T_TO_NO_BRKTS_FREEMAIL=0.01] autolearn=spam
Received: from mail.daemonland.org ([127.0.0.1])
 by mail.daemonland.org (mail.daemonland.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id rjtk3A0Afb2L for ;
 Tue, 28 Aug 2012 03:08:40 +0800 (CST)
Received: from wzr.ug.edu.pl (wzr.ug.edu.pl [153.19.121.200])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mail.daemonland.org (Postfix) with ESMTPS id 516BB30A8EE3
 for ; Tue, 28 Aug 2012 03:08:34 +0800 (CST)
Received: from localhost (localhost [127.0.0.1])
 by wzr.ug.edu.pl (Postfix) with ESMTP id 46AA311F9E9;
 Mon, 27 Aug 2012 20:41:18 +0200 (CEST)
X-Virus-Scanned: Antywirus amavisd-new na wzr.ug.edu.pl
Received: from wzr.ug.edu.pl ([127.0.0.1])
 by localhost (jowita.zr.univ.gda.pl [127.0.0.1]) (amavisd-new, port 10024)
 with SMTP id pfzxGTTCPeR4; Mon, 27 Aug 2012 20:41:18 +0200 (CEST)
Received: from User (rrcs-71-41-65-58.se.biz.rr.com [71.41.65.58])
 by wzr.ug.edu.pl (Postfix) with ESMTPA id AA80611F9DC;
 Mon, 27 Aug 2012 20:40:57 +0200 (CEST)
Reply-To: 
From: "Google"
Subject: ***SPAM*** Attn::User
Date: Mon, 27 Aug 2012 14:41:07 -0400
MIME-Version: 1.0
Content-Type: multipart/mixed;
 boundary="----=_NextPart_000_0071_01C2A9A6.03BBAB76"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
Message-Id: <20120827184057.AA80611F9DC@wzr.ug.edu.pl>
To: undisclosed-recipients:;

This is a multi-part message in MIME format.

------=_NextPart_000_0071_01C2A9A6.03BBAB76
Content-Type: text/plain;
 charset="Windows-1251"
Content-Transfer-Encoding: 7bit

Dear Google User.
Attached to this email is your Official notification of You being among
one of the selected winners in the ongoing GOOGLE USER REWARD
PROMOTION,see attached document for more details.

Eduardo Molestina,
Chief Announcer.
Google INC.

------=_NextPart_000_0071_01C2A9A6.03BBAB76
Content-Type: application/msword;
 name="Google__Reward..doc"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
 filename="Google__Reward..doc"


附件檔案內容
From: Google Inc.
Corporate Headquarters,
1600 Amphitheatre Parkway,
 Mountain View,
 California, 94043,
United States,

Thru: Google Inc,
Google UK,
Manchester, Peter House
Oxford Street, Manchester
M1 5AN
Manchester, United Kingdom.

Dear Google User.

GOOGLE REWARD
Google Incorporation is one company that earns its profit mainly from advertising using their very own Google Ad Sense, Google Chrome browser, Google search engine, Gmail, Gala, Sify, e-mail service Google Maps, Google Apps, Orkut social networking and YouTube video sharing, which are offered to the public and was officially Incorporated on the 4th of September 1998, and our profit margin have been great.

In a bid to reward our numerous users/customers/clients who in one way or the other have contributed to our great success and achievement and had dealings with Google Services, Search Engine, Gmail, YouTube, Translator, Google Map, Google Earth, Ad Sense e.tc, 

The Board of Directors and Management have decided to come up with the Google Reward Scheme (GRS) to show our appreciation and also pray your continual usage of our services. In view of this A GOOGLE REWARD PROGRAM was organized which is first of its kind to encourage our users, and that the interest of our users are dear to us and also to say thank you. This promotion was set-up to encourage the active use of the Google search engine and the Google ancillary services. 

We wish to congratulate you, for being selected as one of our lucky winner in the ongoing GOOGLE REWARD PROGRAM. Hence we do believe with your prize, you will continue to be active in your patronage and loyalty to Google as a total of 6 Winners emerged worldwide, Draw from different continents of the world.

A Bank Cheque have been issued in your favor, hence you have won for yourself the sum of $1,000.000.00 (One Million United States Dollars) and also you have been enlisted as One of the Google Ambassadors. 

We are aware of the rampant abuse of Google as a company  and we are assuring all our users that we are working on this, as Google does not run a lottery and we will bring all those involved to book.

To claim your reward, please contact our Reward  Co-Ordinatoor Dr. Paul Lind by neatly filling the verification and funds release form below, as your payment will be released and arranged by our United Kingdom Office;

VERIFICATION AND FUNDS RELEASE FORM.
*First Name

*Last Name

*Residential Address

*Telephone/ Mobile

*Nationality/Country

*Age

*Sex

*Occupation/Position

*Amount Won

*Alternate Email



To File For Your Claim, Please Contact Our,
REWARD  CO-ORDINATOOR
DR. PAUL LIND
E-MAIL:  USERREWARD2012@GMAIL.COM
PHONE: +44- 702-3073-975/+44- 702-3073-983

Note: You can fill your verification and funds release form by printing and manually filling or you can fill directly on mail, or provide the details on Microsoft Word.

Congratulations from the Staff & Members of Google Incorporation.


Dr. Larry Page.
Chairman of the Board and Chief Executive Officer,
Google Inc.
Corporate Headquarters,
1600 Amphitheatre Parkway,
 Mountain View,
 California, 94043,
United States of America.                                              


寄出郵件的機器資料
Whois
#
# Query terms are ambiguous.  The query is assumed to be:
#     "n 71.41.65.58"
#
# Use "?" to get help.
#

#
# The following results may also be obtained via:
# http://whois.arin.net/rest/nets;q=71.41.65.58?showDetails=true&showARIN=false&ext=netref2
#

NetRange:       71.40.0.0 - 71.43.255.255
CIDR:           71.40.0.0/14
OriginAS:
NetName:        RCSW
NetHandle:      NET-71-40-0-0-1
Parent:         NET-71-0-0-0-0
NetType:        Direct Allocation
RegDate:        2005-04-01
Updated:        2012-02-24
Ref:            http://whois.arin.net/rest/net/NET-71-40-0-0-1


OrgName:        Road Runner HoldCo LLC
OrgId:          RCSW
Address:        13820 Sunrise Valley Drive
City:           Herndon
StateProv:      VA
PostalCode:     20171
Country:        US
RegDate:        2001-09-07
Updated:        2011-07-06
Comment:        Allocations for this OrgID serve Road Runner commercial customers out of the Austin, TX and Tampa Bay, FL RDCs.
Ref:            http://whois.arin.net/rest/org/RCSW

ReferralServer: rwhois://ipmt.rr.com:4321

OrgTechHandle: IPTEC-ARIN
OrgTechName:   IP Tech
OrgTechPhone:  +1-703-345-3416
OrgTechEmail:  abuse@rr.com
OrgTechRef:    http://whois.arin.net/rest/poc/IPTEC-ARIN

OrgAbuseHandle: ABUSE10-ARIN
OrgAbuseName:   Abuse
OrgAbusePhone:  +1-703-345-3416
OrgAbuseEmail:  abuse@rr.com
OrgAbuseRef:    http://whois.arin.net/rest/poc/ABUSE10-ARIN

#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#

Thursday, August 30, 2012

FreeBSD 下 NTFS 效能

由於建立了儲存設備二號機, 之前放在 Windows XP 下磁碟陣列的資料也備份到 USB 外接硬碟. 就想將備份資料放到二號機(FreeBSD 9.0R + ZFS). 心想系統已經有內建的 mount_ntfs 就直接拿來用.
mount_ntfs -C Big-5 /dev/da0s1 /mnt
沒想到 rsync 下去, 速度竟然只有 1MB/s.
檢查確認 da0 是掛在 usbus1(on ehci0) 下

...

usbus1: EHCI version 1.0

usbus1:  on ehci0

...

usbus1: 480Mbps High Speed USB v2.0

...

ugen1.1:  at usbus1

uhub1:  on usbus1

...

ugen1.2:  at usbus1

umass0:  on usbus1

umass0:  SCSI over Bulk-Only; quirks = 0x4000

umass0:13:0:-1: Attached to scbus13

da0 at umass-sim0 bus 0 scbus13 target 0 lun 0

da0:  Fixed Direct Access SCSI-2 device

da0: 40.000MB/s transfers

da0: 1907729MB (3907029168 512 byte sectors: 255H 63S/T 243201C)

看過這篇討論覺得應該嘗試一下 sysutils/fusefs-ntfs.
沒想到效能差這麼多, 同樣使用 rsync 目前速度已經飆到平均 13MB/s , 最大值有看到 22MB/s. 總算可以快速將資料備份. 放在單一硬碟上還真是蠻擔心的.

加入 Project HoneyPot

既然機器三不五時有機器人來嘗試入侵, 就把機器加個小功能 : 升級當蜜罐. 再來看看如何將 sshd ipfw 的資料上傳給類似的計畫. Funny! 感覺好像在 Fallout 中加入 Brother of Steel 鋼鐵兄弟會.

參考資料 (Reference)
1. Project HoneyPot from delphij's Chaos
2. Project HoneyPot 官方網頁

Thursday, August 02, 2012

測試 ZFS copies 在單顆磁碟下的復原能力.

感謝 Kuli 學長提供良好的測試方法 - 使用 memory disk 測試. 測試指令如下 :

1. 準備階段
1.1. 建立 zpool test 及測試 vol 1 及 2
# mdconfig -a -t swap -s 66m
# zpool create test /dev/md0
# zfs create -o copies=1 test/1
# zfs create -o copies=2 test/2

1.2. 編輯測試檔案
# vi /test/1/test1.txt

輸入 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
# vi /test/2/test2.txt

輸入 BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

1.3. 匯出 zpool test 並製作 memory disk dump file.
# zpool export test
# dd if=/dev/md0 of=/root/md0

2. 複製 dump file 並使用 hexedit 搜尋連續的 A 及 B 編輯檔案.
# cp -a /root/md0 /root/md0-edit
# hexedit md0

2.1. 搜尋有 36 個 A 的區塊將 A 改成 C. 進行 3. 觀察 ZFS 的反應.

2.2. 搜尋有 36 個 B 的區塊將 B 改成 D, 但只修改其中一組. 進行 3. 觀察 ZFS 的反應.

2.3. 搜尋有 36 個 B 的區塊, 其中一組將 B 改成 D, 另一組將 B 改成 E. 進行 3. 觀察 ZFS 的反應.

3. 將檔案寫回 memory disk
# dd if=/root/md0-edit of=/dev/md0 bs=1m
# zpool import -d /dev test
# zfs scrub test

4. 觀察結果:
4.1. 進行 2.1. 後, vol 1 出現無法修復的錯誤, 並拒絕使用該檔案.

4.2. 進行 2.2. 後, vol 2 發現錯誤並直接修正錯誤(不需 zfs scrub test), 且匯入後可正常使用該檔案. 此時將 zpool test 匯出後並製作 dump file. 可以觀察到手動變更的 D 已經被修改回 B.

4.3. 進行 2.3. 後, vol 2 出現無法修復的錯誤, 並拒絕使用該檔案.

5. 不過這是可修正的軟體錯誤, 如果發生硬體錯誤會是如何處理. 目前尚未想到好方法測試. 不過可偵測錯誤並顯示出來或許對系統管理就蠻有用的. 至少可以提早發現更換硬碟.

Saturday, July 28, 2012

實做安裝FreeBSD 9.0R root on ZFS

基本上本文幾乎和參考資料相近, 差異在部份分割採用 copies=2. 由於實體機器上並沒有光碟機, 所以使用 USB 外接光碟機啟動 LiveCD.
0. 啟動安裝環境
0.1.  Boot 選單直接選 1. Boot
0.2. 中途若遇到找不到光碟機, 請下 cd9660:/dev/cd0 ro .
0.3. 出現安裝選項時, 請選 LiveCD.

1. 硬體
1.1. 系統硬碟 16GB SLC SSD. 快取硬碟 MLC 80GB SSD. 資料硬碟 1TB * 3.
1.2. 預計在系統上啟用 copies=2.

2. 建立 GPT 分割及 root on ZFS
gpart show ada0
gpart destroy ada0
gpart create -s GPT ada0
gpart add -b 34 -s 128 -t freebsd-boot ada0
gpart add -t freebsd-zfs -l disk0 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
gnop create -S 4096 /dev/gpt/disk0
zpool create -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache zroot /dev/gpt/disk0.nop
zpool export zroot
gnop destroy /dev/gpt/disk0.nop
zpool import -o altroot=/mnt -o cachefile=/var/tmp/zpool.cache zroot
zpool set bootfs=zroot zroot
zfs set checksum=fletcher4 zroot
3. 建立子目錄
zfs create zroot/usr
zfs create zroot/usr/home
zfs create zroot/var

zfs create -o compression=on   -o exec=on  -o setuid=off zroot/tmp
zfs create -o compression=lzjb -o setuid=off zroot/usr/ports
zfs create -o compression=off  -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o compression=off  -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/usr/src
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash

zfs create -o exec=off -o setuid=off -o copies=2 zroot/var/db
zfs create -o exec=off -o setuid=off zroot/var/empty
zfs create -o exec=off -o setuid=off zroot/var/run

zfs create -o compression=lzjb -o exec=on  -o setuid=off -o copies=2 zroot/var/db/pkg
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off -o copies=2 zroot/var/mail
zfs create -o compression=lzjb -o exec=on  -o setuid=off zroot/var/tmp
4. 建立 swap
zfs create -V 4G zroot/swap
zfs set org.freebsd:swap=on zroot/swap
zfs set checksum=off zroot/swap
5. 處理暫存目錄及home目錄
chmod 1777 /mnt/tmp
cd /mnt; ln -s usr/home home
chmod 1777 /mnt/var/tmp
6. 複製系統檔案
sh
cd /usr/freebsd-dist
export DESTDIR=/mnt
for file in base.txz lib32.txz kernel.txz doc.txz ports.txz src.txz; do(cat $file | tar --unlink -xpJf - -C ${DESTDIR:-/}); done
cp /var/tmp/zpool.cache /mnt/boot/zfs/zpool.cache
7. 處理開機設定
echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf
echo 'zfs_load="YES"' >> /mnt/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf
touch /mnt/etc/fstab
8. /tmp 改用 TMPFS
8.1. 重新開機後進 single user mode
mount -u /
swapon -a
zfs mount -a
zfs rename zroot/tmp zroot/tmpold
mkdir -p /tmp
chmod 1777 /tmp
8.2. 在 /etc/fstab 加上
# Device        Mountpoint      FStype  Options         Dump    Pass#
tmpfs           /tmp            tmpfs   rw              0       0

參考資料 :
1. ROOT ON ZFS FREEBSD 9 (NON LEGACY MOUNTPOINT – 4K OPTIMIZED)

Thursday, July 26, 2012

ZFS zpool 匯出(export)及匯入(import)

為啥會出現這個問題, 並不是因為換機器. 而是之前 BIOS 都是使用 SATA mode, 導致有兩個 port 的硬碟跑 Ultra DMA 133. 因為不熟 ZFS, 不敢亂玩(其實是怕資料沒了).
今天就大膽一點直接上了, 重開機後發現 BIOS 只抓到四個硬碟, 心裡涼了一半. 不過在 Unix* 環境似乎還可以期待一下系統自己辨識硬碟. 進了 single user mode 看了一下 dmesg 及 /dev, 發現硬碟全數找到了. 只是硬碟代號有部份異動, 剛好動到的都是 ZFS 的硬碟.
立馬回頭改回原設定, 一樣也是進 single user mode 下
zpool export POOL_NAME

再做一次轉換為 AHCI, 換好了以後再進 single user mode. 此時用 zpool import 去偵測發現只找到一個 STO_NAME. 一時原因無法理解, 本來想要就放棄了. 仔細比較磁碟代號, 發現有重複的磁碟代號出現(adX之前是 POOL1 現在變成是 POOL2). 趕緊先執行
zpool import POOL1

重新開機後再用 zpool import 掃描, 果真出現 POOL2. 再執行
zpool import POOL2

收工.

參考資料 :
1. Migrating ZFS Storage Pools Chapter 4. Managing ZFS Storage Pools

ZFS儲存設備更換硬碟

只能說硬碟不能放在太熱的環境(硬碟工作溫度攝氏 45 ~ 58 度), 否則企業級硬碟還是一樣會出問題.這個硬碟倒不是完全讀不到, 只是三不五時就會出現 smartd 的警告訊息.

SMART error (CurrentPendingSector) detected on host: XXXXXX.daemonland.org

This email was generated by the smartd daemon running on:

   host name: XXXXXX.daemonland.org
  DNS domain: daemonland.org
  NIS domain: 

The following warning/error was logged by the smartd daemon:

Device: /dev/ad10, 3 Currently unreadable (pending) sectors


For details see host's SYSLOG.

You can also use the smartctl utility for further investigation.
No additional email messages about this problem will be sent.

只好將備用的 HITACHI 2TB 硬碟上線, 換 WD 硬碟下來維修. 順便打開系統的 AHCI.


1.更換 ZFS RAIDZ1 硬碟
因為沒熱插拔可用所以就重新開機進 single user mode. 然後依照步驟進行:
# zpool offline POOL_NAME DISKID
關機後, 更換故障硬碟.
# zpool online POOL_NAME DISKID
# zpool replace POOL_NAME DISKID

經過漫長的 resilvering (14小時, 可以執行 zpool status POOL_NAME 檢查進度), 終於完成更換硬碟. 沒有機會測試在多使用者模式下, 系統效能會是什麼情景.

2. 設定 AHCI
很簡單只要在 /boot/loader.conf 加上
ataahci_load="YES"
很訝異重新開機後磁碟機代號竟然沒有改變(沒有從 adX 變成 adaX).

參考資料 :
1. ZFS: how to replace a dead disk?
2. [Solved] ZFS cannot replace failed drive

Tuesday, July 24, 2012

使用國高研發的 Clonezilla 複製 SSD

之前不慎買到美光的倒數計時 SSD(5000 小時), 上頭已經裝好了 Ubuntu. 基於懶人性格, 就不想要再安裝一次. 此時想到 clonezilla-live. 趕緊採購 Intel SSD 更換.

剛開始使用一般模式都讀到 99% 就直接吐錯誤訊息, 只好改用專家模式硬著頭皮上, 沒想到還是不行. 但是看到有個 RAW 複製的選項後面接著效率差的警語, 心想效率差總比沒有好. 沒想到竟然成功複製(複製速度 1.1GB/min), 還好 SSD 只是從 60GB 擴展複製到 120GB, 所以耐心等了不少時間終於完成複製 ext4 檔案系統. (圖片稍候再附上)

接下來是要將儲存設備的暫存 SSD(ufs) 從 160GB 換到 180GB, 也是再做一次, 不過這次不用做複製 GRUB.一切搞定.

Friday, July 20, 2012

[備忘] RSA公開金鑰加密演算法

紀錄一下 1. MIT 金鑰產生演算法的理論公式 :

p,q : large prime
n = p * q
z = (p-1) * (q-1)
GCD(d,z) = 1
(e*d) mod z = 1
Public Key : (n,e)
Private Key : (n,d)
2. RSA公開金鑰加密演算法 :

E(M) = M ^ e mod n = C
D(C) = C ^ d mod n = M

Wednesday, March 28, 2012

使用 rsnapshot 備份及比較設定檔

1. 安裝 rsnapshot
在 Ubuntu 下直接用 apt-get install rsnapshot.
在 FreeBSD 下請到 /usr/ports/sysutils/rsnapshot 下 make install clean. (在我的 FreeBSD 實體及虛擬環境中並沒有實際安裝測試, 以下僅用 Ubuntu 作為範例.)

2. 調整設定檔 /etc/rsnapshot.conf
# diff /etc/rsnapshot.conf /etc/rsnapshot.conf.20120322
27c27
< snapshot_root /.rsnapshot/
---
> snapshot_root /var/cache/rsnapshot/
97c97
< retain                hourly  8
---
> retain                hourly  6
100c100
< retain                monthly 6
---
> #retain       monthly 3
124c124
< logfile       /var/log/rsnapshot.log
---
> #logfile      /var/log/rsnapshot.log
230c230
< #backup       /home/          localhost/
---
> backup        /home/          localhost/
232,235c232
< backup        /usr/local/etc/ localhost/
< backup        /var/log/       localhost/
< backup        /var/mail/      localhost/
< backup        /root           localhost/
---
> backup        /usr/local/     localhost/
註 : 因為 /home 已將大部分資料放上 NFS, 已經有 ZFS snapshot. 就不浪費時間多做 snapshot.

3. 設定定期執行
0 */3 * * * /usr/bin/rsnapshot hourly
0 23 * * * /usr/bin/rsnapshot daily
15 23 * * 1 /usr/bin/rsnapshot weekly
30 23 1 * * /usr/bin/rsnapshot monthly
0 0 * * 1 /PATH/TO/REPORT/PROG/report_CHK_ETC_diff.sh

4. CHK_ETC_diff.sh
#!/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        CHKPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSKEYWORD="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSBASE="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 check_path snapshot_keyword snapshot_base_path"
        echo "Example: $0 /etc Weekly /SNAPSHOT/System"
        exit 1
else
        echo "Collect difference between $CHKPATH and $SSBASE/${SSKEYWORD}*$CHKPATH."
fi

date +"%Y/%m/%d %H:%M:%S"
LAST_DIR="/"
CHKPATH=`echo $CHKPATH | sed -e 's/\///'`

for CONF in `find $LAST_DIR$CHKPATH -type f -name '*'`
do
        if [ `ls $SSBASE | grep ${SSKEYWORD} |wc -l` -lt 1 ]; then
                echo "$SSBASE/${SSKEYWORD}* does not existed!"
                exit 1
        fi
        ls -d $SSBASE/${SSKEYWORD}* | while read DIR
        do
                if [ $LAST_DIR = "/" ]; then
                        HOST=""
                else
                        HOST="/localhost"
                fi
                if [ -f ${LAST_DIR}${HOST}${CONF} ]&&[ -f ${DIR}/localhost${CONF} ]; then
                        if [ $LAST_DIR = "/" ]; then
                                LAST_DIR=""
                        fi
                        DIFFC=`diff ${LAST_DIR}${HOST}${CONF} ${DIR}/localhost${CONF} | wc -l`
                        if [ $DIFFC -gt 0 ]; then
                                echo "[[[ Difference between ${LAST_DIR}${HOST}${CONF} and ${DIR}/localhost${CONF} ]]]"
                                diff ${LAST_DIR}${HOST}${CONF} ${DIR}/localhost${CONF}
                                echo ""
                        fi
                        DIFFC=0
                else
                        if [ ! -f ${LAST_DIR}${HOST}${CONF} ]; then
                                echo "File ${LAST_DIR}${HOST}${CONF} does not existed."
                        fi
                fi

                LAST_DIR=$DIR
        done
done
echo ""
date +"%Y/%m/%d %H:%M:%S"

5. report_CHK_ETC_diff.sh
#!/bin/bash

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:${PATH}

CHKPROG="/PATH/TO/PROG/CHK_ETC_diff.sh /etc daily /.rsnapshot"
REPORTPATH="/PATH/TO/STORE/REPORT/report_CHK_ETC_diff"
REPORTKEYWORD="etcdiff"
REPORTTITLE="Check Difference of ETC Report"
REMAILADDR="ACCOUNT@DOMAIN.NAME"

mkdir -p $REPORTPATH/`date +"%Y"`
mv $REPORTPATH/${REPORTKEYWORD}_*.log $REPORTPATH/`date +"%Y"`
$CHKPROG > $REPORTPATH/${REPORTKEYWORD}_`hostname`_`date +"%Y%m%d"`.log
cat $REPORTPATH/${REPORTKEYWORD}_`hostname`_`date +"%Y%m%d"`.log | mail -s "$REPORTTITLE" $REMAILADDR

參考資料:
淺談linux下的快照備份軟體—rsnapshot

Wednesday, March 21, 2012

應用 UFS snapshot 進行比對 /etc 及 /usr/local/etc 下的檔案內容

1. 這隻程式運作的前提是已經完成 UFS snapshot 並且已經正常掛載. 沒有掛載 snapshot 只會看到出現很多找不到檔案的訊息.

2. 程式碼
#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        CHKPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSKEYWORD="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SSBASE="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 check_path snapshot_keyword snapshot_base_path"
        echo "Example: $0 /etc Weekly /SNAPSHOT/System"
        exit 1
else
        echo "Collect difference between $CHKPATH and $SSBASE/${SSKEYWORD}*$CHKPATH."
fi

date +"%Y/%m/%d %H:%M:%S"
LAST_DIR="/"
CHKPATH=`echo $CHKPATH | sed -e 's/\///'`

for CONF in `find $LAST_DIR$CHKPATH -type f -name '*'`
do
        if [ `ls $SSBASE | grep ${SSKEYWORD} |wc -l` -lt 1 ]; then
                echo "$SSBASE/${SSKEYWORD}* does not existed!"
                exit 1
        fi
        ls -d $SSBASE/${SSKEYWORD}* | while read DIR
        do
                if [ -f ${LAST_DIR}${CONF} ]&&[ -f ${DIR}${CONF} ]; then
                        if [ $LAST_DIR = "/" ]; then
                                LAST_DIR=""
                        fi
                        DIFFC=`diff ${LAST_DIR}${CONF} ${DIR}${CONF} | wc -l`
                        if [ $DIFFC -gt 0 ]; then
                                echo "[[[ Difference between ${LAST_DIR}${CONF} and ${DIR}${CONF} ]]]"
                                diff ${LAST_DIR}${CONF} ${DIR}${CONF}
                                echo ""
                        fi
                        DIFFC=0
                else
                        if [ ! -f ${LAST_DIR}${CONF} ]; then
                                echo "File ${LAST_DIR}${CONF} does not existed."
                        fi
                fi

                LAST_DIR=$DIR
        done
done
echo ""
date +"%Y/%m/%d %H:%M:%S"

3. 使用範例
/PATH/TO/CHK_ETC_diff.sh /PATH/TO/SRC SNAPSHOT_KEYWORD /PATH/TO/SNAPSHOT_ROOT

Sunday, March 18, 2012

在 FreeBSD 8 對 UFS filesystem 做 snapshot

1. 在 crontab 定期做 snapshot
### UFS snapshot
0    1      *       *       *       root    /usr/local/sbin/snapshot make -g7 /SRCPATH:Daily
0      2      *       *       1       root    /usr/local/sbin/snapshot make -g4 /SRCPATH:Weekly

2. 掛載 UFS snapshot 的 shell script
#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        SSSRCPATH="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        SnapshotName="$1"
        shift
else
        CHKPARA=1
fi

if [ -n "$1" ]; then
        MountPath="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 Snapshot_Src_Path SnapshotName MountPath"
        echo "Example: $0 /:Weekly.0 /mnt"
        exit 1
else
        echo "Try to mount $SSSRCPATH:$SnapshotName to $MountPath now."
fi

MC=`df -h |grep $MountPath |wc -l`

if [ -d $SSSRCPATH ]; then
        if [ -f $SSSRCPATH/.snap/$SnapshotName ]; then
                if [ -d $MountPath ]; then
                        if [ $MC -gt 0 ]; then
                                echo "MountPath $MountPath had been mounted!"
                        else
                                snapshot mount $SSSRCPATH:$SnapshotName $MountPath
                                df -h |grep $MountPath
                                echo "MountPath $MountPath was mounted."
                        fi
                else
                        echo "MountPath $MountPath does not existed!"
                fi
        else
                echo "Snapshot $SSSRCPATH/.snap/$SnapshotName does not existed!"
        fi
else
        echo "Snapshot source path $SSSRCPATH does not existed!"
fi

3. 掛載範例
# mkdir -p /SNAPSHOT/Mount/Path/Weekly0

/PATH/TO/mount_UFS_snapshot.sh /SRCPATH Weekly.0 /SNAPSHOT/Mount/Path/Weekly0

4. 卸載 UFS snapshot 的 shell script
#!/usr/local/bin/bash

PATH=$PATH
CHKPARA=0

if [ -n "$1" ]; then
        MountPath="$1"
        shift
else
        CHKPARA=1
fi

if [ $CHKPARA -eq 1 ]; then
        echo "Usage: $0 MountPath"
        echo "Example: $0 /mnt"
        exit 1
else
        echo "Try to umount $MountPath now."
fi

MC=`df -h |grep $MountPath |wc -l`

if [ -d $MountPath ]; then
        if [ $MC -gt 0 ]; then
                snapshot umount $MountPath
                df -h |grep $MountPath
                echo "MountPath $MountPath had been umounted!"
        else
                echo "MountPath $MountPath had not been mounted!"
        fi
else
        echo "MountPath $MountPath does not existed!"
fi

5. 卸載範例
/PATH/TO/umount_UFS_snapshot.sh /SNAPSHOT/Mount/Path/Weekly0

6. UFS 做 snapshot 速度還蠻慢的. (相較於 ZFS 的快速完成 snapshot)

參考資料
FreeBSD UFS/ZFS Snapshot Management Environment

Saturday, March 03, 2012

在 FreeBSD 及 Ubuntu 啟動 MLS SSD 支援 TRIM

目前手邊的機器每一台都至少裝了一顆 MLC SSD. 不論是當系統碟或當暫存區使用, 都有需要啟動 TRIM. 剛好 FreeBSD 8.2 及 Ubuntu 11.10 的 kernel 都有支援 TRIM. 所以就參考文件設定.

[[[ FreeBSD 8.2 ]]]
1. Reboot and login in Single User Mode
2. tunefs -p /dev/adXXsXX
3. tunefs -t enable /dev/adXXsXX
4. Reboot the machine
5. Check the setting
HHHHH(XXXXXX)/root# tunefs -p /dev/adXsXX
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         enabled
tunefs: maximum blocks per file in a cylinder group: (-e)  2048
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

[[[ Ubuntu 11.10 ]]]
1. Edit /etc/fstab and add "discard,noatime,nodiratime,".
root@HOSTNAME:~# diff /etc/fstab.bak /etc/fstab
10c10
< UUID=dXXXdXXX-5899-XXXX-XXXd-XXXXcXXX8XXX /               ext4    errors=remount-ro 0       1
---
> UUID=dXXXdXXX-5899-XXXX-XXXd-XXXXcXXX8XXX /               ext4    discard,noatime,nodiratime,errors=remount-ro 0       1

2. Edit /etc/rc.local and add following statements before 'exit 0'.
### For tweak SSD
echo noop > /sys/block/sda/queue/scheduler
echo 1 > /sys/block/sda/queue/iosched/fifo_batch

3. Reboot the machine

4. 參考這篇做測試. HOWTO: Check If TRIM On Ext4 Is Enabled And Working On Ubuntu And Other Distributions


參考資料 Reference:
TRIM support on UFS2 FreeBSD 9
Racklin's 阿土伯程式大觀園: 為 Thinkpad T61 Debian Linux 換上 SSD
HOWTO: Configure Ext4 to Enable TRIM Support for SSDs on Ubuntu and Other Distributions
HOWTO: Check If TRIM On Ext4 Is Enabled And Working On Ubuntu And Other Distributions
How to Tweak Your SSD in Ubuntu for Better Performance

在 FreeBSD, PC-BSD, Ubuntu, Mint, CentOS, Fedora, OpenSUSE, Debian 上使用 tmpfs 掛載 /tmp

首先說明一下, 以下是給懶人專用, RAM Disk 大小都是預設最大到 50% 記憶體. 如果有需要調整空間使用上限或在大型伺服器上實做, 請務必研究一下附件還有 tmpfs. 另外 NetBSD 5, OpenSolaris 及 Indiana 就不用傷腦筋了, 系統預設已經使用 tmpfs(在 NetBSD 使用 mfs) 掛載 /tmp.

由於現代的機器的記憶體基本上都是 GB 等級. 讓記憶體閒著也是一種浪費. 而更重要的原因是不想讓讀寫 /tmp 的動作大量存取 MLC SSD(雖然已經啟動了 TRIM). 再加上 guest VM 常常讀寫, 多跑幾台對 VM host 還是有效能上的影響. 就這樣看著大家的文件跟著做. 也順手將手邊的測試用 guest VM 加上這個簡單的設定. 以下就是實際上在各平台上的設定方式:

[[[ FreeBSD 8, PC-BSD 8 & 9 beta ]]]
1. Edit /etc/fstab
tmpfs  /tmp  tmpfs  rw 0 0

2. Reboot the machine

[[[ Ubuntu 11.X 12.04 LTS, Mint 11 ]]]
1. Edit /etc/init.d/ramtmp.sh
#!/bin/sh
# RamDisk tmp
PATH=/sbin:/bin:/usr/bin:/usr/sbin

mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp
mount --bind /dev/shm/tmp /tmp

2. chmod 755 /etc/init.d/ramtmp.sh
3. cd /etc/rcS.d; ln -s /etc/init.d/ramtmp.sh SXXramtmp
XX : 必須在所有需要 /tmp 目錄的服務之前. 請注意英文字母排序.
簡單的作法就是在 /etc/rcS.d 下 grep tmp ./* 看有哪些檔案需要 /tmp 目錄.
4. Reboot the machine

[[[ CentOS 6, Fedora 15, ScientificLinux 5.6 ]]]
1. Edit /etc/rc.d/init.d/ramtmp
#!/bin/bash
# chkconfig: 35 80 70
# description: RAMDisk4Tmp
#
# History:
### 2012/02/25 RamDisk tmp

PATH=/sbin:/bin:/usr/bin:/usr/sbin

mkdir /dev/shm/tmp
chmod 1777 /dev/shm/tmp
mount --bind /dev/shm/tmp /tmp

2. chmod 755 /etc/rc.d/init.d/ramtmp
3. chkconfig --add ramtmp
4. Reboot the machine

[[[ OpenSUSE 11.4 ]]]
1. Edit /etc/fstab
tmpfs  /tmp  tmpfs  defaults 0 0

2. Reboot the machine

[[[ Smoothwall express 3.0 sp2 ]]]
1. Edit /etc/fstab
none  /tmp  tmpfs  defaults 0 0

2. Reboot the machine

[[[ ArchLinux 3.0 ]]]
1. Edit /etc/fstab
tmpfs  /tmp  tmpfs  nodev,nosuid 0 0

2. Reboot the machine

[[[ Debian 6 ]]]
1. Edit /etc/fstab
tmpfs /tmp tmpfs defaults,nodev,nosuid 0 0

2. Reboot the machine

[[[ Windows 7 Ultimate, Windows server 2003, Windows XP ]]]
請安裝 Dataram_RAMDisk_V3.5.130R22, 詳細步驟請看參考資料.

參考資料 Reference:
FreeBSD : 【FreeBSD】用 tmpmfs 或 tmpfs 建立 Ramdisk
Ubuntu : Firefox & Chrome Cache on RAM Drive -Fedora / Ubuntu
                 在 Ubuntu 將 Firefox 網頁快取檔,設到 Ramdisk 的方法
CentOS : [筆記]Linux Tmpfs 系統加速實作
OpenSUSE : SDB:SSD performance
Debian : HOWTO: Using tmpfs for /tmp
Windows : [桌面系統]Windows 7 (32bit/64bit) 使用RamDisk 外加ReadyBoost 增加效能