Shell常用脚本整理

发表于 | 更新于: | 分类于 ， |
字数统计: 1k | 阅读时长 ≈ 5

读取文件每一行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#!/bin/bash
# 方法1
while read line
do
    echo $line
done < filename(待读取的文件)

# 方法2
cat filename(待读取的文件) | while read line
do
    echo $line
done

# 方法3
for line in `cat filename(待读取的文件)`
do
    echo $line
done

变量自增

写循环时，常常要用到变量的自增，现在总结一下整型变量自增的方法：

1
2
3
4
5
6
7
8
9
i=`expr $i + 1`

let i+=1;

((i++));

i=$[$i+1];

i=$(( $i + 1 ))

字符串去空格

（1）去行首空格

sed 's/^[ \t]*//g'

（2）去行尾空格

sed 's/[ \t]*$//g'

（3）去所有空格

sed 's/[[:space:]]//g'

shell参数扩展

  • 如果parameter为null或者未设置，整个参数替换表达式值为word 

    1
    ${parameter:-word}

  • 如果parameter为null或者未设置，整个参数替换表达式值为word，并且parameter参数值设置为word 

    1
    ${parameter:=word}

  • 如果parameter为null或者未设置，则打印出错误信息。否则，整个参数替换表达式值为$parameter 

    1
    ${parameter:?word}

  • 如果parameter不为null或者未设置，则整个参数替换表达式值为word 

    1
    ${parameter:+word}
  • 获得字符串的长度 
    1
    ${#parameter}

  • 从尾开始扫描word，将匹配word正则表达式的字符过滤掉, %为最短匹配，%%为最长匹配 

    1
    2
    ${parameter%word}
    ${parameter%%word}

  • 从头开始扫描word，将匹配word正则表达的字符过滤掉, #为最短匹配，##为最长匹配 

    1
    2
    ${parameter#word}
    ${parameter##word}

  • 截取字符串,截取parameter的值的子字符串 

    1
    2
    ${parameter:offset}   从offset到结束
    ${parameter:offset:length}   从offset开始截取length个

  • 字符串替换，将parameter对应值的pattern字符串替换成为string字符串, /表示只替换一次，//表示全部替换 

    1
    2
    ${parameter/pattern/string}
    ${parameter//pattern/string}

Supervise进程监控

supervise 监控进程，如果挂了，重启进程

1
2
3
4
5
6
7
-p,  状态路径，supervise运行时会在该路径下保存一些文件
-f,   启动新的进程命令
-F,   配置文件路径 the config file path, if not given, supervise will use the default value "supervise.conf"
-r,   while the service exits, supervise will excute the program RESTART_SH before restart the service, and the exiting times be passed          to RESTART_SH as the only argument.
 -t,   after TIME_LIMIT seconds, the counter for the exiting times will be reset.
 -v,  print the version of supervise and exit.
 -h,  print the help message and exit.

示例：

1
supervise -p ../supervise/status/hdfs-dumper -f "sh ./control.sh start" -F ../supervise/conf/supervise.conf

远程操作

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#!/bin/bash

LIST_FILE=hdfsdumper.num.list

size=`cat ./$LIST_FILE | wc -l`

line_index=1
while [ $line_index -le $size ]
do
    machine=`cat ./$LIST_FILE | sed -n -e "${line_index}p" | awk -F' ' '{print $1}'`
    number=`cat ./$LIST_FILE | sed -n -e "${line_index}p" | awk -F' ' '{print $2}'`
    nohup expect -c "
        spawn ssh root@$machine;
        expect {
        \"Are you sure you want to continue connecting (yes/no)? \" {send \"yes\r\";exp_continue}
        \"*password: \" {send \"password\n \"; exp_continue}
        \"]# \" {send \"su work; exit\n\"; exp_continue}
        \"bash-4.1$*\" {send \"cd /home/work/xxx && sh control.sh start; exit\n\"; exp_continue}
        \"]$ \" {send \"cd /home/work/xxx && sh control.sh start; exit\n\"; exp_continue}
        }" &
    line_index=`expr $line_index + 1`
done

远程同步文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#!/bin/bash

LIST_FILE=hdfsdumper.num.list

size=`cat ./$LIST_FILE | wc -l`

file_name=./dumper/jdk.tar.gz

line_index=1
while [ $line_index -le $size ]
do
    machine=`cat ./$LIST_FILE | sed -n -e "${line_index}p" | awk -F' ' '{print $1}'`
    number=`cat ./$LIST_FILE | sed -n -e "${line_index}p" | awk -F' ' '{print $2}'`
    expect -c "
        spawn rsync -avz $file_name root@$machine:/home/work/opencrawler
        expect {
        \"Are you sure you want to continue connecting (yes/no)? \" {send \"yes\r\";exp_continue}
        \"*password: \" {send \"password\n \"; interact}; }";
    line_index=`expr $line_index + 1`
done

switch case

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
cmd=$1
case $cmd in
start)
    echo "in start command."
    if [ -f "running_tag.txt" ];then
        echo "running_tag file is exist, already running!"
        exit 1
    fi  
    nohup java -Xms1000m -Xmx1000m -Xmn500m -XX:+UseParallelGC ${MAIN_CLASSNAME} >>log/crontabservice_terminal.log 2>&1 &
    ret=$?
    if [ $ret -eq 0 ];then
        echo "start [OK]."
    else
        echo "start [FAILED]."
    fi
    ;;
stop)
    echo "in stop command."
    ret=`ps aux | grep "${MAIN_CLASSNAME}" | grep -v "grep" | awk '{print $2}'`
    if [ ! $ret ];then
        echo "MODULE:${MAIN_CLASSNAME} not started."
        exit 1
    fi
    echo "pid:$ret"
    kill -9 $ret
    if [ $? -eq 0 ];then
        rm -f running_tag.txt
    fi
    rm -f running_tag.txt
    echo "Stop [OK]."
    ;;
*)
    help
    exit 1
    ;;
esac