一、xsync同步脚本
xsync脚本是复制单个文件或者目录到其他服务器相同目录的快捷高效的脚本,我们需要在在/usr/local/bin
目录下创建xsync
文件。如果我们的服务器的hostname是hadoop001、hadoop002、hadoop003,那么文件内容如下写便可:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
| #! /bin/bash #1. 判断参数个数 if [ $# -lt 1 ] then echo 请输入需要同步到集群的文件名! exit; fi
#2. 遍历集群所有机器 for host in hadoop001 hadoop002 hadoop003 do echo ======================== $host ======================== #3. 遍历所有目录,挨个发送文件 for file in $@ do #4. 判断文件是否存在 if [ -e $file ] then #5. 获取父目录 pdir=$(cd -P $(dirname $file); pwd) # 获取当前文件的名称 fname=$(basename $file) ssh $host "mkdir -p $pdir" rsync -av $pdir/$fname $host:$pdir else echo ${file}文件不存在! fi done done
|
对xsync文件赋予执行权限:
假如我们需要将hive目录同步到其他节点,则在hive的父目录下执行如下命令(注意,使用前提是先配置好多个节点的ssh互访):
1
| /usr/local/bin/xsync hive
|
二、xcall脚本
xcall脚本是类型在各个节点执行jps的查看jvm进程的效果。在/usr/local/bin
目录下创建文件xcall
,同样我们假定服务器节点的hostname是hadoop001、hadoop002、hadoop003,那么文件内容如下写便可:
1 2 3 4 5
| #!/bin/bash for host in hadoop001 hadoop002 hadoop003 do echo =================== $host ================= ssh $host jps
|
对xcall文件赋予执行权限:
我们查看上面三个节点中的jvm进程:
效果如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61
| =================== hadoop001 ================= 28384 Bootstrap 262148 Kafka 15395 udp-agent-1.0.0.jar 273504 ThriftServer 50406 WorkerServer 515439 ResourceManager 879749 RunJar 270445 TaierApplication 511310 NodeManager 202600 NameNode 880636 RunJar 22068 HttpFSServerWebServer 341811 udp-server-1.0.0.jar 20086 QuorumPeerMain 50423 LoggerServer 273715 HRegionServer 754579 EmbeddedServer 274430 HMaster 20540 JournalNode 926355 DataNode 317754 Jps 21631 DFSZKFailoverController =================== hadoop002 ================= 28096 NameNode 28288 Kafka 32546 HistoryServer 49893 HttpFSServerWebServer 51109 DataNode 36521 NodeManager 37326 ResourceManager 26671 zkui.jar 493431 RunJar 46866 Elasticsearch 27349 WorkerServer 35446 ApplicationHistoryServer 978196 Jps 44826 HMaster 52186 JournalNode 8060 udp-agent-1.0.0.jar 24349 HistoryServer 498522 RunJar 27902 DFSZKFailoverController 26911 QuorumPeerMain =================== hadoop003 ================= 8161 Jps 969133 JobHistoryServer 846112 WorkerServer 845155 AlertServer 846147 LoggerServer 746084 -- process information unavailable 818300 Kafka 825406 HRegionServer 19443 QuorumPeerMain 968248 NodeManager 13335 udp-agent-1.0.0.jar 844630 ApiApplicationServer 351679 Elasticsearch 19867 JournalNode 846736 MasterServer 20670 DataNode
|
最后,我们通过xsync把xsync和xcall两个脚本同步到其他节点,这样在上述三个节点任意执行这两个命令,均可生效。
1 2
| cd /usr/local/bin xsync xsync xcall
|