本文共 4074 字,大约阅读时间需要 13 分钟。
需要使用Docker虚拟化Hadoop/Spark等测试环境,并且要可以对外提供服务,要求是完全分布式的部署(尽量模拟生产环境)。那么我们会遇到几个问题:
Container IP 是动态分配的
Container IP 是内部IP,外部无法访问(如对外提供HDFS服务可能会遇到Client无法访问DataNode,因为DataNode注册的是内部IP)
针对第一个问题有不少的方案,可以指定静态的IP,对第二个问题,我们可以使用--net=host解决,但这会导致对外只有一个IP,集群各个Slave的端口都要修改。至于pipework简单地看了下,好像也解决不了。
所以目前看上去只能使用看上去不是很优雅的方案解决:*为Docker宿主网卡绑定多个IP,把这些IP分配给不同的容器。
//这是示例,我是在windows下用Docker toolbox运行的
root@default:~# ifconfigdocker0 Link encap:Ethernet HWaddr 02:42:8C:8E:80:F1inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0UP BROADCAST MULTICAST MTU:1500 Metric:1RX packets:0 errors:0 dropped:0 overruns:0 frame:0TX packets:0 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:0RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)eth0 Link encap:Ethernet HWaddr 08:00:27:24:D1:F5
inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0inet6 addr: fe80::a00:27ff:fe24:d1f5/64 Scope:LinkUP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1RX packets:222 errors:0 dropped:0 overruns:0 frame:0TX packets:164 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:32277 (31.5 KiB) TX bytes:28136 (27.4 KiB)eth1 Link encap:Ethernet HWaddr 08:00:27:76:1D:9B
inet addr:192.168.99.100 Bcast:192.168.99.255 Mask:255.255.255.0inet6 addr: fe80::a00:27ff:fe76:1d9b/64 Scope:LinkUP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1RX packets:66 errors:0 dropped:0 overruns:0 frame:0TX packets:64 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:9001 (8.7 KiB) TX bytes:10469 (10.2 KiB)lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0inet6 addr: ::1/128 Scope:HostUP LOOPBACK RUNNING MTU:65536 Metric:1RX packets:16 errors:0 dropped:0 overruns:0 frame:0TX packets:16 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:0RX bytes:1152 (1.1 KiB) TX bytes:1152 (1.1 KiB)//eth1网卡是可以与外部交互,所以我们添加IP到这个网卡上
//第一步:添加了两个IProot@default:~# ifconfig eth1:0 192.168.99.10 netmask 255.255.255.0 uproot@default:~# ifconfig eth1:1 192.168.99.11 netmask 255.255.255.0 up//再次查看,多了两个IProot@default:~# ifconfig...eth1 Link encap:Ethernet HWaddr 08:00:27:76:1D:9B
inet addr:192.168.99.100 Bcast:192.168.99.255 Mask:255.255.255.0inet6 addr: fe80::a00:27ff:fe76:1d9b/64 Scope:LinkUP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1RX packets:2258 errors:0 dropped:0 overruns:0 frame:0TX packets:1685 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:207033 (202.1 KiB) TX bytes:209587 (204.6 KiB)eth1:0 Link encap:Ethernet HWaddr 08:00:27:76:1D:9B
inet addr:192.168.99.10 Bcast:192.168.99.255 Mask:255.255.255.0UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1eth1:1 Link encap:Ethernet HWaddr 08:00:27:76:1D:9B
inet addr:192.168.99.11 Bcast:192.168.99.255 Mask:255.255.255.0UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1...//第二步:运行容器,指定IP,这里的示例容器开启的SSH服务,后面拿它测试
root@default:~# docker run -d -p 192.168.99.10:222:22 --name ssh1 gudaoxuri/scala-2.11-envroot@default:~# docker run -d -p 192.168.99.11:222:22 --name ssh2 gudaoxuri/scala-2.11-envroot@default:~# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESab024af9c954 gudaoxuri/scala-2.11-env "/usr/sbin/sshd -D" 4 seconds ago Up 3 seconds 192.168.99.11:222->22/tcp ssh2259351134d16 gudaoxuri/scala-2.11-env "/usr/sbin/sshd -D" 15 seconds ago Up 14 seconds 192.168.99.10:222->22/tcp ssh1//测试连接,在Docker宿主机上SSH到第一个容器
root@default:~# ssh 192.168.99.10 -p222The authenticity of host '[192.168.99.10]:222 ([192.168.99.10]:222)' can't be established.RSA key fingerprint is ac:fe:4b:89:f8:51:b7:e9:9c:34:62:f9:80:38:4b:bf.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added '[192.168.99.10]:222' (RSA) to the list of known hosts.root@192.168.99.10's password:Last login: Wed Oct 7 13:12:35 2015 from 192.168.99.1//成功进入[root@259351134d16 ~]#//在第一个容器中SSH到第二个容器[root@259351134d16 ~]# ssh 192.168.99.11 -p222root@192.168.99.11's password:Last login: Wed Oct 7 13:14:53 2015 from 172.17.42.1//也OK了[root@ab024af9c954 ~]#利用Docker搭建大数据处理集群
转载于:https://blog.51cto.com/lookingdream/2057050