• 1. Dockerizing SequoiaDBJiang yong  55041099@qq.com2015.5
  • 2. Docker实战心得 为何使用Docker? Docker技巧 Docker实战遇到的坑 Coreos实战心得 为何使用Coreos? Etcd集群使用心得 Flannel构建docker集群内容提要2SequoiaDB实战心得 为何使用SequoiaDB? 为何使用Docker+Coreos部署SequoiaDB? SequoiaDB Docker镜像定制 SequoiaDB Docker下的使用及维护 SequoiaDB的性能优化 后续工作
  • 3. 为何使用Docker?3Docker vs. Virtual Machine
  • 4. Docker Technology4libvirt: Platform Virtualization LXC (LinuX Containers): Multiple isolated Linux systems (containers) on a single host Layered File System
  • 5. Run Platforms5
  • 6. Image Container Daemon Container Mount Volumes Publish Port Link Containers Dockerfile Docker HubTerminology6
  • 7. Docker技巧7Docker vs. Virtual Machine
  • 8. Docker实战遇到的坑8Docker镜像过大,加载缓慢 Docker镜像中文支持较少,需要自定义 Docker Container IP地址动态变化 Docker Container内部通信 Docker Container访问互联网 Docker Container挂掉导致服务不可用 ……
  • 9. Lightweight OS based on Gentoo Linux Has a distributed key-value store at the core Read-only rootfs. Writeable /etc All services are in containers 为何使用Coreos?
  • 10. 采用etcd构建内部dnsEtcd集群使用心得10FROM jiangyong/docker-alpine RUN apk add git go \ && git config --global http.sslverify "false" \ && GOPATH=/go go get github.com/mrwilson/helixdns \ && cd /bin \ && GOPATH=/go go build github.com/mrwilson/helixdns \ && rm -rf /go \ && apk del go git \ && rm -rf /var/cache/apk/* /usr/local/src/* EXPOSE 53 CMD [ "/bin/helixdns", "-port", "53", "-etcd-address", "127.0.0.1:4001" ]
  • 11. 将dns作为CoreOS系统服务自动运行Etcd集群使用心得11 - name: hgdns.service command: start content: | [Unit] Description=DNS Service After=docker.service Requires=docker.service [Service] EnvironmentFile=/etc/network-environment ExecStartPre=-/usr/bin/docker kill hgdns ExecStartPre=-/usr/bin/docker rm -f hgdns ExecStart=/usr/bin/docker run --name=hgdns --net=host --privileged jiangyong/docker-helixdns /bin/helixdns -port 53 -etcd-address http://127.0.0.1:2379 ExecStop=/usr/bin/docker stop hgdns Restart=on-failure RestartSec=5
  • 12. Etcd集群使用心得12
  • 13. One CIDR subnet per machine, like Kubernetes Host 1: 10.10.10.0/24 Host 2: 10.10.11.0/24 No Docker port-based mapping Containers reach each other through IP Peer network configs exchanged over etcd Packets encapsulated using UDP, and soon VxLANFlannel构建docker集群13
  • 14. (本页无文本内容)
  • 15. Build flannel on each host Set key in etcd for network configInstructions to Run Flannel15$ curl -L http://127.0.0.1:4001/v2/keys/coreos.com/network/config -XPUT -d value='{ "Network": "10.0.0.0/8", "SubnetLen": 20, "SubnetMin": "10.10.0.0", "SubnetMax": "10.99.0.0", "Backend": {"Type": "udp", "Port": 7890}}$ git clone https://github.com/coreos/flannel.git $ cd flannel $ docker run -v `pwd`:/opt/flannel -i -t google/golang /bin/bash -c "cd /opt/flannel && ./build"
  • 16. Start flannel. flanneld port created and route is set for the full flat IP range. Instructions to Run Flannel (contd.)16$ sudo ./bin/flanneld & Output: I1219 17:34:41.159822 00809 main.go:247] Installing signal handlers I1219 17:34:41.160030 00809 main.go:118] Determining IP address of default interface I1219 17:34:41.160579 00809 main.go:205] Using 192.168.111.14 as external interface I1219 17:34:41.212157 00809 subnet.go:83] Subnet lease acquired: 10.12.224.0/20 I1219 17:34:41.217829 00809 main.go:215] UDP mode initialized I1219 17:34:41.218953 00809 udp.go:239] Watching for new subnet leases I1219 17:34:41.219349 00809 udp.go:264] Subnet added: 10.13.128.0/20 core@coreos-05 ~ $ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.111.1 0.0.0.0 UG 1024 0 0 eth0 10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 flannel0 10.12.224.0 0.0.0.0 255.255.240.0 U 0 0 0 docker0 192.168.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
  • 17. Restart docker daemon with appropriate bridge IP Instructions to Run Flannel (contd.)17$ source /run/flannel/subnet.env $ sudo ifconfig docker0 ${FLANNEL_SUBNET} $ sudo docker -d --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} &
  • 18. Ping between two bash containers on two different hosts succeeds. The traffic on wire is encapsulated by flanneldFlannel构建docker集群18 192.168.111.14Docker0 10.12.224.1bash 192.168.111.13Docker0 10.13.128.1bash$ docker run -i -t ubuntu /bin/bash root@36484def3b03:/# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 02:42:0a:0c:e0:02 inet addr:10.12.224.2 Bcast:0.0.0.0 Mask:255.255.240.0 root@36484def3b03:/# ping 10.13.128.2 Success!$ docker run -i -t ubuntu /bin/bash root@e0b9dd20d146:/# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 02:42:0a:0d:80:02 inet addr:10.13.128.2 Bcast:0.0.0.0 Mask:255.255.240.0
  • 19. IP address overlap not possible VxLAN not used to create container groups User-space encapsulation and forwarding Potential performance bottleneck Limitations19
  • 20. SequoiaDB vs. MongoDB vs. Cassandra vs. Hbase 高性能 支持事务 开源为何使用SequoiaDB?20
  • 21. 为何使用Docker+Coreos+SequoiaDB?21零配置 一键使用
  • 22. FROM centos:7 RUN yum install -y net-tools gcc tar wget openssh-server passwd initscripts RUN yum swap -y fakesystemd system #修改密码 RUN groupadd -r sdbadmin && useradd -r -g sdbadmin sdbadmin RUN echo 'root:sdbadmin' | chpasswd RUN echo 'sdbadmin:sdbadmin' | chpasswd RUN rm -f /etc/sysctl.conf ADD sysctl.conf /etc/sysctl.conf RUN rm -f /etc/security/limits.conf RUN rm -f /etc/security/limits.d/90-nproc.conf ADD etc/security/limits.conf /etc/security/limits.conf ADD etc/security/limits.d/90-nproc.conf /etc/security/limits.d/90-nproc.conf SequoiaDB Docker镜像定制22
  • 23. RUN mkdir -p /opt/sequoiadb RUN wget -P /opt http://192.168.1.130/docker/sequoiadb.tar.gz RUN tar -xzvf /opt/sequoiadb.tar.gz -C /opt RUN rm -f /opt/sequoiadb.tar.gz RUN chmod u+x /opt/sequoiadb/sequoiadb ADD etc/default/sequoiadb /etc/default/sequoiadb RUN cp /opt/sequoiadb/sequoiadb /etc/init.d/sdbcm RUN chkconfig --add sdbcm #开放端口 EXPOSE 8000 11780 11785 11790 11890 22 8080 VOLUME ["/opt/sequoiadb/database"] SequoiaDB Docker镜像定制23
  • 24. #设置时区 RUN rm -f /etc/localtime RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime # 添加新的数据库实例 RUN mkdir /opt/sequoiadb/conf/local/11890 ADD hgsdb/sdb.conf /opt/sequoiadb/conf/local/11890/sdb.conf #设置启动点 ADD entrypoint.sh /entrypoint.sh ENTRYPOINT ["/entrypoint.sh"]SequoiaDB Docker镜像定制24
  • 25. #!/bin/bash #fix file permissions chown -R root:root /opt/sequoiadb/database /etc/init.d/sdbcm start && /opt/sequoiadb/bin/sdbwsart -S 0.0.0.0:8080 && /usr/sbin/sshd -Dentrypoint.sh25
  • 26. - name: hgsdb.service command: start content: | [Unit] Description=Sequoiadb Service After=docker.service Requires=docker.service [Service] EnvironmentFile=/etc/network-environment ExecStartPre=-/usr/bin/docker kill hgsdb ExecStartPre=-/usr/bin/docker rm -f hgsdb ExecStartPre=/usr/bin/docker run -d --dns=${DEFAULT_IPV4} --add-host=hostip:${DEFAULT_IPV4} --name=hgsdb -h=sdb.hg.com -p 8000:8000 -p 11790:11790 -p 11890:11890 -p 2481:8080 -v /opt/hgsdb:/opt/sequoiadb/database jiangyong/docker-sequoiadb ExecStart=/usr/bin/docker attach hgsdb ExecStartPost=/opt/bin/addDNSRecord.sh hgsdb sdb.hg.com ExecStop=/usr/bin/docker stop hgsdb Restart=on-failure RestartSec=5SequoiaDB Docker下的使用及维护26
  • 27. - name: hgsdb.service command: start content: | [Unit] Description=Sequoiadb Service After=docker.service Requires=docker.service [Service] EnvironmentFile=/etc/network-environment ExecStartPre=-/usr/bin/docker kill hgsdb ExecStartPre=-/usr/bin/docker rm -f hgsdb ExecStartPre=/usr/bin/docker run -d --dns=${DEFAULT_IPV4} --add-host=hostip:${DEFAULT_IPV4} --name=hgsdb -h=sdb.hg.com -p 8000:8000 -p 11790:11790 -p 11890:11890 -p 2481:8080 -v /opt/hgsdb:/opt/sequoiadb/database jiangyong/docker-sequoiadb ExecStart=/usr/bin/docker attach hgsdb ExecStartPost=/opt/bin/addDNSRecord.sh hgsdb sdb.hg.com ExecStop=/usr/bin/docker stop hgsdb Restart=on-failure RestartSec=5SequoiaDB Docker下的使用及维护27
  • 28. SequoiaDB Docker下的使用及维护28
  • 29. SequoiaDB Docker下的使用及维护29
  • 30. 在docker中搭建SequoiaDB的集群 测试海量数据下SequoiaDB的性能后续工作30
  • 31. Thank you.jiang yong © 2015 Copyright Reserved