最近在Bluemix的虛擬機上玩docker,但是發現默認安裝使用的是loop-lvm的模式做後端存儲,這個肯對對後面的實驗會造成影響,而且docker官方也不建議在生產環境下使用loop-lvm,下圖為docker官方給出的存儲方案優劣對比:
所以今天開題分享一下如何更改docker的devicemapper存儲方式為direct-lvm。
docker 最先是跑在ubuntu和debian上的,使用aufs存儲器. 由於docker越來越流行,許多公司希望在RHEL上使用,但是上游內核中沒有包括aufs,所以rhel不能使用aufs. 最終開發者們開發了一個新的後端存儲引擎devicemapper,基於已有的Device Mapper技術,並且使docker支持可插拔,現在全世界有很多真實案例在生產環境使用devicemapper。Device Mapper是Linux系統中基於內核的高級卷管理技術框架。Docker的devicemapper存儲驅動就是基於該框架的精簡置備和快照功能來實現鏡像和容器的管理。
devicemapper是Red Hat Enterprise Linux下Docker Engine的默認存儲驅動,它有兩種配置模式:loop-lvm和direct-lvm,loop-lvm是默認的模式,但如果是在生產環境的部署Docker,官方不推薦使用該模式。我們使用docker info命令可以看到以下警告:
WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `–storage-opt dm.thinpooldev` or use `–storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
direct-lvm是Docker推薦的生產環境的推薦模式,他使用塊設備來構建精簡池來存放鏡像和容器的數據。
本文的操作系統是Centos7.2,使用docker版本為1.12,devicemapper版本為device-mapper-1.02.107-5.el7.x86_64。首先運行docker info 查看當前系統的docker配置:
# root at bastion.shanker in ~ [15:03:10] # docker info Containers: 6 Running: 0 Paused: 0 Stopped: 6 Images: 40 Server Version: 1.10.3 Storage Driver: devicemapper Pool Name: docker-253:0-11668417-pool Pool Blocksize: 65.54 kB Base Device Size: 10.74 GB Backing Filesystem: xfs Data file: /dev/loop0 Metadata file: /dev/loop1 Data Space Used: 3.641 GB Data Space Total: 107.4 GB Data Space Available: 103.7 GB Metadata Space Used: 5.972 MB Metadata Space Total: 2.147 GB Metadata Space Available: 2.142 GB Udev Sync Supported: true Deferred Removal Enabled: true Deferred Deletion Enabled: true Deferred Deleted Device Count: 0 Data loop file: /var/lib/docker/devicemapper/devicemapper/data WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning. Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata Library Version: 1.02.107-RHEL7 (2015-10-14) Execution Driver: native-0.2 Logging Driver: journald Plugins: Volume: local Network: bridge null host Kernel Version: 4.7.9-200.fc24.x86_64 Operating System: Fedora 24 (Twenty Four) OSType: linux Architecture: x86_64 Number of Docker Hooks: 2 CPUs: 8 Total Memory: 14.99 GiB Name: bastion.shanker ID: SWHZ:KSZ3:CQMS:W5HN:F33Z:HWU2:2GUE:OQII:7BSE:J62P:6EMX:RHL4 Username: shanker Registry: https://index.docker.io/v1/ Registries: docker.io (secure)
發現Storage Driver是Devicemapper,Data File和Metadata File都是loop設備,下面我們將docker停掉:
#systemctl stop docker
查看新添加的磁盤
fdisk -l /dev/sdb Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes
創建pv
# root at model.shanker in ~ [14:36:01] # pvcreate /dev/sdb Physical volume "/dev/sdb" successfully created
創建vg
# root at model.shanker in ~ [14:37:49] # vgcreate docker /dev/sdb Volume group "docker" successfully created
創建data lv
# root at model.shanker in ~ [14:41:00] # lvcreate --wipesignatures y -n thinpool docker -l 95%VG Logical volume "thinpool" created.
創建metadata lv
# root at model.shanker in ~ [14:41:06] # lvcreate --wipesignatures y -n thinpoolmeta docker -l 1%VG Logical volume "thinpoolmeta" created.
注意作為meta的pool大小不能超過16GB!!!
將pool 轉換為thin-pool
將 thinpool lv 的 chunksize 改為 512KB,並且將前 4KB 字節清零。
root at model.shanker in ~ [14:42:31] # lvconvert -y --zero n -c 512k --thinpool docker/thinpool --poolmetadata docker/thinpoolmeta WARNING: Converting logical volume docker/thinpool and docker/thinpoolmeta to pool's data and metadata volumes. THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.) Converted docker/thinpool to thin pool.
創建一個thinpool的profile
# root at model.shanker in /etc/lvm/profile [14:45:42] # vi docker-thinpool.profile # root at model.shanker in /etc/lvm/profile [14:45:50] # pwd /etc/lvm/profile # root at model.shanker in /etc/lvm/profile [14:45:53] # cat docker-thinpool.profile activation { thin_pool_autoextend_threshold=80 thin_pool_autoextend_percent=20 }
應用配置
# root at model.shanker in /etc/lvm/profile [14:45:56] # lvchange --metadataprofile docker-thinpool docker/thinpool Logical volume "thinpool" changed.
注意: docker-thinpool 即剛才創建的 profile 文件名的前綴,不需要加.profile,而且要在