apt-get install libssl-dev build-essential zlib1g-dev git git clone http://www.slac.stanford.edu/~abh/bbcp/bbcp.git cd bbcp/src vi Makefile Line : @+make makeLinux`/bin/uname -i`\\ Replace by : @+make makeLinuxx86_64 make Linux cd .. sudo cp bin/amd64_linux/bbcp /bin sudo chown root:root /bin/bbcp sudo chmod 755 /bin/bbcp ===== Howto use ===== http://pcbunn.cithep.caltech.edu/bbcp/using_bbcp.htm\\ doc : http://www.slac.stanford.edu/~abh/bbcp\\ gist : https://gist.github.com/brianv0/a3bc98bf1fb6fe63d16a7a7cc2ab2a96 ===== Upload ===== # bbcp --ipv4 --port 7777 --verbose --progress 5 --streams 60 --windowsz 128K --append --gross --mkdir --readable --recursive --symlinks keep --sync dd --xfrrate 50M /home/user/isos/plop/tmp/14393.0.161119-1705.RS1_REFRESH_SERVER_EVAL_X64FRE_FR-FR.ISO root@example.com:/root/tmp/ bbcp: Indexing files to be copied... bbcp: Copying 0 files and 0 links in 0 directories. bbcp: Appending to /root/tmp/14393.0.161119-1705.RS1_REFRESH_SERVER_EVAL_X64FRE_FR-FR.ISO at offset 0 bbcp: 180515 15:52:30 2% done; 38.2 MB/s, avg 38.2 MB/s bbcp: 180515 15:52:34 5% done; 38.8 MB/s, avg 38.5 MB/s bbcp: 180515 15:52:38 7% done; 38.9 MB/s, avg 38.6 MB/s ===== Download ===== # bbcp --reverse --ipv4 --port 7777 --verbose --progress 5 --streams 60 --windowsz 512K --append --gross --mkdir --readable --recursive --symlinks keep --sync dd --xfrrate 50M root@example.com:/root/tmp/vzdump-qemu-60001-2018_05_15-07_16_43.vma.lzo /home/user/isos/plop/tmp/ bbcp: Indexing files to be copied... bbcp: Copying 0 files and 0 links in 0 directories. bbcp: Sink I/O buffers (552960K) > 25% of available free memory (126160K); copy may be slow bbcp: Appending to /home/user/isos/plop/tmp/vzdump-qemu-60001-2018_05_15-07_16_43.vma.lzo at offset 0 bbcp: 180515 16:09:31 12% done; 46.8 MB/s, avg 46.8 MB/s bbcp: 180515 16:09:35 21% done; 41.3 MB/s, avg 44.3 MB/s bbcp: 180515 16:09:39 30% done; 44.5 MB/s, avg 44.4 MB/s bbcp: 180515 16:09:43 40% done; 46.4 MB/s, avg 44.9 MB/s bbcp: 180515 16:09:47 52% done; 52.6 MB/s, avg 46.3 MB/s bbcp: 180515 16:09:51 63% done; 54.3 MB/s, avg 47.6 MB/s bbcp: 180515 16:09:55 76% done; 57.5 MB/s, avg 49.0 MB/s bbcp: 180515 16:09:59 87% done; 51.6 MB/s, avg 49.3 MB/s bbcp: 180515 16:10:03 98% done; 50.6 MB/s, avg 49.5 MB/s ===== Sans limit rate download ===== # bbcp --reverse --ipv4 --port 7777 --verbose --progress 5 --streams 30 --windowsz 512K --append --gross --mkdir --readable --recursive --symlinks keep --sync dd root@example.com:/root/tmp/vzdump-qemu-60001-2018_05_15-07_16_43.vma.lzo /home/user/isos/plop/tmp/ bbcp: Indexing files to be copied... bbcp: Copying 0 files and 0 links in 0 directories. bbcp: Appending to /home/user/isos/plop/tmp/vzdump-qemu-60001-2018_05_15-07_16_43.vma.lzo at offset 0 bbcp: 180515 16:16:06 14% done; 64.0 MB/s, avg 64.0 MB/s bbcp: 180515 16:16:10 29% done; 69.7 MB/s, avg 66.8 MB/s bbcp: 180515 16:16:14 43% done; 65.5 MB/s, avg 66.4 MB/s bbcp: 180515 16:16:18 58% done; 67.3 MB/s, avg 66.6 MB/s bbcp: 180515 16:16:22 72% done; 65.5 MB/s, avg 66.4 MB/s bbcp: 180515 16:16:26 86% done; 66.8 MB/s, avg 66.5 MB/s File /home/user/isos/plop/tmp/vzdump-qemu-60001-2018_05_15-07_16_43.vma.lzo created; 1944213734 bytes at 66.6 MB/s 1 file copied at effectively 63.1 MB/s ===== Sans limit rate upload ===== # bbcp --ipv4 --port 7777 --verbose --progress 5 --streams 30 --windowsz 512K --append --gross --mkdir --readable --recursive --symlinks keep --sync dd /home/user/isos/plop/tmp/vzdump-qemu-60001-2018_05_15-07_16_43.vma.lzo root@example.com:/root/tmp/ bbcp: Source I/O buffers (46080K) > 25% of available free memory (132084K); copy may be slow bbcp: Indexing files to be copied... bbcp: Copying 0 files and 0 links in 0 directories. bbcp: Appending to /root/tmp/vzdump-qemu-60001-2018_05_15-07_16_43.vma.lzo at offset 0 bbcp: 180515 16:17:58 7% done; 32.0 MB/s, avg 32.0 MB/s bbcp: 180515 16:18:02 14% done; 30.6 MB/s, avg 31.4 MB/s bbcp: 180515 16:18:06 21% done; 33.0 MB/s, avg 31.9 MB/s bbcp: 180515 16:18:10 28% done; 31.3 MB/s, avg 31.7 MB/s bbcp: 180515 16:18:14 35% done; 32.4 MB/s, avg 31.9 MB/s bbcp: 180515 16:18:18 42% done; 35.0 MB/s, avg 32.4 MB/s bbcp: 180515 16:18:22 49% done; 32.9 MB/s, avg 32.4 MB/s bbcp: 180515 16:18:26 57% done; 33.6 MB/s, avg 32.6 MB/s bbcp: 180515 16:18:30 63% done; 30.6 MB/s, avg 32.4 MB/s bbcp: 180515 16:18:34 70% done; 32.6 MB/s, avg 32.4 MB/s bbcp: 180515 16:18:38 77% done; 32.7 MB/s, avg 32.4 MB/s bbcp: 180515 16:18:42 85% done; 34.1 MB/s, avg 32.6 MB/s bbcp: 180515 16:18:46 92% done; 31.4 MB/s, avg 32.5 MB/s bbcp: 180515 16:18:50 99% done; 35.8 MB/s, avg 32.7 MB/s File /root/tmp/vzdump-qemu-60001-2018_05_15-07_16_43.vma.lzo created; 1944213734 bytes at 32.2 MB/s 1 file copied at effectively 31.4 MB/s --idfile fname specifies the name of the ssh identity file if one has been specifically created for bbcp. The identity filename, prefixed by –i, is included in the ssh command line when starting the source and target nodes.