Scp -pr on a large number of small files to get rid of the slowness.
$ du -hs 201703
2.3G 201703
$ find . -name '*.gz'|wc -l
32736
So, 2.3GB, transfer of 30,000 files. Obediently scp -pr
$ time scp -pr 201703 [email protected]:~/tmp/
Password:
...
...
real 6m46.450s
user 0m28.824s
sys 0m58.958s
And it was about 5.8MB / s.
If you devise this
$ time tar cf - 201703 | ssh [email protected] 'cd tmp; tar xf -'
Password:
real 1m50.797s
user 0m16.753s
sys 0m25.793s
Improved to 21.4MB / s.
Recommended Posts