Moving Files Using netcat and tar
Published: June 25, 2011
Time for an upgrade. I've got this (really old) box running OpenBSD 4.9:
real mem = 234561536 (223MB) avail mem = 226095104 (215MB) mainbus0 at root: SUNW,SPARCstation-5 cpu0 at mainbus0: MB86904 @ 85 MHz, on-chip FPU
And a new one running Debian that is a lot faster...
CPU0: Intel(R) Pentium(R) M processor 1.60GHz stepping 06 503MB LOWMEM available.
I need to copy around 100GB of data from the SPARCstation to the new server. The SPARCstation is able to push around 300KB/sec tops unencrypted over the network. Normally I'd be using rsync over ssh, but in this case, it will actually slow things down. So slow that it would literally take over a week to copy this data! The fastest way I’ve found is to use netcat and tar together. By using this combination I'm freeing up the CPU on the SPARCstation to do more important things like read data off of the disks and push it over the network!
Notice that I'm not using compression with tar... once again, sparky just isn't fast enough :)
#Sender (SPARCStation) tar cf - * | netcat receiver_host_name_or_ip 7000 #Receiver (new computer) netcat -l -p 7000 | tar xv