At 60East, we’re always helping teams analyze complex, interconnected software systems. One thing that’s easy to take for granted is file sharing – something that’s not always easy to do between teams in large enterprises. There are often great reasons for this, but for transmission of log files containing no sensitive information, it can be a real burden. The best way to share files is by using a shared host or file system, but if you have two teams that need to share files that don’t have any shared systems, then there are more efficient ways other than e-mail or file sharing services.

Let’s look at two of my favorite alternatives on Linux for sharing files within enterprise teams with no easy way to share files. In this example, a user on system “a.com” wants to share a file named “big.log” with another user on system “b.com”.

Method 1

The easiest way to share a file is for the file owner to open a simple HTTP server while the other fetches the file using a web browser or wget.

Open an HTTP Server on port 8080 on a.com, using Python:

1 python -m SimpleHTTPServer 8080

Fetch the file from b.com (or from any WebBrowser):

1 wget a.com:8080/symbol_store-20150315.tar.gz
2 
3 Connecting to a.com:8080... connected.
4 HTTP request sent, awaiting response... 200 OK
5 Length: 29627512439 (28G) [application/octet-stream]
6 Saving to: “big.log”
7  1% [                             ] 309,681,284 26.3M/s  eta 18m 24s

Wait for the file to download to complete. The user on b.com should now have the big.log file!

Method 2

The second method uses netcat (aka nc) and is great for file sharing Ninjas that may need to tweak for additional flexibility. The simple form of this method is to open netcat in listening mode on b.com and push the file to the other user from a.com.

On b.com, open the netcat listener on port 8080:

1 nc -l 8080 > big.log

On a.com, push the file over the network to the listening side:

1 cat big.log | nc b.com 8080

Wait for the file transfer to complete. The user on b.com should now have the big.log file!

Bonus: Add Inline Compression

If your file is a big text file, then you could benefit from compressing the data before sending it across. Here’s how to edit the previous command to compress the bytes:

On b.com, open the netcat listener on port 8080:

1 nc -l 8080 | gzip -d > big.log

On a.com, push the file over the network to the listening side:

1 cat big.log | gzip | nc b.com 8080

Super Bonus: Add a Progress Bar

Transferring large files can be frustrating when there’s no indication of how long it’s going to take. Plus, after the weather chit-chat, the progress indicators give introverted engineers something to talk about during the long pauses of silence when the bits are flying! :-) Let’s add some progress bars to our compressed netcat file transfers!

On b.com, open the netcat listener on port 8080, this is the same:

1 nc -l 8080 | gzip -d > big.log

On a.com, push the file over the network to the listening side (just replace “cat” with the “pv –tpreb” command):

1 pv -tpreb big.log | gzip | nc b.com 8080
2 386MB 0:00:10 [27.9MB/s] [>                        ]  1% ETA 0:16:19

Transfer At Will

With these tricks in your toolbox, we hope you’ll never miss dinner again due to the hours spent trying to send a large file to a colleague sitting in the cubicle across from you.

Do you have a great file transfer technique that gets you home on time? Let us know in the comments!