Dtyjlui

I have an 18GB EC2 instance that has run out of space. I wanted to zip and transfer files off the server to a local drive. There is just about 1% free space remaining on it. This means that I cannot compress anything on the drive because it keeps giving me zip I/O error: No space left on device error. Is there a way I can compress everything without using additional drive space?

You can give a command to compress to stdout over ssh and redirect stdout localy to a file. Something like:

ssh user@host "tar c /mydir | gzip -f" > myarchive.tar.gz

The following tools are available in Ubuntu, and I have checked that rsync, gzip and tar are also available in MacOS.

• 
  rsync which can copy files and/or directory trees locally and via a network


• 
  gzip which compresses single files


• 
  tar which can create an archive with many files and directory trees, and compress, if you specify it

Change directory

Change directory with

cd path-to-source-directory

to the directory that you want to compress.

rsync

I don’t have physical access to the server. I always communicate via
ssh. I’ll look up rsync. Thanks.

rsync is a powerful copy tool, and it has a built-in check, that the transfer is correct.

• It can copy files and/or directory trees locally and via a network


• It is often used for backup
  
  
  
  
  • locally to an external drive or
  
  
  • via a network connection to a server or between servers
  
  
  
  

It is straightforward to use rsync, if you have Ubuntu at both ends of the connection, and I checked that there is an rsync version also in MacOS.

I like the following command line where the option

• 
  -H takes hard links into account (and avoids double transfers/copies); if there are no hard links, you should remove this option.


• 
  -a 'archive' makes a copy that suits backup or synchronizing


• 
  -v 'verbose' creates output of all files to be copied with -n and all files copied in the real case (without -n)


• 
  

  -n makes it a 'dry run', just showing what it 'wants to do'

  
  
  

  rsync -Havn source/ target
  
  

  
  

In your case the source is in the server, and you run via the ssh connection. So, in the client (your Mac computer), run

rsync -avn user-id@ip-address:/path-to-source-directory/ path-to-target-directory

Please notice the trailing slash after the source directory.

If it looks good, you can let it do the transfer with the following command (remove the n for 'dry run')

rsync -av user-id@ip-address:/path-to-source-directory/ path-to-target-directory

Tips and comments

• After the transfer, you can do what you want with the copy in the target directory. I think you want to compress it, and I suggest that you use tar for that purpose and create a tarball.




• If you cannot run rsync or gzip or tar in your MacOS, you can boot your Mac computer from a USB pendrive or DVD disk with Ubuntu, and run the programs that way. (The advice to boot the computer from a USB pendrive or DVD disk with Ubuntu applies also to a computer with Windows.)




• You can read the built-in manual man rsync, man gzip and man tar in your Ubuntu 16.04 LTS, and I am sure that you can find good tutorials via the internet.

gzip

Change directory with cd path-to-source-directory to the directory that you want to compress.

Compress single files with

gzip -c file > path-to-external-directory/file.gz

Change directory to where you want to extract the file and run gunzip to uncompress

cd to-where-you-want-to-extract-the-files

gunzip -c path-to-external-directory/file.gz > file

tar

Change directory with cd path-to-source-directory to the directory that you want to compress.

Compress a group of files to a 'tarball', for example

tar -cvzf path-to-external-directory/file.tar.gz file1 file2 file3

or if there is space enough in the target partition on the external drive for the whole directory

tar -cvzf path-to-external-directory/file.tar.gz .

The space and final dot are important.

You can 'look into' the tar file with the command

tar -tvf path-to-external-directory/file.tar.gz

Extract the compressed files from the tarball with the following commands

cd to-where-you-want-to-extract-the-files

tar -xvf path-to-external-directory/file.tar.gz