Download Files from the Web via the Mac OS X Command Line
I am frequently developing web pages and often times I find it frustrating when I can not restart my browser because I’m in the middle of a download. So when I need to download a large file and I don’t want to have to worry about whether or not Safari, Chrome, or Firefox continues to run with out interruption, I turn to the trusty Terminal! In fact, you can easily download any files from the web by using the command line on a Mac.
The next time you have a file you want to download, just copy the URL into your clipboard, then open a Terminal window and use the ‘curl’ command.
Curl is easy to use for downloading files, at it’s simplest form the syntax would be:
curl -O [filenameURL]
The file destination URL should be prefixed with http for the web. By default this will download the requested URL to the present working directory, using the same name for the saved file as that it was on the remote server (in other words, if the file is called “filename.zip” on the remote server, the name will stay the same when it downloads.
You must use the -O (capital o) flag with curl to get the filename to stay the same. A lowercase -o flag will change the name. curl –help can explain more.
With the basics covered, let’s do something a bit more useful though and specify where the file will save from curl by using a specific example.
First you’ll want to change directories to where to save the file to, this is done with the ‘cd’ command. We’ll use the Desktop as an example:
Now that we have changed our directory to the “Desktop” (for convenience) we can start our download. For the download we are going to use a built in utility called “curl”.
curl -O http://www.exampleURL.com/downloads/Example/DoesNotExist.sit
Curl will instantly download the file. If the file is large enough, you’ll get a progress bar indicating how long it’s taking to download.
You can even combine the above command strings into a single command, if you’d like:
cd ~/Desktop; curl -O http://remote-server-IP/file.zip
Of course, curl has lots of other uses beyond downloading files from the web, so don’t miss our other posts on using curl.
If you have any other helpful tips or tricks for using this method of downloading files from the web, share them with us in the comments below!
Hi…losing my mind trying to get this to load, I keep getting syntax (like “Eof,etc.) looks like it”s right to me. Any idea why it won’t populate the file? thanks for any help!
curl —L -o quotes.txt “http://download.finance.yahoo.com/d/quotes.csv$s=mav,nhi,vtn,ekq,nrp,pfe,csco,mhi,ohi,abbv&f=sl1poc1p2qr2s7r7v1yw1a2dee8m3m4t8e9kk4wr1”
The line does break at the csv point with the $ on the next line. But, that shouldn’t matter if the whole url is in “, right?
wget is meant for file downloads. If you specify option ‘-c’, wget makes the download ‘resumable’. Meaning, even if there is a loss of network connectivity (e.g. while your laptop switches between wifi hotspots), you can always resume the download.
Is there an addon or a command I can use to download all files from a directory or a page ?
wget will do that for you
Thanks Bro.. My google start with wget and end to your post.
Thanks, for your share.I’m happy to use Terminal download ISO file :)
I also use wget. you have to download it and compile it. or install it using fink or macports.
nice, this really helped me tonight
I use wget
I have never used the curl command but this is pretty good
You might want to include a few extra bits in that post. First, always include the -L (or –location) switch, since that will allow curl to follow any redirects (if the file gets moved, but there is a redirect for it). Also, if the URL’s file part (the section after the last slash) is not pretty, you can give a name for the downloaded file by using a lower-case -o (instead of -O), followed by a space and the name in quotes. Also, always put the URL in single-quotes – ampersands and a few other characters will break on the command line if they aren’t in quotes. Here’s an example of those ideas combined:
curl -L -o ‘myfile.dmg’ ‘http://www.somewebsite.com/files/getdmg?id=24’
Thanks for adding that, Dan. That was the crucial tip I needed, especially the reminder about the quotes.