Search

Top Posts

Comments

  • jc: @echo_host thanks for the updates… here are my questions 1. after you did that, what does it show in system...
  • iphone: Really informative. thanks.
  • RCB: Works great at Six Flags. Never had any moisture get in after drenching rides like Thunder River, Log Flume, etc.
  • TvvitterBug: Nice article! I think the Twitter client landscape has really changed in 2010. New approaches and design...
  • DistortedLoop: I’ve been putting electronics in ziplock bags for years as a quick and dirty water-RESISTANT...

Categories

Recent Posts


How to easily mirror an entire web site locally

March 19th, 2009 - Command Line, Mac OS X, Tips & Tricks

It’s very easy to mirror an entire web site on your local machine thanks to the terminal command wget, here’s how to do it:

Launch the Terminal and type the following command, replacing guimp.com (a tiny website) with the URL you desire to mirror locally.

wget -m http://www.guimp.com/

This will download the entire site on your local drive in a directory named the websites URL… note that this isn’t a particularly effective way to truly backup a website and it’s functionality, it’s simply mirroring it locally.

Note: various readers pointed out that wget is not installed in Mac OS X by default and that you’ll need to install it using MacPorts, you can check out our past article on installing MacPorts or visit the developer website at macports.org. MacPorts is a very useful tool allowing you to easily install open source software packages and is highly recommended for moderate to advanced OS X users, or those interested in the command line.

Posted by: David Mendez

Share

Save big on Mac Deals from Amazon.com

Comments:

Comments: 12

Comment from saintberry
Time: March 19, 2009, 2:15 pm

I didn’t think wget was included in 10.5 by default. Well it wasn’t for me at least, OS X is missing a tone of great *nix command line tools.

If the above command gives you a command not found error you will need to download and install a wget binary or compile it from source. The latter can be done easily with a package manager like macports.

Comment from james
Time: March 19, 2009, 4:07 pm

sudo port install wget

Comment from CB
Time: March 19, 2009, 4:12 pm

When I type sudo port install wget, I get

sudo: port: command not found

Comment from CB
Time: March 19, 2009, 4:43 pm

Got Deep Vacuum… GUI to wget

Comment from james
Time: March 20, 2009, 12:20 am

Of course first you have to istall macports

Comment from Jerid Hill
Time: March 20, 2009, 9:01 am

Couple of questions:

1. If I mirror my site locally, can I make changes locally and sync it using terminal?

2. What if my site is using a database on a separate server?

Comment from saintberry
Time: March 22, 2009, 10:58 am

Jerid, if you want to sync local and remote directories use rsync. It is actually included in 10.5 by default. If you are game with the command line man rsync, otherwise check the internet for tutorials.

Comment from Jerid Hill
Time: March 23, 2009, 7:58 am

Thanks! Adding products to a shopping cart takes a little too long and I’d rather do it locally!

Comment from best websites
Time: April 3, 2009, 2:00 pm

Great tip, I will try it, thanks!

Comment from praca magisterska
Time: April 26, 2009, 2:12 pm

hi,

I always used wget with -rxk options, result similar :)

Comment from Joe
Time: May 1, 2009, 10:52 pm

httrack is much better for mirroring websites.
Also available through MacPorts.
Personally, I’ve downloaded the entire internet. It is *much*
faster to browse that way.

Comment from medspharmacysupport
Time: July 16, 2009, 4:39 pm

This will download the entire site on your local drive in a directory named the websites URL… note that this isn’t a particularly effective way to truly backup a website and it’s functionality, it’s simply mirroring it locally.

Write a comment






March 19th, 2009