Get HTTP Header Info from Web Sites Using curl
The easiest way to get HTTP header information from any website is by using the command line tool curl. The syntax to retrieve a website header goes like this:
curl -I url
That is a capital ‘i’ not a lowercase L, the capital i extracts only the header information.
Try it out yourself with a sample URL, here’s an example syntax string using Google.com as the website header to retrieve:
curl -I www.google.com
Again, it’s important to note that capitalized I if you only want the site header. Using a lowercase i will give you a ton of minified HTML along with the header, just scroll up in the terminal window to the lines directly succeeding the curl command to find the HTTP header information.
An example of HTTP header details retrieved by curl -I might look something like this:
HTTP/1.1 200 OK
Date: Thu, 07 Jul 2014 22:15:57 GMT
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=741dreb25486514f:FF=0:TM=13154488957:LM=15526957:S=kmFi3jKGDujg; expires=Sat, 06-Jul-2013 22:15:57 GMT; path=/; domain=.google.com
iYjF09QhiCq2MdM3PKJDSFlkJalkaPHAU4JQy7MM8MKDQKEFLPqzoTSBPLKJLKMmdILlkdjel; expires=Fri, 06-Jan-2012 22:15:57 GMT; path=/; domain=.google.com; HttpOnly
X-XSS-Protection: 1; mode=block
curl -iD httpheader.txt www.apple.com && open httpheader.txt
This is the same curl command as before with a few modifiers. The use of the double ampersand tells the command to only open the file if the header was successfully downloaded. Using ‘open‘ will open httpheader.txt in the default GUI text editor, which is generally Text Edit, but you could use vi, nano, or any of your preferred command line tools:
curl -iD httpheader.txt www.apple.com && vi httpheader.txt
curl is a powerful utility that is worth familiarizing yourself with. Anyone involved with the web should get some good use out of the header trick, and web developers can also use curl to copy all the HTML and CSS from a website very quickly. The other advantage to curl is that it’s widely available for virtually every operating system out there, it’s bundled with just about every version of Mac OS X and Linux, and you can also find versions for Windows and even Android and iOS through individualized apps. Because curl has a long history and the commands are universal across platforms, it’s really the ideal choice for pulling header details, and is a valuable tool for systems administrations, network admins, web developers, and many other technical professions.
Update: Updated flags from -i to -I by reader recommendation, thanks everyone!