Marketing Avenue Belgium

Posted by great scott!

The avalanche-like flow of special guest Whiteboard Fridays continues this week with another installment featuring our beloved London SEO expert, Richard Baxter (anchor text, y’all). Last week Richard helped us all learn how to get our fresh content indexed licketty-split, and this week he’s back to help us learn how to identify which areas of our sites are working hardest for us.

Whether you have multiple types of content on your site (maybe a blog, tools, articles, etc.), or you have limited content types across different topics (blog posts about cats, kittens, evil cats, ninja kittens, evil ninja kitten cats, etc.), wouldn’t it be nice to know which content types or topics bring you the most and best traffic?  Never fear, Richard’s here to explain his handy-dandy system to do just that!  By the end of this video you’ll know exactly which stats to pull from your analytics to create a so-shiny-it’s-practically-chromed spreadsheet that will let you peer deep into the inky black heart of your site and know the stars, the slackers, and the shiftless hobos among your content.

Wow! It’s like the future is now! And, since thinking of the future always makes me think of ‘Flash’, and thinking of ‘Flash’ reminds me that those of you without Adobe Flash can’t watch the video, I’ll try to summarize Richard’s bard-like musings on content segmentation and performance analysis.

In order to track and analyze the performance of your individual content, you’ll want to segment out your analytics data by content type. This is really, really easy to do if you have good, clean site structure (which you have, right? RIGHT?!). You can just pull Richard’s data points (below) for the different sections or subfolders of your site. If you were lazy and thought the best way to organize your site was to throw all of the pages into a virtual bucket, dump them out, name them by throwing your keyboard at a stump, and call it a day, you’ll have to get a little more involved with how you filter your segments. No matter what though, you might consider segments like all blog posts (perhaps a ‘CONTAINS /blog’ filter), all tools, all content written by Belverd Needles, III (/authors/belverd), etc. 

Once you have your segment filters in place, you just need to pull the data that Richard suggests and you’ll be able to see exactly how Belverd’s content compares to that of his bloggitty arch-nemesis, Marmaduke Huffsworth, Esq. (/authors/marmaduke). What data you say? This data:

1. Number of Pages per Segment  Richard advocates crawling your site using something like Link Sleuth to get this number; you’ll use it for all sorts of fun calculations. Yes, calculations can be fun. If you don’t believe me, just ask these racially diverse, embroidered youths.

2. Number of Keywords Sending Traffic  You can pull this from your analytics. Don’t worry so much about the words themselves here, you just want to know how many different keyword terms are delivering one or more visits to each segment.

3. Number of Pages Getting Entries from Search Engines  How many pages within the segment received one or more visits from a search engine (pick an engine, any engine, or all of them, whatever matters to you…so Google, basically).

4. Total Visits from Google Search Engines  Like it says on the tin, this is just the total number of visits to the segment from search traffic.

5. Percentage of Total Visits that Performed a Conversion Action  This will require that you have some conversion actions setup in your analytics, but it’s a key data point if you want to figure out your strongest content.

So what can all of this stuff tell you? LOTS! By tracking these numbers, you’ll be able to quickly identify which content is working hardest for you. You’ll be able to know whether Marmaduke or Belverd is better at drawing high-converting traffic. You’ll know which subjects and content types are most deserving of your precious time and the investment of your hard-bilked pennies. You’ll know who put the bop in the bop shoo bop, who moved your cheese, and why birds suddenly appear every time I’m near (it’s because my pockets are full of birdseed). You’ll be 12.7-29.4% awesomier than you were before, and you’ll smell delightful ALL THE TIME!

Now aren’t you glad Richard stopped by and shared his magic secrets with you? Thanks, Richard!

Do you like this post? Yes No

Posted by Lindsay

When I was an in-house SEO I hired outside SEO consultants. Now as the outside SEO consultant I often work with in-house SEOs. In the comments of my most recent post, an interesting question came up, "…why would a company who has an in-house SEO expert hire an external company?"

Here are 8 excellent reasons why talented in-house SEOs often bring in outside help.

1. Specialized Expertise

Not too long ago, SEO was a niche marketing specialization. I remember when even Internet Marketing was considered a highly niche specialization. In fact, my college marketing instructor tried to talk me out of Internet Marketing because it was too niche and I ran the risk of limiting my prospects down the line.

Times have sure changed. As the search engines have matured and the SEO industry has evolved along with them, it is becoming increasingly difficult to be on top of every SEO related factor. Even something as specific as SEO is segmenting into specializations. Experts have emerged in social media promotion, local SEO, mobile SEO, copy-writing for SEO, link-building, and so on.

"I hired the external consultants simply because they had more experience in the area I needed support in. Everyone needs to learn new things, so you’re rarely an expert in everything at once. Hiring the external consultant gets around a lot of hurdles and ramps up your program much quicker. Their deeper domain expertise allowed me to focus in areas I was strong in, while our entire SEO effort moved forward at the desired pace. Why reinvent the wheel when someone else already has an established, productive program that can benefit you?"

Duane Forrester is an in-house SEO with Microsoft, running their program for MSN.  He is also the author of How To make Money With Your Blog and Turn Clicks into Customers. In his spare time, he writes for Search Engine Land.

I like what Duane said about the hiring of external consultants ramping up your program quicker. By knowing and doing what you do best and outsourcing other tasks, you can super-charge your site’s SEO and get closer to your potential traffic level.

If I worked for a national business comprised on thousands of brick-and-mortar locations (think Burger King), I’d definitely look at retaining the services of someone like David Mihm to ensure I had all the right pieces in place. I doubt that many people reading this post are as well versed on the intricacies of Local SEO as David.

How about mobile? You have the choice to either delve into the details yourself or do as other talented in-house SEOs have done and hire someone like Cindy Krum who wrote the book on Mobile Marketing. Literally.

"Mobile SEO is a niche within a niche, and it is pretty specialized. Top in-house SEO’s have brought me in to help with mobile SEO, simply because they don’t have time to learn the niche. There is a lot to know, and it is easy to make mistakes. Mobile is still a small part of most in-house SEO’s traffic, so they want to know that things are set up correctly, but they don’t have enough bandwidth to devote to learning the niche or even shepherding the project."

Cindy Krum is the CEO and Founder of Rank-Mobile, LLC, and author of Mobile Marketing: Finding Your Customers No Matter Where They Are. She also hosts a weekly radio show called Mobile Presence, acts as an SEOmoz Associate, responding to Q&A about mobile SEO.

Why bumble around yourself on such specialized niches when you can focus on the pieces you know best and outsource those pieces to a more qualified expert? You don’t need to be everything SEO all the time. Give yourself a break!

2. Too Much to Do. Too Little Time.

Effective SEO is a lot of work. Managing the internal politics can be a full-time job unto itself! Perhaps you are confident that you have the strategy nailed down but you just can’t get your projects through the pipeline fast enough. In order to keep things moving while you consider the next big project it can help to hire an outside consultant.

"I outsource as necessary for specific tasks, not for general consulting or strategy. Specific examples include content creation for new pages on a site, link building, and social promotion of blog content. This has generally worked out well as I’m able to shape efforts and budget across all aspects of Internet marketing while having a specific challenge or need addressed by the consulting company."

John Santangelo is an Internet marketing professional based in Jacksonville, FL and currently works in-house as the Search Marketing Manager for a staffing firm.

Once you’ve established what needs to be done, hiring an SEO consultant can help you push through a task list and get closer to your goals.

3. Fresh Perspective

Working on the same website for years on end can get mighty boring. You can only come up with so many interesting articles related to nylons, and if you have to rewrite the homepage title tag one more time you’re going to scream. With boredom comes creative stagnation. Bringing in the right SEO consultant can help get the creative juices flowing again. Fresh eyes bring fresh ideas to help your business grow.

At SEOmoz we used to provide whirlwind audits in our boardroom. The client would bring along their best and brightest SEOs, marketing folks, and development staff. We’d go through their site and point out areas for improvement. One particular client comes to mind; well known brand, important website, talented SEO expertise… They’d blocked an important directory in the robots.txt. Sometimes when you are too close to a problem you can miss little details like a line in your robots.txt or an important redirect.

4. Educational Purposes

At SEOmoz we often sold an educational component along with our site audits. We’d go in with slide decks and teach anywhere from one to dozens of in-house resources some of our knowledge. This empowers the in-house team to move forward on their own, knowing a little more. Training can be formal or otherwise. Topher describes his outsourced project as a learning experience.

"As the in-house at CNN.com I have used a agency (Bruce Clay) and have brought in an outside consultant. I think a good SEO has to know what they don’t know and I do not know mobile SEO well at all. I went and asked about for a mobile SEO expert and Cindy Krum’s name came up all over the place so I brought her in and she was great. I am still not an expert on Mobile SEO but I for sure know a heck of a lot more now then I did before because of her."

Topher Kohan is the SEO Coordinator for CNN. He joined CNN, a division of Turner broadcasting and a Time Warner company, in early 2008 after two years at the Centers for Disease Control and Prevention.

5. Validation

SEO enhancements can be expensive to implement and sometimes take months or even years to complete. Based on high level experience across more web properties, an outsourced consultant can help you prioritize your enhancements and validate your project plan to ensure you make the most of the development investment.

"Outside SEO consultants typically have very broad experiences with a variety of websites and industries. Our role is to come along side the in-house team and help them manage the process of inserting SEO into the overall marketing and web production schedules and tackle the different hurdles associated with that. The in-house SEOs are our biggest allies to help us navigate the internal roadblocks and in return we are their biggest allies for getting their projects implemented."

Todd Friesen in the Vice President of Search for Position Technologies Inc. and has been working in SEO and online marketing since 1999 with many high profile clients such as Nike and the NCAA.

At SEOmoz we enjoyed working with strong in-house SEO individuals or teams for our consulting gigs. I suspect that this is true for most SEO consultants that specialize more on strategy and less on implementation.

6. Collaboration

As in-house SEOs, a lot of folks work independently. It can be refreshing and rewarding to expand on the one-man show. Marty describes how he and his employer benefit from expanding his team from time to time to meet a need.

"It really benefits me to be able to divvy up the responsibilities for things like site architecture, internal linking, etc. to an outside firm/person I trust while I focus on other important tasks like content migrations and cleanup with our internal web team. I find it very useful to spread the workload in order to be able to launch a redeveloped site sooner rather than later and in most instances it is also more cost effective in the time savings."

Marty Martin is an SEM/SEO with a broad range of experience working for colleges and universities, regional and state tourism, government and business. He is employed currently as an in-house SEO for Leisure Publishing Co., Inc. in Virginia.

7. Overcome Internal Politics

Of course you know your stuff when it comes to SEO. That is how you got your in-house SEO job, right? Then why do you spend so much of your time selling the value of your projects and negotiating for resources? One challenge that a lot of in-house SEOs face is finding the time to do actual SEO work. External consultants can help pave the way to get home grown ideas implemented.

	"Sometimes in-house SEO departments need help convincing another department that their ideas are solid. We do a lot of consulting that helps the different departments learn how to play together throughout the development life cycle."
Jessica Bowman is an SEO Expert, international speaker, member of the SEMPO Board of Directors and works with companies to figure out what they need to build a successful in-house SEO program.

8. Breadth of Knowledge

As an in-house SEO for a growing business, the challenges you face for the first time have more often than not been considered and successfully addressed by another SEO somewhere out there in cyberspace.

	"A number of our clients have in-house SEO teams and we love working alongside them. There’s quite a range of reasons why we’d be brought in. One of the most common reasons is because we have specific experience across a range of sites or in solving a specific tough problem."
Will Critchlow is the Director of Distilled, an SEO and internet marketing firm in London and Seattle.

Lets say you’ve inadvertently landed yourself a Google penalty. How do you diagnose the problem, get it fixed, and request forgiveness with a successful outcome? A consultant who has helped other websites work their way out of a penalty situation can be invaluable.

There are plenty of less dramatic examples. How do you implement a WordPress powered blog as a sub-folder of a .Net site? How do you handle millions of constantly expiring pages (as is common with job boards and classified ad sites)? How will you write a compelling link bait piece?

Action Items

The next time you get push back when proposing to hire an SEO consultant, choose from the reasons outlined in this post to support your case.

1. We need specialized expertise.
2. We have too much to do. We’ll get this project moving faster if I can get some help.
3. We can learn a lot from an outside expert.
4. We want to double check our strategies before we get going.
5. We would benefit from collaboration with other SEOs.
6. A consultant can help us work through the concerns of marketing/IT/executives.
7. We need the help of someone who has done (insert complicated initiative) before.

In-house SEOs hire outside assistance for all kinds of things from strategy, implementation, retainer, special projects and more. Are you an in-house SEO that has worked with external SEO experts? I’d love to hear your experience.

Happy optimizing!

Do you like this post? Yes No

Posted by randfish

This past week during the SMX Advanced conference in Seattle, I presented some correlation data alongside Janet Driscoll-Miller, Sasi Parthasarathy of Bing & Matt Cutts of Google. Matt in particular was quite vocal in expressing a desire to see additional data points from our research, primarily around the prominence/visibility of particular elements in the results. This post is intended to help make that available.

I must say that I don’t agree with Matt on the importance of the raw visibility/counts over the ranking correlations. My feeling is that SEOs in these spaces are more interested in answering the question – "what features predict a result will rank higher vs. lower on page 1?" – rather than the more straightforward – "does this feature appear more frequently on page 1 at Google or Bing?" However, I certainly agree that both are relevant and interesting.

If you’re trying to wrap your head around how to understand this prominence/visiblity data vs. our earlier data on the correlation with rankings, here’s how we’d best describe it:

• Correlation w/ rankings data helps to answer the question, "when this feature appears in results on the first page of Google/Bing, who ranks it higher and by what amount?" Those correlation numbers were derived by looking at the liklihood that a result would rank above another when it contained the target attribute.
• Visibility/prominence of an element helps to answer the question, "is this element more likely to appears on the first page of Google’s/Bing’s results?" This simply looks at the number of times we saw a result (or multiple results) ranking on page 1 containing the target attribute.

We’re looking at the latter one in this post, but before we dive in, there are a few critical items to understand:

• This isn’t correlation data and there’s no standard error or deviation numbers here. It’s simply how many times we saw the element in the results we gathered, divided by the total number of results (SERPs or URLs depending on the chart) to get a percentage. 
• This data is from page 1 of results from 11,351 search results, gathered from Google’s AdWords categories. This means the terms and phrases vary somewhat in search quantity (from sub-100 searches per month to tens or hundreds of thousands) but generally have a commercial focus and a intent. They generally don’t include brand names, long tail phrases or vanityname searches. Overall, we picked them because they’re precisely the kinds of queries most SEOs care about when they’re doing competitive SEO for their companies and clients.  We also ignore the second result in a SERP from the same domain to avoid effects of indented results (which was important for our earlier statistics, but not those in this post). 
• The results were collected the week of May 31st and thus, include post-"Mayday" update SERPs and likely results from after the "caffeine" launch as well (though Google did not announce when exactly that rollout occurred – it may not have much bearing as caffeine supposedly is an infrastructure, rather than an algorithmic change).
• Each feature contains two pie charts, one showing the percentage of results that contained at least 1 URL with this feature and another showing the percentage of total URLs in all results (102,296 for Google and 109,966 for Bing – note that some SERPs will fluctuate the quantity of standard web results they show on page 1). These are labeled as "(feature) in SERPs" and "(feature) in URLs," respectively.

In gathering this data, we did not optimize to share it in this fashion. In fact, Ben & I both feel that if we wanted to do it this way, we should gather the first 3-5 pages of results, not just the 1st page.  The way, one could compare the counts on page 1 with the counts on page 2.  However, since we’ve got the data and Matt, Sasi and several other folks expressed interest, we’re sharing anyway. Hopefully in the future we can do more on this front.

Let’s dive in!

Exact Match Domains

These are domains that precisely matched the keywords in the query – e.g. for the query "dog collars" only a domain that matched *.dogcollars.* would be included.

You can see that Bing has slightly more exact match domains appearing in at least one result of the SERPs we collected and in the overall count of results (all the URLs from all the SERPs).

Exact Match .com Domains

Similar to exact match domains, exact match .com domains had to contain the exact query in the domain name and have a .com TLD extension.

Again, Bing showed a slight preference for displaying results from these sites in the SERPs and URLs we observed.

Exact Match .net Domains

As above, but replace ".com" with ".net."

The similarity is much closer in the number of total URLs we saw with .net exact match, but Bing is showing a preference in the SERPs count.

Exact Match .org Domains

In the .org TLDs, we start to see a bit of what we observed in the ranking correlation data:

This is the first exact match domain TLD where Google actually had more SERPs containing a result of this type. Bing, however, had a very tiny amount more URLs with this feature.

Exact Hyphenated Match Domains

One of Matt Cutts’ complaints centered around how Google vs. Bing handled exact hyphenated match domains. When we observed them in ranking correlations, it appeared that, when Google listed them, they would rank them higher than Bing did when they appeared on that first page of results. However…

As I called out in the presentation and the prior post, Bing has quite a few more SERPs where exact match domains appear and somewhat more URLs, too. This is another data point that should make us all think carefully about the fallacy of presuming correlation = causation. Bing might have a preference for exact hyphenated match domains, but the ranking correlations suggest to me there’s more going on here – maybe something to do with anchor text or where those types of sites tend to get links or something else we haven’t considered?

It’s critical to keep in mind that we’re just looking at individual factors here – not trying to explain why they exist or correlate (at least, not in the data).

Results that Include All Keywords in the Domain Name

Here we looked for domains that contained the keyword query in the domain, even if the match wasn’t exact. For example, mydogcollar.com would now match for the phrase "dog collar."

Again, it’s Bing that shows a higher number of these types of domains in their results.

Results that Include All Keywords in the Subdomain Name

We’ve previously shown some data suggesting that subdomains might have some ranking influence, but not as much as root domains (this was done using our rank modeling / machine learning process). Here’s some raw data on the number of times we observed keyword matching subdomains:

Perhaps not surprisingly, Bing again is showing more of these results in their SERPs and individual URLs.

.com Domains

For this feature and all the TLDs below, we’re just looking at any URL that has the domain extension.

It looks like Bing has very slightly more .coms in their results vs. Google.

.org Domains

Let’s see what happens for .org domains, recalling Google’s apparent preference for them in the ranking correlations.

Oddly, Bing again seems to have more .org pages in the SERPs and URLs.

.net Domains

URLs with .net probably won’t surprise you much:

Yet again, Bing is showing a small number more than their Googly competitors.

.edu Domains

Recall how, in the correlation data, the numbers were small(ish) but negatively correlated? Let’s see what the number of results shows: 

True to the stereotype, Google is slightly ahead on number of .edu domains in the SERPs & URLs.

.gov Domains

Given the previous charts, this one likely won’t surprise you:

Google has more .edus and more .govs, too.

Keywords in the Title Element

Not surprisingly, nearly every set of SERPs had at least one result where the title tag contained the keywords:

Bing shows up with more results that contain title tag to keyword matching. One thing that is worth mentioning is that we didn’t observe the titles the engines chose to show, but rather the page titles from the results themselves. Hence, if a result was showing a DMOZ title or a brand title (which Goole will sometimes insert), we ignored those and just saw the title element on the page itself.

Keywords in the URL

This one actually surprised me, if only because there were even fewer results with keywords in the URL than in the title! 

Bing again has more results with keyword-matching URLs, though remember that some of that is probably from keyword matching domains, too.

Keywords in the H1

The ranking correlations suggested that the H1 tag isn’t much of a differentiator, yet lots of people still swear by them:

The results would bear out that this is a much less frequent item than URLs or Titles for those ranking on page 1. Bing seems to show more of them than Google, though.

Keywords in the Alt Attribute

Alt attributes looked interesting last fall when we collected ranking information and once again provde worth a look in the correlation data from SMX Advanced. Let’s see what the raw couts show:

Bing is showing slightly more of these, but if the positive correlation means something, these numbers certanly suggest there’s lots of opportunity left for good alt attribute practices.

Homepages

Who lists homepages vs. deep pages in the results more?

My word! It’s Google by a good margin. Bing’s show of internal pages actually surprises me a bit, though perhaps that’s an old stereotype I need to abolish.

And with that, we’re done!

One important point to notice is that I’ve not included data on link results, as these would be hard to interpret and likely non-useful. Every page of results had pages with links to them and nearly every individual ranking URL also had links (a good sign for Linkscape’s index, but not super valuable as a data point). There were a few other data pieces like this that wouldn’t make sense here (keyword prominence in the body tag, word tokens in the body tag, domain name length, etc) and have thus been excluded.

I’ve done less analysis on these results in general, as I think the data is a bit less ideal for the purpose, but it’s still interesting and hopefully, illustrative of general prominence. I look forward to seeing your interpretations and discussion!

p.s. If you email Ben at SEOmoz dot org, he will send you a lot of numbers in a TSV which is for each query the metrics for each result that we used in these posts.  You can also find raw results in a public Google spreadsheet doc here. Feel free to play around and let us know if you see anything else cool and interesting.

Do you like this post? Yes No

Posted by MichaelC

First, a quick refresher:  URL prettying and 301 redirection can both be done in .htaccess files, or in your 404 handler.  If you’re not completely up to speed on how URL rewrites and 301s work in general, this post will definitely help. And if you didn’t read last week’s post on RewriteRule’s split personality, it’s probably helpful background material for understanding today’s post.

"URL prettying" is the process of showing readable, keyword-rich URLs to the end user (and Googlebot) while actually using uglier, often parameterized URLs behind the scenes to generate the content for the page.  Here, you do NOT do a 301 redirection. (Unclear on redirection, 301s vs. 302s, etc.?  There’s help waiting for you here in the SEOmoz Knowledge Center.)

301s are done when you really have moved the page, and you really do want Googlebot to know where the new page is. You’re admitting to Googlebot that it no longer exists in the old location. You’re also asking Googlebot to give the new page credit for all the link juice the old page had earned in the past. For example, you may have migrated your website to a new content management system, and all of the pages have somewhat different URLs than then had before the move.

If you’re trigger-happy, you might leap to the conclusion that RewriteRule is the weapon of choice for both URL prettying and 301 redirects.  Certainly you CAN use RewriteRule for these tasks, and certainly the regex syntax is a powerful way to accomplish some pretty complex URL transformations. And really, if you’re going to use RewriteRule, you should probably be using it in your httpd.conf file instead.

The Apache docs have a great summary of when not to use .htaccess.

Fear Not the 404 Handler

First, all y’all who tremble at the thought of creating your very own custom 404 handler, take a Valium.  It’s not that challenging.  If you’ve gotten RewriteRule working and lived to tell the tale, you’re not going to have any difficulty making a custom 404 error handler. It’s just a web page that displays some sort of "not found" message, but it gives you an opportunity to have a look at the page that was requested, and if you can "save it", you redirect the user to the page they’re looking for with just a line or two of code.

If not, the 404 HTTP status gets returned, along with however you’d like the page to look when you tell them you couldn’t find what they were looking for.

By the way, having your own 404 handler gives you the opportunity to entertain your user, instead of just making them feel sorry for themselves. Check out this post from Smashing Magazine on creative 404 pages. 

Having a good sense of humor could inspire love & loyalty from a customer who otherwise might just be miffed at the 404.

Here’s an example of a 404 handler in ASP. Important note: don’t use Response.Redirect – it does a 302, not a 301!

For PHP, you need to add a line to your .htaccess pointing to wherever you’ve put your 404 handler:

• ErrorDocument 404 /my-fabulous-404-handler.php

Then, in that PHP file, you can get the URL that wasn’t found via:

• $request = $_SERVER['REDIRECT_URL'];

Then, use any PHP logic you’d like to analyze the URL and figure out where to send the user.
If you can successfully redirect it, set:

•     header("HTTP/1.1 301 Moved Permanently");
  
•     header ("Location: http://www.acmewidgets.com/purple-gadgets.php");

And here’s where it gets a bit hairy in PHP.  There’s no real way to transfer control to another webpage behind the scenes–without telling the browser or Googlebot via 301 that you’re handing it off to the other page. But you can use call require() on the fly to pull in the code from the target page.  Just make sure to set the HTTP code to 200 first:

•     header(‘HTTP/1.1 200 OK’);

And you’ve got to be careful throughout your site to use include_once() instead of include() to make sure you don’t pull a common file in twice.  Another option is to use curl to grab the content of the target page as if it were on a remote server, then regurgitate the HTML back in-stream by echoing what you get back.  A bit hazardous if you’re trying to drop cookies, though…

And, if you really need to send a 404:

•     header(‘HTTP/1.0 404 Not Found’);

Very Important: be careful to make sure you’re returning the right HTTP code from your 404 handler.  If you’ve found a good content page you’d like to show, return a 200.  If you found a good match, and want Googlebot to know about that pagename instead of what was requested, do a 301. If you really don’t have a good match, be sure you send a 404.  And, be sure to test the actual response codes received–I’m a huge fan of the HttpFox Firefox plug-in.

Ease of Debugging

This is where the 404 handler really wins my affection.  Because it’s just another web page, you can output partial results of your string manipulation to see what’s going on.  Don’t actually code the redirection until you’re sure you’ve got everything else working.  Instead, just spit out the URL that came in, the URL you’re trying to fabricate and redirect to, and any intermediate strings that help you figure it all out. With RewriteRule, debugging pretty much consists of coding your regex expression, putting in the flags, then seeing if it worked. Is the URL coming in in mixed case?  The slashes…forward?  Reverse?  Did I need to escape that character…or is it not That Special?

You’re flying blind. It works, or it doesn’t work. 

If you’re struggling with RewriteRule regular expressions, Rubular has a nice regex editor/tester.

Programming Flexibility

With RewriteRule, you’ve got to get all the work done in the single line of regex. And while regex is elegant, powerful, and should be worshipped by all, sometimes you’ll want to do more complex URL rewriting logic than just clever substitution. In your 404 handler, you can call functions to do things like convert numeric parameters in your source URL to words and vice versa.

Access to Your Database

If you’re working with a big, database-driven site, you may want to look up elements in your database to convert from parameters to words.   And since the 404 handler is just another webpage, you can do anything with your database that you’d do in any other webpage.

 For example, I had a travel website where destinations, islands, and hotels all were identified in the database by numeric IDs. The raw page that displayed content for a hotel also needed to show the country and island that the hotel was on.

The raw URL for a specific hotel page might have been something like:

/hotel.asp?dest=41&island=3&hotel=572

Whereas the "pretty URL" for this hotel might have been something like:

/hotels/Hawaii/Maui/Grand-Wailea/

When the "pretty URL" above was requested by the client, my 404 handler would break the URL down into sections: 

1. looking up the 2nd section in the destinations table (Hawaii = 41)
2. looking up the 3rd section in the island table (Maui = 3)
3. looking up the 4th section in the hotel table (Grand Wailea = 572)

Then, I’d call the ASP function Server.Transfer to transfer execution to /hotel.asp?dest=41&island=3&hotel=572 to generate the content.

Now, keep in mind that you’ll probably want to generate the links to your pretty URLs from the database identifiers, rather than hard-code them. For instance, if you have a page that lists all of the hotels on Maui, you’ll get all of the hotel IDs from the database for hotels where the destination = 41
and island = 3, and want to write out the links like /hotels/Hawaii/Maui/Grand-Wailea/.  The functions you write to do this are going to be very, very similar
to the ones you need to decode these URLs in your 404 handler.

Last but not least:  you can keep track of 404s that surprise you (i.e. real 404s) by having the page either email you or log the 404′ed URLs to a table
in your database.

Performance

For most people, the performance hit of doing the work in .htaccess is not going to be significant. But if you’re doing URL prettying for a massive site, or have renamed an enormous list of pages on your site, there are a few things you might want to be aware of–especially with Google now using page load speed as one of its ranking factors.

All requests get evaluated in .htaccess, whether the URLs need manipulation/redirection or not.

That includes your CSS files, your images, etc.

By moving your rewriting/redirecting to your 404 handler, you avoid having your URL pattern-matching code check against every single file requested from your webserver–only URLs that can’t be found as-is will hit the 404 handler.

Having said that, note that you can pattern-match in .htaccess for pages you do NOT want manipulated, and use the L flag to stop processing early in .htaccess for URLs that don’t need special treatment.

Even if you expect nearly every page requested to need URL de-prettying (conversion to parameterized page), don’t forget about the image files, Javascript files, CSS, etc. The 404 handler approach will avoid having the URLs for those page components checked against your conversion patterns every single time they’re fetched.

A Special Case

OK, maybe this case isn’t all that special–it’s pretty common, in fact. Let’s say we’ve moved to a structure of new pretty URLs from old parameterized URLs.

Not only do we have to be able to go from pretty URL –> parameterized URL to generate the page content for the user, we also want to redirect link juice from any old parameterized URL links to the new pretty URLs.

In the actual parameterized web page (e.g. hotel.asp in the above example), we want to do a 301 redirect to the pretty URL. We’ll take each of the numeric parameters, look up the destination, island, and hotel name, and fabricate our pretty URL, and 301 to that. There, link juice all saved…

But we’ve got to be careful not to get into an infinite loop, converting back and forth and back and forth:

When this happens, Firefox offers a message to the effect that you’ve done something so dumb it’s not going even bother trying to get the page.  They say it so politely though: "Firefox has detected that the server is redirecting the request for [URL] in a way that will never complete."

By the way, it’s entirely possible to cause this same problem to happen through RewriteRule statements–I know this from personal experience

It’s actually not that tough to solve this.  In ASP, when the 404 handler passes control to the hotel.asp page, the query string now starts with "404;http".  So in hotel.asp, we see if the query string starts with 404, and if it does, we just continue displaying the page. If it doesn’t start with 404;http then we 301 to the pretty URL.

Other References

Information on setting up your 404 handler in Apache:

• http://www.plinko.net/404/custom.asp
• http://www.webreference.com/new/011004.html
• http://www.phpriot.com/articles/search-engine-urls/4

Apache documentation on RewriteRule:

• http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#RewriteRule

ASP.net custom error pages:

• http://aspnetresources.com/articles/CustomErrorPages.aspx

Technorati Tags

RewriteRule, 301, htaccess, 404 handler

Do you like this post? Yes No