Last week at SES, I sat in on the Black Hat, White Hat session. The panelists gave their definitions of “white hat” and “black hat” SEO, and then talked about particular techniques, such as paid links, and whether they thought they were OK or not. The panelists talked about shades of gray and the lack of rules and the need for experimentation.

I come from a background of working at a search engine, rather than from a background of being an SEO, so I may see things a bit differently than those on the panel.

White hat = no guidelines violations
Generally speaking, I don’t see a lot of shades of gray in this discussion. The search engines have published guidelines (and while I was at Google, I spent a lot of time expanding the descriptions of those guidelines to help make them clearer). Violate any of those guidelines and you risk having the site removed from the index. That’s pretty black and white.

Related discussions have more shades of gray
I do see shades of gray in different discussions, such as:

• Do all the guidelines make sense in today’s technological environment? For instance, are there valid reasons for cloaking that don’t manipulate search engines and deceive users, such as showing search engine bots a canonical version of a URL and different versions of that URL to users for tracking purposes? For discussions like this, there’s a reasonable debate to be had about whether search engines should consider different rules, but too often, I see the discussion tally up the reasons why a technique isn’t deception then conclude that it is therefore, white hat. But whether a technique “should” be OK and whether it’s white hat are two different discussions. With regard to cloaking, that’s currently against Google guidelines regardless of intent. So justified or not, it’s not white hat with the current set of guidelines.
• Do techniques that violate the guidelines work? Often, I see the discussion of black hat vs. white hat veer into a discussion about what techniques are most effective: which ones work for enterprise sites or affiliate sites or sites in highly competitive areas. This came up at the panel, when someone talked about how they didn’t believe that a completely white hat site in one of the three Ps (porn, pills or poker) could rank highly. Again, this is an interesting discussion, but a different one. Techniques that violate the guidelines may eventually get the site banned, regardless of their initial efficacy, so it’s important to understand the long term goal of the site before engaging in them. As the panelists noted, sites that are in it for the long term likely want the slow and steady approach.
• What about techniques that violate the guidelines but are commonplace? I hear this discussion integrated with the hat discussion as well, often when talking about paid links. Paid links aren’t black hat, I sometimes hear, because everyone uses them and they’re vital for ranking success. But again, whether or not a technique is commonplace is somewhat irrelevant to question people are asking when they want to know what is white hat. Paid links violate the guidelines (at least Google’s — the other engines aren’t quite as strict), so they can’t be considered a white hat technique. A different, but valid, discussion is whether all paid links should be against the guidelines.

While all of these discussions and more are certainly valid and useful, I feel the trouble comes in when people have these discussions as the discussion about what is white hat and what is black hat. When people who aren’t experienced in the intricacies of SEO look for information and they see statements like “these are white hat reasons to cloak” and “all paid links aren’t bad”, they can be led astray and think that those things adhere to search engine guidelines.

What are the guidelines?
As for the SES panel, I respect all of the panelists and felt they all had interesting, useful things to say. But for me, the question at hand is simple to answer. Techniques that violate the guidelines aren’t white hat. They may be effective (at least for a time), commonplace, non-deceptive, or justified, but that doesn’t make them white hat. To me, white hat is anything that doesn’t put the site at risk of being removed from the search index.

When I was at Google, I spent a lot of time expanding the guidelines, detailing examples, and providing options of techniques that didn’t violate the guidelines. Since then, Google’s continued to expand the information and make it even more helpful. Check them out, and in particular, click the links under the “quality guidelines – specific guidelines” if you want to read up on the details.

What is SEO?
So what is white hat SEO? The panlists agreed it was about creating quality content — being the most relevant result for a desired query. I absolutely agree, but SEO is also about making sure the site can be easily crawled and indexed by search engines. From a search engine perspective, the best site in the world is unlikely to rank if the bot can’t extract any content from it. That latter part of SEO is, of course, the motivation behind Jane and Robot (which is just getting started; look for more soon!).

You might also check out:

• Live Search’s webmaster guidelines
• Yahoo’s webmaster guidelines

I’ll be in Dublin in a couple of weeks, speaking at Search Marketing World and I’m staying a few extra days to hang out with friends. I was looking for a hotel downtown and did a Google search for the best hotel to stay in downtown dublin.

Apparently, it’s the Leeson Inn.

(In case you see different search results than I do, 8 of the top 10 results are for that hotel.)

I can’t imagine this is the result of search engine manipulation, since all kinds of sites are showing up, including Yahoo! and Trip Advisor, which as far as I know, are separately owned. (Although in this age of user-contributed content, I suppose we could be seeing a new era of review manipulation for SEO purposes, as Larry points out in the comments.)

Is the Leeson Inn really the definitive answer to my question?

I recently came across a post Alex Bosworth did a while back called “Google is destroying the web and you don’t even know it” in which he said:

“Unfortunately this [that everyone uses Google for search, and therefore online businesses need to be found on Google] means you need to do Search Engine Optimization. SEO is the worst thing ever invented. It’s destroying good web application development.”

I hesitate to even post on this because the “SEO is evil personified” argument has been rehashed to death. For instance, see this post on Search Engine Land that recaps several of the recent debates. Jeremy Schoemaker doesn’t like 95% of SEOs. Jason Calacanis thinks SEOs are snake oil salesmen. This idea of SEO as evil certainly isn’t new.

And his post is from last October, so it’s not new either, but it’s gotten some renewed interest online and I do think it’s still the case that many people don’t think of SEO as part of marketing. They think of it as a necessary evil rather than part of a larger strategy.

I do think that as businesses move online, more and more site owners are going to have the perspective that Alex has and I think that understanding how search engine optimization fits into a holistic marketing plan is important for the long-term vitality of businesses who want to participate in the online space.

So.

What Do I Know, Anyway?
First, let me back way, way up and provide some context on my perspective. My background is in communication. I have spent years working on communicating effectively with your target audience. My degree is in English. After college, I worked on corporate policy: making sure both corporate and store employees knew everything they needed about whatever their jobs entailed — whether that was creating marketing programs, buying fixed assets, or processed defective merchandise. And I did it in whatever ways worked best: written documentation, in-person training classes, software systems (as long as those software systems worked on AS/400 and didn’t require the internet — this was the early 90s after all).

Later, I worked in marketing. I started building web sites, starting with my company’s in 1995. I also spent years writing documentation for developers, doing audience analysis, and thinking about user interfaces (whether those were windows or SDKs), product strategy, and everything involved with a better experience for customers.

Much, much later, as many of you know, I was was the product manager for Google Webmaster Central and really dove into both the search engine perspective and the site owner perspective on search engine optimization. A big part of the search engine perspective is the searcher perspective, so when I look at the SEO issue I’m looking from a point of triangulation. I understand that searchers want the best result as quickly as possible; I understand that search engines want to understand the web so they can deliver the most relevant results; and I understand that site owners want to market their content effectively to the right audience.

While I was at Google and since I’ve left, I’ve reviewed countless sites for SEO troubleshooting, customer engagement, usability, and overall strategy. I’ve spoken and written about all varieties of online marketing: from technical infrastructure problems in the context of SEO to social media engagement.

Which is to say, I have an opinion about online marketing, particularly as it relates to search engine optimization.

The Age Old Debate: Build For Users Or For Search Engines
In his piece, Alex said:

“If you want a huge amount of traffic, the way to get it is not through community features, it is not through great writing and content development, but it is through optimizing the crap out of your site so that Google will send more and more searchers your way. Now the most important thing to you is no longer, “how can I make my site better to use”.”

Sigh. In my view, this misses the point entirely. Yes, most people on the internet find sites through searching. So, yes, you can get a lot of traffic if you rank highly for relevant keywords. But traffic alone is meaningless. You have to look at traffic + engagement. Traffic + bounce rate. Unless your entire goal with the site is to monetize through CPM-based ads on the home page, your site needs to be compelling to the searchers who land there so that they’ll stay. In other words, if you abandon great writing and content development, if you neglect the question of how to make the site better to use, you are simply being short-sighted and are ignoring all the rules of marketing. That’s not Google’s fault. That’s your own fault for not looking at the right metrics.

If you “optimize the crap” out of your site so that you rank #1 for relevant keywords, but your site isn’t compelling to searchers, that ranking will be completely meaningless as those new visitors will click right back to the search results rather than engage with your site.

He then said:

“Do you think having your site name in your page title is a good idea? Google doesn’t.”

I generally don’t like to speak in absolutes, much less speak for Google since I no longer work there, but I can definitely say that this statement is completely false. Google does think having your site name in your page title is a good idea. Of course you want your company name in your title and it makes no sense at all that Google wouldn’t also want you to be found in searches for your company name. But take Google out of the equation for a minute and think of your customers. They’re searching for something. Using words.

Imagine you have a paint store — an actual physical store on the street. And just imagine for a moment there is no internet (it isn’t hard to do). You want people to come to your store to buy paint. You might get customers who see your sign as they drive by or they might look you up in the yellow pages or perhaps you run TV or radio ads. If your store is called “Buffy’s Store” then how the hell do you expect those people driving by to have any idea what you store might contain? You sit there, with your brushes and your rollers and your five gallon buckets and you wonder where everyone is. Then try changing your sign to “Buffy’s Paint Store” and see if your customer traffic patterns improve.

Making your title descriptive isn’t evil Google oppression. It’s common sense.

“Do you think that javascript widget you made for navigating your archives is really awesome, intuitive and innovative? Google disagrees, it thinks it’s a big black hole of nothing.”

Yes, search engines need to evolve and get better at crawling new web technologies. But so do mobile browsers and screen readers. If you want your site to be accessible, you have to make it easy for your audience. I access the web on my phone A LOT. And if I’m trying to navigate your site and all the menus are in javascript so that I can’t get to them on my mobile browser, I can guarantee you that I’m not thinking about how awesome, intuitive, and innovative you are. And I bet blind people aren’t either.

I have had this exact experience several times over the last few weeks — once while stranded at a Caltrain station, trying to figure out when the next train was coming and I can tell you that I was not looking for innovation in my Caltrain menu. I was looking for information. It is not difficult to gracefully degrade your site so that anyone who is visiting it with a browser that doesn’t support your fancy technology can still access your content. If you are a good web developer, and I’m sure you are, you can build your site in a way that is both innovative and universally easy to access.

“And your user community might even die, but who cares, comparatively they are a tiny minority of your overall user base. You’re too busy dealing with scaling your servers to cope with the millions of hits coming from Google to care about those ten thousand monthly visits from loyal users.”

I admit, I don’t understand this at all. What exactly does he want with those millions of hits if not to add them into the user community? I really don’t get it. Is he building a site or creating a made for AdSense page? He’s complaining that Google is sending him too much traffic and thus it’s somehow Google’s fault that he’s chosen to ignore audience engagement? It’s completely baffling to me.

The bottom line is this. Yes, if you want your customers to find you using search, then you have to understand search engine optimization. And you should want your customers to find you using search because search is the entry point on the web. But if you are operating an online business, you absolutely should understand online marketing. I don’t understand people who say it should all just work and they should be able to concentrate on their core business. (Looking at this from a search engine’s perspective, however, I think they should and certainly they are working on ways to make sure it all just works, because it’s in their best interest to provide searchers the best content on the web, whether the owners of that content understand SEO or not, but that doesn’t negate the point.)

If you have an offline business, you have to understand offline marketing and customer engagement. If you are opening new stores and your core skill set is painting, you will likely hire others for other aspects of your business: determining the best location for the store, branding and advertising, merchandising. You will probably ensure your store is attractive, both inside and outside. You’ll arrange merchandise on your shelves so that people know where to find stuff and can easily reach it. You’ll make your aisles wide enough for carts.

You wouldn’t open your paint store with no sign and a broken door in a back alley that had a brick wall blocking the road. Why would you do the same on the internet and then blame Google?