Whitehouse.gov Keeping Searchers From Asking Questions (Inadvertently)
Yesterday on the O’Reilly Radar blog, I posted that a key component of the American government’s initiative to be transparent and open with its data is to ensure content from government web sites is available through the major search engines. Tomorrow, I’ll follow that up with a post that details some of the specific obstacles that are blocking them now and how they might become more search engine-friendly.
While I was working on that post today, I checked out the new section of whitehouse.gov called “Open for Questions“. Since this is a brand new feature and has been talked up quite a bit online, I thought it would be a good study on government efforts do in search.
The trouble with Google Moderator
Open for Questions uses Google Moderator, which as I’ve previously posted, is a black hole for search engines. That’s unfortunate, since people who are searching for information on health care reform, financial stability, or the environment would likely benefit from finding a lively discussion on whitehouse.gov with concerns from fellow citizens. Another of Google Moderator’s drawbacks is that it doesn’t provide a unique URL for each topic, so for instance, if I wanted to email a link to the discussion about retirement security to my mom or post a link to the education discussion on Facebook, I couldn’t. The best I could do is tell people to go to http://www.whitehouse.gov/OpenForQuestions/ and then scroll to the topic and click on it. If I wanted to share a specific question from one of those topics? Not a chance.
The semantic structure of the page
I then noticed that the title tag of the page uses the text “Open For Questions” with no additional clue that this page is part of whitehouse.gov or that these questions are intended to be from citizens to the White House. That’s not great for either search ranking or search acquisition. If searchers see a listing in the search results for “Open For Questions”, it could mean anything. (Much of whitehouse.gov has title tag issues.)
The heading on the page is “Your Questions on the Economy”. That’s kind of confusing, since you can use the topic navigation to ask questions about all kinds of things. And the heading is in a span class, not an H1, so it doesn’t signal clearly to search engines that this is what the page is about. The text mentions “Thursday”, but no date, which makes me wonder if these discussions will no longer be online after tomorrow. If they are, having the date on the page from the start would make the page a lot easier to maintain later.
The real problem: indexability
But as it turns out, all of those obstacles are inconsquential. A search for [white house open for questions] brings up lots of pages that reference the feature, but not the http://www.whitehouse.gov/OpenForQuestions/ page itself.
I was initially surprised because even though much of the content on the page is inaccessible to search engines, the pages has lots of links with descriptive anchor text. And then I realized this isn’t a ranking problem, it’s an indexing problem.
Surely the search engines have seen the page by now. I checked robots.txt but didn’t see anything blocking it. And then I looked at the source code. There it was.
<meta content=”noindex,nofollow“ name=”robots“/>
My guess is that blocking search engines is either accidental (which happens all the time when pages are first launched, since they are often blocked during the development phase) or this page is intended to be only temporary, meant to gather questions for a short period of time, but not keep them as an archive later. I hope the latter isn’t the case, because I think having a repository of the questions Americans care about most can be very useful for a number of reasons.
How can this be fixed?
Check O’Reilly Radar tomorrow for more tips on how the government can make its content more findable.