This post originally appeared on HubSpot’s Agency Post. To read more content like this, subscribe to Agency Post.
I made a big mistake.
I broke up with Google. And it was much harsher than a “let’s just be friends” moment. I told Google to basically “F off.”
The problem is that I didn’t know that I did this terrible thing. So how did our signals get so crossed?
The Makeover and Reveal (With Ugly Results)
Since HubSpot acquired Agency Post in December 2013, the team had been planning to redesign the site and move it to the HubSpot COS platform. During our discussions, we decided to redesign the entire HubSpot Blog to improve discoverability between our three blogs — Marketing, Sales, and Agency — to improve usability and to better support our goals. We also decided to migrate Agency Post from its home on www.agencypost.com to blog.hubspot.com/agency to make it a more cohesive part of the content ecosystem.
I was excited. Agency Post was like that starter home you move into and find yourself living in 20 years later: The furnishings are a little mismatched, care was ignored because you were never going to actually stay, and additions to your family have caused a scarcity of space.
But with this excitement, I also knew there were some risks in my immediate future. Many experts have written about the loss of organic search traffic during a web design and migration (here and here and here and here). We gathered from these stories that, with the proper 301 redirects, we would recover between four weeks and four months post-redesign. In the end, we knew it was worth it — even if we experienced a temporary organic search loss. Besides, we would be moving to a site with a higher domain authority, which could help us even as we suffered a loss in organic search traffic.
So on December 4, 2014, we launched Agency Post on the HubSpot domain. Our December reporting showed that overall traffic had dipped, but this was expected. We also noticed a decrease in organic search traffic.
Pre-relaunch, organic search was largest driver of traffic to the site. (This is in line with our other blogs: The Marketing blog gets 57% of its traffic from organic search, and 40% of the Sales blog’s traffic comes through organic.) But after our first month on the HubSpot domain, organic search had become the second largest source of traffic.
In January, however, we started to get antsy. We assumed we would take that initial December hit, but after that, we should have seen a slow climb upwards to pre-relaunch levels. But as the chart below shows below, only 12% of our traffic was from organic search traffic. This is a 68% decrease in organic search traffic from November (pre-relaunch), and a 52% decrease from the prior month (after the relaunch).
JANUARY
February’s traffic makeup was similar to January’s. At this point, I was just glad the site’s organic search traffic hadn’t plummeted into the single digits.
FEBRUARY
At this point we asked ourselves, “what’s going on?”
A Game of Hide and Seek
Our blog redesign and launch coincided with the start of a third-party SEO audit our marketing team had commissioned. This was comforting news, as we assumed the expert would uncover some changes we needed to make to overall site structure and determine if we had done unintentional harm to Agency Post as a result of the move.
We got an email on the afternoon of February 26 that basically said: While there are some SEO issues we can fix long-term, one really easy way to improve your site’s SEO is to just remove the noindex, nofollow tags on all 1,300+ pages of blog.hubspot.com/agency.
NO index? NO follow?
Those are four scary words we did not expect to hear.
The nofollow attribute, which can be added at both the meta level (meaning it applies to the entire page) and the link level, tells search engines not to follow a link or transfer value to any site being linked to. It’s basically telling Google that you don’t trust the page, so do not “follow” the link. Noindex tells search engines to refrain from indexing the page, meaning it should not show up in search results.
In other words, we inadvertently told search engines: We do not trust any of the posts and links on the agency subdirectory and that while it could read the content, it should not provide searchers with access to those pages.
Basically, we told Google to move along. We built the roads, but you can’t use ’em.
If you were a drinking woman (or man), this would be the moment to grab a straw and get comfortable.
The Not Complicated, Not In-Depth, Not Time-Consuming Way We Fixed It
So how did we fix the problem where we accidentally told search engines to refrain from sending us organic search traffic?
We went into the blog settings, and deleted this phrase from the <head> tag HTML:
<meta name="robots" content="noindex, nofollow">
Then, we clicked the save changes button.
All it took was the deletion of six words and a few punctuation marks.
Discover If Your Site Is Hiding a noindex, nofollow Tag
You can check an article or site for this tag by viewing the webpage’s HTML source code.
In Chrome, go to your top menu bar and select: View > Developer > View Source. In Firefox, you can use the right-click button and select View Page Source. Then, search for the below piece of code by performing a Control+F.
<meta name="robots" content="noindex, nofollow">
If the code has been added to the specific page or the entire site, it will appear within the <head> section.
Developers can also tell search robots to ignore pages, sections of your site, or the entire website by including some code in the robots.txt file. This basically does the same thing as the noindex, nofollow tag. To find this file, type in your URL, and append the it with robots.txt, such as inbound.org/robots.txt. Then search for:
Disallow: This section will outline the pages you are telling search engines to ignore.
We’re Not the Only Ones
If you’ve ever redesigned a site or dealt with moving content to another domain, subdomain, or subdirectory, you know this is an easy mistake to make. Prior to the relaunch of Agency Post on the HubSpot domain, we had imported our content over to test out design elements and fix content issues. We’re smart, so we included a noindex, nofollow tag in the new blog’s settings so that search engines wouldn’t find duplicate content in the weeks leading up to the official launch.
The problem is that in the midst of our relaunch, we forgot to remove the tag. It’s that final little step that makes all the difference — like remembering to pull the ripcord when skydiving.
And from looking at our analytics, we didn’t initially see any red flags to indicate a deeper issue. December in the publishing world is a strange month. Typically, traffic numbers are lower due to people taking holiday vacations and spending time away from their desktops. It wasn’t until January that the drop in organic search traffic was significant. We assumed at this point that search engines had penalized the site for the transfer of the content.
Those with a higher level of expertise in search optimization probably would have simply examined the source pages to find out if anything was amiss. But we assumed that Google had reacted poorly to our change, and while we are a proactive group, we know there’s not much you can do when you fall afoul of Google’s algorithm. We believed we had been faithful to the search engine’s guidelines — direct and implied — so we accepted our “time out.”
The lesson? Make a checklist for your launches. Check it twice. And don’t just assume that Google is the villain in your story.
So, How Are We Looking Now?
Once we removed the noindex, nofollow, we saw an uptick in organic search traffic within the first week.
December 2014 to March 2015 Organic Search Traffic
The below chart shows our organic search volume pre-move, so you can see that the levels are normalizing.
August to November 2014 Organic Search Traffic
Organic search moved into the second leading source of traffic in March, but it still lags behind email at this point.
MARCH
The One Thing We Did to Prevent Extinction
In the end, I didn’t destroy Agency Post.
While our traffic varied month-over-month and we did experience two months of a decline in visits, we did grow post-relaunch. If February were a 30-day month, we would have only seen a decrease of 3%.
We didn’t experience traumatizing levels of decline for one reason: We decided to make our email list an important asset. We invested in growing our list during the past year, which resulted in a 484% increase in subscribers since May 2014.
While we want people to find our content in the wild — through organic search and social — a niche blog built for a specific community of people cannot rely on this alone. Facebook’s algorithm has made “organic” traffic sourced from the site less of a guarantee. Twitter can provide you with high numbers of impressions, but that doesn’t always equal clicks. Derek Thompson at The Atlantic found that his most popular tweets had a clickthrough rate of just 1.7%. Engagement rate and the click percentage were independent, and most of the engagement was happening on Twitter, not on The Atlantic.
“Twitter is worthless for the limited purpose of driving traffic to your website, because Twitter is not a portal for outbound links, but rather a homepage for self-contained pictures and observations,” he wrote.
Sharing does not equal reading, and our goal as a blog is to build a readership by creating content people want to read and learn from. We’re not BuzzFeed. We don’t rely on shareability alone to drive traffic because more traffic doesn’t equal more ad dollars from impressions. As a brand blog, we want to attract and keep the right type of reader.
We’re building a content ecosystem we own, control, and can protect. And while we don’t want to ever break up with Google again, I’ve learned we can survive without it. Just know that it is a hell of a lot easier to do with the search giant on your side.
