optimizing-and-troubleshooting-mobile-first-indexing

Optimizing and Troubleshooting Mobile-First Indexing

Optimizing for Mobile First Indexing

Table of Contents

Google’s Mobile-First Index: What It Is & How It Impacts Your SEO

One of the primary considerations when doing a technical SEO audit is ensuring the website in question is properly optimized for mobile-first indexing.

Mobile-first indexing was officially announced by Google in 2018. The change came about because the majority of people are searching from their mobile devices rather than desktop computers. Prior to that, Google ranked results according to the desktop version of their site. After the announcement, it became clear that websites needed to cater to its mobile users in order to rank higher in the SERP.

In March of 2020, Google announced that they’ll be switching to mobile-first indexing for all websites by September 2020. 

What does this mean for you? Let’s take a look at 7 ways this shift to mobile-first indexing can influence your SEO strategy.

7 Reasons Mobile-First Indexing Should Be an SEO Priority

User Queries are Changing Because of Mobile Search

The kinds of search queries that users make depend on many factors – what kind of information they’re looking for, what their intent is, and even how search has evolved over the years all impact the language that people use. There are different kinds of queries, such as:

  • Local search queries (e.g. restaurants near me)
  • Long-tail queries (e.g. best gaming laptops under $1000)
  • Transactional queries (e.g. buy Maybelline mascara)
  • How to queries (e.g. how do you unclog a toilet)
  • Informational queries (e.g. who sang rain on me)
  • Research queries (e.g. history of musicals)

But as more people search from their mobile phones, the way they search has changed as well. Conversational and personal searches have risen in popularity because of the casual, on-demand nature of mobile search. So when you plan your search engine optimization strategy, you need to take these two into account.

Personal searches center the user – words like “I”, “my”, and “me” are often used. According to Google, searches that included the phrase “for me” (e.g. best hair care products for me) rose by 60% in the last two years. Similarly, searches with “should I” (e.g. should i buy the new iPhone 11) increased by 80% in the same period.

This demonstrates that many users are using mobile search to solve a problem, complete daily tasks, and discover new options and opportunities around them. 

Conversational searches mimic how people naturally use language. Algorithms have become much more sophisticated and are better able to understand natural language, especially as voice search – where users literally talk to their devices – has increased.

Common conversational phrasing includes:

  • “Do I need” searches (e.g. do i need to change my password regularly, how much RAM do i need)
  • “Can I” searches (e.g. can i file my taxes late without penalty, can i transfer money from PayPal to my bank)
  • “Should I” searches (e.g. what should i have for lunch, should i buy a car)

As technology evolves, so does the way that users interact with it. This creates a feedback loop – users change their behavior to adapt to the technology, and in turn, technology adapts to how users are interacting with it. 

If you want to get ahead, you need to adapt to this as well. Build your content around real questions or phrases that people might use to find you. You can increase your traffic just by reworking your content to target personal and conversational searches.

Mobile Search Allows You to Expand Your Audience

Google’s goal as a search engine is to deliver content that addresses the user’s search intent. Many different sites serve different purposes – review sites help consumers make the right purchasing decision, informational sites help users execute a task or learn about a specific topic, and directory sites help connect users to the businesses they’re searching for.

What this tells you is that different users are looking to solve different kinds of problems. Google’s algorithm is just there to rank the results so that the most useful, most relevant, and most satisfying answers are at the top of the SERP. 

The first page can give you great insight into what users are searching for. Certain phrases are more used by mobile users, so the content they want is different – likely shorter, more digestible content compared to their desktop counterparts.

So when you take mobile users into consideration, you’re dramatically broadening the number of people that your content caters to. The mobile-first approach shouldn’t alter your SEO strategy, it should enhance it – understanding where your users are coming from and how that affects their search patterns is the first step to creating satisfying, relevant content.

Mobile-First Indexing Impacts the Type of Content Shown in the SERPs

It’s not only search intent that determines what kind of content your user expects. Even something as seemingly inconsequential as when they searched can tell you a lot about what that user is looking for.  

According to a 2016 Think with Google report, 80% of people use a smartphone at least once a day, with 27% exclusively using mobile devices for search.

But that’s not the only interesting finding. Google also discovered that people use different devices depending on the time of day, and the type of device used can alter user intent. For example, someone searching on mobile is most likely looking for fast, convenient, and bite-sized information compared to a desktop user who is probably looking for more in-depth content.

Google’s research shows that mobile usage is strongest in the morning, late afternoon, and evening. This is consistent with the average person’s daily routine – because mobile use is strongest when you’re on the go, it makes sense that peak hours coincide with the commute to and from work. 

Desktop usage picks up at around 8 a.m. and drops off in the afternoon, again, consistent with work/school hours.

But how does this impact your SEO strategy?

As we’ve already mentioned, you need to adapt to how users behave. It’s not enough to pepper your content with the right keywords, you also need to understand the different factors that affect what’s relevant to a specific user – factors that also affect what Google displays in the SERP

When Google began shifting to a mobile-first algo in 2018, they said that they may occasionally show non-mobile-friendly results if it’s determined that they’re more relevant or useful, based on particular signals. One such signal is the user’s device, of course, but another potential factor is time.

Basically, the time of day influences the device you use, and the device you use affects your search intent and how your content serves users on different devices.

Mobile Search Gives Insight Into User Behavior & Intent

We keep talking about what’s “relevant” to the user, but there is no one thing that determines relevance. Because search results are delivered based on user intent, the definition of “relevant” can vary from case to case.

The mobile-first index doesn’t necessarily change ranking on its own. In fact, we can frame all algorithm updates as an attempt to understand users better, decipher their intent, and deliver satisfying content. As our understanding of users deepen, so should our strategies for creating useful, targeted content.

Interestingly enough, these updates also influence how users search. As search engines become more sophisticated, users are starting to use more personal and natural language in their searches.

Mobile search adds another layer to this understanding. People search for different things on different devices. So what’s relevant to a desktop search could be very different for the same search on mobile, even if they’re conducted by the same user.

You can take it even further. The kind of device used can tell you a lot about the demographics of your users. There are implications of someone using Android vs iOS, mobile vs tablet vs desktop, and so on and so forth.

Age is one factor that influences device usage. In the Think with Google report we mentioned earlier, the search engine found that popular topics varied from device to device even within the same niche. In the beauty industry, for example, tattoos, nail care, and hair care topped mobile searches. In comparison, searches for anti-aging products dominated desktop. The desktop searches point to an older user demographic.

Again, keywords aren’t the only things that determine relevance. You can’t add the right number of keywords and call it a day. Content is relevant when it solves problems. And problems vary depending on the user – who they are, what they want, what makes them tick.

It’s interesting to see SEO professionals lean on CTR as a metric when click-through rate doesn’t show the bigger picture. In the mobile-first index, just because a user clicked on your results, it doesn’t mean that your content solves their problem.

Other page metrics like bounce rate and viewport time are just as (if not more) important if you want to understand how users engage with your content. 

Mobile Search Can Have Implied Search Intent

A lot of people think that the goal of the mobile-first approach is to make websites more friendly to mobile users. While user experience is definitely a part of it, the bigger concern is whether or not you can satisfy a mobile user’s search intent.

You have to ask yourself a few questions when you’re optimizing content for search, especially when you’re targeting phone or tablet users. Is the user looking for long-form content or a quick answer? Is the answer buried under paragraphs of text, or is it easy to find? Can users compare the different options easily?

Machine Learning Determines If Your Content Is Interesting For Mobile Users

We’ve already touched on how CTR can be a misleading metric if you take it out of context. A click still contributes to your ranking, but a more important quality signal is how long a user stays on your page and how much they interact with it. 

Google’s machine learning algorithm collects this data to understand what kind of content users find interesting. By analyzing patterns of user behavior, it can predict what users might want to read depending on their search intent.

So if your content is too long and difficult to understand, the algorithm can pick up on that and lower your rankings accordingly. But if you’re tackling a complex topic that requires a deep dive, then long content that tries to answer all of the important questions would be more useful to users.

The best content targets the most number of people, mobile users included. Of course, it’s impossible to appeal to everyone, but you can’t ignore the large numbers of people accessing search on non-desktop devices.

We really want to push this point home: the most important thing is that you satisfy user intent, not just in the topic that you’re covering but in how you present it. Length, language, tone, and structure can dramatically impact how a user engages with your content.

Promulgation of Mobile Search Has Changed User Behavior

Every SEO article says the same thing – Google’s algorithm is constantly changing. Heck, we’ve said it too. But underneath the surface, Google’s main objective has always remained the same: deliver the results that users want.

How it does that is a different story. Users are always in flux. What they want, when they want it, and how they search for it have mutated over the years. Google updates its algorithm to keep up with these changes, understand users better, and serve more relevant content.

The switch to a mobile-first paradigm is one of the ways that search engines are adapting to new user behavior. Google isn’t “forcing” websites to adhere to its guidelines; instead, it publishes guidelines so that websites can better cater to its users. 

So when you approach the mobile-first index, don’t think of it as a way to appease search engines. Ultimately, search engines are still at the “mercy” of its users, the same users that are increasingly relying on mobile to get the information they need. 

All of this – the emphasis on responsive design, the return to personal/conversational language, and the impact of the device on user intent – is a response to how user behavior has changed and adapted in light of new technology. 

Are Your Mobile First Indexed Pages Decreasing?

A large chunk of SEO is focused on getting your pages indexed.

Why?

You can’t rank if your pages aren’t indexed by search engines.

So if your number of mobile-first indexed pages is going down, that could spell trouble for your website. Oftentimes, it’s a symptom of a much bigger problem.

Let’s take a look at the different reasons your mobile-first indexed pages could be decreasing and how to fix the issue.

Should A Decrease In Mobile Indexed Pages Worry You?

The success of an SEO campaign is measured by metrics like organic traffic, non-branded keyword impressions, CTR, and SERP rankings. Indexed pages aren’t a common KPI, especially since it doesn’t translate into any direct economic value for the business. Because of this, many people completely overlook any decrease in indexed pages.

But indexed pages are essential to ranking. If you have more indexed pages, there are more opportunities for you to rank for more keywords. People won’t see your content in the SERP if they haven’t been indexed, which will negatively impact other critical KPIs like traffic and revenue.

Still, that doesn’t mean that a decrease in indexed pages is necessarily a bad thing. This could point to a problem that you need to fix that you wouldn’t have otherwise noticed. 

Plus, there are some cases in which a decrease in indexed pages demonstrates positive progress. For example, if you had a lot of indexed pages thanks to low-quality, thin, or duplicate content, cleaning that up will lower your indexed pages count. But that’s not a bad thing – after all, quality is way more important than quantity.

So you don’t have to immediately panic if you notice that number going down.

Diagnose the issue first!

How To Check How Many Pages You Have Indexed

There are a few ways to monitor your indexed pages count:

  • Google search: In the search bar, type “site:[your domain]”. For example, “site:example.com”. This will show you all of the results for that domain. If you can’t find your site, or if there are fewer results than your published pages, that’s a sign that Google isn’t indexing your content.
  • Google Search Console: The Google Search Console is an invaluable tool for keeping track of your indexed pages. It’ll even show you how many pages you’ve submitted versus how many are indexed by the search engine. The numbers will never match up 100%, and that’s okay, but you want the difference between the two values to be as little as possible.
  • Other index checker tools: There are plenty of (free and paid) online tools to check your indexation status. Just plug in your domain, run the checker, and get the results of the scan.

Keep in mind that you might get different figures from different scanning methods. This usually isn’t a big deal, unless there’s a huge discrepancy between numbers. You’ll want to run this check regularly to see if any of your pages have been de-indexed since you last ran the scan.

The Reasons Behind Decreasing Mobile-First Pages

If there’s a big difference between your indexed and submitted pages, or if the number has gone down since you last checked, that possibly means that the search engine can’t read your content properly. There are a few common reasons for this:

  • Your site/page has been penalized and has therefore been removed from the SERP.
  • Your page is considered irrelevant, so it’s excluded from indexing.
  • Your page cannot be crawled by search engine bots.

In the next section, we’ll discuss in detail how you can identify the root cause of the index issues, plus a few tips on how to rectify the issues.

Troubleshooting Mobile-First Indexing Issues

 

If your mobile indexed pages count is going down, it might be a sign of a larger issue.

Here are five ways to diagnose and troubleshoot a decrease in mobile-first indexed pages.

A decrease in indexed pages doesn’t always mean that there’s something wrong, but more often than not, there’s a problem that needs to be fixed.

This quick guide will help you troubleshoot mobile-first indexing issues and get your content back on the SERP.

Check #1: Are Your Pages Loading?

Your page has to return a 200 HTTP status code. If your server is down, or if you have an expired domain, that prevents search engines from crawling your content.

Use a header status checker (e.g. SreamingFrog, DeepCrawl, Xenu, Botify, etc.) to identify pages with the wrong status codes. 3xx, 4xx, and 5xx errors are a bad sign, but they at least point you to the source of the problem.

Check #2: Have the URLs Changed?

If you switch content management systems, servers, or other backend infrastructure, that could result in a change of URLs. And since search engines “archive” copies of your old URLs and content, it might prevent your new pages from getting indexed due to duplicate content.

In this case, the best thing to do is to redirect your old URLs to your updated content. Eventually, the old URL will be replaced with the new one after Google processes the redirect.

Check #3: Check for Duplicate Content

As we mentioned earlier, removing duplicate content, adding tags (e.g. canonical, disallow robots.txt, noindex, etc.), or redirecting URLs can cause your indexed pages to go down. But again, this is not a bad thing. Having fewer low-quality pages is always beneficial to your SEO.

You just have to make sure that this is the cause of the de-indexation and not any other factor. If you’ve already double-checked every other possibility, and you’ve recently fixed duplicate or thin content issues, then you don’t really have to do anything else.

Check #4: Are Pages Timing Out?

A page time-out can usually be attributed to server overload. Most sites have limited bandwidth or a setting that blocks IP addresses that try to access your site above a certain threshold. In the case of the latter, this is mostly a security setting to prevent DDOS attacks from taking down your site, but it can also prevent search engine bots from crawling your site.

Double-check your settings and make sure that the threshold for blocking isn’t too low. You should also enable caching to lessen the load on your servers. If that still doesn’t work, consider upgrading your server or physical hardware to accommodate the crawlers

Check #5: Can Search Engine Crawlers Read Your Site?

Just because you can see your content, it doesn’t mean that search engines can. This is why you need to build your website with SEO in mind, so that crawlers can parse and index your pages properly.

Search engine bots can also have a difficult time reading your site if you have cloaked content in an attempt to manipulate the algorithm. Hackers who have redirected your content to a different URL and malware are two other possibilities.

More common than cloaked content is content that requires user interactions (for example, swiping, clicking, or typing) to load.

Additionally, some lazy-loaded content could cause Googlebot trouble rendering the page if the settings are not correct.

You can check what Google sees through the URL Inspection tool in the Google Search Console. Looking at the cached page will also give you an idea of how crawlers see your content, although it’s still possible to cloak content and bypass this checker.

Looking Ahead at Mobile-First Indexing

There’s no debate about whether or not mobile is the future. Recent studies have shown that more and more people are turning to mobile, with some exclusively using their phones/tablets for search. 

And because user searching patterns are changing, it’s necessary to change the way we approach SEO. We’re no longer optimizing for desktop users alone, and our strategies need to reflect this new reality.

Technical SEO Audits

24 Pages of Technical Insights for Under $180