The Power of No-Indexing Non-Core Pages to Focus Google Indexing

When it comes to the pages that Google indexes and displays on the Google Search Engine, you won't need to do much more than show up on the Internet. If you meet Google halfway and set up an account on Google Analytics and Google Search Console, make sure you have an XML Sitemap running on your site, and then submitting it on GSC and Bing Webmaster. That should be enough, I guess; however, if you spend some time exploring your GSC's Page section and dig into which pages Google indexes and which pages Google's discovered but didn't index or has rejected based on issues with canonicalization, with duplication, as well as other issues, you get what you get.

Don't Index No Index No Robots

A lot of sites, including Wordpress and the rest, aren't very granular about which pages they set to "do index, allow robots" as default. So, you set Google free on your site and who knows which pages will win the indexing fight? Will it be your product or you blog page or will the indexed pages of your site be silted up by indexed category pages, tag pages, author pages, year, month, or day pages, search pages, URLs with errant variables and arguments and other hashes and URL detritus? When people see all the pages that are not being indexed by Google compared to the pages that are, in the green bar, ensconced into deeply into the search resources profered by Google, Bing, Duck Duck Go, and even the various and sundry Generative AIs.

I Prefer You Not To

You can define which pages you do and don't want Google to index and I can help you. First, don't confuse the deindexing, or "no index," of your secondary and tertiary pages as making them inaccessible to actual visitors. Even if you no-index or no robots or deindex navigation or UI/.UX pages from Google Search, they'll still be accessible to visitors and won't become 404s or 301s or 302s. One of the tools I really love for doing this in a very big was is Yoast Premium. Using Yoast Premium allows me to no-index all category pages, all author pages, all archive and date pages, all search variable URLs and other secondary URLs, weird template-based document or category type, and any other things, like images and so forth. I generally salt the earth.

(SEO) Health Starts from the Core (Content)

My clients generally only care that their core content is on Google: their blog and news Posts, their site Pages, and their product pages. Most of them don't even want their landing pages to show up on search as they're optimized for ads or for promotionals. You can go even deeper than that and explicityly "no index" page by page explicitly. If there are pages that are so last season or are dead promotions or if there is old news about previous C-Suite executives that have come and gone; instead of deleting them and needing to use Redirection to heal any 404s with a 302, you can just deindex, noindex, no robot them. If you have 300 blog posts, 300 products, and 30 main pages, you should really only have 630 indexed pages, not 10,000 indexed and nonindexed pages, with 500 of them actually accepted into the Google Search Engine Index based on the Green Bar and downloadable CSV and viewable list of Pages that are indexed (you can look through it, eyeball the list, maybe Google did it pretty well based on the Sitemap.xml that Google has already discovered and uses as a rule of thumb both with regards to checking for updates and making sure the URLs, Pages, Posts, Recipes, Products, etc, are listed clearly in your artchitecure and in your main and sub sitemap XML files.

Blocking Bots Not Blocking Visitors

Remember, again: if you block Google and Bing spiders, bots, and robots, you're not blocking the visitors, too. You're also not preventing any of the search engines from spidering the content but you are letting them know that you would prefer they do not pay attention to it as part of their search engine. All platforms have some form of this, I have just become extremely proficient working with Yoast Premium. There is a weird quirk with regards to Yoast these days: browser caching and other caching. If your Yoast SEO changes don't take, try to make the changes behind the veil/wall of Incognito mode. We did a bunch of testing and it seems to make it easier to make sure that any changes to Yoast SEO and Yoast Premium take and don't revert by the next time you check them.

Hire Me!

Anway, be sure to ask me any questions you have and I would be really happy to help you with any SEO need you have. Please drop me a line or hire me over at UpWork.

Taming a Wild Octopus

Navigating the labyrinth of SEO optimization can often feel like trying to tame a wild octopus—slippery and unpredictable. But with a little bit of humor and a lot of patience, you can indeed master the beast. It's essential to not get too bogged down by the technical jargon and to remember that at the end of the day, it’s all about making your site user-friendly for both humans and search engine robots.

SEO Dysmorphic Disorder

Now, while it's vital to manage which pages get indexed, there's also the larger task of ensuring that your site actually shows up for relevant searches. This is where the magic of keyword research comes into play. Just tossing keywords into your content like confetti at a parade isn’t going to cut it. You need to be strategic—think of it more like precision-guided munitions in the battlefield of Google's search results. Tools like SEMRush, Ahrefs, or even Google’s own Keyword Planner can help you discover the terms people are actually using when they wiggle their fingers on the search keys. Once you’ve got your targeted list of keywords, integrate them into your site's content naturally. Force-feeding keywords where they don’t belong will not only turn off your readers but could also earn you a stern side-eye from Google.

Google Hearts Hot Donuts!

Speaking of content, let's not forget the power of compelling, valuable content. Google loves fresh, relevant content. Think of it as the search engine equivalent of humans preferring hot, fresh pizza over a cold, day-old slice. Blogs, how-to articles, videos, infographics—keeping these fresh and updated not only makes your site more appealing to visitors but also to Google’s algorithms.

Social Signals

Social signals are another often overlooked gem. Sure, social media may not directly influence rankings, but it amplifies your visibility, driving organic traffic back to your site. It’s like telling Google, "Hey, look how popular and relevant I am!" Every tweet, share, or like is a small vote of confidence in your site’s content.

Mobile Optimization

Lastly, don't forget about mobile optimization. With more than half of all global web traffic coming from mobile devices, your site better be as mobile-friendly as a kangaroo’s pouch. Google’s Mobile-Friendly Test can tell you where you stand. If your site isn’t up to scratch, you could be losing out on a massive chunk of potential traffic.

Wrapping it Up

To wrap it up, while SEO might seem as daunting as teaching quantum physics to a toddler, with the right tools and strategies, it's entirely manageable. Just remember to keep your user’s experience at the forefront of your efforts. After all, what good is driving traffic if visitors find your site about as appealing as a soggy toast?

Whether you’re a seasoned SEO guru or just dipping your toes in the vast ocean of search engine optimization, remember that it’s a marathon, not a sprint. Changes won’t happen overnight, but with consistent effort and a pinch of SEO savvy, you’ll see your pages climbing the ranks. And as always, if you feel overwhelmed, I'm just a message away—ready to help you navigate through the murky waters of SEO. Reach out, let's optimize together!

FAQ

Q1: Why should I consider no-indexing non-core pages? A1: No-indexing non-core pages like tag, author, or archive pages helps focus search engines on indexing your primary content—products, services, and main articles—improving your site's relevance and search rankings.

Q2: How does no-indexing affect my website's SEO? A2: It streamlines the crawl budget, allowing search engines to concentrate on your more important pages, which can improve site visibility and reduce the chances of duplicate content issues.

Q3: Can no-indexed pages still be accessed by users? A3: Yes, no-indexed pages are still accessible to users who have the URL. They just won't appear in search engine results, which helps maintain the focus on your more significant content.

Q4: What is the best way to implement a no-index strategy? A4: Tools like Yoast Premium offer easy management of no-index settings, allowing you to specify which types of pages should be hidden from search engines without affecting user access.

Q5: Is it necessary to no-index every non-core page? A5: While it’s not mandatory for all sites, no-indexing non-core pages can be beneficial for larger sites with many pages that dilute the focus and SEO performance of essential content.

Glossary of Terms

No-index: A directive that can be used in a page's meta tags or HTTP header to instruct search engines not to include the page in search results, thus not using resources to index these pages.

Core Content: Essential pages of a website, including main articles, product descriptions, and services, which provide significant value to visitors and are crucial for SEO.

Crawl Budget: The number of pages a search engine bot is programmed to crawl and index within a given time frame, often influenced by a site's perceived value and freshness of content.

Yoast Premium: A popular SEO plugin for WordPress that provides advanced features for managing no-index settings and other SEO optimizations tailored to enhance the visibility of primary site content.

SEO (Search Engine Optimization): The practice of increasing the quantity and quality of traffic to your website through organic search engine results.

Google Search Console (GSC): A web service by Google which allows webmasters to check indexing status and optimize visibility of their websites.

XML Sitemap: A file that helps search engines better understand your website structure by listing all your pages along with additional metadata regarding each page (like the last time it was updated).

Canonicalization: The process of choosing the best URL when there are several choices available, often used to prevent problems caused by duplicate content.

Meta Tags: Snippets of text that describe a page’s content; the tags themselves don’t appear on the page but only in the page’s code. Important for SEO, as they help search engines understand the topic of the page and index it accordingly.

Duplicate Content: Blocks of content within or across domains that completely match other content or are appreciably similar. Typically, this is not deceptive in origin, but it can affect search engine rankings.

Robots.txt: A file used to tell search engine crawlers which pages or sections of the site should not be processed or scanned.

Indexing: The process by which search engines organize information before a search to enable super-fast responses to queries.

SERP (Search Engine Results Page): The page displayed by search engines in response to a query by a searcher. The main component of the SERP is the listing of results that are returned by the search engine in response to a keyword query.

Backlink: An incoming hyperlink from one web page to another website. Backlinks are important for SEO because they represent a "vote of confidence" from one site to another, indicating to the search engine that others vouch for your content.

PageRank: An algorithm used by Google Search to rank web pages in their search engine results. It is used to measure the importance of website pages based on the links between them.