For Gold, Peace, and Freedom


How to Avoid Duplicate Content in Wordpress Blogs

April 20th, 2007

As I mentioned at the end of my article about simple SEO strategies, duplicate content can be a real problem for websites and is definitely something that you should avoid whenever you can. Much to my chagrin, I have discovered that Wordpress blogs seem to have their own particular issues with this.

Wordpress (and some other blogging platforms) can cause problems with duplicate content because of the way that the blog’s pages are displayed and indexed. For example, in the default Wordpress format, the same page (or at least a substantial part of it) can show up on the home page, in the monthly archives, on the pages for each category (and sometimes multiple categories), on the permalink page, and in various feeds, including RSS and comment feeds. When the search engine spiders crawl over all of these pages, they will detect lots of duplicates, and this will cause many of your pages to get thrown into the supplemental index or even be blocked from the search engine results entirely. Fortunately, after doing some research, I have found four useful fixes and workarounds for this:

1. The “more” tag is your friend. Use it wisely!

Using the “more” tag, which divides your page text between what is shown on the home page and what is published on the permalink page, can solve much of the problem with duplicates because the home page is often the first page that the search engine spiders begin crawling. In fact, according to what I have read in some of the webmaster forums, some bloggers were able to solve their duplicate issues entirely, even without making any other changes, by simply breaking up their posts with judicious use of the “more” tag feature.

I have found that this practice helps with overall appeal to the visitor as well; by placing only one or two sentences on the actual home page, the visitors do not have to scroll down through a long post to read the titles of previous entries. Meanwhile, if they have any real interest in the topic of the post, they are enticed to click through to the permalink page in order to read the rest of that entry.

2. Use robots.txt to direct search engine spiders to your favorite page

The robots.txt file can be used to block spiders from crawling pages that you do not want them to index. For example, to keep them out of your category pages so that Google does not detect duplicates between these and your permalink pages, you can simply write a line that reads:

Disallow: /category/

You can repeat the “disallow” parameter for other pages as well, such as your admin pages, your login page, feed pages, dynamic pages with “?” in the URL, and any other pages that you probably don’t want or need indexed by a search engine.

3. Make sure that you have unique titles and meta descriptions for your posts

It is important not to have two or more posts with exactly the same title tag, especially since this is one of the first things that Google looks at when ranking your pages. Keywords in the meta description fields also should not be repeated more than once because this could be seen as keyword stuffing, a notorious “black hat” SEO tactic that search engines have learned to penalize.

4. Do not copy and paste whole pages from other websites

Okay, if you have managed to read this far, #4 should be pretty obvious but it still bears repeating because it is so tempting. You should always use original content and avoid outright plagiarism; otherwise, you will be sentenced to the dungeon of “Supplemental Index” and receive practically no traffic. Paraphrasing is OK, although you should still include your own thoughts and experiences whenever possible because it makes your posts look more authentic and believable to your readers. You can also hire a ghostwriter if you do not have the time to write lots of original content, but it is a good idea to make sure that the articles he/she provides have not been copied from somewhere else on the Internet.

So far, these techniques have proved effective; I have managed to get Google to remove almost all of my pages from the supplemental index, and most of the pages that are appearing in the regular index are the “pretty permalink” pages, which are the most desirable ones in terms of SEO and monetization. After applying the modifications, you may need to wait for a few days while Google sorts out all of the changes and begins displaying the correct pages in the search results.

For more information about what Google considers duplicate content and how to handle this issue, you can read their recommendations at the Official Google Webmaster Central Blog.

2 Responses to “How to Avoid Duplicate Content in Wordpress Blogs”

  1. comment number 1 by: bloggernoob

    you really know your stuff. really glad i found your site. do you run other blogs too?

  2. comment number 2 by: Karlonia

    Other blogs? Ha- if only I had the time! Since this blog is just a one-person operation at this point, I seem to have my hands full with this one.

    Eventually I’m hoping to free up some time by accepting submissions from other people who may want to publish their articles or other content here in return for free backlinks and publicity. Once I get to the point where I don’t have to spend so much time researching, writing, editing, proofreading, and optimizing each article on a daily basis, I might be able to build some smaller niche blogs. Then I could “bum market” their content at other article directories, optimize them for search traffic, or even use them as marketing funnels.

Post Your Comments, Opinions, or Suggestions Here:


Email (optional)

Website (optional)