Alden Bates (abates) wrote in suggestions,
Alden Bates

Block search engines from indexing syndicated accounts

Block search engines from indexing syndicated accounts

Short, concise description of the idea
Search engines should be prevented from indexing all pages on

Full description of the idea
Currently LJ allows Google, Yahoo, etc, to index syndicated journals. Since these are duplications of content from other sites, it doesn't make sense for posts in syndicated accounts to be indexed seperately. Additionally posts expire from syndicated accounts after two weeks leaving search engines with blank month and day archive pages.

An ordered list of benefits
  • Cuts down on unnecessary traffic from search engine spiders.
  • Avoids clogging search engines with URLs which won't have any useful content in a short period of time.
  • Less duplication of content in search engines.

An ordered list of problems/issues involved
  • may make it hard to find syndicated feeds via search engines, but the userinfo pages will still be indexed as they're on

An organized list, or a few short paragraphs detailing suggestions for implementation
  • Create a robots.txt file for blocking all spiders, or
  • Hardcode the spider blocking flag for syndicated journals to on.
Tags: external services, syndication, § migrated
  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

← Ctrl ← Alt
Ctrl → Alt →
← Ctrl ← Alt
Ctrl → Alt →