Alden Bates (abates) wrote in suggestions,
Alden Bates
abates
suggestions

Block search engines from indexing syndicated accounts

Title
Block search engines from indexing syndicated accounts

Short, concise description of the idea
Search engines should be prevented from indexing all pages on syndicated.livejournal.com

Full description of the idea
Currently LJ allows Google, Yahoo, etc, to index syndicated journals. Since these are duplications of content from other sites, it doesn't make sense for posts in syndicated accounts to be indexed seperately. Additionally posts expire from syndicated accounts after two weeks leaving search engines with blank month and day archive pages.

An ordered list of benefits
  • Cuts down on unnecessary traffic from search engine spiders.
  • Avoids clogging search engines with URLs which won't have any useful content in a short period of time.
  • Less duplication of content in search engines.

An ordered list of problems/issues involved
  • may make it hard to find syndicated feeds via search engines, but the userinfo pages will still be indexed as they're on www.livejournal.com

An organized list, or a few short paragraphs detailing suggestions for implementation
  • Create a robots.txt file for syndicated.livejournal.com blocking all spiders, or
  • Hardcode the spider blocking flag for syndicated journals to on.
Tags: external services, syndication, § migrated
Subscribe
  • Post a new comment

    Error

    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded 

  • 19 comments
Previous
← Ctrl ← Alt
Next
Ctrl → Alt →
Previous
← Ctrl ← Alt
Next
Ctrl → Alt →