Block search engines from indexing syndicated accounts
Short, concise description of the idea
Search engines should be prevented from indexing all pages on syndicated.livejournal.com
Full description of the idea
Currently LJ allows Google, Yahoo, etc, to index syndicated journals. Since these are duplications of content from other sites, it doesn't make sense for posts in syndicated accounts to be indexed seperately. Additionally posts expire from syndicated accounts after two weeks leaving search engines with blank month and day archive pages.
- Cuts down on unnecessary traffic from search engine spiders.
- Avoids clogging search engines with URLs which won't have any useful content in a short period of time.
- Less duplication of content in search engines.
An ordered list of problems/issues involved
- may make it hard to find syndicated feeds via search engines, but the userinfo pages will still be indexed as they're on www.livejournal.com
An organized list, or a few short paragraphs detailing suggestions for implementation
- Create a robots.txt file for syndicated.livejournal.com blocking all spiders, or
- Hardcode the spider blocking flag for syndicated journals to on.