Robots.txt Disallows All User-Agents
Short, concise description of the idea
Robots.txt disallows all user-agents, which means that my livejournal cannot be indexed by search engines.
Full description of the idea
Despite setting my journal to public, and unchecking "minimize inclusion to search engines," my robots.txt remains the same, it blocks all search engines. What's the use of having a blog if not crawled by search engines anyway? How would one get new visitors?
My LiveJournal URL is: http://skillsfuture.livejournal.com/
- It's user-friendly.
- Robots.txt is not changed despite setting privacy setting to PUBLIC.