robots.txt

HRobertson

New Member
I'm a little nervous about adding restrictions on my robots.txt file. I want to allow all but a couple of pages from being indexed by all robots. Could someone please verify that the following will work:

User-Agent: *
Disallow: /hierarchy1/
Disallow: /westernphilosophy/

Want I need to ensure is that everything But the URLs containing the above mentioned strings are crawled and hopefully indexed. Will this work?

Any help is greatly appreciated. Thank you...
 

vavietnam

New Member
Maybe you've posted to a wrong forum, so that this thread's now have 0 reply.

Plz suggest in the Web Promotion > Search Engine Optimization and Marketing
 
Top