Can I control what webcrawlers have access to?

HTML Help Forum

FAQ

Memberlist

Profile

Register - Not registered yet? Got something to say? Join HTML Code Tutorial!

Can I control what webcrawlers have access to?

HTML Help Forum -> Member Tutorials

View previous topic :: View next topic

Author

Message

mjpliv

Joined: 11 May 2004
Posts: 402
Location: Nova Scotia

Posted: Fri Jun 11, 2004 3:26 pm Can I control what webcrawlers have access to?

There is a meta tag available that will help (or deny) a webcrawler navigation of your website. It should appear in the head section of your HTML with the other meta tags and the syntax is -

Code:

There are several options for the "content" portion of this tag and they are covered quite nicely at this website -

http://www.seoconsultants.com/meta-tags/robots.htm

Here is a FAQ page dealing with most robot issues -

http://www.robotstxt.org/wc/faq.html

Display posts from previous:

	HTML Help Forum -> Member Tutorials	All times are GMT - 8 Hours
Page 1 of 1

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Your Ad Here

DARFUR

Get a Stop the Genocide poster at SaveDarfur.org.
Also see Eyes on Darfur.

HOSTING / DESIGN

$1.99 Domain Registration
Cheap Web Hosting
Website Templates

MAKE MONEY

Sell Domain Names
Sell Website Templates

| | | | |