Google
HTML Tutorial


 Forum HomeForum Home   FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
RegisterRegister - Not registered yet? Got something to say? Join HTML Code Tutorial!
Can I control what webcrawlers have access to?
Post new topic   Reply to topic    HTML Help Forum Index -> HTML FAQ
View previous topic :: View next topic  
Author Message
mjpliv



Joined: 11 May 2004
Posts: 406
Location: Nova Scotia
PostPosted: Fri Jun 11, 2004 3:26 pm     Can I control what webcrawlers have access to? Reply with quote
There is a meta tag available that will help (or deny) a webcrawler navigation of your website. It should appear in the head section of your HTML with the other meta tags and the syntax is -

Code:
<meta name="robots" content="index,follow">


There are several options for the "content" portion of this tag and they are covered quite nicely at this website -

http://www.seoconsultants.com/meta-tags/robots.htm

Here is a FAQ page dealing with most robot issues -

http://www.robotstxt.org/wc/faq.html
Display posts from previous:   
Post new topic   Reply to topic    HTML Help Forum Index -> HTML FAQ All times are GMT - 8 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
HTML Help Archive
Powered by phpBB © 2001, 2005 phpBB Group
 
HOSTING / DESIGN
$1.99 Domain Registration
Cheap Web Hosting
Website Templates
MAKE MONEY
Sell Domain Names
Sell Website Templates

Home   |   Tutorials   |   Forum   |   Quick List   |   About
Copyright ©1997-2002 Idocs and ©2002-2007 HTML Code Tutorial