PDA

View Full Version : why is google so slow


tfgames
09-21-2005, 09:14 PM
My site is one week old and msn hits it every days so far like 240 times, why hasnt google even hit me once yet and how long does it normally take? i have loads of other sites linking to it some with pr5 but still google hasnt come to me yet.
P.S howcome this forum is top of the list when you search for "web design forum" but it has a pr0?

SiteExpress
09-21-2005, 09:27 PM
It took Google weeks to hit some of my sites, even after I manually submitted them. I thinkit is just kind of random.

Kiwiberry
09-21-2005, 09:42 PM
I’m not sure if this is coincidence or not, but I was reading about google using a sitemap.xml to help index websites. So I read about creating the sitemap.xml file (which is very similar to robots.txt). Once I added it, I started getting hits from the googlebot, and I don’t have a lot of traffic to my site. I’ve got two hits this month, and I’m alright with that.

tfgames
09-21-2005, 10:24 PM
Will google still index me if i dont have a robot.txt file or do you have to have this file?

StephanieCordray
09-21-2005, 10:52 PM
yes, they will still index you.. all a robot.txt does is make sure the robots index every page and block other bots you don't want on your site. Since you want google to index you, you won't have a problem as long as they can find your site. The easier it is to find your site, the better.

tfgames
09-21-2005, 10:59 PM
Hey stephanie, some bot called LinkWalker hit my site around 180 times so i checked on google what linkwalker is and what i saw some guy was saying it downloads everything of your site and stuff. So if i make a robot.txt i can stop LinkWalker from coming back right?. Im still new and had never heard of robot.txt before but im gonna read up on it now thanks.