Tips to be crawled by Search Engines

3dmagicaldesign · 10-04-2008, 02:42 PM

If your website is frequently visited by the Search Engine crawler then it is the first sign that your site appeals to that Search Engine. One needs to design a website in such a way that Search Engines find it important and thus love to crawl it quite frequently.
You cannot force a Crawler to visit you website more often, but of course you can do things that would attract crawlers to your website. You can take following measures to increase crawl rate:

1. Keep updating your content regularly - You need to update your website frequently, because Search Engines are looking for website with good amount of activity.
2. Check for Server Errors: When crawlers reach your page you don't want to them to face a server error or an unreachable page, Webmaster tools can help you with this.
3. Page Load Time: Crawlers have limited time to crawl your website; if it takes too long to index a single page then chances are that not all your pages will get crawled. So keep your pages light.
4. Check the site internal Website linking: Be sure there is no duplicate content that is returned via different URLs. Here the more time the crawler spends figuring out your duplicate content.
5. Get back links from websites that are crawled regularly by SERPS.
6. You can also adjust the spider crawl speed using Google Webmaster tools.
7. Use a Sitemap as it will help crawlers to crawl different pages of your website more easily.
8. Check whether your error handling is done properly in case your website encounters an error don't make the crawlers figure out what has happened.
9. Keep your Meta tags and Title Tag unique.
10. Regularly monitor Google, Yahoo crawl rate for your website and try to understand what works and what doesn't.

All these above tips will help to improve crawl rate of your website. This will also ensure that each of your pages get updated in search engine databases regularly.

fromthe5 · 10-08-2008, 03:00 PM

Quote:

Originally Posted by 3dmagicaldesign

If your website is frequently visited by the Search Engine crawler then it is the first sign that your site appeals to that Search Engine.

Well... Sort of. Although there are some really good tips in the above post, the fact is that spiders follow links. If your site has links pointing to it from other sites that are frequently spidered, then the spiders will follow those links to your site. The more links you have pointed to your site from other sites, the more your site will be spidered. 3dmagicaldesign's tips are good to make sure that once the spider gets to your site you will have a much better chance of getting and keeping your site "indexed" in the major search engines.

3dmagicaldesign · 10-08-2008, 03:58 PM

Thanks a lot for your comment, you are also right. I posted this information because it is useful, of course I also agree that when it comes to SEO you can't only use a strategy, you need to try as many strategies as possible. What do you think?

fromthe5 · 10-08-2008, 04:29 PM

With the latest Google update (Not the PR update) which may put a bit more emphasis to "on page factors", the fact remains that inbound links (IBLs) still seem to be the major ranking factor, and this is true to a noticeable extent with Yahoo and MSN also. The more IBLs you have with the correct "targeted link text" the higher that term will rank. From there the value of each link depends on:

1. The relevance to your site of the page carrying the link.

2.The PR of the page carrying the link. (link juice)

3. The age of the link. (I am one of those people who believes that this a a factor for Google.

4. One way links carry a lot more "link juice" than a link exchange.

These are what I consider the most important "link" factors. Undoubtedly there are more, but IBLs are the thing you should be concentrating on.

3dmagicaldesign · 10-08-2008, 04:45 PM

Well..

Right now I am focusing on many factors:

Blogging, link exchanging, meta tags, forums, etc and everything has brought me good results. As I say, as most strategies you apply, the most good results you're gonna achieve.

fromthe5 · 10-08-2008, 06:48 PM

I'm glad that everything is working out well for you. Does that mean that your site is ranking highly in the major search engines for the terms you want to be found for? Nice site you have there, however the link in your sig takes me to the web design page and I can't seem to make the links to other pages work, although the ones on your home page work fine.

3dmagicaldesign · 10-08-2008, 07:03 PM

Right now I am ranking for 3d architectural tours, and increasing my popularity for 3d architectural renderings and 3d architectural renderings. Yes, regarding the web design page, my website is being updated at the moment, it will be fixed in some minutes. Thanks a lot for that.

fromthe5 · 10-08-2008, 09:01 PM

I would think that there are more people searching for "3d architectural renderings" than for "3d architectural tours". According to Yahoo you have about 120 IBLs. The #1 site in Google for "3d architectural renderings" has over 1000 IBLs and the #2 site has over 2500 IBLs. If you submitted to a few hundred directories using your search term as the link text I bet you could jump up pretty high on page #1 though.

3dmagicaldesign · 10-08-2008, 09:10 PM

I am already aware of this

, that's why I am working hard on the website, we were not even in top 500 last month.

walkingbird · 10-10-2008, 06:16 AM

Directory submissions are garbage you want to put your links on pages that actual get visitors - that way you kill 2 birds with one stone - you get click troughs and the anchor text perks.

10-04-2008, 02:42 PM	#1 (permalink)
3dmagicaldesign Silver Member Join Date: May 2008 Posts: 169	Tips to be crawled by Search Engines If your website is frequently visited by the Search Engine crawler then it is the first sign that your site appeals to that Search Engine. One needs to design a website in such a way that Search Engines find it important and thus love to crawl it quite frequently. You cannot force a Crawler to visit you website more often, but of course you can do things that would attract crawlers to your website. You can take following measures to increase crawl rate: 1. Keep updating your content regularly - You need to update your website frequently, because Search Engines are looking for website with good amount of activity. 2. Check for Server Errors: When crawlers reach your page you don't want to them to face a server error or an unreachable page, Webmaster tools can help you with this. 3. Page Load Time: Crawlers have limited time to crawl your website; if it takes too long to index a single page then chances are that not all your pages will get crawled. So keep your pages light. 4. Check the site internal Website linking: Be sure there is no duplicate content that is returned via different URLs. Here the more time the crawler spends figuring out your duplicate content. 5. Get back links from websites that are crawled regularly by SERPS. 6. You can also adjust the spider crawl speed using Google Webmaster tools. 7. Use a Sitemap as it will help crawlers to crawl different pages of your website more easily. 8. Check whether your error handling is done properly in case your website encounters an error don't make the crawlers figure out what has happened. 9. Keep your Meta tags and Title Tag unique. 10. Regularly monitor Google, Yahoo crawl rate for your website and try to understand what works and what doesn't. All these above tips will help to improve crawl rate of your website. This will also ensure that each of your pages get updated in search engine databases regularly. __________________ 3D MAGICAL DESIGNS 3d Architectural Business Development, Web Design email: mauricio@3dmagicaldesigns.com website: www.3dmagicaldesigns.com office: +57 (67) 461991 cell: +57 (317) 3417958 USA: (239) 2475835

10-08-2008, 03:58 PM	#3 (permalink)
3dmagicaldesign Silver Member Join Date: May 2008 Posts: 169	Thanks a lot for your comment, you are also right. I posted this information because it is useful, of course I also agree that when it comes to SEO you can't only use a strategy, you need to try as many strategies as possible. What do you think? __________________ 3D MAGICAL DESIGNS 3d Architectural Business Development, Web Design email: mauricio@3dmagicaldesigns.com website: www.3dmagicaldesigns.com office: +57 (67) 461991 cell: +57 (317) 3417958 USA: (239) 2475835

10-08-2008, 04:45 PM	#5 (permalink)
3dmagicaldesign Silver Member Join Date: May 2008 Posts: 169	Well.. Right now I am focusing on many factors: Blogging, link exchanging, meta tags, forums, etc and everything has brought me good results. As I say, as most strategies you apply, the most good results you're gonna achieve. __________________ 3D MAGICAL DESIGNS 3d Architectural Business Development, Web Design email: mauricio@3dmagicaldesigns.com website: www.3dmagicaldesigns.com office: +57 (67) 461991 cell: +57 (317) 3417958 USA: (239) 2475835

10-08-2008, 07:03 PM	#7 (permalink)
3dmagicaldesign Silver Member Join Date: May 2008 Posts: 169	Right now I am ranking for 3d architectural tours, and increasing my popularity for 3d architectural renderings and 3d architectural renderings. Yes, regarding the web design page, my website is being updated at the moment, it will be fixed in some minutes. Thanks a lot for that. __________________ 3D MAGICAL DESIGNS 3d Architectural Business Development, Web Design email: mauricio@3dmagicaldesigns.com website: www.3dmagicaldesigns.com office: +57 (67) 461991 cell: +57 (317) 3417958 USA: (239) 2475835

10-08-2008, 09:10 PM	#9 (permalink)
3dmagicaldesign Silver Member Join Date: May 2008 Posts: 169	I am already aware of this , that's why I am working hard on the website, we were not even in top 500 last month. __________________ 3D MAGICAL DESIGNS 3d Architectural Business Development, Web Design email: mauricio@3dmagicaldesigns.com website: www.3dmagicaldesigns.com office: +57 (67) 461991 cell: +57 (317) 3417958 USA: (239) 2475835

10-08-2008, 04:29 PM	#4 (permalink)
fromthe5 Bronze Member Join Date: Oct 2008 Location: Cincinnati Posts: 47	With the latest Google update (Not the PR update) which may put a bit more emphasis to "on page factors", the fact remains that inbound links (IBLs) still seem to be the major ranking factor, and this is true to a noticeable extent with Yahoo and MSN also. The more IBLs you have with the correct "targeted link text" the higher that term will rank. From there the value of each link depends on: 1. The relevance to your site of the page carrying the link. 2.The PR of the page carrying the link. (link juice) 3. The age of the link. (I am one of those people who believes that this a a factor for Google. 4. One way links carry a lot more "link juice" than a link exchange. These are what I consider the most important "link" factors. Undoubtedly there are more, but IBLs are the thing you should be concentrating on.

10-08-2008, 06:48 PM	#6 (permalink)
fromthe5 Bronze Member Join Date: Oct 2008 Location: Cincinnati Posts: 47	I'm glad that everything is working out well for you. Does that mean that your site is ranking highly in the major search engines for the terms you want to be found for? Nice site you have there, however the link in your sig takes me to the web design page and I can't seem to make the links to other pages work, although the ones on your home page work fine.

10-08-2008, 09:01 PM	#8 (permalink)
fromthe5 Bronze Member Join Date: Oct 2008 Location: Cincinnati Posts: 47	I would think that there are more people searching for "3d architectural renderings" than for "3d architectural tours". According to Yahoo you have about 120 IBLs. The #1 site in Google for "3d architectural renderings" has over 1000 IBLs and the #2 site has over 2500 IBLs. If you submitted to a few hundred directories using your search term as the link text I bet you could jump up pretty high on page #1 though.

10-10-2008, 06:16 AM	#10 (permalink)
walkingbird Silver Member Join Date: Aug 2008 Posts: 102	Directory submissions are garbage you want to put your links on pages that actual get visitors - that way you kill 2 birds with one stone - you get click troughs and the anchor text perks. __________________ Marshal Finch Regina Web Design Make Money Online to- with seo(search engine optimization) Logo Picture Gallery - rate, submit and review Forum Link Expermient

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode