What is the difference between crawling and indexing?

Discussion in 'Search Engine Optimization and Marketing' started by phpassignmenthelp, Dec 14, 2016.

to remove all ads.
Thread Status:
Not open for further replies.
  1. phpassignmenthelp

    phpassignmenthelp Member

    Messages:
    34
    Hi Everyone !! What is the difference between crawling and indexing?
     
  2. chrishirst

    chrishirst Well-Known Member Staff Member

    Messages:
    2,638
    Crawling is the retrieval of the source code from a URL and storing the data in the 'indexer' queue database.

    Indexing is the process of reading the stored source data and adding the words, along with their positions in the document to an 'inverted index' and associating that data with the document ID for later scoring and weighting.
     
  3. jainteq

    jainteq Member

    Messages:
    42
    Crawling: When Google visits your website for tracking purposes. This process is done by Google’s Spider crawler. Indexing: After crawling has been done, the results get put onto Google’s index (i.e. web search).
     
  4. chrishirst

    chrishirst Well-Known Member Staff Member

    Messages:
    2,638
    "Tracking" has nothing to do with it, unless you are using 'tracking' in whole other way than the usual.

    So:

    if you mean 'tracking' as in "checking the URL for changes to the content" ... ... No it isn't.

    if you mean 'tracking' as in "checking the URL following some 'ping' request" ... ... No it isn't.

    if you mean 'tracking' as in "looking at some visitor statistics" ... ... No it isn't.

    if you mean 'tracking' as in "checking the URL after a new 'link' is discovered" ... ... Not really but that may happen.


    An individual URL, NOT a 'website' is crawled on a schedule determined by Google's system, seemingly based on links and a Google maintained history of when the content at the particular URL has been changed when previously visited.
     
    hariandro95 likes this.
  5. hariandro95

    hariandro95 New Member

    Messages:
    27
    hello phpassignmenthelp , glad to answer this question because i also had this doubt before.Lets know about that..
    Crawling
    Crawling is the process search engines discover updated content on the web, such as new sites or pages, changes to existing sites, and dead links.
    To do this, a search engine uses a algorihthat can be referred to as a 'crawler', ‘bot’ or ‘spider’ which follows an algorithmic process to determine which sites to crawl and how often.As a search engine's crawler moves through your site it will also detect and record any links it finds on these pages and add them to a list that will be crawled later. This is how new content is discovered.

    Indexing
    Once a search engine processes each of the pages it crawls, it compiles a massive index of all the words it sees and their location on each page. It is essentially a database of billions of web pages.
    This extracted content is then stored, with the information then organised and interpreted by the search engine’s algorithm to measure its importance compared to similar pages.
     
  6. Steve robert

    Steve robert New Member

    Messages:
    11
    Crawling means google has crawled your site atleast once and your site goes into their crawled database. Indexing means google has crawled your site and found it valuable enough to put it in their indexed pages database.


    link dropping removed
     
    Last edited by a moderator: Jul 9, 2017
  7. chrishirst

    chrishirst Well-Known Member Staff Member

    Messages:
    2,638
    That is a circular 'definition', you cannot use two verbs derived from the same word to 'explain' the original word.

    Besides that you are incorrect , because crawling is a present tense verb while, crawled is a past tense verb. So crawling is performing the action of a 'crawl' while crawled is after the 'crawl' is completed.


    No again, indexing is purely the operation of reading the retrieved data when the source URL was crawled at this point all indexed URLs are added to their database 'valuable' or not
     
  8. chrishirst

    chrishirst Well-Known Member Staff Member

    Messages:
    2,638
    THAT could well be the very WORST example of automatic "translation" that I have EVER seen, and the article 'spinner' you obviously used then turned it into complete gibberish.


    The word is crawl, not creep or slithers, neither of those can be a synonym for 'crawl' when writing or speaking in a search engine context
     
Thread Status:
Not open for further replies.

Share This Page