Robots in SEO

For a site to have good SEO, its better that the Search engine gets your pages faster & to the relevant content. This can be done by indexing the site. Indexing is important because the spiders or search engine crawlers first visit & search for the robot file in your site. This Robots.txt file tells [...]