To prevent unwanted material while in the research indexes, webmasters can instruct spiders not to crawl certain documents or directories from the conventional robots.txt file in the foundation directory from the domain. Furthermore, a webpage might be explicitly excluded from the online search engine's databases by using a meta tag https://www.facebook.com/acourseinmiraclesdavid/