To avoid undesirable content inside the lookup indexes, site owners can instruct spiders not to crawl selected information or directories through the regular robots.txt file in the foundation Listing from the area. Furthermore, a web page can be explicitly excluded from a internet search engine's database by using a meta https://news.rafeeg.ae