A robots.txt file is utilised primarily to control crawler traffic to your web site, and typically to help keep a file off Google, dependant upon the file variety: robots.txt effect on distinct file types This is a ASCII file Found at the root in the website, and has to be https://seotoolstube.com/