OldRobot: Robots.txt History


Try: Amazon.com, Google.com, PayPal.com

OldRobot scrapes the history of a robots.txt file from the Wayback Machine, removes any duplicates, and lets you see the differences between version. Questions? @KiwiAlec