# /robots.txt for http://www.vlaamsparlement.be # case-insensitive User-agent: wget User-agent: twiceler User-agent: Jyxobot Disallow: / User-agent: * Disallow: /Proteus5/feedbackDetails.action? Crawl-delay: 10 User-agent: * Crawl-delay: 600 Visit-time: 0100 - 0500 Request-rate: 1/30