Validate robots.txt configuration files client-side. Test a target request path and User-Agent parameters against directives to check indexing status.
robots.txt Rules Input
Loading...
Crawler Simulation Parameters
User-Agent to test
Target Path to check
Simulation Audit Outcome
Access Allowed! Web crawlers can index this path.
Matched Directive Details
No matching disallow rule blocked this user-agent. The crawler assumes permission to access the URL by default.
Crawling PermittedDirectives: 2 agents configured