Informational only. Not medical advice.INFORMATIONAL PLATFORM ONLY — NOT MEDICAL ADVICE, DIAGNOSIS, OR TREATMENT
Informational only. Not medical advice.INFORMATIONAL PLATFORM ONLY — NOT MEDICAL ADVICE, DIAGNOSIS, OR TREATMENT
Integration guide for site operators and vendor IT teams.
DisclosedLabsBot is the web crawler operated by Disclosed Labs. It scans peptide vendor websites for FDA compliance issues, identifying language and claims that may trigger warning letters or enforcement actions. Scans are only initiated by verified domain owners.
Every request includes two identifying headers:
User-Agent: DisclosedLabsBot/1.0 (+https://disclosedlabs.com/bot; compliance scanning) From: bot@disclosedlabs.com
Match on either header when creating allow rules. The User-Agent substring DisclosedLabsBot is the simplest match.
Disclosed LabsAllow DisclosedLabsBot.(http.user_agent contains "DisclosedLabsBot")If you also use Cloudflare Bot Fight Mode or Super Bot Fight Mode, you may need to add DisclosedLabsBot to your verified bots list or create a separate skip rule with the same expression under Security → Bots.
Sucuri
In the Sucuri dashboard, go to Access Control → User-Agents and add DisclosedLabsBot to the allowlist.
Wordfence (WordPress)
Go to Wordfence → Firewall → Rate Limiting. Add DisclosedLabsBot under the allowlisted User-Agents. Alternatively, go to All Options → Allowlisted URLs and allowlist by the From header.
Apache / .htaccess
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} DisclosedLabsBot [NC]
RewriteRule .* - [L]Place this before any deny rules in your .htaccess.
Google reCAPTCHA
reCAPTCHA does not support User-Agent allowlisting. If reCAPTCHA is blocking the scanner, the simplest fix is to exempt the bot at the WAF level (see above) so requests never reach the captcha. If that is not possible, contact us to set up authenticated scanning.
hCaptcha
hCaptcha Enterprise supports bot allowlisting. In your hCaptcha dashboard, go to Settings → Advanced and add our User-Agent or From header to the passthrough list. On non-Enterprise plans, bypass at the WAF level instead.
Cloudflare Turnstile
Turnstile challenges are handled by Cloudflare's managed rules. Creating a WAF skip rule (see Cloudflare section above) will bypass Turnstile for DisclosedLabsBot.
If your product pages are behind authentication, contact us at bot@disclosedlabs.com to set up authenticated scanning. We will configure secure, credential-based access to crawl behind your login. Your credentials are encrypted at rest and used only for compliance scans.
DisclosedLabsBot respects robots.txt. If you disallow our bot, we will not crawl those paths. To explicitly allow DisclosedLabsBot while blocking other crawlers:
User-agent: DisclosedLabsBot Allow: / User-agent: * Disallow: /admin/
DisclosedLabsBot is designed to be lightweight. It enforces a minimum 1.5-second delay between requests and will never fetch more than 50 pages per scan. A full scan typically completes in under 90 seconds. If your rate-limiting rules are stricter than 1 request per 1.5 seconds, you may need to add the User-Agent to your rate-limit exemption list.
DisclosedLabsBot runs on serverless infrastructure. Outbound IP addresses rotate and cannot be predicted. Do not rely on IP-based allowlisting. Use User-Agent-based rules as described above. The User-Agent string is present on every request and is the reliable way to identify our bot.
Scan returned 0 pages?
Your site is likely blocking the bot. Check for:
robots.txt disallowing DisclosedLabsBot or all botsFollow the WAF and captcha steps above, then re-run the scan. If the problem persists, contact us with your domain and we will diagnose it.
For integration help, allowlisting questions, or to set up authenticated scanning: bot@disclosedlabs.com