How to Configure Cloudflare WAF to Block Bot Scrapes on Forms

Web forms are the gateway to leads, registrations, and transactions on any site. They also attract automated bots that scrape data, submit junk, or try credential stuffing. A Cloudflare Web Application Firewall (WAF) can block most of this unwanted traffic before it reaches your server, preserving performance and protecting your data.

Why Use Cloudflare WAF for Bot Protection?

• Edge‑level filtering – Requests are examined at Cloudflare’s global network, reducing load on your VPS or dedicated server.
• Fine‑grained rules – Target specific URI paths, HTTP methods, or request headers used by scrapers.
• Managed rulesets – Cloudflare continuously updates bot‑control signatures, so you benefit from the latest threat intelligence without manual effort.
• Zero‑downtime deployment – Changes are applied instantly at the edge, keeping your site online while tightening security.

Prerequisites

1. A Cloudflare account with your domain added and DNS proxied (orange cloud).
2. Access to the Cloudflare dashboard (or an API token with Zone.Zone and WAF permissions).
3. Knowledge of the form endpoint(s) you want to protect (e.g., /contact.php, /api/submit).

1. Enable the Cloudflare WAF and Bot Management

Before creating custom rules, make sure the WAF and Bot Management features are active.

• Log in to the Cloudflare dashboard and select the domain.
• Navigate to Security > WAF. If the toggle is off, click Enable WAF.
• Under Security > Bots, set Bot Fight Mode to On. This adds a baseline layer of automated‑bot detection.

These built‑in protections block many generic scrapers. Targeted bots often mimic legitimate browsers, so custom rules are needed.

2. Create a Custom Firewall Rule to Target Form Submissions

Custom firewall rules let you filter traffic based on URI, request method, and user‑agent patterns.

1. In the dashboard, go to Security > WAF > Managed Rules and click Create Custom Rule.
2. Give the rule a clear name, such as Block Bot Scrapes on Form Submissions.
3. Set the Action to Block (or Challenge if you prefer a CAPTCHA).
4. Define the Expression:

(http.request.uri.path contains "/contact.php" or http.request.uri.path contains "/api/submit")
and http.request.method == "POST"
and (http.user_agent contains "python-requests" or http.user_agent contains "curl" or http.user_agent contains "bot")

This expression does three things:

• Matches the specific form endpoint(s) you want to protect.
• Limits the rule to POST requests, which are typical for form submissions.
• Blocks or challenges requests that identify themselves as common scripting tools or contain the word “bot” in the User‑Agent header.

After entering the expression, click Save. The rule appears with a default priority of 100. If you have other custom rules, ensure this one sits above any Allow rules that might unintentionally bypass it.

3. Fine‑Tune Bot‑Control Using Managed Rulesets

Cloudflare’s Managed Rulesets include specific bot‑control rules that you can toggle on or off. Adjusting them helps reduce false positives.

• Return to Security > WAF > Managed Rules.
• Locate the Cloudflare Bot Management set.
• Enable the following sub‑rules (they are usually off by default):
  • cft_bot_fingerprint – blocks known bad bots based on fingerprinting.
  • cft_bot_behavior – looks for rapid, repetitive requests typical of scrapers.
• If legitimate users are challenged, raise the Bot Sensitivity slider from “Low” to “Medium” or “High” under Security > Bots > Settings. Higher sensitivity catches more bots but may affect some edge cases.

4. Add Rate Limiting for Extra Protection

A determined scraper might rotate IPs and stay under the block threshold. Rate limiting caps the number of requests per IP over a short window.

1. Go to Security > Rate Limiting and click Create Rate Limit.
2. Configure:
   • URL pattern: /contact.php (or the endpoint you protect).
   • Method: POST.
   • Threshold: 5 requests per minute.
   • Action: Block for 30 minutes.
3. Save the rule. Cloudflare now blocks any IP that exceeds five form submissions in a minute, a typical pattern for bots.

5. Test Your Configuration

Before relying on the new rules, verify that legitimate traffic works while bots are blocked.

1. Manual test: Use a browser to submit the form normally. It should succeed.
2. Bot simulation: Run a curl command that mimics a scraper:

curl -X POST -d "name=Test&email=test@example.com" \
     -H "User-Agent: python-requests/2.28.1" \
     https://yourdomain.com/contact.php

You should receive a 403 Forbidden (or a CAPTCHA challenge if you chose that action).

1. Review logs: In the dashboard, open Security > Events. Look for entries matching your custom rule or rate‑limit. Confirm that legitimate requests are logged as “Allowed” and bot‑like requests as “Blocked”.

6. Ongoing Maintenance Tips

• Monitor false positives: Check the Events log weekly. If you see many “Blocked” entries from real users, adjust the User-Agent condition or lower the rate‑limit threshold.
• Update endpoints: Whenever you add a new form or API endpoint, repeat steps 2‑4 to protect it.
• Leverage the API for automation: If you manage many domains, use Cloudflare’s API to deploy the same rule set across them. The API endpoint is POST /zones/:zone_identifier/firewall/rules with a JSON payload matching the expression shown earlier.
• Stay informed: Cloudflare releases new bot‑control signatures regularly. Review the “Managed Rules” page after major updates to ensure new rules are enabled.

Conclusion

By enabling Cloudflare’s WAF, creating a targeted custom rule, fine‑tuning managed bot controls, adding rate limiting, and regularly testing the configuration, you can effectively stop most automated scrapers from abusing your form submissions. The protection happens at the edge, keeping your AtoZNode VPS or dedicated server free from unnecessary load while preserving a smooth experience for genuine users.