Uncovering CloudFlare - HackTricks

# Uncovering CloudFlare

> [!TIP] Learn & practice AWS Hacking:HackTricks Training AWS Red Team Expert (ARTE)
> Learn & practice GCP Hacking: HackTricks Training GCP Red Team Expert (GRTE)

> Support HackTricks
> - Check the subscription plans!
> - Join the 💬 Discord group or the telegram group or follow us on Twitter 🐦 @hacktricks_live.
> - Share hacking tricks by submitting PRs to the HackTricks and HackTricks Cloud github repos.

## Common Techniques to Uncover Cloudflare

- You can use some service that gives you the historical DNS records of the domain. Maybe the web page is running on an IP address used before.
  - Same could be achieve checking historical SSL certificates that could be pointing to the origin IP address.
  - Check also DNS records of other subdomains pointing directly to IPs, as it's possible that other subdomains are pointing to the same server (maybe to offer FTP, mail or any other service).
- If you find a SSRF inside the web application you can abuse it to obtain the IP address of the server.
- Search a unique string of the web page in browsers such as shodan (and maybe google and similar?). Maybe you can find an IP address with that content.
  - In a similar way instead of looking for a uniq string you could search for the favicon icon with the tool: https://github.com/karma9874/CloudFlare-IP or with https://github.com/pielco11/fav-up
  - This won't work be very frequently because the server must send the same response when it's accessed by the IP address, but you never know.

## Tools to uncover Cloudflare

- Search for the domain inside http://www.crimeflare.org:82/cfs.html or https://crimeflare.herokuapp.com. Or use the tool CloudPeler (which uses that API)
- Search for the domain in https://leaked.site/index.php?resolver/cloudflare.0/
- CloudFlair is a tool that will search using Censys certificates that contains the domain name, then it will search for IPv4s inside those certificates and finally it will try to access the web page in those IPs.
- CloakQuest3r: CloakQuest3r is a powerful Python tool meticulously crafted to uncover the true IP address of websites safeguarded by Cloudflare and other alternatives, a widely adopted web security and performance enhancement service. Its core mission is to accurately discern the actual IP address of web servers that are concealed behind Cloudflare's protective shield.
- Censys
- Shodan
- Bypass-firewalls-by-DNS-history
- If you have a set of potential IPs where the web page is located you could use https://github.com/hakluke/hakoriginfinder

    # You can check if the tool is working with
    prips 1.0.0.0/30 | hakoriginfinder -h one.one.one.one

    # If you know the company is using AWS you could use the previous tool to search the
    ## web page inside the EC2 IPs
    DOMAIN=something.com
    WIDE_REGION=us
    for ir in `curl https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.service=="EC2") | select(.region|test("^us")) | .ip_prefix'`; do
        echo "Checking $ir"
        prips $ir | hakoriginfinder -h "$DOMAIN"
    done

## Uncovering Cloudflare from Cloud infrastructure

Note that even if this was done for AWS machines, it could be done for any other cloud provider.

For a better description of this process check:

https://trickest.com/blog/cloudflare-bypass-discover-ip-addresses-aws/?utm_campaign=hacktrics&utm_medium=banner&utm_source=hacktricks

    # Find open ports
    sudo masscan --max-rate 10000 -p80,443 $(curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.service=="EC2") | .ip_prefix' | tr '\n' ' ') | grep "open"  > all_open.txt
    # Format results
    cat all_open.txt | sed 's,.*port \(.*\)/tcp on \(.*\),\2:\1,' | tr -d " " > all_open_formated.txt
    # Search actual web pages
    httpx -silent -threads 200 -l all_open_formated.txt -random-agent -follow-redirects -json -no-color -o webs.json
    # Format web results and remove eternal redirects
    cat webs.json | jq -r "select((.failed==false) and (.chain_status_codes | length) < 9) | .url" | sort -u > aws_webs.json

    # Search via Host header
    httpx -json -no-color -list aws_webs.json -header Host: cloudflare.malwareworld.com -threads 250 -random-agent -follow-redirects -o web_checks.json

## Bypassing Cloudflare through Cloudflare

### Authenticated Origin Pulls

This mechanism relies on client SSL certificates to authenticate connections between Cloudflare’s reverse-proxy servers and the origin server, which is called mTLS.

Instead of configuring it's own certificate, customers can simple use Cloudflare’s certificate to allow any connection from Cloudflare, regardless of the tenant.

> [!CAUTION] Therefore, an attacker could just set a domain in Cloudflare using Cloudflare's certificate and point it to the victim domain IP address. This way, setting his domain completely unprotected, Cloudflare won't protect the requests sent.

More info here.

### Allowlist Cloudflare IP Addresses

This will reject connections that do not originate from Cloudflare’s IP address ranges. This is also vulnerable to the previous setup where an attacker just point his own domain in Cloudflare to the victims IP address and attack it.

More info here.

## Bypass Cloudflare for scraping

### Cache

Sometimes you just want to bypass Cloudflare to only scrape the web page. There are some options for this:

- Use Google cache: `https://webcache.googleusercontent.com/search?q=cache:https://www.petsathome.com/shop/en/pets/dog`
- Use other cache services such as https://archive.org/web/

### Tools

Some tools like the following ones can bypass (or were able to bypass) Cloudflare's protection against scraping:

- https://github.com/sarperavci/CloudflareBypassForScraping

### Cloudflare Solvers

There have been a number of Cloudflare solvers developed:

- FlareSolverr
- cloudscraper Guide here
- cloudflare-scrape
- CloudflareSolverRe
- Cloudflare-IUAM-Solver
- cloudflare-bypass [Archived]
- CloudflareSolverRe

### Fortified Headless Browsers

Use a headless browser that isn't deetcted as an automated browser (you might need to customize it for that). Some options are:

- Puppeteer: The stealth plugin for puppeteer.
- Playwright: The stealth plugin is coming to Playwright soon. Follow developments here and here.
- Selenium: The undetected-chromedriver an optimized Selenium Chromedriver patch.

### Smart Proxy With Cloudflare Built-In Bypass

Smart proxies proxies are continuously updated by specialized companies, aiming to outmaneuver Cloudflare's security measures (as thats their business).

Som of them are:

- ScraperAPI
- Scrapingbee
- Oxylabs
- Smartproxy are noted for their proprietary Cloudflare bypass mechanisms.

For those seeking an optimized solution, the ScrapeOps Proxy Aggregator stands out. This service integrates over 20 proxy providers into a single API, automatically selecting the best and most cost-effective proxy for your target domains, thus offering a superior option for navigating Cloudflare's defenses.

### Reverse Engineer Cloudflare Anti-Bot Protection

Reverse engineering Cloudflare's anti-bot measures is a tactic used by smart proxy providers, suitable for extensive web scraping without the high cost of running many headless browsers.

Advantages: This method allows for the creation of an extremely efficient bypass that specifically targets Cloudflare's checks, ideal for large-scale operations.

Disadvantages: The downside is the complexity involved in understanding and deceiving Cloudflare's deliberately obscure anti-bot system, requiring ongoing effort to test different strategies and update the bypass as Cloudflare enhances its protections.

Find more info about how to do this in the original article.

## References

- https://scrapeops.io/web-scraping-playbook/how-to-bypass-cloudflare/

> [!TIP] Learn & practice AWS Hacking:HackTricks Training AWS Red Team Expert (ARTE)
> Learn & practice GCP Hacking: HackTricks Training GCP Red Team Expert (GRTE)

> Support HackTricks
> - Check the subscription plans!
> - Join the 💬 Discord group or the telegram group or follow us on Twitter 🐦 @hacktricks_live.
> - Share hacking tricks by submitting PRs to the HackTricks and HackTricks Cloud github repos.