THE LAB #14: Scraping Cloudflare Protected Websites (early 2023 version)

In the latest post, we have seen how to scrape a Kasada-protected website, using both free and commercial tools.

Many of you found it useful for their projects, despite Kasada seeming to have a relatively small market share in the business.

Since it’s been a while since I’ve written about Cloudflare solutions and things do evolve rapidly in this industry, I’ve decided to update my old article about scraping Cloudflare-protected websites, using the same format as the Kasada one but with a difference. We’ll test the solutions both on a local environment and on a remote virtual machine on AWS. This is because the website we’re going to analyze has Cloudflare activated probably at the highest levels of paranoia and you can’t even browse it from there.

Browsing Harrods’s website from VM in AWS
Browsing Harrods’s website from VM in AWS

Continue reading