THE LAB #6: Changing Ciphers in Scrapy to avoid bans by TLS Fingerprinting

Understanding TLS Fingerprinting

TLS fingerprinting is a passive (or server-side) fingerprinting technique used by servers to identify the configuration of the clients connecting to it.

The fingerprints are created using the ciphers exchanged when the connection between the client and servers establishes.

To better understand how this technique works, let's borrow the image from this Cloudflare blog post.

HTTP protocol
HTTP protocol

When we connect a client to a server, the first interaction is made by the TCP protocol. It's called Three-way Handshake, where the client and server share their willingness and availability to connect.

Continue reading