Pen Test Blacklisting: What to Do

TL;DR

Getting blocked during a penetration test is common. This guide covers how to identify blacklisting, avoid it where possible, and work around it when it happens.

1. Understanding Blacklisting

Blacklisting means the target system has identified your testing activity as malicious and is actively preventing further interaction. This usually involves blocking your IP address, user agent, or other identifying information. It’s a security measure designed to protect the system.

2. Identifying Blacklisting

1. Check Connectivity: The most obvious sign is an inability to connect to the target. Try pinging it and see if you get replies.
2. HTTP Status Codes: Look for error codes like 403 (Forbidden), 406 (Not Acceptable) or 503 (Service Unavailable). These often indicate blocking.
3. Web Server Logs: If you have access, examine the web server logs for entries related to your IP address and any blocked requests.
4. Network Monitoring: Use tools like Wireshark to capture network traffic and see if packets are being dropped or rejected.

3. Preventing Blacklisting (Before You Start)

1. Scope Definition: Clearly define the scope of your test with the client. This includes allowed IP addresses, testing times, and permitted techniques. Stick to this!
2. Rate Limiting: Avoid overwhelming the target system with requests. Implement delays in your scripts to mimic normal user behaviour. For example:

   import time
   for i in range(10):
       # Your request code here
       time.sleep(2)  # Wait 2 seconds between requests

3. User Agent: Use a common, non-suspicious user agent string. Avoid anything that screams ‘automated tool’.
4. Request Headers: Ensure your requests include standard HTTP headers. Missing or malformed headers can trigger blocking.
5. Initial Reconnaissance: Start with passive reconnaissance to understand the target’s infrastructure and security measures before launching active scans.

4. Working Around Blacklisting (When It Happens)

1. Change IP Address: The simplest solution is often to use a different IP address. This could involve using a VPN, proxy server, or requesting a new IP from your ISP.
2. Rotate User Agents: Cycle through a list of legitimate user agent strings.
3. Proxy Servers: Use a pool of proxy servers to distribute requests and avoid being blocked by a single IP address.

   # Example using Python's Requests library with proxies
   proxies = {
     'http': 'http://your_proxy_ip:port',
     'https': 'https://your_proxy_ip:port'
   }
   response = requests.get(url, proxies=proxies)

4. Delay and Throttling: Significantly reduce the rate of your requests. Increase delays between requests to mimic human behaviour even further.
5. Fragmentation: In some cases, fragmenting packets can bypass basic blocking rules (use with caution). This is a more advanced technique and may be detected by intrusion detection systems.
6. Alternative Ports/Protocols: If possible, try using different ports or protocols to access the target system.
7. Contact Client: If you’re repeatedly blocked despite taking precautions, contact the client to inform them of the issue and request a temporary whitelist for your IP address. This is often the best approach.

5. Document Everything

Keep detailed records of any blacklisting incidents, including the time, affected IP addresses, requests that triggered blocking, and steps taken to resolve the issue. This information can be valuable for future tests.