This repository contains scripts to block a predefined list of IP addresses and subnets using iptables. The scripts are available in three versions: Bash (Shell), Python, and Node.js. Follow the instructions below to set up and run the version that best suits your environment.
Before running these scripts, ensure you have the following installed on your system:
- Shell: No additional software is needed for Bash.
- Python: Python 3.x. Download Python
- Node.js: Node.js 10.x or higher. Download Node.js
Additionally, you will need sudo or root access to modify iptables rules.
Clone this repository to your local machine using:
git clone https://github.com/ThunderDoesDev/CrawlerBlocker.git
cd CrawlerBlocker-
Navigate to the directory containing the
crawler_blocker.sh. -
Make the script executable:
chmod +x crawler_blocker.sh
-
Ensure Python 3.x is installed by running:
python3 --version
-
Navigate to the directory containing the
crawler_blocker.py.
-
Ensure Node.js is installed by running:
node --version
-
Navigate to the directory containing the
crawler_blocker.js.
Run the script with:
sudo ./crawler_blocker.shRun the script with:
sudo python3 crawler_blocker.pyRun the script with:
sudo node crawler_blocker.js-
Bash Script: The Bash script will automatically delete itself after execution using
rm -- "$0". This helps to keep your environment clean, but be sure to keep a backup if you plan to reuse the script. -
Python and Node.js Scripts: Both the Python and Node.js scripts are also designed to delete themselves after successful execution. Similar to the Bash script, ensure you have a backup if needed.
All scripts include robust error handling to ensure that any issues are logged and the script exits gracefully. The logs are stored in /var/log/crawler_blocker.log for your review.
- Error Handling: Each script has mechanisms in place to catch and handle errors, preventing the scripts from failing silently.
- Logging: The scripts log their progress and any errors they encounter, allowing you to monitor the operations and troubleshoot if necessary.
Each script will install iptables-persistent before setting any iptables rules to ensure the rules persist after a reboot.
This project is licensed under the APACHE License - see the LICENSE file for details.
These scripts leverage iptables to block incoming traffic from the specified IP addresses and subnets, which are known sources of many web crawlers and bots. By enforcing these rules, not only is traffic from these entities prevented, but your backend services will also be shielded from potential threats, significantly enhancing your server infrastructure's security. Please ensure you have the appropriate permissions to modify your system's firewall rules and fully understand the implications of these changes on your network's security.
For support, issues, or enhancements, please open an issue in this repository or join our discord support server.