Ranking bot type 1

These bots are retrieving information for the domain ranking. They perform simplistic backlink-counting actions and only store the following information:
  • Domain name.
  • Contents of the robots.txt file in the root directory of each domain.
  • Number of links pointing to each domain from random others.
These bots only visit URLs which aren't forbidden by the corresponding robots.txt file.

As usual, I am the sole author of these bots and developed them completely from scratch (PHP + MySQL/MariaDB).