Ranking bot type 2

These bots are part of the domain ranking. They perform simplistic backlink-counting actions and only store the following information:
  • Domain name.
  • Contents of the robots.txt file in the root directory of each domain.
  • Number of links pointing to each domain from random others.
  • Global assessment of the site/page on account of the aforementioned information and further issues analysed on-the-go.
All of them have also the following features in common:
  • User agent: RankingBot2 -- https://varocarbas.com/bot_ranking2/.
  • They only visit pages which aren't forbidden in the corresponding robots.txt file, by exclusively considering entries expressly referring to them ('RankingBot2') or to any bot ('*').
    While confirming reasonable assumptions of dishonest inter-linking (e.g., group of sites created for the sole purpose of providing backlinks among each other), these bots might ignore the robots.txt indications.
  • As usual, I am the sole author of these bots and developed them completely from scratch (PHP + MySQL/MariaDB).