Is Micro.blog seeing these bots crawling hosted sites? I wish there is a report or something that we can monitor so we have the choice which bots to block. :)
Is Micro.blog seeing these bots crawling hosted sites? I wish there is a report or something that we can monitor so we have the choice which bots to block. :)
@rom Good question, I’ll check. I’ve documented some common user-agents here which we can continue to update.