PASTA now records all “robot” requests to the audit logs. The audit log will now show the phrase “robot:” in the user column, along with the type of robot detected. For example:
robot: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Suspect requests are flagged as a robot if found in a list of known offending user agents provided by the Counter project (https://www.projectcounter.org/), which is updated on a regular basis (about every 2-6 months). This feature was just released this past Wednesday evening (April 12, 2017) during our weekly updates. We are still fine-tuning the list to avoid false positives, but it is now functioning in the production PASTA environment.
