add: robots middleware
#627

markelog wants to merge markelog:robots into apex:master

We do not currently have an analysis of these two commits