rajatoto Secrets
the worth on the name attribute (robots) specifies which the rule relates to all crawlers. both equally the identify plus the material attributes are circumstance-insensitive. to handle a particular crawler, exchange the robots price of the name attribute With all the person agent token from the crawler that you will be addressing. Google supports