( gemnon | 2016. 04. 26., k – 12:26 )

Mondjuk ez nem a site hibája, hogy a Bingbot identitászavarban van. :)

3.2.1 The User-agent line

...

The name token a robot chooses for itself should be sent as part of the HTTP User-agent header, and must be well documented.

...

The robot must obey the first record in /robots.txt that contains a User-Agent line whose value contains the name token of the robot as a substring.

http://www.robotstxt.org/norobots-rfc.txt