www.tifaa.com/robots.txt:
User-agent: *
Disallow: /Ad/AdContact
Disallow: /Ad/ContactForm
Disallow: /ad/chat/*
Disallow: /ad/insta/*
Allow: /
User-agent: Bytespider
Disallow: /
User-agent: Amazonbot
Disallow: /
User-agent: Bingbot
Crawl-delay: 10
User-agent: SpiderLing
Disallow: /
User-agent: msnbot.
Crawl-delay: 60
User-agent: SemrushBot
Disallow: /
User-agent: SemrushBot-SA
Disallow: /
User-agent: Exabot
Disallow: /
User-agent: Speedy
Disallow: /
User-agent: sogou spider
Disallow: /
User-agent: sitebot
Disallow: /
User-agent: Covario
Disallow: /
User-agent: AhrefsBot
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: Yandex
Disallow: /
User-agent: archive.org_bot
Disallow: /
User-agent: Dotbot
Disallow: /
User-agent: AlphaSeoBot
Disallow: /
User-agent: AlphaSeoBot-SA
Disallow: /
User-agent: ICC-Crawler
Disallow: /
Sitemap: https://www.panikad.com/sitemap.xml
в яндексе
Прочее:
Ping/Trace,
История в Веб-архиве,
История владельцев,
Валидность HTML,
Валидность CSS,
Проверка на вирусы.
Запрашиваемая страница недоступна!
|