ryuken
pharmacist
- Joined
- Jun 10, 2024
- Posts
- 5,561
- Reputation
- 8,440
Bot Type
Good crawler
(always identifies itself)
IP Range
Distributed, Worldwide
Obeys Robots.txt
Yes
Obeys Crawl Delay
Yes
Data served at
Majestic.com
Majestic is a UK based specialist search engine used by hundreds of thousands of businesses in 13 languages and over 60 countries to paint a map of the Internet independent of the consumer based search engines. Majestic also powers other legitimate technologies that help to understand the continually changing fabric of the web.
Web site owners can see data about their own websites on majestic.com.
MJ12Bot does not currently cache web content or personal data. Instead it maps the link relationships between websites to build a search engine. This data is available to technologies and the public, either by searching for a keyword or a website at Majestic. Details about the community project behind the crawlers are at Majestic12.co.uk.
What is MJ12bot doing on my site(s)?
We spider the Web for the purpose of building a search engine with a fast and efficient downloadable distributed crawler that enables people with broadband connections to help contribute to, what we hope, will become the biggest search engine in the world. Production of a full text search engine at Majestic-12 is currently in the research phase, funded in part by the commercialisation of research at Majestic.
What happens to the crawled data?
Crawled data (currently only a web graph of links) is added to the largest public backlinks search engine index that we maintain as a dedicated tool called Site Explorer. Learn about your own backlinks from the extensive backlinks index.
Good crawler
(always identifies itself)
IP Range
Distributed, Worldwide
Obeys Robots.txt
Yes
Obeys Crawl Delay
Yes
Data served at
Majestic.com
Majestic is a UK based specialist search engine used by hundreds of thousands of businesses in 13 languages and over 60 countries to paint a map of the Internet independent of the consumer based search engines. Majestic also powers other legitimate technologies that help to understand the continually changing fabric of the web.
Web site owners can see data about their own websites on majestic.com.
MJ12Bot does not currently cache web content or personal data. Instead it maps the link relationships between websites to build a search engine. This data is available to technologies and the public, either by searching for a keyword or a website at Majestic. Details about the community project behind the crawlers are at Majestic12.co.uk.
What is MJ12bot doing on my site(s)?
We spider the Web for the purpose of building a search engine with a fast and efficient downloadable distributed crawler that enables people with broadband connections to help contribute to, what we hope, will become the biggest search engine in the world. Production of a full text search engine at Majestic-12 is currently in the research phase, funded in part by the commercialisation of research at Majestic.
What happens to the crawled data?
Crawled data (currently only a web graph of links) is added to the largest public backlinks search engine index that we maintain as a dedicated tool called Site Explorer. Learn about your own backlinks from the extensive backlinks index.
