Gary Illyes from Google added to the big crawl budget article he published on the webmaster blog in January 2017 a new question and answer. It basically says that if you disallow a URL in your robots.txt file, then those URLs do not affect your crawl budget.
He added this question and answer:
Q: Do URLs I disallowed through robots.txt affect my crawl budget in any way?
A: No, disallowed URLs do not affect the crawl budget.
He posted last night that he added the Q&A for a former Googler, Pierre Far:
Based on a question from @pierrefar , i just updated the Crawl Budget blog post to include the following:
Q: Do URLs I disallowed through robots.txt affect my crawl budget in any way?
A: No, disallowed URLs do not affect the crawl budget.https://t.co/kVYj6XaNn0— Gary “鯨理” Illyes (@methode) June 21, 2019
Forum discussion at Twitter.
This marketing news is not the copyright of Scott.Services – please click here to see the original source of this article. Author: barry@rustybrick.com (Barry Schwartz)
For more SEO, PPC, internet marketing news please check out https://news.scott.services
Why not check out our SEO, PPC marketing services at https://www.scott.services
We’re also on:
https://www.facebook.com/scottdotservices/
https://twitter.com/scottdsmith
https://plus.google.com/112865305341039147737
The post Google: Disallowed URLs Through Robots.txt Does Not Affect Crawl Budget appeared first on Scott.Services Online Marketing News.
source https://news.scott.services/google-disallowed-urls-through-robots-txt-does-not-affect-crawl-budget/
No comments:
Post a Comment