> Microsoft can throw around $60M, and Bing is used by most of the "alternative" search engines.
That's a symptom of the issue, not a solution. Bing is used because having your own crawler is infeasible, partially because you will be literally blocked in many cases.
That's a symptom of the issue, not a solution. Bing is used because having your own crawler is infeasible, partially because you will be literally blocked in many cases.