Search engines decide what we see for a given search query. Since many people are exposed to information through search engines, it is fair to expect that search engines are neutral. However, search engine results do not necessarily cover all the viewpoints of a search query topic, and they can be biased towards a specific view since search engine results are returned based on relevance, which is calculated using many features and sophisticated algorithms where search neutrality is not necessarily the focal point. Therefore, it is important to evaluate the search engine results with respect to bias. In this work we propose novel web search bias evaluation measures which take into account the rank and relevance. We also propose a framework to evaluate web search bias using the proposed measures and test our framework on two popular search engines based on 57 controversial query topics such as abortion, medical marijuana, and gay marriage. We measure the stance bias (in support or against), as well as the ideological bias (conservative or liberal). We observe that the stance does not necessarily correlate with the ideological leaning, e.g. a positive stance on abortion indicates a liberal leaning but a positive stance on Cuba embargo indicates a conservative leaning. Our experiments show that neither of the search engines suffers from stance bias. However, both search engines suffer from ideological bias, both favouring one ideological leaning to the other, which is more significant from the perspective of polarisation in our society.

搜索引擎决定给定搜索查询的内容。由于许多人通过搜索引擎接触到信息，因此可以公平地期望搜索引擎是中立的。但是，搜索引擎结果并不一定涵盖搜索查询主题的所有观点，并且由于基于相关性返回搜索引擎结果，因此搜索引擎结果可能会偏向特定视图，而相关性是使用许多功能和复杂算法（其中搜索中性为不一定是重点。因此，重要的是要评估搜索引擎相对于偏向的结果。在这项工作中，我们提出了新颖的网络搜索偏见评估措施，该措施考虑了排名和相关性。我们还提出了一个框架，使用所提出的措施来评估网络搜索偏向，并基于57个有争议的查询主题（例如堕胎，医用大麻和同性婚姻）在两个流行的搜索引擎上测试我们的框架。我们测量立场偏见（支持或反对），以及意识形态偏见（保守或自由主义）。我们观察到，这种立场不一定与意识形态倾向相关，例如，对堕胎的正面立场表明是自由主义倾向，而对古巴禁运的正面立场表明是保守主义倾向。我们的实验表明，这两个搜索引擎都没有遭受立场偏见的困扰。但是，这两种搜索引擎都存在意识形态偏见，两者都偏向于一种意识形态向另一种意识形态倾斜，这从我们社会的两极分化角度来看更为重要。