To do this, the search engine evaluator must be familiar with the language and the culture of the local search engine user. Typically these positions are bilingual, but there are some opening for English-only search evaluators. The job of search evaluation is typically a work at home position and can go by many names--search evaluator, Internet assessor, ads quality rater or Internet judge.
For more about the subject of search evaluation, search company has a nice explanation of how the complexities of improving search evaluation, which is both an algorithmic and human process.
In 2007 Google launched over 450 improvements to Google search, and every one of them went through a comprehensive evaluation process.
Not surprisingly, Google takes search evaluation very seriously. Precise evaluation enables Google teams to know "which way is up" and evaluating search is difficult for several reasons.
- understanding what a user really wants when they type a query.
- comparing the quality of search engines (whether Google versus competitors, Google versus Google a month ago, or Google versus Google plus the "letter T" hack) is never black and white.
- there are several dimensions to "good" results.
- evaluating Google search quality requires covering an enormous breadth.
To get over above issues, search company employ a variety of evaluation methods and data sources, and one the them is called Human evaluators, also known as search evaluator, search engine evaluator, search engine evaluation job, ad rater, internet assessor, search engine judges.
Search companies like Google or Appen need people to do them. Google itself calls the jobs "ads quality rater." These are one of the only home-based Google jobs that aren't scams. Google makes use of evaluators in many countries and languages. These evaluators are carefully trained and are asked to evaluate the quality of search results in several different ways.
The main objective of search evaluator is giving the highest quality search experience possible.