Map
Index
Random
Help
Notes
Related
th

ThesaHelp: needle-in-a-haystack test of directed search

Thesa topic root > ThesaGroup: search with Thesa


--related--
search with Thesa
a preliminary test of directed search
read-everything test of directed search
information retrieval by searching

--referenced--
all Thesa topics
map of the Thesa web site

Summary

Use the needle-in-a-haystack test to estimate recall and precision for large collections. In a large collection, an exhaustive search is impractical, making recall and precision difficult to measure.

  1. Have someone select random quotations in Thesa, and write questions which the quotations help answer. The question should not use keywords from the quotation.

  2. Ask a subject to locate all material relevant to the question.

  3. If the subject does not select the quotation, ask whether it is relevant and if not, why not.

  4. Repeat this procedure to estimate the recall and precision ratios. If subject looks at 200 items, identifies 70 relevant items, and locates 90% of the selected quotations, the recall is 0.90 and the precision is 0.35.

  5. You may use the same procedure, to test keyword searching.

Related up

ThesaHelp: a preliminary test of directed search
ThesaHelp: read-everything test of directed search
Topic: information retrieval by searching

Updated barberCB 4/06
Copyright © 2002-2023 by C.B. Barber
Thesa, Avev, and thid-... are trademarks of C.B. Barber