Bogeum Choi
Jaime Arguello
Robert Capra
Skip Abstract Section
Abstract
People often search for information to acquire procedural knowledge–“how to” knowledge about step-by-step procedures, methods, algorithms, techniques, heuristics, and skills. A procedural search task might involve implementing a solution to a problem, evaluating different approaches to a problem, and brainstorming on the types of problems that can be solved with a specific resource. We report on a study (N=36) that aimed to better understand how people search for procedural knowledge. Much research has investigated how search task characteristics impact people’s perceptions and behaviors. Along these lines, we manipulated procedural search tasks along two orthogonal dimensions: product and goal. The product dimension relates to the main outcome of the task and the goal dimension relates to task’s success criteria. We manipulated tasks across three product categories and two goal categories. The study investigated four research questions. First, we examined the effects of the product and goal on participants’ (RQ1) pre-task perceptions, (RQ2) post-task perceptions, and (RQ3) search behaviors. Second, regardless of the task product and goal, by analyzing participants’ think-aloud comments and screen activities we closely examined how people search for procedural knowledge. Specifically, we report on (RQ4) important relevance criteria, types of information sought, and challenges.
- [1] . 2020. HealthAid: Extracting domain targeted high precision procedural knowledge from on-line communities. Information Processing & Management 57, 6 (2020).
DOI: Google Scholar
Cross Ref
- [2] . 2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Complete Edition.Google Scholar
- [3] . 2012. User task understanding: A web search engine perspective. (2012). https://www.microsoft.com/en-us/research/publication/user-task-understanding-a-web-search-engine-perspective/
Presentation delivered at the NII Shonan: Whole-Session Evaluation of Interactive Information Retrieval Systems workshop. 8-11 October 2012, Shonan, Japan. Google Scholar - [4] . 2012. An eye-tracking approach to the analysis of relevance judgments on the Web: The case of Google search engine. Journal of the American Society for Information Science and Technology 63, 9 (2012), 1728–1746.Google ScholarDigital Library
- [5] . 1994. User-defined relevance criteria: An exploratory study. Journal of the American Society for Information Science 45, 3 (1994), 149–159.Google ScholarDigital Library
- [6] . 1995. Task complexity affects information seeking and use. Information Processing and Management 31, 2 (1995), 191–213.Google ScholarDigital Library
- [7] . 1988. Task complexity: A review and analysis. The Academy of Management Review 13, 1 (1988), 40–52.Google ScholarCross Ref
- [8] . 2023. How does AI chat change search behaviors? arXiv (2023). https://arxiv.org/abs/2307.03826Google Scholar
- [9] . 2015. Differences in the use of search assistance for tasks of varying complexity. Association for Computing Machinery, New York, NY, USA, 23–32.Google Scholar
- [10] . 2019. A team-based approach to open coding: Considerations for creating intercoder consensus. Field Methods 31, 2 (2019), 116–130.Google ScholarCross Ref
- [11] . 2020. A qualitative analysis of the effects of task complexity on the functional role of information. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval. 328–332.Google ScholarDigital Library
- [12] . 2023. Understanding procedural search tasks “in the wild”. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (
CHIIR’23 ). Association for Computing Machinery, New York, NY, USA, 24–33.DOI: Google ScholarDigital Library - [13] . 2022. Procedural knowledge search by intelligence analysts(
CHIIR ’22 ). Association for Computing Machinery, New York, NY, USA, 169–179.Google Scholar - [14] . 2019. The effects of task complexity on the use of different types of information in a search assistance tool. ACM Trans. Inf. Syst. 38, 1, Article
9 (Dec 2019), 28 pages.DOI: Google ScholarDigital Library - [15] . 2017. Distilling task knowledge from how-to communities. In Proceedings of the 26th International Conference on World Wide Web (WWW ’17). International World Wide Web Conferences Steering Committee, 805–814.
DOI: Google ScholarDigital Library - [16] . 2011. Task and user effects on reading patterns in information search. Interacting with Computers 23, 4 (
05 2011), 346–362.DOI: Google ScholarCross Ref - [17] . 2006. Relevance criteria identified by health information users during Web searches. Journal of the American Society for Information Science and Technology 57, 10 (2006), 1368–1382.Google ScholarDigital Library
- [18] . 2014. Lessons from the Journey: A Query Log Analysis of Within-Session Learning. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining (WSDM ’14). ACM, New York, NY, USA, 223–232.
DOI: Google ScholarDigital Library - [19] . 2011. Procedural instructions, principles, and examples: How to structure instructions for procedural tasks to enhance performance, learning, and transfer. Human Factors 53, 6 (2011), 749–770.Google ScholarCross Ref
- [20] . 2008. The qualitative content analysis process. Journal of Advanced Nursing 62, 1 (2008), 107–115.Google ScholarCross Ref
- [21] . 2009. Conceptual and procedural knowledge construction in computer supported collaborative learning. In Proceedings of the 9th International Conference on Computer Supported Collaborative Learning (CSCL’09). International Society of the Learning Sciences, 137–141.Google ScholarCross Ref
- [22] . 2005. Modeling the information behaviour of software engineers using a work-task framework. Proceedings of the American Society for Information Science and Technology (2005).Google ScholarCross Ref
- [23] . 2022. “What can I cook with these ingredients?” — understanding cooking-related information needs in conversational search. ACM Transactions of Information Systems 40, 4, Article
81 (2022).Google ScholarDigital Library - [24] . 1986. Procedural knowledge. Proc. IEEE 74, 10 (1986), 1383–1398.
DOI: Google ScholarCross Ref - [25] . 2019. Self-assessment accuracy in higher education: The influence of gender and performance of university students. Active Learning in Higher Education 20, 2 (
July 2019), 101–114. .Publisher: SAGE Publications. Google ScholarCross Ref - [26] . 2015. Development and evaluation of search tasks for IIR experiments using a cognitive complexity framework. In Proceedings of the 2015 International Conference on The Theory of Information Retrieval (ICTIR ’15). Association for Computing Machinery, New York, NY, USA, 101–110.Google ScholarDigital Library
- [27] . 2020. Inductive content analysis. The Application of Content Analysis in Nursing Science Research (2020), 13–21.Google ScholarCross Ref
- [28] . 1986. Do written instructions need examples? Cognition and Instruction 3, 1 (1986), 1–30.Google ScholarCross Ref
- [29] . 2008. A faceted approach to conceptualizing tasks in information seeking. Information Processing & Management 44, 6 (2008), 1822–1837.Google ScholarDigital Library
- [30] . 2010. Search behaviors in different task types. In Proceedings of the 10th Annual Joint Conference on Digital Libraries (JCDL ’10). Association for Computing Machinery, New York, NY, USA, 69–78.Google ScholarDigital Library
- [31] . 2013. Examining the effects of task topic familiarity on searchers’ behaviors in different task types. In Proceedings of the 76th ASIS&T Annual Meeting: Beyond the Cloud: Rethinking Information Boundaries (ASIST ’13). American Society for Information Science, USA.Google ScholarDigital Library
- [32] . 2000. Learning a procedural task: Effectiveness of multimedia presentations. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition 14, 6 (2000), 555–575.Google ScholarCross Ref
- [33] . 2019. Recent trends in natural language understanding for procedural knowledge. In 2019 International Conference on Computational Science and Computational Intelligence (CSCI). 420–424.
DOI: Google ScholarCross Ref - [34] . 2019. Search and justification behavior during multimedia web search for procedural knowledge. In Companion Publication of the 10th ACM Conference on Web Science (WebSci ’19). ACM, New York, NY, USA, 17–20.Google ScholarDigital Library
- [35] . 2018. Learning procedures from text: Codifying how-to procedures in deep neural networks. In Companion Proceedings of The Web Conference 2018 (WWW ’18). International World Wide Web Conferences Steering Committee, 351–358.
DOI: Google ScholarDigital Library - [36] . 2017. Mining alternative actions from community Q&A corpus for task-oriented web search. In Proceedings of the International Conference on Web Intelligence (WI ’17). ACM, New York, NY, USA, 607–614.
DOI: Google ScholarDigital Library - [37] . 2021. The living codebook: Documenting the process of qualitative data analysis. Sociological Methods & Research (2021), 0049124120986185.Google Scholar
- [38] . 2006. User-defined relevance criteria in web searching. Journal of Documentation (2006).Google ScholarCross Ref
- [39] . 2010. The developmental relations between conceptual and procedural knowledge: A multimethod approach. Developmental Psychology 46, 1 (2010), 178.Google ScholarCross Ref
- [40] . 2012. Extraction of procedural knowledge from the web: A comparison of two workflow extraction approaches. In Proceedings of the 21st International Conference on World Wide Web (WWW ’12 Companion). ACM, New York, NY, USA, 739–747.
DOI: Google ScholarDigital Library - [41] . 1994. Procedural knowledge and processing strategies in social cognition. Handbook of Social Cognition 2 (1994), 99–152.Google Scholar
- [42] . 2012. User relevance criteria choices and the information search process. Information Processing & Management 48, 1 (2012), 136–153.Google ScholarDigital Library
- [43] . 1999. Procedural and declarative knowledge: An evolutionary perspective. Theory & Psychology 9, 5 (1999), 605–624.
DOI: Google ScholarCross Ref - [44] . 1999. Procedural and declarative knowledge: An evolutionary perspective. Theory & Psychology 9, 5 (1999), 605–624.Google ScholarCross Ref
- [45] . 2022. Understanding the “pathway” towards a searcher’s learning objective. ACM Transactions of Information Systems 40, 4 (2022).Google ScholarDigital Library
- [46] . 2020. The effects of learning objectives on searchers’ perceptions and behaviors(
ICTIR ’20 ). ACM, New York, NY, USA, 77–84.DOI: Google ScholarDigital Library - [47] . 2015. What users ask a search engine: Analyzing one billion Russian question queries. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM ’15). ACM, New York, NY, USA, 1571–1580.
DOI: Google ScholarDigital Library - [48] . 2012. Answers, not links: Extracting tips from yahoo! answers to address how-to web queries. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining (WSDM ’12). ACM, New York, NY, USA, 613–622.
DOI: Google ScholarDigital Library - [49] . 2014. Untangling search task complexity and difficulty in the context of interactive information retrieval studies. Journal of Documentation 70, 6 (
Oct. 2014), 1118–1140.Google ScholarCross Ref - [50] . 1999. The effects of understanding rules and a worked example on the acquisition of procedural knowledge and task performance. Accounting & Finance 39, 2 (1999), 177–203.Google ScholarCross Ref
- [51] . 2006. Relevance judgment: What do information users consider beyond topicality? Journal of the American Society for Information Science and Technology 57, 7 (2006), 961–973.Google ScholarDigital Library
- [52] . 2015. Leveraging procedural knowledge for task-oriented search(
SIGIR ’15 ). ACM, New York, NY, USA, 513–522.DOI: Google ScholarDigital Library - [53] . 2012. Automatically extracting procedural knowledge from instructional texts using natural language processing. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC ’12). European Language Resources Association (ELRA), Istanbul, Turkey, 520–527.Google Scholar
Index Terms
- Better Understanding Procedural Search Tasks: Perceptions, Behaviors, and Challenges
Recommendations
Understanding Procedural Search Tasks “in the Wild”
CHIIR '23: Proceedings of the 2023 Conference on Human Information Interaction and RetrievalPeople often search online for procedural (i.e., “how-to”) knowledge. A procedural search task might involve a do-it-yourself project, cooking a dish, fixing a problem, or learning a new skill. Prior research has studied procedural search tasks from ...
Attitudes and Perceptions of Older Chinese People in Hong Kong Towards Silver Gaming
Human Aspects of IT for the Aged Population. Applications in Health, Assistance, and EntertainmentAbstractThis preliminary study aimed to explore the perceptions and attitudes of older Chinese people (over 60 years of age) in Hong Kong towards gaming by adopting a qualitative research method and to investigate the factors that motivate them to ...
What do exploratory searchers look at in a faceted search interface?
JCDL '09: Proceedings of the 9th ACM/IEEE-CS joint conference on Digital librariesThis study examined how searchers interacted with a web-based, faceted library catalog when conducting exploratory searches. It applied eye tracking, stimulated recall interviews, and direct observation to investigate important aspects of gaze behavior ...
Comments