Abstract
Recent advances in the field of artificial intelligence bring promising new capabilities that can substantially improve our ability to manage complex and evolving situations in the face of uncertainty. Humanitarian crises exemplify such situations, and the pervasiveness of social media renders it one of the most abundant sources of real-time information available. However, it is quite a difficult task to condense a body of social media posts into useful information quickly. In this paper we consider the challenge of using social media reports to provide a reliable, real-time situational awareness in the management of humanitarian crises. Effectively addressing this challenge requires extracting only the relevant information out of text and images in individual social media posts, fusing this information together into actionable information points for decision makers, and providing an assessment of the trustworthiness of this information. We propose a general solution framework and discuss a system developed in collaboration with NATO which combines state-of-the-art deep learning, natural language processing, computer vision, and information fusion models to provide a reliable, actionable, real-time situational awareness for supporting decision making in humanitarian crisis logistics. In addition to the technical approach, we also discuss important practical aspects of this project including the development and validation process, challenges encountered along the way, and key lessons learned.
Funding source: North Atlantic Treaty Organization
Award Identifier / Grant number: 194001532
Award Identifier / Grant number: 204000469
Acknowledgement
This research was funded by a NATO ACT Innovation Hub; contract numbers 194001532 and 204000469. The authors are grateful for the technical and management support of NATO’s personnel Serge Da Deppo, Jose Moreira, Andrei Mititelu, Jacques-Olivier Carrasset, and Menno Vanderbijl. We are indebted to NATO’s Crisis Management and Disaster Response Centre of Excellence, Sofia, Bulgaria, namely Orlin Nikolov, Plamen Milanov, Kostadin Lazarov, and Genadi Kolev for volunteering as end users. We would also like to recognize the software development team at POLARes LLC, Rashed Omar, Yanal Otaibi and Iyad Dasouqi.
References
Alam, F., F. Ofli, and M. Imran. 2018. “Processing Social Media Images by Combining Human and Machine Computing During Crises.” International Journal of Human-Computer Interaction 34 (4): 311–27. https://doi.org/10.1080/10447318.2018.1427831.Search in Google Scholar
Alam, F., F. Ofli, and M. Imran. 2020. “Descriptive and Visual Summaries of Disaster Events Using Artificial Intelligence Techniques: Case Studies of Hurricanes Harvey, Irma, and Maria.” Behaviour & Information Technology 39 (3): 288–318. https://doi.org/10.1080/0144929x.2019.1610908.Search in Google Scholar
Ankerst, M., M. M. Breunig, K. Hans-Peter, and J. Sander. 1999. “OPTICS: Ordering Points to Identify the Clustering Structure.” In Proceedings of the SIGMOD/PODS99: International Conference on Management of Data and Symposium on Principles of Database Systems, Philadelphia, 49–60.10.1145/304182.304187Search in Google Scholar
Avvenuti, M., S. Cresci, F. Del Vigna, T. Fagni, and M. Tesconi. 2018. “CrisMap: A Big Data Crisis Mapping System Based on Damage Detection and Geoparsing.” Information Systems Frontiers 20 (5): 993–1011. https://doi.org/10.1007/s10796-018-9833-z.Search in Google Scholar
Devlin, J., M. W. Chang, K. Lee, and K. Toutanova. 2018. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805.Search in Google Scholar
Figure Eight. 2020. Multilingual Disaster Response Messages. https://www.figure-eight.com/dataset/combined-disaster-response-data (accessed January 8, 2020).Search in Google Scholar
Folium. 2020. Python Data, Leaflet.Js Maps. https://python-visualization.github.io/folium/ (accessed March 23, 2021).Search in Google Scholar
Huang, X., Z. Li, Y. Jiang, X. Li, and D. Porter. 2020. “Twitter Reveals Human Mobility Dynamics during the COVID-19 Pandemic.” PLoS One 15 (11): e0241957, https://doi.org/10.1371/journal.pone.0241957.Search in Google Scholar
iHELP. 2020. Intelligent Holistic Emergency Logistics Platform. Also available at www.ihelplogistics.com.Search in Google Scholar
Imran, M., C. Castillo, F. Diaz, and S. Vieweg. 2015. “Processing Social Media Messages in Mass Emergency: A Survey.” ACM Computing Surveys 47 (4): 67. https://doi.org/10.1145/2771588.Search in Google Scholar
Imran, M., P. Mitra, and C. Castillo 2016. “Twitter as a Lifeline: Human-Annotated Twitter Corpora for NLP of Crisis-Related Messages.” In Proceedings of the 10th Language Resources and Evaluation Conference (LREC), 1638–43. Portorož.Search in Google Scholar
Imran, M., F. Alam, U. Qazi, S. Peterson, and F. Ofli. 2020a. “Rapid Damage Assessment Using Social Media Images by Combining Human and Machine Intelligence.” In Proceedings of the 17th ISCRAM Conference. Blacksburg.Search in Google Scholar
Imran, M., F. Ofli, D. Caragea, and A. Torralba. 2020b. “Using AI and Social Media Multimodal Content for Disaster Response and Management: Opportunities, Challenges, and Future Directions.” Information Processing & Management 57 (5): 1638–43. https://doi.org/10.1016/j.ipm.2020.102261.Search in Google Scholar
Kersten, J., A. Kruspe, M. Wiegmann, and F. Klan. 2019. “Robust Filtering of Crisis-Related Tweets.” In Proceedings of the 16th ISCRAM Conference. Spain: ISCRAM.Search in Google Scholar
Kropczynski, J., R. Grace, J. Coche, S. Halse, E. Obeysekare, A. Montarnal, F. Benaben, and A. Tapia. 2018. “Identifying Actionable Information on Social Media for Emergency Dispatch.” In Proceedings of the ISCRAM Asia Pacific.Search in Google Scholar
Li, H., D. Caragea, C. Caragea, and N. Herndon. 2018. “Disaster Response Aided by Tweet Classification with a Domain Adaptation Approach.” Journal of Contingencies and Crisis Management 26: 16–27. https://doi.org/10.1111/1468-5973.12194.Search in Google Scholar
Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692.Search in Google Scholar
Loshchilov, I., and F. Hutton. 2017. Decoupled Weight Decay Regularization. arXiv:1711.05101.Search in Google Scholar
Manzoor, S. I., J. Singla, and Nikita. 2019. “Fake News Detection Using Machine Learning Approaches: A Systematic Review.” In Proceedings of the 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), 230–4. Tirunelveli.10.1109/ICOEI.2019.8862770Search in Google Scholar
Mazloom, R., H. Li, D. Caragea, and M. Imran. 2019. “A Hybrid Domain Adaptation Approach for Identifying Crisis-Relevant Tweets.” International Journal of Information Systems for Crisis Response and Management 11 (2): 1–19. https://doi.org/10.4018/ijiscram.2019070101.Search in Google Scholar
Mazloom, R., H. Li, D. Caragea, M. Imran, and C. Caragea. 2018. “Classification of Twitter Disaster Data Using a Hybrid Feature-Instance Adaptation Approach.” In Proceedings of the 15th ISCRAM Conference. Rochester.Search in Google Scholar
Ofli, F., F. Alam, and M. Imran. 2020a. “Analysis of Social Media Data Using Multimodal Deep Learning for Disaster Response.” In Proceedings of the 17th ISCRAM Conference. Blacksburg.Search in Google Scholar
Ofli, F., M. Imran, and F. Alam. 2020b. “Using Artificial Intelligence and Social Media for Disaster Response and Management: An Overview.” In AI and Robotics in Disaster Studies Book. Part of the Disaster Research and Management Series on the Global South, edited by: T. V. V. Kumar, and K. Sud, 63–81. Singapore: Palgrave Macmillan.10.1007/978-981-15-4291-6_5Search in Google Scholar
Purohit, H., C. Castillo, M. Imran, and R. Pandey. 2018. “Social-EOC: Serviceability Model to Rank Social Media Requests for Emergency Operation Centers.” In Proceedings of the 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 119–26.10.1109/ASONAM.2018.8508709Search in Google Scholar
Rabadi, G. 2021. “Simulation and Optimization for Rapid Response after a Disaster.” Industrial and Systems Engineering 53 (5): 32–6.Search in Google Scholar
Resch, B., F. Usländer, and C. Havas. 2018. “Combining Machine-Learning Topic Models and Spatiotemporal Analysis of Social Media Data for Disaster Footprint and Damage Assessment.” Cartography and Geographic Information Science 45 (4): 362–76. https://doi.org/10.1080/15230406.2017.1356242.Search in Google Scholar
Reuter, C., A. L. Hughes, and M. A. Kaufholda. 2018. “Social Media in Crisis Management: An Evaluation and Analysis of Crisis Informatics Research.” International Journal of Human-Computer Interaction 34 (4): 280–94. https://doi.org/10.1080/10447318.2018.1427832.Search in Google Scholar
Reuter, C., S. Stieglitz, and M. Imran. 2020. “Social Media in Conflicts and Crises.” Behaviour & Information Technology 39 (3): 241–51. https://doi.org/10.1080/0144929x.2019.1629025.Search in Google Scholar
Ritchie, H. 2014. Natural Disasters. Published Online at Our World In Data.org. [Online Resource]. Also available at https://ourworldindata.org/natural-disasters.Search in Google Scholar
Rudra, K., P. Goyal, N. Ganguly, M. Imran, and P. Mitra 2019. “Summarizing Situational Tweets in Crisis Scenarios: An Extractive-Abstractive Approach.” IEEE Transactions on Computational Social Systems 6 (5): 981–93. https://doi.org/10.1109/TCSS.2019.2937899.Search in Google Scholar
Simonyan, K., and A. Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv:1409.1556.Search in Google Scholar
Stowe, K., J. Anderson, M. Palmer, L. Palen, and K. Anderson. 2018. “Improving Classification of Twitter Behavior during Hurricane Events.” In Proceedings of the Sixth International Workshop on Natural Language Processing for Social Media, 67–75. Melbourne.10.18653/v1/W18-3512Search in Google Scholar
Vaswan, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I. Polosukhin. 2017. “Attention Is All You Need.” In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17), 6000–10. Red Hook: Curran Associates Inc.Search in Google Scholar
Wang, Z., and X. Ye. 2018. “Social Media Analytics for Natural Disaster Management.” International Journal of Geographical Information Science 32 (1): 49–72. https://doi.org/10.1080/13658816.2017.1367003.Search in Google Scholar
Weber, E., N. Marzo, D. Papadopoulos, A. LaperdrizaLapedriza, F. Ofli, M. Imran, and A. Torralba. 2020. “Detecting Natural Disasters, Damage, and Incidents in the Wild.” In Computer Vision – ECCV 2020. Lecture Notes in Computer Science, Vol. 12364, edited by A. Vedaldi, H. Bischof, T. Brox, and J. M. Frahm. Cham: Springer.10.1007/978-3-030-58529-7_20Search in Google Scholar
Zade, H., K. Shah, V. Rangarajan, P. Kshirsagar, M. Imran, and K. Starbird. 2018. “From Situational Awareness to Actionability: Towards Improving the Utility of Social Media Data for Crisis Response.” In Proceedings of the ACM on Human-Computer Interaction, Vol. 2, 1–18: CSCW.10.1145/3274464Search in Google Scholar
Zahra, K., M. Imran, and F. Ostermann. 2020. “Automatic Identification of Eyewitness Messages on Twitter during Disasters.” Journal of Information Processing & Management 57 (1): 102107, https://doi.org/10.1016/j.ipm.2019.102107.Search in Google Scholar
© 2023 Walter de Gruyter GmbH, Berlin/Boston
