Skip to main content
SpringerLink
Log in

Decentralized multiple hypothesis testing in Cognitive IOT using massive heterogeneous data

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

An emerging area of study known as Cognitive IoT (CIoT) has emerged as a result of recent efforts to include cognition in the design of the Internet of Things (IoT). Several features and challenges from the IoT are carried over to the CIoT. A lot of applications in CIoT generate massive heterogeneous data, and they require inferential tasks with less computational overhead. Therefore, this study suggests a decentralized computing approach to handle multiple hypothesis testing in less computational time. The first stage is to minimize error at the cluster node by total variance regularization through the alternating direction method of multipliers. Subsequently, the fusion center handles the heterogeneity of data, minimizes model error, extracts the most informative data, and adjusts the p-value for multiple hypotheses testing. The experimental evaluation and cross-validation on six-month traces of the carbon monoxide dataset reveal the efficacy of the proposed algorithm over competing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  1. Asthon, K.: International telecommunication union. RFiD J 22, 97–114 (2010)

    Google Scholar 

  2. Jararweh, Y., Fatima, S., Jarrah, M., Alzu’bi, S.: Smart and sustainable agriculture: fundamentals, enabling technologies, and future directions. Comput Electr Eng 110, 108799 (2023). https://doi.org/10.1016/j.compeleceng.2023.108799

    Article  Google Scholar 

  3. Andronie, M., Lăzăroiu, G., Karabolevski, O.L., et al.: Remote big data management tools, sensing and computing technologies, and visual perception and environment mapping algorithms in the internet of robotic things. Electronics 12, 22 (2022). https://doi.org/10.3390/electronics12010022

    Article  Google Scholar 

  4. Tu, S., Yu, H., Badshah, A., et al.: Secure internet of vehicles (iov) with decentralized consensus blockchain mechanism. IEEE Trans. Veh. Technol. 72, 11227–11236 (2023). https://doi.org/10.1109/TVT.2023.3268135

    Article  Google Scholar 

  5. Atzori, L., Iera, A., Morabito, G.: The internet of things: a survey. Comput. Netw. 54(15), 2787–2805 (2010). https://doi.org/10.1016/j.comnet.2010.05.010

    Article  Google Scholar 

  6. Perera, C., Zaslavsky, A., Christen, P., Georgakopoulos, D.: Context aware computing for the internet of things: a survey. IEEE Commun. Surv. Tutorials 16, 414–454 (2014). https://doi.org/10.1109/SURV.2013.042313.00197

    Article  Google Scholar 

  7. Palattella, M.R., Accettura, N., Vilajosana, X., et al.: Standardized Protocol Stack for the Internet of (Important) Things. IEEE Commun. Surv. Tutorials 15, 1389–1406 (2013). https://doi.org/10.1109/SURV.2012.111412.00158

    Article  Google Scholar 

  8. Vlacheas, P., Giaffreda, R., Stavroulaki, V., et al.: Enabling smart cities through a cognitive management framework for the internet of things. IEEE Commun. Mag. 51, 102–111 (2013). https://doi.org/10.1109/MCOM.2013.6525602

    Article  Google Scholar 

  9. Zhang, M., Zhao, H., Zheng, R., et al.: Cognitive internet of things: concepts and application example. Int. J. Comput. Sci. Issues 9, 151 (2012)

    Google Scholar 

  10. Rouse M, Shea S, Haughn M. Iot devices (internet of things devices). Dosegljivo. 2018 Mar.

  11. Bui, N., Castellani, A., Casari, P., Zorzi, M.: The internet of energy: a web-enabled smart grid system. IEEE Netw. 26, 39–45 (2012). https://doi.org/10.1109/MNET.2012.6246751

    Article  Google Scholar 

  12. Tan, S., De, D., Song, W.-Z., et al.: Survey of security advances in smart grid: a data driven approach. IEEE Commun. Surv. Tutorials. 19, 397–422 (2017). https://doi.org/10.1109/COMST.2016.2616442

    Article  Google Scholar 

  13. (2017) High-assurance SPIRAL: end-to-end guarantees for robot and Car control. IEEE Control. Syst. 37:82–103. https://doi.org/10.1109/MCS.2016.2643244

  14. Benjamini, Y., Hochberg, Y.: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B 57, 289–300 (1995). https://doi.org/10.1111/j.2517-6161.1995.tb02031.x

    Article  MathSciNet  Google Scholar 

  15. Storey, J.D.: A direct approach to false discovery rates. Soc. Ser. B Statistical. Methodol. 64, 479–498 (2002). https://doi.org/10.1111/1467-9868.00346

    Article  MathSciNet  Google Scholar 

  16. Efron, B.: Large-scale simultaneous hypothesis testing. J. Am. Stat. Assoc. 99, 96–104 (2004). https://doi.org/10.1198/016214504000000089

    Article  Google Scholar 

  17. Li, S., Wang, X.: Fully distributed sequential hypothesis testing: algorithms and asymptotic analyses. IEEE Trans. Inf. Theory 64, 2742–2758 (2018). https://doi.org/10.1109/TIT.2018.2806961

    Article  MathSciNet  Google Scholar 

  18. Veeravalli, V.V., Basar, T., Poor, H.V.: Decentralized sequential detection with a fusion center performing the sequential test. IEEE Trans. Inf. Theory 39, 433–442 (1993). https://doi.org/10.1109/18.212274

    Article  Google Scholar 

  19. Tsitsiklis, J.N.: Decentralized detection. Adv. Stat. Signal Proc. 2, 297–344 (1993)

    Google Scholar 

  20. Fellouris, G., Moustakides, G.V.: Decentralized sequential hypothesis testing using asynchronous communication. IEEE Trans. Inf. Theory 57, 534–548 (2011). https://doi.org/10.1109/TIT.2010.2090249

    Article  MathSciNet  Google Scholar 

  21. Li, S., Li, X., Wang, X., Liu, J.: Decentralized sequential composite hypothesis test based on one-bit communication. IEEE Trans. Inf. Theory 63, 3405–3424 (2017). https://doi.org/10.1109/TIT.2017.2693156

    Article  ADS  MathSciNet  Google Scholar 

  22. Helbing, D.: The automation of society is next: how to survive the digital revolution. SSRN Electron. J. (2015). https://doi.org/10.2139/ssrn.2694312

    Article  Google Scholar 

  23. Allakany A, Yadav G, Paul K, Okamura K (2020) Detection and mitigation of lfa attack in sdn-iot network. In: Workshops of the International Conference on Advanced Information Networking and Applications. Springer, pp 1087–1096

  24. Wu, J., Wang, C., Yu, Y., et al.: Sequential fusion to defend against sensing data falsification attack for cognitive Internet of Things. ETRI J. 42, 976–986 (2020). https://doi.org/10.4218/etrij.2019-0388

    Article  Google Scholar 

  25. Li, F., Xie, R., Wang, Z., et al.: Online distributed iot security monitoring with multidimensional streaming big data. IEEE Int. Things J. 7, 4387–4394 (2020). https://doi.org/10.1109/JIOT.2019.2962788

    Article  Google Scholar 

  26. Kassab R, Simeone O, Popovski P (2020) Fog-based detection for random-access iot networks with per-measurement preambles. In: 2020 IEEE 21st International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). IEEE, pp 1–5

  27. Walshe, M., Epiphaniou, G., Al-Khateeb, H., et al.: Non-interactive zero knowledge proofs for the authentication of IoT devices in reduced connectivity environments. Ad. Hoc. Netw. 95, 101988 (2019). https://doi.org/10.1016/j.adhoc.2019.101988

    Article  Google Scholar 

  28. Ukil, A., Bandyopadhyay, S., Pal, A.: IoT-Privacy: To be private or not to be private. In: 2014 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, (2014) pp 123–124

  29. Tarighati, A., Gross, J., Jalden, J.: Decentralized hypothesis testing in energy harvesting wireless sensor networks. IEEE Trans. Signal Process. 65, 4862–4873 (2017). https://doi.org/10.1109/TSP.2017.2716909

    Article  ADS  MathSciNet  Google Scholar 

  30. Ullah, A., Azeem, M., Ashraf, H., et al.: Secure critical data reclamation scheme for isolated clusters in iot-enabled WSN. IEEE Internet Things J. 9, 2669–2677 (2022). https://doi.org/10.1109/JIOT.2021.3098635

    Article  Google Scholar 

  31. Sun, M., Tay, W.P.: On the relationship between inference and data privacy in decentralized iot networks. IEEE Trans. Inf. Forensics Secur. 15, 852–866 (2020). https://doi.org/10.1109/TIFS.2019.2929446

    Article  Google Scholar 

  32. Leonard, M.R., Stiefel, M., Fauss, M., Zoubir, A.M.: Robust sequential testing of multiple hypotheses in distributed sensor networks. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, (2018) pp 4394–4398

  33. List, J.A., Shaikh, A.M., Xu, Y.: Multiple hypothesis testing in experimental economics. Exp. Econ. 22, 773–793 (2019)

    Article  Google Scholar 

  34. Lee, S., Lee, D.K.: What is the proper way to apply the multiple comparison test? Korean J. Anesthesiol. 71, 353–360 (2018). https://doi.org/10.4097/kja.d.18.00242

    Article  PubMed  PubMed Central  Google Scholar 

  35. Yu, C., Zelterman, D.: Distributions associated with simultaneous multiple hypothesis testing. J. Stat. Distrib. Appl. 7, 6 (2020). https://doi.org/10.1186/s40488-020-00109-6

    Article  Google Scholar 

  36. Qian, X., Brutti, A., Lanz, O., et al.: Multi-speaker tracking from an audio-visual sensing device. IEEE Trans. Multimed. 21, 2576–2588 (2019). https://doi.org/10.1109/TMM.2019.2902489

    Article  Google Scholar 

  37. Yong Rui, Yunqiang Chen, Rui Y, Chen Y Better proposal distributions: object tracking using unscented particle filter. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. IEEE Comput. Soc, pp II-786-II–793

  38. Kidron, E., Schechner, Y.Y., Elad, M.: Cross-modal localization via sparsity. IEEE Trans. Signal Process. 55(4), 1390–1404 (2007). https://doi.org/10.1109/TSP.2006.888095

    Article  ADS  MathSciNet  Google Scholar 

  39. Sargın, M.E., Member, S., Yemez, Y., et al.: Audio-visual synchronization and fusion using canonical correlation analysis. Speech Signal Process. Proc. (2006). https://doi.org/10.1109/ICASSP.2006.1660095

    Article  Google Scholar 

  40. Butz, T., Thiran, J.: From error probability to information theoretic (multi-modal) signal processing. Signal Proc. 85, 875–902 (2005). https://doi.org/10.1016/j.sigpro.2004.11.027

    Article  Google Scholar 

  41. Fisher, J.W., Darrell, T.: Speaker association with signal-level audiovisual fusion. IEEE Trans. Multimed. 6, 406–413 (2004). https://doi.org/10.1109/TMM.2004.827503

    Article  Google Scholar 

  42. Besson, P., Kunt, M.: Hypothesis testing for evaluating a multimodal pattern recognition framework applied to speaker detection. J. Neuro Eng. Rehabilit. 8, 1–8 (2008). https://doi.org/10.1186/1743-0003-5-11

    Article  Google Scholar 

  43. Prati, A., Vezzani, R., Benini, L., et al.: An integrated multi-modal sensor network for video surveillance. In: Proceedings of the third ACM international workshop on Video surveillance & sensor networks - VSSN ’05. ACM Press, New York, New York, USA, p 95 (2005)

  44. Siegmund, D.O., Zhang, N.R., Yakir, B.: False discovery rate for scanning statistics. Biometrika 98, 979–985 (2011). https://doi.org/10.1093/biomet/asr057

    Article  MathSciNet  Google Scholar 

  45. Sun W, Reich BJ, Tony Cai T, et al (2015) False discovery control in large-scale spatial multiple testing. J R Stat Soc Ser B (Statistical Methodol 77:59–83. https://doi.org/10.1111/rssb.12064

  46. Kirk, J.T., Cain, S.C.: Multi-hypothesis test detection for star tracking systems. In: AIAA Scitech 2021 Forum. American Institute of Aeronautics and Astronautics, Reston, Virginia (2021)

  47. Lánczky, A., Győrffy, B.: Web-based survival analysis tool tailored for medical research (kmplot): development and implementation. J. Med. Int. Res. 23, e27633 (2021). https://doi.org/10.2196/27633

    Article  Google Scholar 

  48. Golz, M., Zoubir, A.M., Koivunen, V.: Multiple hypothesis testing framework for spatial signals. IEEE Trans Signal Inf Process over Networks 8, 771–787 (2022). https://doi.org/10.1109/TSIPN.2022.3190735

    Article  MathSciNet  Google Scholar 

  49. Herrmann, M., Piroli, A., Strohbeck, J., et al (2020) LMB filter based tracking allowing for multiple hypotheses in object reference point association. In: 2020 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI). IEEE, pp 197–203

  50. Liu, X., Zhang, Z., Wang, L.: Bayesian hypothesis testing of mediation: Methods and the impact of prior odds specifications. Behav. Res. Methods (2022). https://doi.org/10.3758/s13428-022-01860-1

    Article  PubMed  PubMed Central  Google Scholar 

  51. Gilani, A., Belhadj Amor, S., Salehkalaibar, S., Tan, V.Y.F.: Distributed hypothesis testing with privacy constraints. Entropy 21, 478 (2019). https://doi.org/10.3390/e21050478

    Article  ADS  MathSciNet  PubMed  PubMed Central  Google Scholar 

  52. Negm, E.: Internet of Things (IoT) acceptance model – assessing consumers’ behavior toward the adoption intention of IoT. Arab Gulf J. Sci. Res. (2023). https://doi.org/10.1108/AGJSR-09-2022-0183

    Article  Google Scholar 

  53. Khan, S., Thapa, C., Durrani, S., Camtepe, S.: Access-based Lightweight Physical Layer Authentication for the Internet of Things Devices. (2023) 1–13 https://doi.org/10.48550/arXiv.2303.00307

  54. Šidák, Z.: Rectangular confidence regions for the means of multivariate normal distributions. J. Am. Stat. Assoc. 62, 626–633 (1967). https://doi.org/10.1080/01621459.1967.10482935

    Article  MathSciNet  Google Scholar 

  55. Holm, S.: Board of the foundation of the scandinavian journal of statistics a simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65–70 (1979)

    Google Scholar 

  56. Simes, R.J.: An improved Bonferroni procedure for multiple tests of significance. Biometrika 73, 751–754 (1986). https://doi.org/10.1093/biomet/73.3.751

    Article  MathSciNet  Google Scholar 

  57. Hochberg, Y.: A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75, 800–802 (1988). https://doi.org/10.1093/biomet/75.4.800

    Article  MathSciNet  Google Scholar 

  58. Hommel, G.: A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 75, 383–386 (1988). https://doi.org/10.1093/biomet/75.2.383

    Article  Google Scholar 

  59. Moulin, P., Veeravalli, V.V.: Sha. Stat Inference. Eng Data Sci 50, 54–70 (2018). https://doi.org/10.1017/9781107185920.005

    Article  Google Scholar 

  60. Benjamini, Y.: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B 57, 289–300 (1995)

    MathSciNet  Google Scholar 

  61. Zhang, C., Lee, H., Shin, K.G.: Efficient distributed linear classification algorithms via the alternating direction method of multipliers. J. Mach. Learn. Res. 22, 1398–1406 (2012)

    Google Scholar 

  62. Zhao, P., Yang, J., Zhang, T., Li, P.: Adaptive stochastic alternating direction method of multipliers. Int. Conf. Mach. Learn. 1, 69–77 (2015)

    Google Scholar 

  63. Samuel, M.D.D.K., Dominique, D.M., Samuel K.: Correlation and dependence. World Scientific, (2001)

  64. Hurtado, J.E.: Structural reliability: statistical learning perspectives. Springer Science & Business Media; 2004 May 13.

  65. Joe, H., Xu, J.J.: The estimation method of inference functions for margins for multivariate models. Tech Rep no 166, Dep Stat Univ Br Columbia 1–21 (1996)

  66. Nelsen R.B.: An introduction to copulas. New York, USA. Lect Notes Stat 1999

  67. Kurowicka, D., Cooke, R.M.: Uncertainty analysis with high dimensional dependence modelling. John Wiley & Sons (2006)

    Book  Google Scholar 

  68. Bouyé, E., Durrleman, V., Nikeghbali, A., et al.: Copulas for finance - a reading guide and some applications. SSRN Electron. J. (2000). https://doi.org/10.2139/ssrn.1032533

    Article  Google Scholar 

  69. Genest, C., Ghoudi, K., Rivest, L.-P.: A semiparametric estimation procedure of dependence parameters in multivariate families of distributions. Biometrika 82, 543–552 (1995). https://doi.org/10.1093/biomet/82.3.543

    Article  MathSciNet  Google Scholar 

  70. Shihl, J.H., Louis, T.A.: Inferences on the association parameter in copula models for bivariate survival data. Biometrics 51, 1384–1399 (1995)

    Article  MathSciNet  Google Scholar 

  71. Akaike, H.: A new look at the statistical model identification. IEEE Trans. Automat. Contr. 19, 716–723 (1974). https://doi.org/10.1109/TAC.1974.1100705

    Article  ADS  MathSciNet  Google Scholar 

  72. Akaike, H.: Likelihood of a model and information criteria. J Econom 16, 3–14 (1981). https://doi.org/10.1016/0304-4076(81)90071-3

    Article  Google Scholar 

  73. Lehmann E.L., Casella G.: Theory of point estimation. Springer Science & Business Media (2006)

  74. Kullback S. Information theory and statistics. Courier Corporation; 1997 Jul 7.

  75. Bozdogan, H.: Model selection and Akaike’s information criterion (AIC): the general theory and its analytical extensions. Psychometrika 52, 345–370 (1987)

    Article  MathSciNet  Google Scholar 

  76. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. J. R Stat. Soc. Ser. B 39, 1–22 (1977). https://doi.org/10.1111/j.2517-6161.1977.tb01600.x

    Article  MathSciNet  Google Scholar 

  77. McLachlan, G.J., Thriyambakam, K., and Ng, K.: The EM algorithm. 24: 2004 (2004)

  78. Statistics M: On the Convergence Properties of the EM Algorithm. Institute of Mathematical Statistics Stable (1983) pp 95–103

  79. Figueiredo, M.A.T., Jain, A.K.: Unsupervised learning of finite mixture models. IEEE Trans. Patt. Anal. Mach. Intell. 24, 381–396 (2002). https://doi.org/10.1109/34.990138

    Article  Google Scholar 

  80. Hinton G, Sejnowski TJ (1999) Unsupervised Learning: Foundations of Neural Computation. MIT press

  81. Dunnett, C.W.: A multiple comparison procedure for comparing several treatments with a control. J. Am. Stat. Assoc. 50, 1096–1121 (1955)

    Article  Google Scholar 

  82. Armstrong, R.A.: When to use the B onferroni correction. Ophthalmic. Physiol. Opt. 34(5), 502–508 (2014). https://doi.org/10.1111/opo.12131

    Article  PubMed  Google Scholar 

  83. Streiner, D.L., Norman, G.R.: Correction for multiple testing. Chest 140, 16–18 (2011). https://doi.org/10.1378/chest.11-0523

    Article  PubMed  Google Scholar 

  84. Benjamini, Y., Hochberg, Y.: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Royal Stat. Soc. 57, 289–300 (1995)

    MathSciNet  Google Scholar 

  85. Sarkar, S.K., Chang, C.K., Chang, C.K.: The simes method for multiple hypothesis testing with positively dependent test statistics. J. Am. Stat. Assoc. 92, 1601–1608 (1997). https://doi.org/10.1080/01621459.1997.10473682

    Article  MathSciNet  Google Scholar 

  86. Hussien, M., Nguyen, K.K., Cheriet, M.: A learning framework for bandwidth-efficient distributed inference in wireless IoT. IEEE Sens. J. 23, 17656–17666 (2023). https://doi.org/10.1109/JSEN.2023.3283923

    Article  ADS  Google Scholar 

  87. Gölz M, Zoubir AM, Koivunen V (2023) Spatial Inference Using Censored Multiple Testing with Fdr Control. In: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 1–5

  88. Chen, L., Zhou, J., Lin, L.: Hypothesis testing for populations of networks. Commun. Stat. Theory Methods 52, 3661–3684 (2023). https://doi.org/10.1080/03610926.2021.1977961

    Article  MathSciNet  Google Scholar 

  89. Vieira F, Leenders R, McFarland D, Mulder J (2023) A Bayesian actor-oriented multilevel relational event model with hypothesis testing procedures. Springer Japan

  90. Wu, Y., Jing, T., Gao, Q., et al.: Game-theoretic physical layer authentication for spoofing detection in internet of things. Digit Commun. Netw. (2023). https://doi.org/10.1016/j.dcan.2022.12.016

    Article  Google Scholar 

Download references

Funding

The authors have not disclosed any funding.

Author information

Author notes

  1. Vidyapati Jha and Priyanka Tripathi have contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Applications, National Institute of Technology, Raipur, Chattisgarh, India

    Vidyapati Jha & Priyanka Tripathi

Authors
  1. Vidyapati Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Priyanka Tripathi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Vidyapati Jha designed the model and carried out the implementation with the computational framework. Priyanka Tripathi performed the analysis and wrote the manuscript. Both authors were in charge of overall direction and planning.

Corresponding author

Correspondence to Vidyapati Jha.

Ethics declarations

Conflict of interest

The authors have not disclosed any competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jha, V., Tripathi, P. Decentralized multiple hypothesis testing in Cognitive IOT using massive heterogeneous data. Cluster Comput (2024). https://doi.org/10.1007/s10586-024-04324-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10586-024-04324-7

Keywords

Search

Navigation