Skip to main content
SpringerLink
Log in

Image Processing Pipeline for Fluoroelastomer Crystallite Detection in Atomic Force Microscopy Images

  • Thematic Section: 7th World Congress on Integrated Computational Materials Engineering
  • Published:
Integrating Materials and Manufacturing Innovation Aims and scope Submit manuscript

Abstract

Phase transformations in materials systems can be tracked using atomic force microscopy (AFM), enabling the examination of surface properties and macroscale morphologies. In situ measurements investigating phase transformations generate large datasets of time-lapse image sequences. The interpretation of the resulting image sequences, guided by domain-knowledge, requires manual image processing using handcrafted masks. This approach is time-consuming and restricts the number of images that can be processed. In this study, we developed an automated image processing pipeline which integrates image detection and segmentation methods. We examine five time-series AFM videos of various fluoroelastomer phase transformations. The number of image sequences per video ranges from a hundred to a thousand image sequences. The resulting image processing pipeline aims to automatically classify and analyze images to enable batch processing. Using this pipeline, the growth of each individual fluoroelastomer crystallite can be tracked through time. We incorporated statistical analysis into the pipeline to investigate trends in phase transformations between different fluoroelastomer batches. Understanding these phase transformations is crucial, as it can provide valuable insights into manufacturing processes, improve product quality, and possibly lead to the development of more advanced fluoroelastomer formulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Nguyen-Tri P, Ghassemi P, Carriere P, Nanda S, Assadi AA, Nguyen DD (2020) Recent applications of advanced atomic force microscopy in polymer science: a review. Polymers 12(5):1142

    Article  CAS  Google Scholar 

  2. Binnig G, Quate CF, Gerber C (1986) Atomic force microscope. Phys Rev lett 56(9):930

    Article  CAS  Google Scholar 

  3. Maiti M, Bhowmick AK (2006) New insights into rubber-clay nanocomposites by AFM imaging. Polymer 47(17):6156–6166

    Article  CAS  Google Scholar 

  4. Ornaghi FG, Bianchi O, Ornaghi HL Jr, Jacobi MA (2019) Fluoroelastomers reinforced with carbon nanofibers: a survey on rheological, swelling, mechanical, morphological, and prediction of the thermal degradation kinetic behavior. Polym Eng Sci 59(6):1223–1232

    Article  CAS  Google Scholar 

  5. Ameduri B, Boutevin B, Kostov G (2001) Fluoroelastomers: synthesis, properties and applications. Prog Polym Sci 26(1):105–187

    Article  CAS  Google Scholar 

  6. Améduri B (2020) The promising future of fluoropolymers. Macromol Chem Phys 221(8):1900573. https://doi.org/10.1002/macp.201900573

    Article  CAS  Google Scholar 

  7. Hobbs JK, Farrance OE, Kailas L (2009) How atomic force microscopy has contributed to our understanding of polymer crystallization. Polymer 50(18):4281–4292. https://doi.org/10.1016/j.polymer.2009.06.021

    Article  CAS  Google Scholar 

  8. Wang Y, Yao Q, Kwok JT, Ni LM (2020) Generalizing from a few examples: a survey on few-shot learning. ACM Comput Surv CSUR 53(3):1–34

    Google Scholar 

  9. Gaponenko I, Tückmantel P, Ziegler B, Rapin G, Chhikara M, Paruch P (2017) Computer vision distortion correction of scanning probe microscopy images. Sci Rep 7(1):669

    Article  Google Scholar 

  10. Wang Y, Lu T, Li X, Wang H (2018) Automated image segmentation-assisted flattening of atomic force microscopy images. Beilstein J Nanotechnol 9(1):975–985

    Article  CAS  Google Scholar 

  11. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48

    Article  Google Scholar 

  12. Giergiel M, Zapotoczny B, Czyzynska-Cichon I, Konior J, Szymonski M (2022) AFM image analysis of porous structures by means of neural networks. Biomed Signal Process Control 71:103097. https://doi.org/10.1016/j.bspc.2021.103097

    Article  Google Scholar 

  13. Ge M, Su F, Zhao Z, Su D (2020) Deep learning analysis on microscopic imaging in materials science. Materials Today Nano 11:100087. https://doi.org/10.1016/j.mtnano.2020.100087

    Article  Google Scholar 

  14. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  15. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFS. IEEE Trans Pattern Anal Mach Intell 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  16. Wang Y, Yao Q (2019) Few-shot learning: a survey. CoRR arXiv: 1904.05046

  17. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  CAS  Google Scholar 

  18. Mandelkern L, Martin G, Quinn F Jr (1957) Poly-(vinylidene fluoride), and their copolymers. J Res Natl Bur Stand 58(3):137

    Article  CAS  Google Scholar 

  19. Ameduri B (2009) From vinylidene fluoride (VDF) to the applications of VDF-containing polymers and copolymers: recent developments and future trends. Chem Rev 109(12):6632–6686

    Article  CAS  Google Scholar 

  20. Kelly K, Brown G, Anthony S (2020) Quantifying CTFE content in FK-800 using ATR-FTIR and time to peak crystallization. Int J Polym Anal Charact 25(8):621–633. https://doi.org/10.1080/1023666X.2020.1827859

    Article  CAS  Google Scholar 

  21. Willey TM, DePiero SC, Hoffman DM (May 2009) A comparison of new TATBs, FK-800 binder and LX-17-like PBXs to legacy materials. Technical report LLNL-CONF-412929, Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States). https://www.osti.gov/biblio/966908 Accessed 2021-12-14

  22. Orme CA (2018) Progress summary: developing experimental methods to quantify the degree of crystallinity in fluoropolymer binders. Tech Rep Lawrence Livermore Nat Lab. https://doi.org/10.2172/1476198

    Article  Google Scholar 

  23. Cady W, Caley L (1977) Properties of Kel F-800 polymer. Technical report UCRL-52301, Lawrence Livermore National Lab. (LLNL), Livermore CA, USA. https://doi.org/10.2172/5305005. http://www.osti.gov/servlets/purl/5305005/ Accessed 2021-11-08

  24. Crist B, Schultz JM (2016) Polymer spherulites: a critical review. Prog Polym Sci 56:1–63. https://doi.org/10.1016/j.progpolymsci.2015.11.006

    Article  CAS  Google Scholar 

  25. Su Y, Liu G, Xie B, Fu D, Wang D (2014) Crystallization features of normal alkanes in confined geometry. Accounts Chem Res 47(1):192–201. https://doi.org/10.1021/ar400116c

    Article  CAS  Google Scholar 

  26. George L (2011) HBase the definitive guide. O’Reilly, Sebastopol, CA. http://shop.oreilly.com/product/0636920014348.do Accessed 2013-04-18

  27. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG (2009) Research electronic data capture (redcap)-a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42(2):377–381. https://doi.org/10.1016/j.jbi.2008.08.010

    Article  Google Scholar 

  28. Girshick R (2015) Fast r-cnn. In: 2015 IEEE International conference on computer vision (ICCV), pp 1440–1448. https://doi.org/10.1109/ICCV.2015.169

  29. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. CoRR arXiv:1804.02767

  30. Wang CY, Liao HYM, Yeh IH, Wu YH, Chen PY, Hsieh JW (2019) CSPNet: A new backbone that can enhance learning capability of CNN. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

  31. He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Computer vision—ECCV 2014, pp 346–361. Springer, Berlin. https://doi.org/10.1007/978-3-319-10578-9_23

  32. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8759–8768. Salt Lake City, UT. https://doi.org/10.1109/CVPR.2018.00913

  33. Misra D (2020) Mish: A self regularized non-monotonic activation function

  34. Wang CY, Bochkovskiy A, Liao HYM (2022) YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors

  35. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR), pp 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965

  36. Chen LC, Papandreou G, Schroff F, Adam H (2017) Rethinking Atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587

  37. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122

  38. Russell BC, Torralba A, Murphy KP, Freeman WT (2008) Labelme: a database and web-based tool for image annotation. Int J Comput Vision 77(1):157–173

    Article  Google Scholar 

  39. Ester M, Kriegel HP, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the second international conference on knowledge discovery and data mining. KDD’96, pp 226–231. AAAI Press, Pomona

  40. Schubert E, Sander J, Ester M, Kriegel HP, Xu X (2017) DBSCAN revisited, revisited: why and how you should (still) use DBSCAN. ACM Trans Database Syst TODS 42(3):1–21. https://doi.org/10.1145/3068335

    Article  Google Scholar 

  41. Avrami M (2004) Kinetics of phase change I general theory. J Chem Phys 7(12):1103–1112. https://doi.org/10.1063/1.1750380

    Article  Google Scholar 

  42. Long Y, Shanks RA, Stachurski ZH (1995) Kinetics of polymer crystallisation. Prog Polym Sci 20(4):651–701. https://doi.org/10.1016/0079-6700(95)00002-W

    Article  CAS  Google Scholar 

  43. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88:303–338

    Article  Google Scholar 

  44. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge. http://www.deeplearningbook.org

  45. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: a metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 658–666

  46. Bradski G (2000) The opencv library. Dr Dobb’s J Softw Tools Prof Program 25(11):120–123

    Google Scholar 

  47. Cantor B (2020) The Avrami equation: phase transformations. In: Cantor, B. (ed.) The equations of materials, pp. 180–206. Oxford University Press, Oxford. https://doi.org/10.1093/oso/9780198851875.003.0009

Download references

Acknowledgements

This material is based upon research in the Materials Data Science for Stockpile Stewardship Center of Excellence (MDS3−COE), and supported by the Department of Energy’s National Nuclear Security Administration under Award Number(s) DE-NA0004104. CO acknowledges useful discussions with En Ju Cho, Chami Swaminathan, Xiaojie Xu, and James Lewicki. Work performed by CO was under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52- 07NA27344. This work made use of the High Performance Computing Resource in the Core Facility for Advanced Research Computing at Case Western Reserve University.

Author information

Authors and Affiliations

  1. Materials Data Science for Stockpile Stewardship: Center of Excellence, Case Western Reserve University (CWRU), Cleveland, OH, 44106, USA

    Mingjian Lu, Sameera Nalin Venkat, Jube Augustino, David Meshnick, Jayvic Cristian Jimenez, Pawan K. Tripathi, Arafath Nihar, Roger H. French, Laura S. Bruckman & Yinghui Wu

  2. Department of Materials Science and Engineering, CWRU, Cleveland, OH, 44106, USA

    Sameera Nalin Venkat, Jube Augustino, Jayvic Cristian Jimenez, Pawan K. Tripathi, Roger H. French & Laura S. Bruckman

  3. Department of Computer and Data Sciences, CWRU, Cleveland, OH, 44106, USA

    Mingjian Lu, David Meshnick, Arafath Nihar & Yinghui Wu

  4. Physical and Life Sciences Directorate, Lawrence Livermore National Laboratory, Livermore, CA, 94550, USA

    Christine A. Orme

Authors
  1. Mingjian Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sameera Nalin Venkat
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jube Augustino
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Meshnick
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jayvic Cristian Jimenez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pawan K. Tripathi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Arafath Nihar
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Christine A. Orme
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Roger H. French
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Laura S. Bruckman
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Yinghui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinghui Wu.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, M., Venkat, S.N., Augustino, J. et al. Image Processing Pipeline for Fluoroelastomer Crystallite Detection in Atomic Force Microscopy Images. Integr Mater Manuf Innov 12, 371–385 (2023). https://doi.org/10.1007/s40192-023-00320-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40192-023-00320-8

Keywords

Search

Navigation