Elsevier

Applied Soft Computing

Volume 112, November 2021, 107759
Applied Soft Computing

Accurate multi-class image segmentation using weak continuity constraints and neutrosophic set

https://doi.org/10.1016/j.asoc.2021.107759Get rights and content

Highlights

  • The method segments an image using the concept of weak continuity constraints.

  • The constraints in neutrosophic domain help to manage uncertainties.

  • The method automatically segments an image without prior knowledge about the number of classes.

Abstract

In this paper, we propose a multi-class image segmentation method based on uncertainty management by weak continuity constraints and neutrosophic set (NS). To manage the uncertainties in the segmentation process, an image is mapped into the NS domain. In the NS domain, the image is represented as true, false, and indeterminate subsets. In the proposed method, accurate segmentation is achieved by minimizing an energy function in the NS domain. The theory of weak continuity constraints is integrated into the NS domain to generate the energy function. The weak continuity constraints take into account the spatial and boundary information of the segments to manage the uncertainties in the segmentation process. The proposed method can automatically segment an image iteratively without any prior knowledge about the number of classes. The performance of the proposed method is compared with state-of-the-art methods and it is found to be quite satisfactory. The proposed method’s performance under noise perturbations is statistically validated using a modified Cramer–Rao bound. The bound predicts the performance of image segmentation algorithms and serves as a benchmark for segmentation results.

Introduction

The modern age is the age of automation. In this modern era, there is a lot of applications of image processing in metal industries [1], agriculture industries [2], robotics [3], and medical sciences [4]. The application varies from automatic fault detection, quality detection to MRI and CT scan processing, etc. In various stages of image processing, image segmentation especially the segmentation of a multi-class image is one of the essential task [5]. In the multi-class image segmentation process, an image is segmented into different regions based on different features. For proper segmentation, the variations among the pixels in the same segment should be decreased with the increase in the variation among different segments. Moreover, an efficient segmentation technique should also be able to determine the boundaries between the segments accurately. The localization of the segment boundaries with the detection of segments is an ill-posed problem [6]. This is due to the uncertainties involved in the process of segmentation [7]. The automatic detection of the number of segments in an image without having any pre-defined knowledge is also a difficult task. As both of these problems emerge in highly uncertain image patterns, they must be addressed properly for an efficient segmentation algorithm to be developed.

The literature on different types of segmentation techniques is quite rich. Currently, the segmentation techniques based on the edge, graph, region, and energy are quite popular. Zhang et al. [8] proposed a graph-based technique for fast image segmentation. Here, an image was segmented based on the partitioning of the graph. In the paper, Bragantini et al. [9] extended the concept of Image Foresting Transform (IFT) for the graph-based classification of different objects in an image. The graph-based dominant set concept was utilized by Mequanint et al. [10] for the segmentation of an image into different classes. Niu et al. [11] proposed a region-based method based on local similarity for the segmentation of regions in an image. The method required no pre-processing and thus preserved the image structure accurately. Edge or contour-based methods were used by the authors in [12], [13]. In the edge-based methods, the gradient was utilized to generate the proper segments. Among all the conventional methods, segmentations based on thresholding are very popular due to the simplicity of the techniques. Multi-level thresholding based on Tsallis–Havrda–Charvat entropy was proposed by Borjigin et al. [14]. Upadhyay et al. [15] proposed multilevel image thresholding using Kapur’s entropy. Several unconventional techniques were also utilized by some researchers for the segmentation of an image. Algebraic topology-based image segmentation was proposed by Assaf et al. [16]. A persistent homology technique of algebraic topology was utilized here for the segmentation purpose. Wang et al. [17] presented a geometric flow-based bandlet transform for segmentation. The particle replanting algorithm was employed here to deal with the region merging or separating in an image.

The methods mentioned above gave no attention to manage the uncertainties that arise in the segmentation process. Thus, the methods were less tolerant of the ambiguities due to complex image patterns, different types of perturbations etc. [18].

In the literature on segmentation, a lot of methods used uncertainty handling techniques for the accurate classification of regions in an image. They mainly used fuzzy set [19], [20], [21], rough set [22] or neutrosophic set [23] for uncertainty handling. In the paper [24] we proposed a type-2 fuzzy set-based multi-class image segmentation. Out of the three sets, the neutrosophic set (NS) is relatively new and it is very popular to handle various types of uncertainties. The applications of NS in the field of image segmentation were found to be quite effective [25], [26].

Due to high computational complexity in multi-class image segmentation, several researchers used different meta-heuristic search algorithms to compute the thresholds. In the paper proposed by Li et al. [27] authors used fuzzy coyote optimization algorithm for multi-class segmentation. They used Otsu and fuzzy entropy as their objective functions. They did not consider the boundary information between the regions of an image. Wunnava et al. [28] proposed an adaptive Harris hawks optimization method for multi-class image segmentation. Multi-level thresholding by minimizing Tsallis fuzzy entropy was proposed by Raj et al. [29]. Here they utilized a differential evolution algorithm to search the thresholds. The authors in the paper [30] investigated butterfly optimization and gases Brownian motion optimization for multi-class segmentation. Bat algorithm and type-2 fuzzy-based image segmentation was proposed by us in the paper [31]. The limitation of the meta-heuristic search-based algorithms is that, in most cases, they are not fully automatic. That means, the number of the thresholds should be pre-defined. Moreover, a large number of parameters should be initialized in those algorithms.

One ideal segmentation technique should have some important properties. The technique should consider the spatial information along with the gray level information for capturing the texture of a segment. The method should also consider the boundary information with the utilization of some proper uncertainty handling tools.

Some of the methods found in the literature incorporate local information for segmentation [14], [32]. But the methods have no provision for capturing the boundary information between the segments. Moreover, in most of the methods, the number of classes or segments are ad-hoc or fixed. That means they should have prior knowledge about the number of segments in an image. Additionally, the conventional methods, which do not have any uncertainties handling tools could not reduce the ambiguities in an image. These limitations restrict the methods to reach the highest level of accuracy and they also leave room for improvement of the methods for practical applications.

Because of the above limitations, we propose an automatic multi-class image segmentation method based on weak continuity constraints. The weak continuity constraints help to localize the boundaries between the segments in the image. It also takes into account local or spatial information during segmentation. In the field of computer vision, weak continuity constraints were efficiently used for image reconstruction [33], [34]. Weak continuity constraints in a set of data are the constraints that can be violated but with a penalty. The property is used to localize the discontinuities in a set of reconstructed values. One can find in [34] that weak continuity constraints are powerful tools for discontinuity detection. Conventional methods for discontinuity detection blur the original signal. No such problem arises in weak continuity constraints and they detect the discontinuities robustly and accurately without any prior information about the position of discontinuities. The boundaries between the classes in a segmented image may not be strong due to low-intensity change, i.e low gradient at the boundary area between the two classes. This may increase the uncertainties in the segmentation process. Thresholding depending only on the intensity values of the pixels may disconnect the weak boundaries. The thresholding using weak continuity constraints helps to localize the segmentation boundaries, and thus it is useful for uncertainty management.

As already mentioned recently, we have proposed a segmentation method using weak continuity constraints in the type-2 fuzzy set domain [24] with high accuracy. But, the limitation of a type-2 fuzzy set is that it requires a type-reduction, which has high computational complexity. Given the above limitations, in this paper, we propose weak continuity constraints-based segmentation in the NS domain.

To manage the uncertainties in the segmentation process accurately, we apply the weak continuity constraints in the NS domain. The NS was proposed by Florentin Smarandache [23]. An NS is represented by three subsets. They are true, indeterminate, and false subsets. For an image segmentation process by NS, the image is converted into a neutrosophic image. This is followed by the minimization of the uncertainties to generate the segments [35], [36], [37].

The novelty of the proposed method is that here we propose an energy function in the NS domain based on weak continuity constraints for image segmentation. The energy function acts as an objective function for segmentation in the NS domain. The objective function can express the gray level and spatial ambiguity in an image in the NS domain and the minimization of the function minimizes the ambiguities. The proposed method automatically determines the segments iteratively in an image without any prior knowledge. The iteration stops when a proposed base condition is satisfied. The schematic diagram of the proposed method is shown in Fig. 1.

We have already mentioned that we incorporated the weak continuity constraints in the fuzzy set domain for segmentation [21], [24]. In image segmentation using a fuzzy set, one pixel can have different membership for representing the belongingness of the pixel into different classes. But, it has no provision for representing the portion in between the classes explicitly. Logically, the portion should be represented separately by another set. In NS, the set is called an indeterminate subset. Thus, the representation of an image into an NS domain is significantly different from the representation of the image into fuzzy domain. In this paper, we incorporate the weak continuity constraints in the NS domain for automatic multi-class segmentation. We use the indeterminate subset of the NS for boundary regions and the true subset of the NS for non-boundary regions. It is expected that the two subset representation of NS will give better performance than that of the conventional fuzzy set in the segmentation process.

The rest of the paper is as follows: The brief introduction of the neutrosophic set (NS), mapping of an image into the NS domain, the theory of weak continuity constraints, the application of the weak continuity constraints in the NS domain for segmentation, and proposed algorithm are described in Section 2. The results are discussed in Section 3. The section includes different quantitative measures, comparison with the other methods on different datasets, and statistical validation by modified Cramer–Rao bound.

Section snippets

Neutrosophic set

Let A be a neutrosophic set (NS). An element x from the union of discourse U will belong to A as x(T,I,F). It means that the belongingness of x to true subset T is t%, indeterminate subset I is i% and false subset F is f%. T, I and F represents the standard or non-standard subsets, which have the open interval represented by ]0,1+[. The elements of T, I and F can have any value between [01]. For practical use it is assumed T,I,F[01] with t+f=1 and i[01]. Details theory about NS can be

Results and discussion

In this section, we discuss the performance of the proposed method on different datasets. For this, at first in the next subsection, we describe different performance measures and a technique for statistical validation using modified Cramer–Rao bound.

Conclusions and future works

Management of uncertainties in an image pattern is one of the major problems in image segmentation. The current measures of uncertainties do not take into account the boundary information. In this paper, we have proposed an NS-based multi-class segmentation technique in combination with the theory of weak continuity constraints. The weak continuity constraints take care of the boundary information and help to localize the segmentation boundaries accurately. By doing this, the gray level and

CRediT authorship contribution statement

Soumyadip Dhar: Methodology, Writing - review & editing. Malay K. Kundu: Conceptualization, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (55)

  • WunnavaA. et al.

    An adaptive Harris hawks optimization technique for two dimensional grey gradient based multilevel image thresholding

    Appl. Soft Comput.

    (2020)
  • RajA. et al.

    Multi-level thresholding based on differential evolution and Tsallis Fuzzy entropy

    Image Vis. Comput.

    (2019)
  • SowjanyaK. et al.

    Investigation of butterfly optimization and gases Brownian motion optimization algorithms for optimal multilevel image thresholding

    Expert Syst. Appl.

    (2021)
  • DharS. et al.

    A novel method for image thresholding using interval type-2 fuzzy set and Bat algorithm

    Appl. Soft Comput.

    (2018)
  • LiuC. et al.

    An improved edge-based level set method combining local regional fitting information for noisy image segmentation

    Signal Process.

    (2017)
  • DharS. et al.

    Accurate segmentation of complex document image using digital shearlet transform with neutrosophic set as uncertainty handling tool

    Appl. Soft Comput.

    (2017)
  • SengurA. et al.

    Color texure segmentation based on neutrosophic set and wavelet transform

    Comput. Vis. Image Underst.

    (2011)
  • Zhang.Y.

    A survey on evaluation methods for image segmentation. Image segmentation evaluation: A survey of unsupervised methods

    Pattern Recognit.

    (1996)
  • PengR. et al.

    On performance limit of image segmentation algorithms

    Comput. Vis. Image Underst.

    (2015)
  • WangD. et al.

    An efficient iterative thresholding method for image segmentation

    J. Comput. Phys.

    (2017)
  • GuoY. et al.

    NCM: Neutrosophic c-means clustering algorithm

    Pattern Recognit.

    (2015)
  • JhaS. et al.

    Neutrosophic image segmentation with dice coefficients

    Measurement

    (2019)
  • KarnikN.N. et al.

    Centroid of a type-2 fuzzy set

    Inform. Sci.

    (2001)
  • MartínezS.S. et al.

    Quality inspection of machined metal parts using an image fusion technique

    Measurement

    (2017)
  • RazmjooyN. et al.

    Applications of Image Processing and Soft Computing Systems in Agriculture

    (2019)
  • DoughertyG.

    Digital Image Processing for Medical Applications

    (2009)
  • RaduA. et al.

    Industrial applications of image processing

    Acta Univ. Cibiniensis

    (2014)
  • Cited by (16)

    • Colorectal polyp region extraction using saliency detection network with neutrosophic enhancement

      2022, Computers in Biology and Medicine
      Citation Excerpt :

      Both NS-based decision-making [52] and similarity measure [49,53] are frequently employed for dealing with information fusion, as well as finding the solution deemed to be satisfactory. To transfer the information into the NS domain, the corresponding T (Truth), I (Indeterminacy), and F (Falsity) functions are usually defined [43–45,47,48]. Due to the ability of NS in tackling uncertainty problems, it has shown its effectiveness when confronting noisy circumstances [47].

    View all citing articles on Scopus
    View full text