Accurate multi-class image segmentation using weak continuity constraints and neutrosophic set
Introduction
The modern age is the age of automation. In this modern era, there is a lot of applications of image processing in metal industries [1], agriculture industries [2], robotics [3], and medical sciences [4]. The application varies from automatic fault detection, quality detection to MRI and CT scan processing, etc. In various stages of image processing, image segmentation especially the segmentation of a multi-class image is one of the essential task [5]. In the multi-class image segmentation process, an image is segmented into different regions based on different features. For proper segmentation, the variations among the pixels in the same segment should be decreased with the increase in the variation among different segments. Moreover, an efficient segmentation technique should also be able to determine the boundaries between the segments accurately. The localization of the segment boundaries with the detection of segments is an ill-posed problem [6]. This is due to the uncertainties involved in the process of segmentation [7]. The automatic detection of the number of segments in an image without having any pre-defined knowledge is also a difficult task. As both of these problems emerge in highly uncertain image patterns, they must be addressed properly for an efficient segmentation algorithm to be developed.
The literature on different types of segmentation techniques is quite rich. Currently, the segmentation techniques based on the edge, graph, region, and energy are quite popular. Zhang et al. [8] proposed a graph-based technique for fast image segmentation. Here, an image was segmented based on the partitioning of the graph. In the paper, Bragantini et al. [9] extended the concept of Image Foresting Transform (IFT) for the graph-based classification of different objects in an image. The graph-based dominant set concept was utilized by Mequanint et al. [10] for the segmentation of an image into different classes. Niu et al. [11] proposed a region-based method based on local similarity for the segmentation of regions in an image. The method required no pre-processing and thus preserved the image structure accurately. Edge or contour-based methods were used by the authors in [12], [13]. In the edge-based methods, the gradient was utilized to generate the proper segments. Among all the conventional methods, segmentations based on thresholding are very popular due to the simplicity of the techniques. Multi-level thresholding based on Tsallis–Havrda–Charvat entropy was proposed by Borjigin et al. [14]. Upadhyay et al. [15] proposed multilevel image thresholding using Kapur’s entropy. Several unconventional techniques were also utilized by some researchers for the segmentation of an image. Algebraic topology-based image segmentation was proposed by Assaf et al. [16]. A persistent homology technique of algebraic topology was utilized here for the segmentation purpose. Wang et al. [17] presented a geometric flow-based bandlet transform for segmentation. The particle replanting algorithm was employed here to deal with the region merging or separating in an image.
The methods mentioned above gave no attention to manage the uncertainties that arise in the segmentation process. Thus, the methods were less tolerant of the ambiguities due to complex image patterns, different types of perturbations etc. [18].
In the literature on segmentation, a lot of methods used uncertainty handling techniques for the accurate classification of regions in an image. They mainly used fuzzy set [19], [20], [21], rough set [22] or neutrosophic set [23] for uncertainty handling. In the paper [24] we proposed a type-2 fuzzy set-based multi-class image segmentation. Out of the three sets, the neutrosophic set (NS) is relatively new and it is very popular to handle various types of uncertainties. The applications of NS in the field of image segmentation were found to be quite effective [25], [26].
Due to high computational complexity in multi-class image segmentation, several researchers used different meta-heuristic search algorithms to compute the thresholds. In the paper proposed by Li et al. [27] authors used fuzzy coyote optimization algorithm for multi-class segmentation. They used Otsu and fuzzy entropy as their objective functions. They did not consider the boundary information between the regions of an image. Wunnava et al. [28] proposed an adaptive Harris hawks optimization method for multi-class image segmentation. Multi-level thresholding by minimizing Tsallis fuzzy entropy was proposed by Raj et al. [29]. Here they utilized a differential evolution algorithm to search the thresholds. The authors in the paper [30] investigated butterfly optimization and gases Brownian motion optimization for multi-class segmentation. Bat algorithm and type-2 fuzzy-based image segmentation was proposed by us in the paper [31]. The limitation of the meta-heuristic search-based algorithms is that, in most cases, they are not fully automatic. That means, the number of the thresholds should be pre-defined. Moreover, a large number of parameters should be initialized in those algorithms.
One ideal segmentation technique should have some important properties. The technique should consider the spatial information along with the gray level information for capturing the texture of a segment. The method should also consider the boundary information with the utilization of some proper uncertainty handling tools.
Some of the methods found in the literature incorporate local information for segmentation [14], [32]. But the methods have no provision for capturing the boundary information between the segments. Moreover, in most of the methods, the number of classes or segments are ad-hoc or fixed. That means they should have prior knowledge about the number of segments in an image. Additionally, the conventional methods, which do not have any uncertainties handling tools could not reduce the ambiguities in an image. These limitations restrict the methods to reach the highest level of accuracy and they also leave room for improvement of the methods for practical applications.
Because of the above limitations, we propose an automatic multi-class image segmentation method based on weak continuity constraints. The weak continuity constraints help to localize the boundaries between the segments in the image. It also takes into account local or spatial information during segmentation. In the field of computer vision, weak continuity constraints were efficiently used for image reconstruction [33], [34]. Weak continuity constraints in a set of data are the constraints that can be violated but with a penalty. The property is used to localize the discontinuities in a set of reconstructed values. One can find in [34] that weak continuity constraints are powerful tools for discontinuity detection. Conventional methods for discontinuity detection blur the original signal. No such problem arises in weak continuity constraints and they detect the discontinuities robustly and accurately without any prior information about the position of discontinuities. The boundaries between the classes in a segmented image may not be strong due to low-intensity change, i.e low gradient at the boundary area between the two classes. This may increase the uncertainties in the segmentation process. Thresholding depending only on the intensity values of the pixels may disconnect the weak boundaries. The thresholding using weak continuity constraints helps to localize the segmentation boundaries, and thus it is useful for uncertainty management.
As already mentioned recently, we have proposed a segmentation method using weak continuity constraints in the type-2 fuzzy set domain [24] with high accuracy. But, the limitation of a type-2 fuzzy set is that it requires a type-reduction, which has high computational complexity. Given the above limitations, in this paper, we propose weak continuity constraints-based segmentation in the NS domain.
To manage the uncertainties in the segmentation process accurately, we apply the weak continuity constraints in the NS domain. The NS was proposed by Florentin Smarandache [23]. An NS is represented by three subsets. They are true, indeterminate, and false subsets. For an image segmentation process by NS, the image is converted into a neutrosophic image. This is followed by the minimization of the uncertainties to generate the segments [35], [36], [37].
The novelty of the proposed method is that here we propose an energy function in the NS domain based on weak continuity constraints for image segmentation. The energy function acts as an objective function for segmentation in the NS domain. The objective function can express the gray level and spatial ambiguity in an image in the NS domain and the minimization of the function minimizes the ambiguities. The proposed method automatically determines the segments iteratively in an image without any prior knowledge. The iteration stops when a proposed base condition is satisfied. The schematic diagram of the proposed method is shown in Fig. 1.
We have already mentioned that we incorporated the weak continuity constraints in the fuzzy set domain for segmentation [21], [24]. In image segmentation using a fuzzy set, one pixel can have different membership for representing the belongingness of the pixel into different classes. But, it has no provision for representing the portion in between the classes explicitly. Logically, the portion should be represented separately by another set. In NS, the set is called an indeterminate subset. Thus, the representation of an image into an NS domain is significantly different from the representation of the image into fuzzy domain. In this paper, we incorporate the weak continuity constraints in the NS domain for automatic multi-class segmentation. We use the indeterminate subset of the NS for boundary regions and the true subset of the NS for non-boundary regions. It is expected that the two subset representation of NS will give better performance than that of the conventional fuzzy set in the segmentation process.
The rest of the paper is as follows: The brief introduction of the neutrosophic set (NS), mapping of an image into the NS domain, the theory of weak continuity constraints, the application of the weak continuity constraints in the NS domain for segmentation, and proposed algorithm are described in Section 2. The results are discussed in Section 3. The section includes different quantitative measures, comparison with the other methods on different datasets, and statistical validation by modified Cramer–Rao bound.
Section snippets
Neutrosophic set
Let be a neutrosophic set (NS). An element from the union of discourse will belong to as . It means that the belongingness of to true subset is , indeterminate subset is and false subset is . , and represents the standard or non-standard subsets, which have the open interval represented by . The elements of , and can have any value between . For practical use it is assumed with and . Details theory about NS can be
Results and discussion
In this section, we discuss the performance of the proposed method on different datasets. For this, at first in the next subsection, we describe different performance measures and a technique for statistical validation using modified Cramer–Rao bound.
Conclusions and future works
Management of uncertainties in an image pattern is one of the major problems in image segmentation. The current measures of uncertainties do not take into account the boundary information. In this paper, we have proposed an NS-based multi-class segmentation technique in combination with the theory of weak continuity constraints. The weak continuity constraints take care of the boundary information and help to localize the segmentation boundaries accurately. By doing this, the gray level and
CRediT authorship contribution statement
Soumyadip Dhar: Methodology, Writing - review & editing. Malay K. Kundu: Conceptualization, Supervision.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (55)
- et al.
Applications of image processing in robotics and instrumentation
Mech. Syst. Signal Process.
(2019) - et al.
Revisiting graph construction for fast image segmentation
Pattern Recognit.
(2018) - et al.
Robust noise region-based active contour model via local similarity factor for image segmentation
Pattern Recognit.
(2017) - et al.
Active contours driven by local pre-fitting energy for fast image segmentation
Pattern Recognit. Lett.
(2018) - et al.
Active contours driven by local image fitting energy
Pattern Recognit.
(2010) - et al.
Color image segmentation based on multi-level Tsallis–Havrda–Charvát entropy and 2D histogram using PSO algorithms
Pattern Recognit.
(2019) - et al.
Persistent homology for object segmentation in multidimensional grayscale images
Pattern Recognit. Lett.
(2018) - et al.
Image segmentation algorithm based on geometric flow Bandelets transformation particle replanting
Pattern Recognit. Lett.
(2018) - et al.
Image thresholding using restricted equivalence function and minimizing the measures of similarity
Fuzzy Sets and Systems
(2007) - et al.
A novel image segmentation algorithm based on neutrosophic similarity clustering
Appl. Soft Comput.
(2014)
An adaptive Harris hawks optimization technique for two dimensional grey gradient based multilevel image thresholding
Appl. Soft Comput.
Multi-level thresholding based on differential evolution and Tsallis Fuzzy entropy
Image Vis. Comput.
Investigation of butterfly optimization and gases Brownian motion optimization algorithms for optimal multilevel image thresholding
Expert Syst. Appl.
A novel method for image thresholding using interval type-2 fuzzy set and Bat algorithm
Appl. Soft Comput.
An improved edge-based level set method combining local regional fitting information for noisy image segmentation
Signal Process.
Accurate segmentation of complex document image using digital shearlet transform with neutrosophic set as uncertainty handling tool
Appl. Soft Comput.
Color texure segmentation based on neutrosophic set and wavelet transform
Comput. Vis. Image Underst.
A survey on evaluation methods for image segmentation. Image segmentation evaluation: A survey of unsupervised methods
Pattern Recognit.
On performance limit of image segmentation algorithms
Comput. Vis. Image Underst.
An efficient iterative thresholding method for image segmentation
J. Comput. Phys.
NCM: Neutrosophic c-means clustering algorithm
Pattern Recognit.
Neutrosophic image segmentation with dice coefficients
Measurement
Centroid of a type-2 fuzzy set
Inform. Sci.
Quality inspection of machined metal parts using an image fusion technique
Measurement
Applications of Image Processing and Soft Computing Systems in Agriculture
Digital Image Processing for Medical Applications
Industrial applications of image processing
Acta Univ. Cibiniensis
Cited by (16)
Image completion based on segmentation using neutrosophic sets
2024, Expert Systems with ApplicationsSemantic segmentation using Firefly Algorithm-based evolving ensemble deep neural networks
2023, Knowledge-Based SystemsInformation measures based on similarity under neutrosophic fuzzy environment and multi-criteria decision problems
2023, Engineering Applications of Artificial IntelligenceNeutrosophic set based local binary pattern for texture classification
2022, Expert Systems with ApplicationsColorectal polyp region extraction using saliency detection network with neutrosophic enhancement
2022, Computers in Biology and MedicineCitation Excerpt :Both NS-based decision-making [52] and similarity measure [49,53] are frequently employed for dealing with information fusion, as well as finding the solution deemed to be satisfactory. To transfer the information into the NS domain, the corresponding T (Truth), I (Indeterminacy), and F (Falsity) functions are usually defined [43–45,47,48]. Due to the ability of NS in tackling uncertainty problems, it has shown its effectiveness when confronting noisy circumstances [47].
Risk assessment method for large-section tunnel using neutrosophic numbers similarity measure based on arcsine function
2023, Journal of Intelligent and Fuzzy Systems