Abstract
With people’s higher and higher spiritual requirements, fiber art is more common in people’s lives. From ordinary fiber materials to handicrafts and daily necessities, there are shadows of fiber art. This paper aims to study the application of two-dimensional images in the visual and tactile dimensions of fiber art design. This paper proposes a three-dimensional simulation of two-dimensional images of art based on deep learning technology to achieve the purpose of convenient display and also analyzes the feature extraction in fiber art. In order to comprehensively compare the performance of the algorithms, this paper compares the performance of four categories: reconstruction speed, number of reconstructed point clouds, diversity of reconstructable categories, and reconstruction stability. The experimental results show that the number of point clouds obtained by the algorithm proposed in this paper far exceeds the other two reconstruction processes, and the average time is 34.4 min. This also shows that such algorithmic processes are more robust to clothing diversity.
1. Introduction
In the increasingly diversified and open modern civilization, the cloth is taken as the research object, and the visual texture and tactile perception characteristics of the cloth are researched with regard to the basic attributes of the fiber material such as color, texture, texture, and pattern. Researchers discuss the expression techniques and expressions of fabrics in fiber art design and creation under modern civilization and the trend of the times, breaking the traditional design thinking mode of fabrics. It is of great significance to study the application of two-dimensional images in fiber art.
For the research on fiber art, this paper has the following two innovations: (1) The first is innovation in perspective. The content of this article is the visual and tactile application of fiber art. Such a perspective pays more attention to the artistic aspect of fiber art, rather than paying too much attention to the material of fiber art itself. (2) The second is innovation in method. The paper proposes a feature extraction algorithm and a three-dimensional reconstruction algorithm for the two-dimensional images of fiber art. Such an algorithm can transform a 2-dimensional image into a 3-dimensional image, thereby more intuitively showing the visual and tactile dimensions of fiber art.
2. Related Work
For today’s fiber art, many scholars have conducted in-depth research, especially into the innovation of fiber art. Firth analyzed digital fiber art in combination with digitization technology and analyzed the problem of growing digital photos with users not wanting to delete them [1]. Zhao and Wang introduced the application of FORS in identifying pigments and dyes, analyzing adhesives, and studying preservation outcomes and preservation environments. They discuss detection limits, reproducibility, and signal characterization methods and summarize the future development of FORS in the study of artefacts [2]. Jin and Xia believe that image art analysis and computer-generated art are frontier application hotspots in the field of visualization. They found that with the gradual deepening of the application of digital technology represented by artificial intelligence in the field of art, the integration of art and digital technology will provide a broader space and prospects for the development of art [3]. Liu analyzed the principle and characteristics of computer image recognition technology, analyzed the existing problems and solutions, and discussed the application of computer intelligent image recognition technology in various fields [4]. Li et al. found that previously presented image matting algorithms might fail to produce favorable results since most of them concentrate on the similarity between the neighboring pixels while neglecting the corresponding spatial relationship. To address this issue, an end‐to‐end image matting framework through leveraging deep learning mechanism and graph theory is proposed [5]. Wang analyzed the development of traditional hand-woven craftsmanship into modern three-dimensional works, which had a positive impact on people’s lives, for example, the application of fiber art in specific products [6]. Zhang introduced new testing methods and typical instruments for fibers and textiles developed at home and abroad, including fiber length, fineness, crimp, heat shrinkage, maturity, irregularity, and fiber structure analysis and testing; yarn fineness, strength, twist, hairiness, unevenness, and defect testing; textile color, gloss, flexural stiffness, drape, wrinkle resistance, heat and moisture transfer, breathability, flame retardancy, color fastness, blending ratio, and UV and radiation protection; and antistatic, antibacterial, anti-mite, anti-moth-eaten, ecological, and safety performance tests. In addition, the study discussed the related technologies of fiber structure analysis, including electron microscope analysis, infrared spectrum analysis technology, X-ray diffraction analysis, and thermal analysis technology. The research specifically mentions that computer image processing technology is widely used in the textile industry and the specific applications mainly include textile fiber detection, textile yarn detection, and textile fabric detection [7]. However, it can also be found from relevant research that many scholars lack corresponding technical support for the study of fiber art, and they are more concerned with analyzing the aesthetic significance of art.
3. Vision and Touch of Fiber Art Design
3.1. Fiber Art in China
The term “fiber art” first appeared in the United States in the 1970s. In China, people are used to calling it “dyeing and weaving art,” “weaving art,” and “fabric art.” In foreign countries, words such as “tapestry” (English) and “textile” (French) are mostly used. With the development of history, the art form with weaving as the main technique has begun a new transformation and extension. Artists began to explore new fiber materials and fabrication techniques, creating new works.
To understand the development history of Chinese fiber art, we have to mention another long-established fiber art form—Chinese carpet. In China, wool fabrics did not appear as early as silk fabrics. The ethnic minorities living in the ancient northwest region were good at using animal wool to spin into woolen yarn and to weave woolen cloth or blankets. As early as the Zhou Dynasty in China, there were records of the manufacture and use of felt. The wool fabrics unearthed at the Wubao site in Hami, Xinjiang, dating back 2,300 years, are already exquisite. There are two kinds of plain weave, twill weave and jacquard fine wool fabrics woven with colored threads. During the Sui and Tang Dynasties, the weaving of carpets had a high level of skill, and tapestries also developed and prospered locally. The kesi tapestries of the Tang Dynasty in China were passed to Japan. The Xinjiang wool fabrics unearthed from the ancient tomb of Zhahong Luke in Qiemo County include printed wool fabrics, painted wool fabrics, wool fragments, silk fabric fragments, and wool fabrics. In the Dunhuang Grottoes, there are a large number of carpet pictures in the murals of the Tang and Song Dynasties, which are exquisite in craftsmanship and highly ornamental and decorative. It can be seen from this that China’s fiber art has a long history of development and is very prosperous. In addition to Chinese silk and carpets, Chinese kesi, embroidery, and other techniques also have a long history. They have developed in all dynasties and generations and are also an important part of the history of Chinese fiber art [8, 9].
With the progress of the times, modern fiber art has achieved rapid development in China. In the early 1980s, the textile works of this period were mainly tapestries. They were produced by the traditional flocking process. The design of the works was mainly based on the reproduction of photos and master paintings, and the aim was a realistic and realistic effect. Moreover, most of the fiber art designers in this period were those who used to work in the carpet industry, so they were still mainly flat in form. Until the spring of 1981, American fiber art students had come to the Central Academy of Industrial Fine Arts to study abroad. Through study and exchange, the research and development of Chinese modern fiber art began to internationalize [10, 11].
Under the influence of the artistic trend of the 20th century, fiber art has already belonged to a new expression of modern art form. It has been difficult for traditional titles to accurately cover this art. Whether traditional or modern, the decisive factor in this art is the fiber material. Works made from fibers belong to the category of this art. It is therefore reasonable to call this art “fiber art.”
3.2. Visual Affinity
Vision is the most direct way of communicating and perceiving information. People use the eyes as a medium to observe and examine the fabric. This forms the intuitive impression and psychological feeling of the fabric, which is the visual texture of the fabric. The influencing factors of the visual texture effect mainly include the appearance pattern design of the fabric, the structure of the fabric, and the internal texture of the fiber element. Appearance patterns of fabrics refer to the appearance patterns attached to the surface of fabrics by means of printing, hand-painting, spray painting, etc. during fabric processing. Different attachment drawing techniques and patterns of different colors, shapes, shades, thicknesses, and densities bring different psychological feelings to the audience. For example, “in the 1990s, the patterns attached to the fabrics were mostly printed. Common patterns are repaints of natural features, such as animals, flowers, landscapes, etc. This realistic and natural appearance pattern is vivid under the drawing technique of printing. Its full and rich imitation reproduces the vitality and agility of the natural ecological environment.”
The weave structure of the fabric refers to the organization law formed by the interweaving of the warp and weft threads in the woven fabric according to certain rules. As components of the fabric system, the weaving technique, yarn shape, yarn texture, luster, etc. of the fabric will affect the visual artistic effect of the fabric. For example, in plain weave, the warp and weft are alternately woven up and down (as shown in Figure 1). It forms a dense arrangement of warp and weft yarns, and the fabric under the plain weave is a point-like composition surface from a visual perspective. The surface of the fabric is smooth and regular, and the style is stiff. If dark-colored yarns are used, the fabric gloss and visual perception fineness under the plain weave method are not high. Under the light and dark light, the unevenness of the fabric surface will present a three-dimensional sense and spatial hierarchy.
3.3. Tactile Texture
For cloth, texture is the touch of the cloth. The human body surface skin is used as a medium to stroke the cloth, and the mutual contact and friction between the human body surface texture and the cloth surface texture are used. It senses the fibrous elements and tissue structure inside the cloth by tactile nerves. This arouses the intuitive psychological feeling of the toucher on the fabric and forms an intuitive impression of the texture effect of the fabric. It is further deepening the artistic perception of cloth and further rendering the artistic appeal.
The texture of the fabric is an important natural attribute and a unique aesthetic form of the fabric, which constitutes the unique artistic charm of the fabric. Different fabrics have different textures in terms of touch, and the texture effects and psychological feelings brought to the touch are also different. Generally speaking, fabrics with a smooth surface texture and no obvious abrupt texture feel smooth to the touch. For example, the striped texture on the surface of the cloth will give the touch a sense of fluidity and regularity. The cotton fabric is soft and delicate to the touch, and the silk is light and smooth to the touch, which will bring softness and comfort to the touch. They make the toucher feel the natural pleasure of tranquility, peace, and serenity and are usually used for summer cool quilts or intimate clothing designs. Although the fabric with rough surface texture and heavy wool texture is lacking in touch, it can bring warmth and fulfillment to the touch. For example, bumps on the surface will give the toucher a sense of three-dimensionality and irregularity. Under light mapping, there will be obvious spatial hierarchy and directionality. The hemp fabric is light in quality but has a rough surface that can tingle when touched. It has a poor affinity with the skin and is mainly used to design clothing for hot seasons. The cloth mainly made of animal fur has a rough surface texture and touch, but it can give people a warm feeling and spiritual comfort like hugging and soothing pets.
Therefore, the texture of the fabric is not only the internal combination of fiber elements, but also the emotional characteristics of “humanization.” When choosing fabrics, modern people not only consider the visual affinity, but also perceive the texture that the fabric gives the touch. They seek suitable fabrics that meet the modern people’s pursuit of texture and conform to modern people’s psychological emotions.
4. Feature Extraction of Fiber Art Based on Two-Dimensional Images
4.1. Feature Detection and Extraction
The first step in the motion structure recovery process is feature extraction, and the extracted features need to be used in the matching in subsequent steps and inference of depth and camera position. The desired feature extraction algorithm in this paper can be as invariant as possible to scaling and rotation, as well as lighting and camera angles. It maintains the stability and consistency of feature points under different viewing angles. The scale-invariant feature transform matching algorithm [12] is the most widely used and spread feature matching algorithm. It is used in most kinematic structure restoration components.
The SIFT algorithm is used to find key points in different scale spaces, and the scale space of the image can be obtained by Gaussian blurring of the image. This paper defines this process as function , which can be obtained by convolving the original image with Gaussian filter , as shown in the following formula:
The two-dimensional filtering expression is defined as follows:
The effective detection of stable feature points is to perform extreme point detection on the difference of Gaussian constructed by the adjacent scale-space difference after convolving with the image Gaussian [13]. The spatial factors of different scales differ by k times, and the differential pyramid formulas are expressed as follows:
The composition of the Gaussian pyramid is shown in Figure 2. Pyramid has multiple groups (octave), and each group has multiple layers. The scales of multiple layers in a group are different; that is, parameter is different. The scales between the two layers differ by a scaling factor k [14]. If each group contains S layers, there is the relationship of the following formula:
Whenever the value of parameter is doubled, the image of scale 2 is downsampled by a factor of 2 as the bottommost image of the next set.
In order to find the extreme points in the DoG scale space, each point in the DoG is compared with 8 points around it and 9 points in the upper and lower layers. Only the point that is the maximum or minimum value of the 27 points [15] will be recognized as a feature point, as shown in Figure 3.
In the figure, X represents the current point, and the blue clouds represent the surrounding points.
The difference pyramid constructed according to the previous steps can search for local extreme points from the discrete spaces. The so-called discrete spaces refer to the results obtained by sampling from a three-dimensional continuous space. The extreme point searched in the discrete spaces may not be the final extreme point. It also needs to filter the searched feature points in the subsequent steps and eliminate the points that do not meet the constraints [16]. After the feature points are selected, appropriate filtering processing can be performed to fit the nearby data to find the position, scale, principal curvature, and so on. This step can filter out feature points with low contrast and unstable response to edges. The realization of this process is to perform Taylor series expansion on the original Gaussian difference, as shown in the following formula:
The in (6) represents the offset of the sample point. Since x is an extreme point on , deriving the above formula, let its derivative be 0, and get the following formula:
It then brings the obtained into the Taylor expansion of to obtain the following formula:
It sets the contrast threshold T, usually 0.5. If , the feature point is retained; otherwise, it is eliminated.
The SIFT algorithm reacts strongly to image edge regions. Therefore, removing these feature points along the edge of the image is also necessary to improve the stability of the algorithm. The principal curvature of the edge gradient direction is much larger than the principal curvature along the edge direction, which can be considered as an inferior feature point, which is also a point that must be deleted in this paper [17]. The of the candidate feature points can be simply calculated with the second-order Hessian matrix H, because the eigenvalues of the Hessian matrix are proportional to the principal curvature . H is defined as follows:
Among them, , , and are obtained from the corresponding position difference of the candidate point neighborhood, respectively. This paper does not care about the specific values of the eigenvalues but only focuses on the ratio of the eigenvalues.
In , and represent eigenvalues in different directions. The sum of eigenvalues can be represented by the trace of the matrix [18]. The product of eigenvalues can be represented by the value of the determinant of the matrix, so there is the relationship of the following formula:
Among them, is the trace of the matrix, and is the determinant of the matrix. When the eigenvalues in different directions are equal, the numerical result of this formula is the smallest and has a positive correlation with the ratio [19]. Therefore, only by judging the following formula, it can be concluded whether the principal curvature is smaller than the threshold value .
If (11) holds, this means that the principal curvature is greater than the threshold, that is, the inferior feature points mentioned above. After this series of filtering, this paper can obtain the same feature points from different perspectives. Now, this paper can restore the three-dimensional information of the scene through feature point matching under multiple views. It should be pointed out here that the SIFT algorithm is not the only way to find feature points.
The SIFT algorithm also assigns the scaling and rotation of the feature points. It uses the gradient magnitude and direction of the image scale space where the feature is located to determine the relevant parameters. These values are used in subsequent feature descriptor calculations. This is beneficial for this paper to calculate the correspondence of features in different images [20].
In order to ensure the rotation invariance of the rotation vector, the original coordinate axis needs to be rotated around the feature point. After defining the rotation angle size as θ, the new coordinates of the image pixel points can be expressed in the form of matrix multiplication, as shown in the following formula:
When the coordinate axis is rotated to be consistent with the main direction of the feature point, it takes the feature point as the center to take a window with a length of 8 pixels, as shown on the left of Figure 4. The center of the window represents the location of the key point, and each square represents a pixel of its neighborhood in the specified scale space. The arrows in the squares represent the direction of the pixel gradient, and the length indicates the magnitude of the pixel gradient. It then performs a weighted operation on the pixel gradient through Gaussian filtering, and finally a square with a length of 4 pixels can be counted into a gradient histogram of 8 dimensions. After calculating the cumulative value of each dimension, seed points in 8 directions can be formed, as shown on the right of Figure 4. The results obtained in this way combine the information of the pixels around the feature points. It can effectively enhance the anti-noise ability of the algorithm and is more robust to matching pairs with positioning errors.
The previous matching stage in this paper finds the common points in the corresponding image pairs. However, this paper cannot guarantee that the matching pairs found are 3D corresponding points in the real scene, and there may be outliers. It is necessary to find a geometric transformation that can map enough common points in the two images. If such a geometric transformation exists, the two images are considered geometrically validated. This means that the geometry of these points in the scene also needs to have a one-to-one correspondence between these two graphs. Depending on the spatial information obtained in the images, different methods can be used to describe the geometric relationship between them. A homography transformation can be used to describe the transformation between two images of a camera that acquires a flat scene. Its epipolar geometry is used to represent the geometric relationships that exist between the same spatial point, different camera positions, and projections on the camera’s image plane. Figure 5 shows the spatial relationship between the spatial point and the projection plane. Point represents a point in three-dimensional space, and represent the position of the optical center of the camera at different shooting angles, and and are called epipolar lines. The essential matrix E is a 3 × 3 sized matrix containing the epipolar geometry. If the camera calibration internal parameters are known, the epipolar geometry allows the motion of the camera to be described by the essential matrix E, and the relationship between the two epipolar lines of the binocular image is obtained. The essential matrix is solved as follows.
R and T represent the rotation matrix and translation matrix experienced by the camera from to , respectively. It defines an obliquely symmetric matrix, as in the following formula:
In Figure 5, vectors and represent the positions in their respective camera coordinate systems, and the relationship is as shown in the following formula:
Multiplying the left and right sides of (16) by yields the following formula:
It usually defines formula (18) as the essential matrix, resulting in the following formula:
Since the feature correspondences in the feature matching stage always contain outliers, it is necessary to use robust estimation methods to screen feature points in geometric validation, such as Random Sample Consensus. The output of this stage is referred to herein as the scene graph, and the nodes of the graph represent images. The edges of the graph connect image pairs that are geometrically validated.
4.2. Feature Extraction Effect
In order to obtain a 3D model with better reconstruction effect, this paper compares three mainstream reconstruction algorithm benchmarks. The three methods of motion structure recovery and multi-view reconstruction are VisualSFM + PMVS, COLMAP + OpenMVS, and OpenMVG + OpenMVS, as shown in Table 1.
As shown in Table 2, in order to comprehensively compare the performance of the algorithms, this paper compares the four performances of reconstruction speed, reconstructed point cloud number, reconstructable category diversity, and reconstruction stability. In this paper, four different types of clothing were selected in the experiment, and 60 images were taken evenly around the model. The paper uses the image input of the pattern to combine different reconstruction processes to compare the reconstruction effect.
From the statistical results of the number of dense point clouds in Figure 6, among the results of the four types of clothing, the reconstruction process of OpenMVG + OpenMVS has the largest number of point clouds. Among them, regardless of the reconstruction process, the number of point clouds reconstructed by pure black long sleeves is much lower than those of the other three types of clothing. Combined with the analysis of the reconstruction effect above, for solid-color clothing or clothing lacking texture information, the use of 2D images to reconstruct 3D models is more prone to the problem of missing model point clouds. For casual suits, although the color is single, the surface of the clothing is rougher than the black long sleeves. There are many rough fibers attached to its surface, which is more conducive to finding feature points. This means that clothing with richer texture or color information is more suitable for reconstructing the 3D model of clothing in this way. A careful observation of the data in Figure 6 shows that for casual suits, there are differences in the number of point clouds obtained by the three reconstruction processes. However, the number of point clouds can basically reach the order of millions, and the gap is not obvious. However, when reconstructing clothing types with less texture information, such as sports jackets or striped T-shirts, the number of point clouds obtained by OpenMVG + OpenMVS far exceeds those of the other two reconstruction processes. This also shows that such algorithmic processes are more robust to clothing diversity.
The hardware processor used in this experiment is configured as Intel Core i5-6600K. Figure 7 shows that the increasing trend of reconstruction time is consistent with the number of point clouds. In order to explore the reconstruction efficiency of various reconstruction processes, this paper uses statistics to reconstruct the time spent on each million point clouds of various clothing models and then compares the average values. The calculation shows that the reconstruction process OpenMVG + OpenMVS takes an average of 34.4 minutes to reconstruct a million point clouds, which is the shortest among the three. For the exploration of the reconstruction stability performance, in this experiment, all kinds of experiments were repeated three times, and the reconstruction time was nearly the same as the reconstructed point cloud effect. This shows that there is no significant difference in the reconstruction stability of the three reconstruction processes, they are all relatively stable, and the data results are no longer given separately.
5. Development and Innovation of Fiber Art Design
5.1. Pseudo 3D Display Technology Based on 2D Images
Showing fiber art with its two-dimensional images lacks authenticity. The product can only be observed from a fixed angle, and the interaction between the user and the artistic design cannot be realized. The use of 2D images to build 3D models can effectively make up for these shortcomings. But this method has many steps from shooting to 3D reconstruction. This cannot meet the requirements of rapid deployment and is more suitable for offline modeling application scenarios.
The product display based on image photo stream has been studied by scholars as early as ten years ago. This method is based on the object to be displayed and rotates the object by 360 degrees. It takes multiple images according to different precisions, and through image analysis and processing and adding interactive functions, the pseudo three-dimensional display effect of commodities is realized. In fact, if the avatar in this photo stream can be replaced with the user’s avatar, the fitting effect will be closer to the photo stream effect of the user’s real fitting. In view of this, this section first introduces the realization of clothing display based on photo stream. It then describes how to achieve a personalized display of the dressed human body through “face swapping” technology.
In this experiment, a mobile device is used to collect the target image, so this paper first imports the captured image into the PC and then uses GraphCut to remove the background of the image. It only retains models wearing clothing and adjusts details such as color and light to avoid poor display effects due to changes in light during shooting. This display method can be easily embedded into the HTML file to achieve interaction with the user. The specific implementation process is shown in Figure 8.
5.2. Training and Transformation Analysis
Usually, in such an encoding and decoding process, the neural network is trained to reduce the error between the predicted value and the true value. This results in a weight matrix that satisfies the transformation requirements, the training process of which is shown in Figure 9.
Different from the evaluation indicators of traditional neural networks, the “transformation” task cannot simply use the magnitude of the loss value to determine the quality of the model. Therefore, it is common to define encoder A and encoder B losses separately. The values of both represent how well the network reconstructs the input image. The details are as follows, where functions and represent the encoder and decoder, respectively; x is the image input to the encoder; and h is the intermediate vector obtained by the encoder. represents the output obtained by decoding with h as input, and the autoencoder loss is defined as the difference of between the input x of the model and the output. The calculation formulas are as follows:
And for the transformation task, the model needs to restore image B from the vector encoded by image A. Since there is no so-called ground-truth label, this experiment cannot define a numerical loss for how good the conversion result is. It can only judge the quality of model training by continuously observing the conversion effect. That is, it is judged whether the model has learned an effective weight matrix by observing the loss value. Figure 10 shows the loss curve during the training process of this experiment. It can be seen that when the number of iterations exceeds 20,000, the value of the loss function has not decreased significantly. At this point, the gradient has basically stopped decreasing, which means that the training can be terminated.
Based on a large number of experiments, the mini-batch used for training is finally set to 64. The initial learning rate is set to 0.001, and the learning rate is set to decay to 10% of the learning rate itself every 10 epochs.
6. Conclusions
The design of art is to innovate, and the same is true of fiber art. Today, with the development of network technology, the innovation of fiber art design should also be combined with the technology of the times. Starting from the current research background of fiber art, this paper puts forward the research significance of this paper from the visual and tactile dimensions of fiber art. Afterwards, a comprehensive analysis of the visual and tactile dimensions of fiber art is carried out. Furthermore, a related feature selection algorithm is proposed based on the design of fiber art, and the effectiveness of the algorithm is proved. Finally, for the design exhibition of fiber art, a three-dimensional simulation display of two-dimensional images is proposed. The experimental results show that the algorithm in this paper is effective. However, there is also room for improvement in the research; that is, the fiber art can be classified into different periods of art, and the results can be more universal.
Data Availability
Data can be obtained by contacting the corresponding author under reasonable requirements.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported by 2021 Liaoning Provincial Department of Education Basic Scientific Research Project (Youth Project), “Recyclable + Cocreation Model,” Low-Carbon Economy Research on the Realization of School-Enterprise Joint Creation Fiber Art Design (No. LJKQR2021033).