Physically Motivated Deep Learning to Superresolve and Cross Calibrate Solar Magnetograms

To train our SR architecture, we leverage overlapping observation periods between MDI and HMI (2010–2011) and between GONG and HMI (2010–2019), which provides us with ∼9000 (∼19,000) MDI-HMI (GONG-HMI) magnetogram pairs. We split the data into training/validation/test sets by randomly allocating 10 months to the training set, 1 month to the validation set, and 1 month to the test set for each overlapping year (see Table 1).

In the case of GONG-HMI, we only use even years (2010, 2012, 2014, 2016, and 2018) for this work to keep the data volume manageable. The test set comprises magnetograms taken at a 96 minute cadence for MDI, and a 10 minute cadence for GONG. Across all experiments, we choose 2010 June and 2011 March as our test months.

Each full-disk magnetogram is split into small patches (discussed below) to ensure that model training is done on spatial scales and features that evolve over hours. This helps break any correlations that may happen from one solar rotation to the next. Because of this, we do not provide a time buffer between the training, validation, and test months to ensure data independence. For other tasks involving larger areas, or for performing full-disk conversions, the slow evolution of the global magnetic field could unintentionally leak into the test set if a sufficient time buffer is not provided.

2.1.1. Data Preprocessing

The observations used in this work cover one solar cycle, meaning that our data set contains systematic polarity orientations for both positive and negative fields and their relative distributions across the solar disk. These polarity orientations change from cycle to cycle and from hemisphere to hemisphere, increasing the risk of our network learning unintended structures and patterns. To avoid this, we augment data through random polarity flips, as well as north–south, and east–west reflections.

The deep-learning model used in this work was adapted from the HighRes-net model (Deudon et al. 2020) ¹² (see Figure 2), as this model has shown great performance for SR of Earth observation data. Our model input consists of two channels, a magnetogram patch and a location channel. The location channel captures the normalized radial distance of each pixel to the disk center and provides the network with the necessary information for estimating projection and foreshortening effects at the solar limb. The data is encoded into 64 channels through a series of convolution operations shown in the encode block of Figure 2. In the decoding operation, the size of each encoded patch is increased through bilinear upsampling to a target patch size of 128 × 128 pixels, before passing it through a final convolutional layer.

Zoom In Zoom Out Reset image size

We use reflection padding to retain constant dimensions as the different convolutional layers are applied. We also experimented with zero padding and constant padding, but found reflection padding to result in better performance.

Training a neural network involves the minimization of an objective function that quantifies how well the transformation of the input matches the target. This objective function is typically referred to as the loss function (${ \mathcal L }$). As SR is an ill-posed problem (i.e., a one-to-many operation), multiple SR outputs can explain the same LR input. For scientifically useful applications of SR, the model output should capture the physical properties of the target, and cannot just be perceptually convincing. To better respect the physical properties of the target HR magnetograms, we construct a loss function that combines four terms, each of which aims to capture a different aspect of what makes a magnetogram physically plausible. Importantly, compared to recent advances in developing physics informed neural networks (e.g., Raissi et al. 2019), our model does not explicitly draw upon physical laws, but instead relies on terms that are common for SR and image processing and that also capture physical quantities of interest for magnetograms.

The loss function used in this work is

$$\begin{eqnarray}\begin{array}{rcl}{ \mathcal L } & = & {{ \mathcal L }}_{l2}+{w}_{\mathrm{grad}}{{ \mathcal L }}_{\mathrm{grad}}\\ & & +\,{w}_{\mathrm{hist}}{{ \mathcal L }}_{\mathrm{hist}}+{w}_{\mathrm{ssim}}{{ \mathcal L }}_{\mathrm{ssim}},\end{array}\end{eqnarray} \tag{ 1 }$$

with the following terms:

• 1.  
  ${{ \mathcal L }}_{l2}$ penalizes the mean squared error (MSE) between the superresolved output and the target, and captures pixel-based differences in the signed flux.
• 2.  
  ${{ \mathcal L }}_{\mathrm{grad}}$ penalizes the mean squared difference between pixel gradients of the superresolved output and the target. The gradients are approximated using a Sobel operator (Pingle 1969). ${{ \mathcal L }}_{\mathrm{grad}}$ aims to capture the gradients present at the boundaries between positive and negative polarities.
• 3.  
  ${{ \mathcal L }}_{\mathrm{hist}}$ penalizes an approximation of the total variation distance between magnetic field distributions of the output and target magnetograms. For that, we calculate a differentiable pixel histogram using the method described in Wang et al. (2018). Formally, if the magnetic field of a magnetogram is divided into K bins, given a value of the magnetic field B_i,j at pixel (i, j), we assign a differentiable weight ψ_k (B_i,j ) to bin k = 1,...,K, where

  $$\begin{eqnarray*}&&{\psi }_{k}({B}_{i,j})=\max \{0,1-\displaystyle \frac{1}{{\gamma }_{k}}| {B}_{i,j}-{\mu }_{k}| \},\end{eqnarray*}$$

  where μ_k and γ_k are learnable parameters. Then, for each bin k, we compute a batch differentiable count of magnetic field values that falls within [μ_k − γ_k , μ_k + γ_k ] as

  $$\begin{eqnarray*}&&\displaystyle \sum _{n=1}^{N}\displaystyle \sum _{(i,j)\in {p}_{n}}{\psi }_{k}({B}_{i,j}),\end{eqnarray*}$$

  where N is the number of 128 × 128 patches p_n in the batch.
• 4.  
  ${{ \mathcal L }}_{\mathrm{ssim}}$ measures the structural similarity (SSIM; Wang et al. 2004) between regions surrounding each pixel, including similarities in contrast, unsigned flux, and variance. Formally, given a superresolved 128 × 128 patch ${\hat{B}}_{n}=\{{\hat{B}}_{i,j,n}\}$, its target B_n = {B_i,j,n } and weights α_i,j , the structural similarity is defined as

  $$\begin{eqnarray*}&&\mathrm{SSIM}({B}_{n},{\hat{B}}_{n})=\displaystyle \frac{(2{\mu }_{B}{\mu }_{\hat{B}}+{C}_{1})(2{\sigma }_{B\hat{B}}+{C}_{2})}{({\mu }_{B}^{2}+{\mu }_{\hat{B}}^{2}+{C}_{1})({\sigma }_{B}^{2}+{\sigma }_{\hat{B}}^{2}+{C}_{2})},\end{eqnarray*}$$

  where

  $$\begin{eqnarray*}&&{\mu }_{B}=\displaystyle \sum _{(i,j)}{w}_{i,j}{B}_{i,j},\end{eqnarray*}$$

  $$\begin{eqnarray*}&&{\sigma }_{B}^{2}=\displaystyle \sum _{(i,j)}{w}_{i,j}{\left({B}_{i,j}-{\mu }_{B}\right)}^{2},\end{eqnarray*}$$

  and

  $$\begin{eqnarray*}&&{\sigma }_{B\hat{B}}=\displaystyle \sum _{(i,j)}{w}_{i,j}({B}_{i,j}-{\mu }_{B})({\hat{B}}_{i,j}-{\mu }_{\hat{B}}).\end{eqnarray*}$$

  ${\mu }_{\hat{B}}$ and ${\sigma }_{\hat{B}}$ are defined similarly as μ_B and σ_B . In practice, the weights are defined by an 11 × 11 circular Gaussian weighting scheme. And the constants C₁ and C₂ are set to 0.0001 and 0.009, respectively.

We initialized the values of the loss coefficients w to scale each term's contribution to the same order of magnitude as the ${{ \mathcal L }}_{l2}$ term. We then refined the scaling factors by conducting a partial grid search to determine which ${w}_{\mathrm{grad}},{w}_{\mathrm{hist}}$, and w_ssim minimize ${{ \mathcal L }}_{2}+{w}_{\mathrm{grad}}{{ \mathcal L }}_{\mathrm{grad}}$, ${{ \mathcal L }}_{2}+{w}_{\mathrm{hist}}{{ \mathcal L }}_{\mathrm{hist}}$ and ${{ \mathcal L }}_{2}+{w}_{\mathrm{ssim}}{{ \mathcal L }}_{\mathrm{ssim}}$, respectively (see Table 2). We subsequently used the values of the weights in Table 2 to minimize the loss function included as Equation (1). These coefficients vary by instrument because of differences in the properties of GONG and MDI, e.g., resolution, noise, and saturation levels.

Table 3 shows the values of the hyperparameters used to train the models. All magnetic fields are normalized by 3500 to avoid overflow issues. We use an Adam optimizer (Kingma & Ba 2014), with a constant learning rate (10⁻⁴) without an annealing schedule.

As mentioned previously, compared to natural images, which often consist of three color channels with integer pixel values, LOS estimates of the solar magnetic field can be positive or negative and span multiple orders of magnitude. Additionally, given that the Sun is a three-dimensional object projected onto a two-dimensional image, LOS measurements show projection effects that are location dependent. More specifically, the solar limb shows significantly larger projection effects than areas close to the center of the Sun. This highlights the need to consider pixel locations when evaluating the performance of SR approaches (as discussed in more detail later).

To measure the performance of any SR or cross-calibration operation of solar magnetograms, it is essential to approach them as scientific measurements rather than standard images. We propose to use the following quantities to compare the performances of SR/cross-calibration approaches. Note that these quantities are post-mortem measurements that, we believe, should be reported for any SR/cross-calibration methods of solar magnetograms and astronomical data in general.

We denote ${\hat{B}}_{i,j,n}$ as the superresolved magnetic field at pixel (i, j) and patch n; and B_i,j,n as the ground-truth target magnetic field value for the corresponding patch and at the same location. Each patch n, unless specified otherwise, refers to an area of 128 × 128 pixels, corresponding to 1/1024 of a full-disk HMI magnetogram. We find that patches of 128 × 128 pixels are large enough to encompass a whole active region, while being small enough to allow fast matrix computations and obtain an approximate flux balance (Mackay et al. 2011).

• 1.  
  Correlations: We follow Liu et al. (2012) and measure how superresolved magnetograms are cross-calibrated to their HR counterpart by measuring the Pearson correlation coefficient between $\hat{B}=\{{\hat{B}}_{i,j,n}\}$ and B = {B_i,j,n } across all pixels (i, j) and patches n:

  $$\begin{eqnarray}&&\rho =\displaystyle \frac{\displaystyle \sum _{i,j,n}({B}_{i,j,n}-\overline{B})({\hat{B}}_{i,j,n}-\overline{\hat{B}})}{\sqrt{\displaystyle \sum _{i,j,n}{\left({B}_{i,j,n}-\overline{B}\right)}^{2}\displaystyle \sum _{i,j,n}{\left({\hat{B}}_{i,j,n}-\overline{\hat{B}}\right)}^{2}}},\end{eqnarray} \tag{ 2 }$$

  where $\overline{B}$ and $\overline{\hat{B}}$ are the average ground-truth and superresolved magnetic field across all pixels and patches (ρ takes value between 0 and 1). The larger ρ the better is the cross-calibration of the superresolved magnetograms to their HR counterpart.
• 2.  
  Signed fluxes: Magnetic fields are divergence-free, i.e., the integration of the radial magnetic field over the solar surface sums to zero. To evaluate how an SR technique conserves the signed flux, we calculate the signed flux of a pixel by converting the LOS field into the radial field and correcting for area foreshortening:

  $$\begin{eqnarray}&&{E}_{i,j,n}^{\mathrm{flux}}={\hat{\phi }}_{i,j,n}-{\phi }_{i,j,n}.\end{eqnarray} \tag{ 3 }$$
• 3.  
  Extreme values: Regions with extreme magnetic field values occupy areas that are smaller than the area covered by a pixel of a magnetogram. This is particularly true for lower-resolution instruments (e.g., MDI, GONG). We expect that the filling factor (i.e., the ratio of the area occupied by the magnetic field to the total area) is larger at HR than at LR. Therefore, particularly for the study of sunspots and active regions, it is of interest to assess the ability of an SR technique to generate extreme values that occupy an area smaller than an LR pixel. Moreover, extreme values of the magnetic field have low frequency. Therefore, they may be described less confidently by an SR technique that learns its predictions from data with a limited number of occurrences of extreme values. To measure the ability of an SR technique to reproduce the tail of the magnetic field distribution, we compute the absolute difference between the minimum/maximum magnetic field over each 128 × 128 patch n:

  $$\begin{eqnarray}&&{E}_{n}^{\max }=\left|\max {B}_{n}-\max {\hat{B}}_{n}\right|\end{eqnarray} \tag{ 4 }$$

  and

  $$\begin{eqnarray}&&{E}_{n}^{\min }=\left|\min {B}_{n}-\min {\hat{B}}_{n}\right|.\end{eqnarray} \tag{ 5 }$$
• 4.  
  Gradients: We expect large magnetic field values to occupy a smaller number of pixels in HR magnetograms than in lower-resolution magnetograms. Pixel-level gradients of magnetic field values can quantify variations of the magnetic field within an LR pixel. This also helps to evaluate how an SR technique captures polarity inversion and defines boundaries between positive and negative regions. We compute ${E}_{i,j,n}^{g}$ as the (i, j) pixel of the image gradient of the difference ${\hat{B}}_{n}-{B}_{n}$ between the predicted and true magnetogram of patch n:

  $$\begin{eqnarray}\begin{array}{rcl}{E}_{i,j,n}^{g} & = & g{({\hat{B}}_{n}-{B}_{n})}_{i,j},\quad \mathrm{where}\\ g{(I)}_{i,j} & = & \sqrt{{g}_{x}(I{)}_{i,j}^{2}+{g}_{y}(I{)}_{i,j}^{2}},\quad \mathrm{with}\\ {g}_{x}(I) & = & {G}_{x}\ast I,\quad \mathrm{and}\\ {g}_{y}(I) & = & {G}_{y}\ast I.\end{array}\end{eqnarray} \tag{ 6 }$$

  Here, g is the (i, j) pixel of the output image of the Sobel operator g on image I. G_x and G_y are 3 × 3 kernels ¹³ that convolve an image to produce the smoothed finite difference on the x and y image dimensions, respectively.

To measure the performance of SR, we compute the signed flux and extreme values at small spatial scales using patches of size 4 × 4, 8 × 8, 16 × 16, and 32 × 32 pixels. In addition, we also calculate the Pearson correlation as a function of magnetic field strength and location on the surface of the Sun. This allows us to understand how the performance of any SR technique applied to the solar magnetic field depends on the spatial scale and strength of the magnetic field.

To benchmark our deep-learning approach, we compare it against a bicubic upsampling baseline. Bicubic upsampling interpolates only the information contained in the LR image, and does not add new information to the higher-resolution counterpart.

We follow the method presented in Liu et al. (2012) and apply a cross-calibration factor to MDI and GONG. We perform a linear regression of LR magnetic fields (MDI or GONG) against the HR magnetic field (HMI) of the form MDI/GONG = a + b × HMI. We find that b = 1.3 ¹⁴ for MDI and b = 0.7 for GONG. We construct our baseline by bicubic upsampling, and then, scaling of MDI by 1/1.3 and GONG by 1/0.7.

In addition to our baseline, we also compare our work to results achieved with the same neural network but employing a loss function that is only based on the MSE between the model output and target. This allows us to highlight the need for including physically motivated terms in the loss function when handling scientific data.

Our first trained models only contained the MSE loss (see Section 3.2). However, it became apparent that this loss was not sufficiently nuanced to capture important properties of the HR magnetograms. In this section, we show the results of the gradual addition of the loss terms described (see Section 3.2) roughly in the order in which they were added. In total, we added three additional terms that addressed the need for better polarity inversion lines (${{ \mathcal L }}_{\mathrm{grad}}$, Grad), well-balanced positive and negative polarities (${{ \mathcal L }}_{\mathrm{hist}}$, Hist), and more concentrated and detailed magnetic field features (${{ \mathcal L }}_{\mathrm{SSIM}}$, SSIM). These were the only additional loss terms that we experimented with and they resulted in superresolved magnetograms that better match the statistical properties of HMI. However, we cannot rule out the possibility that there are other even better loss terms, or combinations of them, that could further improve the results.

Figure 3 shows full-disk images of our input data (top row) and the best results of our deep-learning SR network (bottom row). The insets show one of the 1024 patches used for splitting the Sun during training. These results were achieved with a loss function that features a combination of differentiable physically motivated terms including the MSE, magnetic field gradients, pixel histograms, and self-similarity penalties, as discussed earlier. The superresolved full-disk magnetograms of MDI and GONG have noise levels, texture, and relative magnetic field intensity akin to those of the HMI target. Zooming in closer, the insets show higher-resolution structures for the model's outputs, which better match those of the HMI target. The improvement is especially significant for GONG, with a striking difference in small-scale structures between the input and output patches.

Zoom In Zoom Out Reset image size

Figure 4 compares superresolved magnetograms obtained with different loss functions to the input (MDI/GONG), the target (HMI), and our bicubic upsampling baseline. The first row (third row) in Figure 4 shows the same patch of the Sun with MDI (GONG) as the input. The second and last rows show the calculated difference between the upsampled magnetograms and the target.

Zoom In Zoom Out Reset image size

Starting with our baseline, the bicubic upsampled MDI magnetogram still shows the salt-and-pepper-like noise structure that is present in the MDI input in the lower left corner of the magnetogram patch. This is because simple upsampling techniques extrapolate the magnetic field, including its noise, to the higher-resolution image. Moreover, bicubic upsampling cannot leverage the information present in the whole data set of magnetograms. Bicubic upsampling of GONG increases the sharpness of edges around active regions, but the large patch-like features do not increase in detail.

Using our deep-learning model with a simple MSE loss removes the noise floor of the MDI input image. In addition, we start to recover small-scale features in and around active regions. Adding optimization penalty terms to the MSE loss modifies details in the HR reconstructions. It also visibly reduces the characteristic size of the structures in the difference images (Figure 4, second row). We see this as evidence that the additional loss terms allow the CNN to better capture the structure of the target magnetic field. However, a purely visual inspection of the images is not enough to find significant differences or distinguish which loss function is best at recovering HR features.

Figure 5 is an ablation study that compares the effect of each component included in the loss function on the reconstruction of the magnetic field. We compare performances by evaluating the post-mortem measurements introduced earlier and calculate (a) differences in extreme magnetic field values, (b) the Pearson correlation coefficient, (c) differences in image gradients, and (d) differences in the signed flux of the target and deep-learning output magnetograms. All metrics are calculated on a pixel-to-pixel basis across our test set, which contains approximately 1 million patches for MDI and 8 million patches for GONG. Figure 5 shows the results obtained for MDI input magnetograms.

Zoom In Zoom Out Reset image size

As mentioned above, a simple MSE loss succeeds at creating visually pleasing magnetogram outputs that show a higher level of detail than the input magnetograms (see Figure 4). However, an objective function based exclusively on MSE is unable to reconstruct extreme values of the magnetic field (i.e., the strongest positive and negative magnetic fields in a patch) properly as shown in Figure 5(2-a)). Looking at how well extreme values are reconstructed, we observe double peaks centered around ±100 Gauss in the bicubic baseline (Figure 5(1-a)), and when using an MSE loss (Figure 5(2-a)). With MSE alone, the neural network consistently underestimates the magnitude of extreme values, leading to a peak centered around +100 Gauss for the maximum magnetic field values, and a second peak centered around −100 Gauss for the minimum magnetic field values when comparing the target and the deep-learning output. Additionally, the MSE loss produces a clear asymmetry in the signed flux (Figure 5(2-d)). This is highly problematic considering that the solar magnetic field must be divergence-free, and thus in flux balance.

Including a gradient penalty term in the loss function (indicated in the third row of Figure 5 as MSE + Grad) removes the double peaks and centers the distribution around zero (red line in Figure 5(3-a)). Taking image gradients into account is a measure often used in computer vision to improve edge detection and texture matching (Forsyth & Ponce 2002). For the application to magnetograms, edge detection aids in defining boundaries around active regions, and texture matching helps to recover detailed features in the HR image. Despite these improvements, maximum fields are still slightly underestimated, as indicated by the fact that the distribution of extreme values is asymmetrically skewed toward positive values for an MSE + gradient loss function (Figure 5(3-a)).

On average, the sum of the magnetic field values on the surface of the Sun is expected to be close to zero. Deviations from zero only occur when the leading part of an active region comes into view of the instrument, and the following cancellation of the magnetic field cannot be viewed yet. Biases in reconstructing positive or negative fields in the superresolved magnetic field would violate Gauss' law (which states that the magnetic field must be divergence-free). The histogram penalty (MSE + Grad + Hist in the fourth row of Figure 5) manages to mostly correct the skewed distribution of extreme values (Figure 5(4-a)) while also slightly shifting the discrepancies in image gradients (Figure 5(4-c)) closer to zero.

We further improve model performance by adding a similarity penalty term (SSIM, see Section 3.2) that forces the model to learn spatial structures of the solar magnetic field (MSE + Grad + Hist + SSIM in the fifth row of Figure 5). This combination of terms also produces the solution with the best balance between positive and negative fluxes (Figure 5(4-d)).

This ablation study shows that the structure of the loss function we optimize for generates trade-offs for the physical properties of the superresolved magnetograms. While MSE alone achieves better performances in terms of Pearson correlations, adding a gradient, histogram, and SSIM penalty terms significantly improves how the model captures extreme values of the solar magnetic field, as well as achieving the flux balance critical to ensuring that the recovered magnetic field is divergence-free.

In the following, we demonstrate the value added to using superresolved magnetograms over their LR counterparts. Specifically, we investigate small- and large-scale structures, homogenization properties, and temporal patterns of the superresolved magnetic fields.

In Figure 6, the first (third) row shows a pixel-to-pixel correlation plot between target magnetograms and SR output for the entire test set for MDI (GONG). The test set contains ≈25 (≈125) million pixels for MDI (GONG). The orange lines highlight regression lines between the output and target. To put these results into perspective, Figure 6 also shows a comparison of the correlation between the bicubic upsampling baseline and the target magnetograms (purple graph on the right). Correlation plots aligning with the 45° diagonal or small residuals indicate good cross-calibration. Our deep-learning approach centers the correlation plots more on the 45° diagonal, thereby improving the cross-calibration between MDI (GONG) and HMI. This is clearly visible for GONG (Figure 6, third row) and less strongly observable for MDI (Figure 6, first row).

Zoom In Zoom Out Reset image size

Figure 6 suggests great performance of both the bicubic baseline (purple) and our deep-learning approach (orange) when only looking at the correlation plots as both comparisons are strongly centered around x = y. The second and fourth rows in Figure 6 show further quantitative visualizations of both the cross-calibration and SR approaches, showing that deep learning is more effective at performing SR. We measure this improvement by investigating the relative average deviation of the output and target across a 4 × 4 pixel area compared to the corresponding LR pixel. In the case of MDI, the deep-learning algorithm is able to fix the well-known saturation error at strong magnetic fields. In the case of GONG, the deep-learning algorithm is able to overcome the large disagreement between GONG and HMI magnetograms present around 1000 G.

These results highlight one of the main challenges of finding suitable quantities to capture SR. When looking simply at averages, it is easy to become overly confident and misrepresent the quality of results. Our work encourages benchmarking SR techniques with a quantitative assessment that directly measures the reconstruction of small-scale structures of the magnetic field.

Table 4 replicates the quantitative assessment in Tables 1 and 2 of Liu et al. (2012) and compares the Pearson correlation coefficient between superresolved MDI and GONG magnetograms across different radial regions of the Sun and different values of the magnetic field. For both MDI and GONG, the Pearson coefficient is computed on our test set of magnetograms from 2011 March. Our results show that our deep-learning approach generates magnetograms that contain information present in HMI magnetograms, but not in their LR counterparts. Across all radial regions and field values, the Pearson correlation coefficient between superresolved and HMI magnetograms increases by 5%–7% relative to the correlation between lower-resolution and HMI magnetograms.

In Table 5, we compare statistics of the magnetic field between HMI and superresolved MDI/GONG over kernels of various pixel sizes. We benchmark our results against our baseline approach (bicubic upsampling with linear rescaling). Our results show that the difference in gradient between HMI and deep-learning output over small kernels (2–4 pixels) is 30% (for MDI) and 4% (for GONG) smaller than between HMI and the baseline outputs. Similar improvements are observed for extreme values of the magnetic field within kernels of different sizes. This confirms that our deep-learning SR captures details that are averaged out at lower resolution. Remarkably, improvements in small-scale patterns extend to structures of size larger than the 4× upscaling factor.

Finally, in this section, we show the value added of superresolved magnetograms to study time series of unsigned fluxes and gradients within SHARPs (Bobra et al. 2014). ¹⁵ The SHARPs data series allows us to systematically track solar active regions at a 12 minute cadence over the lifetime of each region, and is among the most widely used products for connecting the evolution of individual magnetic regions and space weather events (see Asensio Ramos et al. 2023, and the references therein). Each region tracked through SHARPs has an associated HMI Active Region Patch (HARP) number (HARPNUM), and a number of associated calculated quantities used for space weather forecasts, with the average unsigned flux and average unsigned gradient being the only two that can be calculated using LOS magnetograms. The main objective of this exercise is to evaluate whether superresolving GONG and MDI magnetograms is adding value to the resulting magnetograms, or whether pure calibration, i.e., adjusting for instrumental differences without upscaling, either using our baseline approach or via deep learning, results in better end products.

Figure 7 focuses our analysis on HARP region 407 (NOAA AR 11169). This region was chosen out of the 36 HARPNUM observed during 2011 March (our test set) because it is large, long-lived, and has relatively good time coverage by both the MDI and GONG instruments. The top panels (Figure 7(a)) show five snapshots of the evolution of HARP region 407 as observed by our target instrument (HMI). Figure 7(b) shows the unsigned magnetic flux for the HMI target (red triangles), compared to ML SR or bicubic upscaling of MDI (green/beige dots) or of GONG (purple/light blue dots). The colored vertical lines indicate the timestamp of the snapshots shown in Figure 7(a). Figure 7(c) shows the ratio between the ML or bicubic upscaling outputs and the HMI targets.

Zoom In Zoom Out Reset image size

We find that the magnitude of the unsigned flux, and its evolution as the region moves through the instrument's field of view, are well reproduced in the ML outputs of both MDI and GONG. Looking at the ratio comparison with HMI, we find the calibration to be centered around the target value of 1.0, with two main characteristics. The first one is the underestimation of the unsigned flux during the emerging phase of the HARP region, which stabilizes after the region is fully developed (Figure 7(c)). This behavior arises from the fact that the ML algorithm denoises the magnetogram, thereby reducing the relative amount of unsigned flux in the ML output compared to the HMI target. As the region grows (and most unsigned magnetic flux is in the region itself), this stops being an issue. One of the main differences between the ML output (green circles and purple squares) and the bicubic upsampling baselines (tan circles and blue squares) is the ability of the ML algorithm to address known systematic issues introduced by projection effects due to area foreshortening close to the solar limb. While the bicubic baselines systematically underestimate flux as the HARP region rotates out of view, the ML output is generally stable through the lifetime of the HARP.

The second observable characteristic in Figure 7(c) is a clear 24 hr modulation that is a known systematic issue in HMI measurements due to HMI's fast orbital speed (see Figure 9). Neither GONG, being ground based, nor MDI, having a stable orbit at L1, suffer from this problem. Interestingly, the ML output retains the stability of the input instruments. This likely arises because time is not included as an input parameter, and therefore no direct information is provided for the model to learn temporal dependencies. While we have not explored this further, this hints at the possibility of using GONG and MDI to remove this outstanding systematic issue from HMI in the future.

Figure 7(d) shows a detailed comparison of the magnetic field and gradients calculated on the HMI target, and the MDI and GONG outputs using bicubic upscaling and ML-based SR at the same point in time. Each panel also contains the numerical measurement of the average unsigned magnetic flux and average unsigned gradient. The quantitative and qualitative superiority of the superresolved output over the baseline bicubic upsampling is evident.

Figure 8 further illustrates the superiority of the ML-based SR output (ML-4X; light blue squares) over both the baseline (purple dots) and an ML-based calibration approach (ML-1X; green squares). The latter was obtained by training our ML model on HMI magnetograms that have been downsampled to the resolution of MDI/GONG. In this figure, we also show results from all SHARPS regions and time stamps available during our test period.

Zoom In Zoom Out Reset image size

Figures 8(a)–(d) show the time evolution of the unsigned flux and the average unsigned gradient as estimated by the different algorithms applied to HARP region 407 as it crosses the solar disk. Figures 8(e) and (f) show scatter plots of the HMI target against the different outputs for all 36 regions (and all their time stamps during 2011 March; a total of 2261 SHARPs compared). The closest match to the target HMI measurements (for both the average unsigned flux and average unsigned gradient) is always the SR output (ML-4X). While all algorithms underestimate the unsigned average gradient (Figures 8(c), (d), (g), and (h)), the improvement brought by SR is evident. This improvement is especially remarkable for GONG, which is currently the world's main space weather operations magnetograph.

In the case of average unsigned flux, we also find the best match to be the SR outputs. Interestingly, bicubic upscaling overestimates the unsigned flux by enhancing the contribution of noise (Figures 8(a) and (b)). In comparison, our SR approach not only properly calibrates the inputs to the target instrument, but also denoises the magnetograms, ensuring an almost perfect match with the target HMI magnetograms. Table 6 in Appendix A.3 shows that the benefits of SR extend to all SHARPs. Taken all together, we see this as evidence that ML-driven SR is superior to simple upscaling by all relevant metrics and performance quantities. Considering that this is one of the first implementations of ML-based SR to solar magnetograms, it is reasonable to assume that these algorithms will only perform better in the future.

The SDO/HMI, and SOHO/MDI data set used during the current study are available from the Joint Science Operations Center at http://jsoc.stanford.edu/. GONG magnetograms are available from the NSO website at https://gong2.nso.edu/archive/patch.pl. To prepare the original data for our deep-learning pipeline, we follow the preprocessing steps outlined in the Methodology section (Section 3). The code used in this work is available at https://gitlab.com/frontierdevelopmentlab/living-with-our-star/super-resolution-maps-of-solar-magnetic-field. All questions regarding the code should be directed to the corresponding author. A simplified version of the repository is available via Muñoz-Jaramillo et al. (2021a), containing (1) the necessarily functions to process the original data into a machine-learning-ready format, and (2) inference-based magnetogram converters using our best models. An example data set of our upscaled and calibrated magnetograms is available via Muñoz-Jaramillo et al. (2021b).

As an extension to Figure 7, in Figure 9 we show the HMI and SR MDI time series along with the ratio of time series, and the radial velocity of HMI. The observed oscillations arise from a Doppler shift in the spectral line due to the orbital variation of the spacecraft (Couvidat et al. 2016).

Zoom In Zoom Out Reset image size

Our results indicate that the effectiveness of the neural network to cross-calibrate and superresolve the magnetic field is sensitive to the signal-to-noise ratio within a magnetogram. The signal-to-noise ratio is affected by (1) the strength of the magnetic field itself, and (2) the proximity to the solar limb. HMI's noise level is 15 (Hoeksema et al. 2014). Figure 10 compares the Pearson correlation metric calculated for superresolved MDI magnetograms as a function of patch location and magnetic field value. The gray-shaded histograms were calculated for patches across the full solar disk, while the blue-shaded histograms were calculated for patches that lie within 90% of the radius of the solar disk. In addition, we compare the Pearson correlation for all patches (left column), and those that have an average unsigned field larger than 15 G (right column). Looking at all magnetic field values, we can see that the distribution of Pearson correlation coefficients shows two peaks around 0.25 and 0.75 when patches across the entire solar disk are considered (Figure 10, left column, gray histogram). Discarding patches that lie outside of the central 90% of the radius of the solar disk removes the double peak and shifts the distribution to be asymmetrically centered around 0.8 (Figure 10, left column, blue histogram).

Zoom In Zoom Out Reset image size

When we also disregard patches that show magnetic field strengths close to HMI's noise level, we see that the distribution of Pearson correlation coefficients becomes substantially narrower and is more symmetrically centered around 0.8. This observation supports our finding that magnetogram patches with field strengths around the noise level are harder to align and less reliable for the neural network to learn. In addition, near the solar limb, the magnetic field intensity weakens due to projection effects and a reduction of the effective resolution of the instruments. In the post-mortem evaluation of our results, we therefore focus on patches that lie within 90% of the radius of the solar disk, unless otherwise specified.

Table 6 in Appendix A.3 shows that the benefits of SR extend to all SHARPs.