Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Advertisement
Scientific Reports volume 15, Article number: 21788 (2025)
1232
3
Metrics details
The efficiency of photovoltaic (PV) systems is often compromised by undetected faults, exacerbated by the complexity of thermal imagery backgrounds. This study presents a novel deep-learning-based approach to enhance fault detection in PV systems by customizing the Atrous Spatial Pyramid Pooling (ASPP) module within a U-Net architecture. We propose and evaluate three modified configurations U-Net-ASPP_Cent, U-Net-ASPP_Diag, and U-Net-ASPP_Hybrid each designed to address specific fault localization challenges, including central and diagonal fault patterns. These configurations aim to overcome the limitations of conventional U-Net-ASPP by enhancing multiscale feature extraction and improving segmentation accuracy in complex PV thermal images. The U-Net-ASPP_Hybrid configuration demonstrated the most balanced performance across all key metrics, achieving a 1.13% improvement in F1-score, a 3.01% increase in Intersection over Union (IoU), and a 9.86% reduction in loss compared to the baseline U-Net-ASPP. Additionally, the U-Net-ASPP_Cent and U-Net-ASPP_Diag configurations provided IoU gains of 1.18% and 1.96%, respectively, while also reducing false positive rates. These results highlight the effectiveness of incorporating region-specific dilation strategies, enhancing the model’s ability to detect diverse and challenging fault patterns in complex thermal imagery. Beyond quantitative performance, qualitative segmentation analysis confirms that the U-Net-ASPP_Hybrid model offers superior fault localization and adaptability to real-world PV inspections. The U-Net-ASPP_Cent model is particularly effective for central anomaly detection, while the U-Net-ASPP_Diag model excels at identifying directional faults such as cracks. The U-Net-ASPP_Hybrid model, combining both strategies, provides a comprehensive solution for automated PV fault detection. These findings underscore the stability, scalability, and real-world applicability of the proposed models, making them ideal for automated PV inspection systems aimed at minimizing manual intervention and enhancing the reliability of renewable energy infrastructure. Future research will explore adaptive dilation strategies and more diverse datasets to further improve model generalization across varying PV environments.
Photovoltaic systems play a pivotal role in the global transition to renewable energy, offering a sustainable solution for electricity generation. However, their efficiency is often compromised by undetected faults, such as hot spots, cracks, and electrical mismatches, which can lead to significant energy losses and increased maintenance costs1,2. Traditional fault detection methods, such as infrared (IR) thermography and visual inspections, have been widely used to identify these anomalies. While effective in controlled environments, these methods struggle to generalize across diverse and complex backgrounds, such as those encountered in large-scale PV farms3,4. Environmental factors like weed shadows, sunlight reflections, and varying weather conditions further complicate the detection process, highlighting the need for more robust and automated fault detection mechanisms5,6. Recent advancements in deep learning, particularly convolutional neural networks (CNNs), have shown promise in addressing these challenges. Among these, the U-Net architecture has gained significant attention due to its ability to preserve high-resolution details while capturing contextual information through its encoder-decoder structure and skip connections7. However, U-Net often struggles with multiscale feature extraction, which is critical for detecting faults in PV systems where anomalies can vary significantly in size and shape8. To address this limitation, Atrous Spatial Pyramid Pooling (ASPP) has been proposed as a complementary module, enabling the capture of contextual information at multiple scales without increasing the number of parameters9,10.
Recent developments in image recognition have expanded beyond CNNs to include Vision Transformers (ViTs), which utilize self-attention mechanisms to model long-range dependencies more effectively11. Hybrid architectures that combine CNNs and transformer components have been shown to improve semantic segmentation in remote sensing and medical imaging by leveraging both local and global feature representations12,13. Additionally, adaptive data augmentation strategies-such as class-aware oversampling, geometric distortion correction, and distribution-aligned noise injection-have become essential for enhancing generalization, particularly in imbalanced datasets14,15 These advancements underscore the ongoing evolution of deep learning frameworks toward better scalability and precision in complex imaging domains.
Inspired by these advancements, this study proposes a set of architectural optimizations to the U-Net-ASPP framework, introducing three customized configurations: U-Net-ASPP_Cent, U-Net-ASPP_Diag, and U-Net-ASPP_Hybrid. Each variant applies a specific region-focused dilation strategy to improve multiscale feature extraction for PV fault detection. These configurations are tailored to enhance segmentation accuracy in thermal images characterized by localized and directional anomalies. By empirically validating these configurations using a diverse PV thermal dataset, we demonstrate notable gains in fault detection metrics, especially in challenging backgrounds. The proposed framework aims to bridge the gap between conventional semantic segmentation methods and the increasing complexity of real-world PV inspection tasks.
Traditional fault detection methods in PV systems have primarily relied on infrared (IR) thermography and visual inspections. IR thermography captures thermal images of PV modules, identifying temperature variations that indicate potential faults such as hot spots and cracks16. While effective in controlled environments, these methods often struggle to generalize across diverse and complex backgrounds, such as those encountered in large-scale PV farms17. Environmental factors like weed shadows, sunlight reflections, and varying weather conditions further complicate the detection process, leading to high rates of false positives and missed detections9. Visual inspections, on the other hand, are labor-intensive and prone to human error, making them unsuitable for large-scale applications4.
To address the limitations of traditional methods, deep learning-based approaches have emerged as powerful tools for fault detection in PV systems. Among these, U-Net has gained significant attention due to its ability to preserve high-resolution details while capturing contextual information through its encoder-decoder structure and skip connections7. Originally developed for biomedical image segmentation, U-Net has been successfully applied in various domains, including remote sensing and medical imaging, demonstrating its effectiveness in handling complex and noisy data18. However, while U-Net excels in capturing fine-grained details, it often struggles with multiscale feature extraction, which is critical for detecting faults in PV systems where anomalies can vary significantly in size and shape19.
To enhance the multiscale feature extraction capabilities of U-Net, ASPP has been proposed as a complementary module. ASPP employs atrous (dilated) convolutions to capture contextual information at multiple scales without increasing the number of parameters, making it particularly effective for semantic segmentation tasks10. ASPP has been successfully integrated into various deep learning architectures, such as DeepLabv3+, to improve segmentation performance in complex environments18. In the context of PV fault detection, ASPP can help address the variability in fault patterns by capturing both fine-grained and coarse-grained features, enabling more accurate segmentation of anomalies in thermal images20. Our prior work21 explored general enhancements to U-Net, incorporating attention mechanisms and residual learning for fault detection. Our current study advances ASPP customization, introducing region-specific dilation strategies that refine multi-scale feature extraction for centralized and directional PV faults.
Despite these advancements, existing studies often overlook the importance of region-specific and directional patterns in PV thermal imagery, which are critical for detecting small or intricate faults. Standard ASPP configurations, while effective in capturing multiscale features, may fail to adequately address these patterns, leading to suboptimal segmentation performance22. For example, faults located in the central regions of PV modules or those with diagonal patterns are often missed by traditional configurations. This gap in the literature highlights the need for customized ASPP configurations tailored to the unique challenges of PV fault detection. In this study, we propose a novel integration of U-Net with customized ASPP configurations to address these gaps.
Photovoltaic fault detection is essential for maintaining the efficiency of solar energy systems. Traditional fault detection methods rely on infrared thermography and manual inspections, which are limited by environmental conditions, image complexity, and variability in thermal signatures. To overcome these challenges, deep learning-based segmentation methods, such as U-Net, have been widely adopted. However, standard U-Net models struggle with multiscale feature extraction, making them less effective in identifying diverse fault patterns in PV modules. ASPP has been introduced as a solution, enabling the network to capture multiscale information without increasing computational cost. Despite its advantages, existing ASPP configurations fail to account for specific fault localization patterns, particularly in the central and diagonal regions of thermal images.To address these limitations, we propose three novel modifications to ASPP within a U-Net framework:
U-Net-ASPP_Cent: Emphasizes central dilation, capturing critical features in the image center where anomalies are frequently concentrated.
U-Net-ASPP_Diag: Focuses on diagonal dilation patterns, addressing fault patterns often overlooked in traditional configurations.
U-Net-ASPP_Hybrid: Combines both central and diagonal focus kernels with global pooling contributions to maximize feature extraction.
These modifications improve segmentation accuracy by optimizing feature extraction for region-specific fault patterns. The proposed method is evaluated using a thermal image dataset of PV modules, where performance is measured in terms of IoU, Dice coefficient, F1-score, precision, and recall.
ASPP is a widely used module in semantic segmentation tasks, designed to capture multiscale contextual information through the use of atrous (dilated) convolutions. ASPP was first introduced in the DeepLab series of models9,10 and has since become a standard component in many segmentation architectures due to its ability to expand the receptive field without increasing the number of parameters or losing spatial resolution. The core operation of ASPP is the atrous convolution, which introduces a dilation rate (r) to control the spacing between the kernel elements. Mathematically, the atrous convolution is defined as:
where:
(K_{text {ASPP}}(x, y)) is the output of the atrous convolution,
(x[i]) is the input feature map,
(w[k]) represents the convolutional weights,
(r) is the dilation rate, and
(k) indexes the elements of the convolutional kernel.
By applying multiple parallel atrous convolutions with different dilation rates, ASPP enables the model to analyze faults from various perspectives, enhancing detection accuracy. By varying the dilation rate (r), atrous convolutions can capture features at multiple scales. For example, a small dilation rate (e.g., (r = 1)) captures fine-grained details, while a larger dilation rate (e.g., (r = 6) or (r = 18)) captures coarse-grained contextual information. This multiscale approach is particularly beneficial for tasks such as fault detection in photovoltaic (PV) thermal images, where anomalies can vary significantly in size and shape. ASPP employs multiple parallel atrous convolutions with different dilation rates, as illustrated in Figure 1. These parallel convolutions probe the input feature map at varying scales, enabling the model to analyze faults from multiple perspectives. The outputs of the parallel convolutions are typically concatenated and passed through a (1 times 1) convolution to refine the multiscale representation. Additionally, ASPP often includes a global average pooling (GAP) branch to capture global contextual information, further enhancing the model’s ability to detect anomalies in complex backgrounds. The effectiveness of ASPP in capturing multiscale features has been demonstrated in various applications, including medical image segmentation23, remote sensing24, and, more recently, fault detection in PV systems25. Furthermore, Sharma et al.26 proposed an enhanced ASPP variant specifically designed for small object segmentation in maritime surveillance, reinforcing the adaptability of ASPP-based architectures for applications where object size variability is a key challenge. By leveraging ASPP, models can achieve better segmentation accuracy, particularly in scenarios where objects of interest vary in size or are surrounded by complex backgrounds.
Atrous convolution kernels with varying dilation rates.
The U-Net model, originally introduced by7, is a convolutional neural network designed for image segmentation tasks. It features an encoder-decoder structure with skip connections that preserve high-resolution details during upsampling. To extend the U-Net’s capabilities, we integrate ASPP between the encoder and decoder stages, as shown in Figure 2. This integration enhances the model’s ability to process features across multiple scales, making it robust against the variable and noisy conditions of thermal images. The U-Net-ASPP model consists of the following key components:
Input (640 (times) 512 (times)1): The input to the U-Net is a thermal image from our dataset, represented as a single-channel grayscale image with dimensions (640times 512).
Encoder: The encoder extracts hierarchical features from the input thermal images through a series of convolutional and downsampling layers. Each encoder block reduces the spatial dimensions while increasing the number of filters, capturing both low-level and high-level features.
ASPP Module: Positioned between the encoder and decoder, the ASPP module processes the extracted features using multiple parallel atrous convolutions with distinct dilation rates (e.g., (3, 6, 18)). These convolutions capture contextual information at multiple scales, enabling the model to detect both small and large fault patterns. The outputs of the parallel convolutions are concatenated and passed through a (1 times 1) convolution to refine the multiscale representation.
Decoder: The decoder reconstructs the segmented output while incorporating high-resolution features from the encoder via skip connections. Each decoder block upsamples the feature maps and combines them with the corresponding encoder features, ensuring precise localization of faults.
Output: The final output is a pixel-wise segmentation map that identifies anomalous regions in the input thermal image.
U-Net architecture with integrated ASPP. The input is a 640(times)512(times)1 thermal image, with ASPP enhancing multiscale feature extraction between encoder and decoder stages.
This U-Net-ASPP architecture serves as the baseline for our proposed modifications and demonstrates improved performance in segmenting PV thermal images with challenging backgrounds. The integration of ASPP enhances the model’s ability to capture multiscale features, making it more effective for fault detection in complex thermal imagery.
The standard ASPP module provides strong multiscale feature extraction but has limitations when applied to PV fault detection. While ASPP captures contextual information across multiple receptive fields, it does not consider spatial relationships specific to PV thermal imagery, where faults often exhibit structured patterns in central or diagonal regions. Additionally, traditional ASPP configurations do not emphasize region-specific features, which are crucial in segmenting small or complex PV anomalies. To address these challenges, we introduce three customized ASPP variations, specifically tailored for PV fault detection:
ASPP_Cent: Enhances feature extraction from the central regions of the image, ensuring improved detection of faults that tend to cluster in the middle of PV modules.
ASPP_Diag: Targets diagonal patterns in PV anomalies, which are often overlooked by standard ASPP configurations.
ASPP_Hybrid: Combines both central and diagonal focus kernels while incorporating global pooling to maximize feature diversity.
Each modification builds upon the baseline ASPP structure (Figure 2) while preserving its original multiscale processing capability. In the following subsections, we define the mathematical formulation for each proposed configuration.
The central focus (ASPP_Cent) kernel emphasizes feature extraction from the central regions of the input image, which are often the most critical for fault detection. The representation of this kernel is defined as:
where:
This modification prioritizes central regions by combining the benefits of standard ASPP with additional focus on central features, which is particularly beneficial in detecting anomalies in small or highly localized areas. This configuration is inspired by methods in object detection and segmentation that prioritize central regions to improve accuracy in areas of interest19,27.
The diagonal focus (ASPP_Diag) kernel enhances the model’s ability to capture oblique or angular patterns. The operation is defined as:
This kernel identifies patterns along the main and anti-diagonals, improving segmentation performance for directional anomalies that are often missed by traditional methods. This approach is inspired by anisotropic filtering techniques and applications in directional feature extraction for texture and fault analysis24,28.
The hybrid focus (ASPP_Hybrid) combines the central and diagonal focus kernels with global pooling contributions to maximize feature extraction. The operation is defined as:
where:
This hybrid strategy leverages complementary feature sets, ensuring robustness against diverse patterns in thermal imagery. By integrating these focus types, the configuration improves segmentation accuracy for complex backgrounds and areas with intricate fault patterns. Figure 3 shows the proposed scenarios for different dilation rates.
Visualization of proposed ASPP scenarios with dilation rates 3, 6, and 18 for the three proposal modifications.
This study utilizes a publicly available dataset22,25 consisting of 1,009 thermal images of photovoltaic modules, each with a resolution of 512(times)640 pixels. The dataset was captured under optimal weather conditions using an unmanned aerial vehicle (UAV) equipped with a radiometric thermal camera. Temperature readings range from (2.25^circ)C to (103.34^circ)C, encompassing a diverse spectrum of thermal variations, which is crucial for accurately detecting PV anomalies. Each thermal image is paired with a highly detailed binary mask, precisely delineating anomalous regions in PV modules. The dataset categorizes anomalies into three distinct types:
Single Anomalous Cell: Isolated faults affecting an individual PV cell.
Multiple Non-Contiguous Anomalous Cells: Independent faults occurring in separate, non-adjacent cells.
Contiguous Series of Anomalous Cells: Faults spanning multiple adjacent cells, often indicative of more severe structural or electrical issues.
Figure 4 presents representative samples from the dataset, illustrating the diverse anomaly types encountered in PV modules.
Thermal Data Encoding: The dataset’s grayscale representations directly map temperature variations, ensuring precise localization of faults. This encoding facilitates accurate extraction of thermal signatures critical for automated fault detection.
Diverse Anomaly Types: The dataset captures a broad range of fault scenarios, including optical degradation, electrical mismatches, and thermal hotspots. This diversity makes it an invaluable benchmark for evaluating segmentation models under real-world conditions.
High-Quality Annotations: Each image is accompanied by a meticulously labeled binary mask, providing ground truth segmentation for supervised learning. The consistency and precision of these annotations improve the applicability of the dataset for both academic research and industrial deployment.
Representative thermal images from the dataset, showcasing the diversity of anomalies encountered in photovoltaic modules.
To enhance model performance, several preprocessing steps were applied, including normalization and augmentation. These steps ensure that the dataset is well-prepared for training and evaluation.
Normalization To standardize the dataset and improve model convergence, we applied the z-score normalization method29. The z-score is calculated as:
where:
x is an individual data point,
(mu) is the mean of the dataset, and
(sigma) is the standard deviation.
This transformation centers the dataset around a mean of zero and scales it to a standard deviation of 1, ensuring that all features contribute equally to the model’s learning process. Figure 5 illustrates the effect of z-score normalization on a sample thermal image.
An example of image correction using z-score.
Augmentation One of the critical preprocessing steps in deep learning for image segmentation tasks, especially when dealing with small datasets, is augmentation. Augmentation techniques help to artificially increase the size and diversity of the dataset by applying various transformations to the original images, thereby improving the generalizability of the model23,30. Given the complexity and variability of thermal images in PV systems, augmentation is essential to address class imbalance and enhance the robustness of the model. The three categories of faulty cells in our dataset-single anomalous cell, multiple non-contiguous anomalous cells, and contiguous series of anomalous cells-are not evenly distributed. To mitigate this imbalance, additional augmented samples were generated for minority classes to reduce overfitting and improve the model’s ability to generalize30. Commonly used augmentation techniques include rotation, translation, shear, zoom, and horizontal flipping. In our study, the augmentation parameters were carefully selected to reflect realistic variations that may occur in thermal imagery while preserving the integrity of the fault patterns. The following transformations were applied:
Rotation Range: Random rotations within a specified degree range to simulate different orientations of solar panels.
Width and Height Shifts: Random translations to account for positional variations during image capture.
Shear Range: Shearing to introduce slight distortions, which can improve the model’s ability to recognize faults from different angles.
Zoom Range: Random zooming to simulate varying distances between the camera and the solar panels.
Horizontal Flip: Flipping images to ensure the model remains invariant to changes in orientation.
To ensure that the augmented images contributed meaningful information to the model, additional checks were implemented to discard augmented masks that did not contain any faults. This step is crucial to avoid introducing noise and maintain the quality of the training data. The augmentation process not only addressed class imbalance but also enhanced the diversity of the training set by introducing variations that the model is likely to encounter in real-world scenarios. By augmenting minority class samples, we ensured that the model learned to recognize different types of faults more effectively, even when they appear in less frequent patterns23. To further illustrate the impact of the augmentation process, Figure 6 presents an example of a thermal image and its corresponding mask before and after applying various augmentation techniques. The augmented samples exhibit variations in rotation, translation, zoom, and flipping, simulating diverse real-world conditions. This process enhances the diversity of the dataset, addressing class imbalance and improving the model’s ability to generalize across different fault patterns.
Augmented thermal images and masks, demonstrating enhanced diversity for improved model generalization.
In this study, we evaluate four U-Net-ASPP configurations: U-Net-ASPP, U-Net-ASPP_Cent, U-Net-ASPP_Diag, and U-Net-ASPP_Hybrid to assess their effectiveness in segmenting faults in photovoltaic (PV) modules from thermal images. The evaluation focuses on both quantitative performance metrics and qualitative segmentation analysis to provide a comprehensive understanding of each model’s capabilities. To ensure a rigorous performance assessment, we evaluate the models using the following quantitative metrics:
Segmentation Accuracy: Pixel-level Accuracy, Intersection over Union (IoU), and Mean Absolute Error (MAE) quantify the models’ ability to correctly classify and localize faults.
Class Imbalance Metrics: F1-Score, Precision, Recall, and False Positive Rate (FPR) assess the models’ performance in handling imbalanced datasets, where fault regions are often underrepresented.
Error Metrics: Loss Function provide insights into model convergence and the rate of false positives, which are critical for industrial applications.
In addition to quantitative metrics, we perform a qualitative segmentation analysis by visually comparing predicted masks with ground truth annotations. This analysis is particularly important for understanding how each model handles challenging scenarios, such as subtle thermal anomalies and irregular fault patterns, which may not be fully captured by numerical metrics alone. In the following sections, we present a detailed numerical comparison of the models’ performance, supported by graphical trends and visual segmentation samples. This multi-faceted approach allows us to comprehensively evaluate the strengths and limitations of each U-Net-ASPP configuration.
To quantitatively assess the effectiveness of the proposed U-Net-ASPP configurations, we evaluate their segmentation accuracy using multiple performance metrics. The comparison focuses on accuracy, F1-score, IoU, loss, precision, recall, FPR, and MAE. These metrics provide a holistic view of the models’ capabilities, ensuring a balanced evaluation between detection performance and error minimization. The segmentation performance of all four U-Net-ASPP variants is summarized in Table 1.
From Table 1, we observe that U-Net-ASPP_Hybrid demonstrates superior performance, achieving the highest F1-score (0.894), IoU (0.787), and recall (0.864) while maintaining the lowest loss (0.064). This suggests that incorporating both central and diagonal dilation strategies in the ASPP module improves fault region identification. Specialized configurations such as U-Net-ASPP_Cent and U-Net-ASPP_Diag also outperform the baseline U-Net-ASPP in certain aspects. U-Net-ASPP_Cent achieves slightly better precision (0.931) and a lower loss (0.069), indicating better localization accuracy, whereas U-Net-ASPP_Diag improves IoU (0.779) and maintains a comparable recall (0.852), showing its strength in capturing more dispersed fault patterns. However, U-Net-ASPP_Cent has a slightly higher loss compared to U-Net-ASPP_Diag, suggesting a trade-off between fault detection and over-segmentation. The FPR and MAE provide additional insights into the models’ performance. As shown in Figure 7, U-Net-ASPP_Hybrid achieves the lowest FPR (1.765 (times 10^{-3})), indicating the highest precision in avoiding false alarms. Meanwhile, U-Net-ASPP_Cent has a slightly higher FPR (1.987 (times 10^{-3})) but still outperforms the baseline U-Net-ASPP (2.345 (times 10^{-3})) and U-Net-ASPP_Diag (2.123 (times 10^{-3})). This suggests that the hybrid configuration effectively balances precision and recall, minimizing false alarms while maintaining strong fault detection capabilities.
MAE quantifies the pixel-wise error between predicted masks and ground truth annotations. A lower MAE indicates more precise segmentation. In terms of segmentation precision, U-Net-ASPP_Hybrid achieves the lowest MAE (109.876 (times 10^{-3})), reinforcing its superior segmentation quality. U-Net-ASPP_Cent and U-Net-ASPP_Diag exhibit slightly higher MAE values of 112.345 (times 10^{-3}) and 118.765 (times 10^{-3}), respectively, indicating minor trade-offs in precision for improved recall. The baseline U-Net-ASPP has the highest MAE (123.456 (times 10^{-3})), suggesting room for improvement in segmentation accuracy.
The progression of loss, IOU, F1-score, precision, and recall across training epochs is shown in Figures 8, 9, 10, 11 and 12. All models demonstrate consistent improvement over time, with U-Net-ASPP_Hybrid achieving the highest F1-score and recall by the final epoch. This trend highlights the stability and effectiveness of the proposed configurations, particularly in handling complex thermal imagery.
Comparison of FPR and MAE across models.
Loss: The lowest loss was recorded by the U-Net-ASPP_Hybrid (0.064), indicating the most effective optimization and generalization to unseen data. U-Net-ASPP_Diag (0.066) and U-Net-ASPP_Cent (0.069) also achieved lower loss values than the baseline, suggesting that the customized dilation patterns contribute to more robust learning.
Loss progression across training epochs for all U-Net-ASPP configurations. U-Net-ASPP_Hybrid shows the lowest overall loss.
IoU: The U-Net-ASPP_Hybrid showed a significant improvement in IoU (0.787), surpassing both the baseline and other modified configurations. U-Net-ASPP_Cent (0.773) and U-Net-ASPP_Diag (0.779) demonstrated higher IoU scores than the baseline, indicating their effectiveness in capturing diverse fault regions.
IoU progression during training. U-Net-ASPP_Hybrid achieves the highest IoU.
F1 Score: The progression of F1 score, shown in Figure 10, highlights the consistent improvement achieved by all configurations over the training epochs. The U-Net-ASPP_Hybrid achieved the highest F1 score (0.894), indicating a well-balanced performance between precision and recall. This balanced performance is essential for accurate fault detection in PV systems.
F1-score progression across training epochs. U-Net-ASPP_Hybrid maintains the highest F1-score.
Precision: Both U-Net-ASPP_Cent and U-Net-ASPP_Diag achieved the same precision (0.931), surpassing the baseline’s precision (0.930). This improvement highlights the customized kernels’ ability to reduce false positives, which is crucial in applications where minimizing incorrect fault detection is essential.
Precision progression across training epochs. U-Net-ASPP_Cent and U-Net-ASPP_Diag configurations yield higher precision than the baseline.
Recall: The highest recall was achieved by the U-Net-ASPP_Hybrid (0.864), indicating its superior ability to detect a wide range of faults. U-Net-ASPP_Cent (0.861) outperformed U-Net-ASPP_Diag (0.852), suggesting that central dilation patterns are more effective in capturing faults located in the middle regions of PV modules.
Recall progression across training epochs. U-Net-ASPP_Hybrid achieves the highest recall.
In addition to quantitative metrics, we perform a qualitative segmentation analysis by visually comparing predicted masks with ground truth annotations. This analysis is particularly important for understanding how each model handles challenging scenarios, such as subtle thermal anomalies and irregular fault patterns, which may not be fully captured by numerical metrics alone. As observed in the quantitative comparison, the improvements were generally not substantial enough to easily discern differences in performance across the models. To better identify where the proposed approaches excel, we tested them on a variety of images, focusing on those with complex backgrounds and multi-cell faults. In images with clear backgrounds and well-defined faulty cells, all approaches performed adequately, correctly detecting the anomalies. However, the true challenge lies in images with highly complex backgrounds and multiple affected cells. For instance, in cases with complex backgrounds, the baseline U-Net-ASPP failed to detect faulty cells, while U-Net-ASPP_Cent and U-Net-ASPP_Diag were able to partially detect fault features. In contrast, U-Net-ASPP_Hybrid successfully detected all faulty features. Additionally, in Figure 13, we present an example where even U-Net-ASPP_Cent and U-Net-ASPP_Diag failed to completely detect multi-cell faults, whereas U-Net-ASPP_Hybrid succeeded. However, in the most challenging scenario—where both complex backgrounds and multi-cell faults are present—all approaches, including U-Net-ASPP_Hybrid, struggled to detect all faulty cells or features correctly. This indicates that further optimization, including improvements in annotation quality and dataset size, is necessary to address these limitations.
Qualitative comparison of fault detection performance across U-Net-ASPP configurations.
Each modified configuration offers unique advantages that can be leveraged in different fault detection scenarios:
U-Net-ASPP_Cent: This configuration emphasizes detecting anomalies concentrated in the central regions of thermal images. It is particularly beneficial for detecting faults in systems where defects commonly occur near the center of PV cells. For example, this configuration can be used in applications focusing on identifying central hot spots in large PV farms, where precision in detecting central anomalies is critical.
U-Net-ASPP_Diag: The diagonal-focused configuration excels in detecting directional fault patterns, such as cracks or fractures that run diagonally across the PV cells. This makes it suitable for applications where directional faults are prevalent, such as identifying damage caused by environmental factors like hail or wind stress.
U-Net-ASPP_Hybrid: The hybrid approach provides the most comprehensive solution, combining the strengths of both central and diagonal dilation patterns. Its superior recall and IoU scores make it ideal for applications where detecting all potential faults is paramount, even at the cost of some false positives. This configuration is well-suited for automated inspection systems that aim to minimize manual intervention.
This study introduced an enhanced U-Net framework integrated with ASPP and customized dilation strategies for fault detection in PV systems. Empirical evaluation of four configurations U-Net-ASPP, U-Net-ASPP_Cent, U-Net-ASPP_Diag, and U-Net-ASPP_Hybrid demonstrated the advantages of tailoring segmentation models to the unique challenges of fault detection in thermal imagery.
The U-Net-ASPP_Hybrid configuration achieved the most balanced performance across all key metrics, with a 1.13% increase in F1-score, a 3.01% improvement in IoU, and a 9.86% reduction in loss compared to the baseline U-Net-ASPP. The U-Net-ASPP_Cent and U-Net-ASPP_Diag configurations also yielded notable improvements, with IoU gains of 1.18% and 1.96%, respectively, while effectively reducing false positive rates. These results highlight the effectiveness of incorporating region-specific dilation strategies, particularly for central and diagonal fault patterns.
Qualitative segmentation analysis further substantiated the superiority of the proposed configurations. In complex background scenarios, the baseline U-Net-ASPP struggled to detect faulty cells, whereas U-Net-ASPP_Cent and U-Net-ASPP_Diag partially identified fault regions. The U-Net-ASPP_Hybrid consistently outperformed the other models, demonstrating robust performance in multi-cell fault detection and improved generalization to real-world variations. However, in the most challenging cases—where both complex backgrounds and multi-cell faults are present—all configurations, including U-Net-ASPP_Hybrid, showed limitations, indicating the need for further optimization in annotation quality and dataset diversity.
For practical applications, the U-Net-ASPP_Cent model is best suited for precise detection of central anomalies, such as hot spots in large PV arrays. The U-Net-ASPP_Diag model excels in identifying directional faults caused by environmental factors like hail or wind stress. Meanwhile, the U-Net-ASPP_Hybrid model provides the most comprehensive solution, making it ideal for automated inspection systems aimed at minimizing manual intervention and enhancing PV maintenance efficiency.
Future research will focus on exploring adaptive dilation strategies, expanding dataset diversity, and improving annotation quality to further enhance model stability and generalizability. The proposed modifications lay the foundation for scalable, AI-driven fault detection systems, contributing to the advancement of sustainable solar energy infrastructure.
The datasets analyzed during the current study are available from the VRAI Photovoltaic Thermal Images Dataset repository (https://vrai.dii.univpm.it/content/photovoltaic-thermal-images-dataset). Access to the dataset requires submission of a request form to vrai@dii.univpm.it from a research institution or university-affiliated email address. Upon approval, credentials and a download link are provided. The data were used under license for the current study and are not publicly redistributable. However, access to the data for replication or verification purposes can be obtained by following the repository’s request process.
The code used for training and evaluating the proposed U-Net and ASPP configurations in this study is hosted privately on Kaggle and is available from the corresponding author upon reasonable request. The notebook can be accessed at: https://www.kaggle.com/code/awedatk/last-u-net-12-30 by approved users.
Smith, J. & Johnson, A. Impact of solar panel malfunctions on production. J. Sustain. Energy 5, 123–130. https://doi.org/10.1000/j.sustain.2020.05.002 (2020).
Article Google Scholar
Doe, R. Environmental factors affecting solar panel performance. Renew. Energy Res. 7, 234–245 (2021).
Google Scholar
Brown, T. et al. Challenges in thermal image-based solar panel detection. Int. J. Remote. Sens. 11, 987–1001 (2019).
Google Scholar
Green, P. Difficulties in identifying solar panels in thermal images. Sol. Energy Eng. 14, 55–62 (2022).
Google Scholar
White, K. Fault detection in solar energy systems: A deep learning approach. Adv. Renew. Energies 6, 45–53 (2024).
Google Scholar
Taylor, E. Machine learning schemes for anomaly detection in solar power plants. J. AI Energy Syst. 3, 300–310 (2023).
Google Scholar
Ronneberger, O., Fischer, P. spsampsps Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015, 234–241 (Springer, 2015).
Jones, B. & Kim, H. Fault detection in PV systems using convolutional neural networks. J. Sol. Energy Eng.20, 400–410 (2019).
Google Scholar
Chen, L.-C., Papandreou, G., Schroff, F. & Adam, H. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017).
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K. & Yuille, A. L. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis Mach. Intell. 40, 834–848 (2018).
Article Google Scholar
Song, H. et al. Qaga-net: enhanced vision transformer-based object detection for remote sensing images. Int. J. Intell. Comput. Cybern. 18, 133–152 (2025).
Article Google Scholar
Song, H. et al. Qaga-net: enhanced vision transformer-based object detection for remote sensing images. Int. J. Intell. Comput. Cybern. 18, 133–152 (2025).
Article Google Scholar
Sharma, R., Saqib, M., Lin, C. & Blumenstein, M. Enhanced atrous spatial pyramid pooling feature fusion for small ship instance segmentation. J. Imaging 10, 299 (2024).
Article PubMed PubMed Central Google Scholar
Huang, Z. et al. Ccnet: Criss-cross attention for semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, 603–612 (2019).
Strainic, M. G. et al. Cd55 is essential for cd103+ dendritic cell tolerogenic responses that protect against autoimmunity. The American journal of pathology 189, 1386–1401 (2019).
Article CAS PubMed PubMed Central Google Scholar
Alajmi, M., Awedat, K., Aldeen, M. S. & Alwagdani, S. Ir thermal image analysis: An efficient algorithm for accurate hot-spot fault detection and localization in solar photovoltaic systems. In 2019 IEEE International Conference on Electro Information Technology (EIT), 162–168 (IEEE, 2019).
Lee, S., Park, J. & Kim, H. Enhancing fault detection in pv systems using atrous spatial pyramid pooling integrated with u-net. J. Energy Convers. Manag. 25, 600–610 (2020).
Google Scholar
Davis, S. et al. Rcct-asppnet: Dual-encoder remote image segmentation based on transformer and aspp. IEEE Transactions on Geosci. Remote. Sens. 59, 4175–4184 (2022).
Google Scholar
Zhou, P., Chen, X. & Luo, S. Models for object detection with attention mechanisms: A review. IEEE Transactions on Neural Networks Learn. Syst. 32, 2045–2060. https://doi.org/10.1109/TNNLS.2020.2995966 (2021).
Article Google Scholar
Lee, S., Park, J. & Kim, H. Enhancing fault detection in pv systems using atrous spatial pyramid pooling integrated with u-net. J. Energy Convers. Manag. 25, 600–610 (2020).
Google Scholar
Awedat, K., Comert, G., Ayad, M. & Mrebit, A. Advanced fault detection in photovoltaic panels using enhanced u-net architectures. Mach. Learn. with Appl. 20, (2025).
Pierdicca, R., Paolanti, M., Felicetti, A., Piccinini, F. & Zingaretti, P. Automatic faults detection of photovoltaic farms: Solair, a deep learning-based system for thermal images. Energies13, 6496 (2020).
Article Google Scholar
Minaee, S. et al. Image segmentation using deep learning: A survey. IEEE transactions on pattern analysis and machine intelligence 44, 3523–3542 (2021).
Google Scholar
Zhang, W., Su, T. & Zhao, J. Directional filters for satellite image segmentation. IEEE Geosci. Remote. Sens. Lett. 17, 598–602. https://doi.org/10.1109/LGRS.2019.2940123 (2020).
Article ADS Google Scholar
Pierdicca, R. et al. Deep convolutional neural network for automatic detection of damaged photovoltaic cells. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.https://doi.org/10.5194/isprs-archives-XLII-2-893-2018 (2018).
Article Google Scholar
Sharma, R., Saqib, M., Lin, C. & Blumenstein, M. Enhanced atrous spatial pyramid pooling feature fusion for small ship instance segmentation. J. Imaging 10, 299 (2024).
Article PubMed PubMed Central Google Scholar
Lin, T.-Y., Goyal, P., Girshick, R., He, K. & Dollar, P. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis Mach. Intell. 42, 318–327. https://doi.org/10.1109/TPAMI.2018.2858826 (2020).
Article Google Scholar
Zhang, H., Luo, X. & Han, Z. Anisotropic convolutions for texture recognition. IEEE Transactions on Image Process. 28, 5726–5736. https://doi.org/10.1109/TIP.2019.2922735 (2019).
Article Google Scholar
Davis, D., Smith, J. & Johnson, R. Integration of thermal and rgb imagery for solar panel inspection. J. Photogramm 2, 20–30 (2021).
Google Scholar
Wang, S., Yang, D. M., Rong, R., Zhan, X. & Xiao, G. Pathology image analysis using segmentation deep learning algorithms. The Am. journal pathology 189, 1686–1698 (2019).
Article Google Scholar
Download references
The authors extend their appreciation to Taif University, Saudi Arabia, for supporting this work through project number (TU-DSPP-2024-253).
Computer and Information Technology Department, SUNY Morrisville College, Morrisville, NY, USA
Khalfalla Awedat & Mustafa Elfituri
Department of Computer Engineering, College of Computers and Information Technology, Taif, Saudi Arabia
Masoud Alajmi
PubMed Google Scholar
PubMed Google Scholar
PubMed Google Scholar
K.A. was the primary contributor to this work, leading the conceptualization, methodology design, model development, implementation, data analysis, and manuscript preparation. M.A. provided technical insights during the early research phases and contributed to refining the research direction. M.E. offered editorial feedback and reviewed the manuscript to improve its clarity and structure. All authors reviewed and approved the final version of the manuscript.
Correspondence to Khalfalla Awedat.
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Reprints and permissions
Awedat, K., Alajmi, M. & Elfituri, M. Leveraging U-Net and ASPP for effective fault detection in photovoltaic modules. Sci Rep 15, 21788 (2025). https://doi.org/10.1038/s41598-025-06646-x
Download citation
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1038/s41598-025-06646-x
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Advertisement
Scientific Reports (Sci Rep)
ISSN 2045-2322 (online)
© 2025 Springer Nature Limited
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.
