An Advanced Deep Learning Method for Component Failure Prediction

Recently, wind turbine operators have begun to recognize the importance of reducing O&M costs by adopting a predictive maintenance approach instead of a reactive or run to failure approach. Advances in artificial intelligence have increased the number of predictive maintenance use cases and allow automated data driven failure forecast/prediction of many wind turbine components. A typical modern wind turbine has dozens of sensors from which we can collect raw data. At Ensemble, we focus on building an asset management platform that turns raw data into valuable insights. This article discusses a specific deep learning algorithm implementation, one of many methods that we have developed and deployed for predictive maintenance.

A Deep Learning-Based Approach
At Ensemble, we use a combination of machine learning and physics-based features for failure prediction. However, it can be challenging to develop a generalized model for different purposes since feature engineering is specific to the problem scenario and the available data which varies from one case to the other. An interesting challenge in applying deep learning in the predictive maintenance domain is the ability to automatically extract the right features from the data without manual feature engineering. Deep learning algorithms have been widely used in various implementations: sentiment analysis, object detection, among other things. Sensor data coming from wind turbines has a time stamp and is often transmitted on the order of every few seconds. For time-series problem, Recurrent Neural Networks (RNN) are widely used as they are designed to capture patterns in a sequence. Another type of neural network is known as Convolutional Neural Networks (CNN), which has been the state-of-the-art in image classification for a while. Researchers have been developing CNN algorithms for years. The first generation of Inception, which was first developed in 2015, was an important milestone in CNN development. Ensemble is implementing Inception into 1D time-series problems and combining the model with an RNN. Figure 1 shows an example of such a combined deep learning model developed by Ensemble.

In image classification, CNN algorithm is usually a stack of convolution layers. On each convolution layer, a kernel (or a 2D window smaller than the input image) slides over an image and performs certain calculations, and results are passed to the next layer. The first few layers are composed of low-level features (lines, edges, etc.). As we go into deeper layers, the model reconstructs a higher-level representation of the image, such as eyes and nose in human face recognition. This excellent video by Yosinski et al. (2014) (https://youtu.be/AgkfIQ4IGaM) provides a visual overview of how CNN layers respond to an input. In time-series problem, CNN works in a similar way by scanning a sequence of features with a 1D window and extracting information from the data. And very much like a CNN, an RNN learns a range of simple patterns (slope, peak, etc.) along with more complex ones (such as long-dependency rolling mean and other complex 1D pattern). At the end of the network, an output layer is responsible for assigning a probability measure. With a certain threshold, we can transform the probability into binary classes. In our case, it is either the machine is failing or not.

Application to NASA Turbofan Dataset

This dataset contains data from 26 sensors readings from 100 different turbofan units each in training and test set. The problem is framed as a supervised-learning problem; given sensor readings at a given time, the model predicts how likely it is that the turbofan is going to fail. The actual Remaining Useful Life (RUL) was given in the test set to validate the model.

The deep learning architecture was tested in test set and showed good performance. Figure 2 shows results of the Inception based model used to predict failures in the data. The blue line is drawn based on actual failure data; 10 cycles before actual failure is marked as 1, otherwise 0. Note that the last entry does not always correspond to the unit failure. The orange line is the model calculated probability of failure. As can be seen in the figure, despite the rigid zero-one labeling, the model can identify that the failure probability is slowly transitioning over time from zero to one and is even able to predict failures 10 – 30 cycles ahead of time. The same algorithm can be applied for multiclass classification and RUL regression by modifying the last few layers. This is a very promising application of a deep learning-based algorithm for predictive maintenance. Given the right data and features, we can create value for wind operators by preventing equipment failure from occurring, or by allowing operators to plan for failures, reducing their cost and operational impact.

References:

1. Yosinski et al, “Understanding Neural Networks Through Deep Visualization”, ICML DL Workshop 2014.
2. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2015). Rethinking the Inception Architecture for Computer Vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2818-2826.
3. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A. (2015) Going Deeper with Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, 7-12 June 2015, 1-9.