Short-Term Daily Peak Load Forecasting Using Fast Learning Neural Network

Gul Muhammad Khan Department of Electrical Engineering UET Peshawar Pakistan Email: gk502@nwfpuet.edu.pk Shahid Khan Department of Electrical Engineering UET Peshawar Pakistan Email: k shahid66@hotmail.com Fahad Ullah Comp. System Engg. Department, UET Peshawar Pakistan Email: fahadullah@nwfpuet.edu.pk

AbstractLoad forecasting has been an inevitable issue in electric power supply in past. It is always desired to predict the load requirements in order to generate and supply electric power efciently. In this research, a neuro-evolutionary technique known as Cartesian Genetic Algorithm evolved Articial Neural Network (CGPANN) has been deployed to develop a peak load forecasting model for the prediction of peak loads 24 hours ahead. The proposed model presents the training of all the parameters of Articial Neural Network (ANN) including: weights, topology and functionality of individual nodes. The network is trained both on annual as well as quarterly bases, thus obtaining a unique model for each season. Keywords-Short Term Load Forecasting, Articial Neural Networks, Genetic Programming, Neuro-evolution.

Genetic Programming evolved Articial Neural Network (CGPANN). It is intended to forecast daily peak loads with load times of 1 to 7 days through the use of previous ten days data history. Section II of the paper presents the literature review with reects on necessary concepts such as load forecasting itself, Cartesian Genetic Programming, Articial Neural Networks and Neuro-Evolution. Section III describes CGPANNs in detail. Section IV reects on the Application of CGPANN to load forecasting. This section contains the Experimental setup as well as the Results and Analysis. Section V shows the concluding remarks and the future work. II. LITERATURE REVIEW A. Load Forecasting Load forecasting, like any other forecasting model is an efcient planning of supplying electricity from power stations while keeping in mind the additional capacity associated with the load escalation in future. Load forecasting is important for the reason that energy storage on large scale either isnt possible or undoubtedly problematic. Hence, it is inevitable to implement a forecasting system so that the production levels conform to the demand ones. A number of techniques are adopted in order to efciently forecast the future load for power systems [11], [22], [14], [4], [1], [5], [21]. Adaptive load forecasting [14] automatically sets the system parameters when the load condition changes. In Autoregressive model [11] load is taken as a combination of the previously used loads and hence the forecasting is achieved. Autoregressive Moving Average (ARMA) model [4] and its variant ARMA with Exogenous variable (ARMAX) model [22] implements a non-linear regression algorithm approach in load forecasting. Abbas and Arif used seven SVM models for daily peak load demand [1]. Genetic algorithms were used for the optimization of the parameters in this research. Xu Tao et al. [21] implemented SVM to predict short-term load. El-Attar et al. [5] proposed a hybrid approach that used both support vector regression and local prediction framework for load forecasting. A not very recent yet much popular idea is the deployment of Articial Neural Networks (ANNs) for load forecasting [2], [19], [7], [6], [17], [12], [18], [13], [10], [3]. Peng et al. [18] used neural networks,

I. I NTRODUCTION The price of electric unit and hence electricity itself depends upon the efcient and optimal operation of an electrical power generation plant. This efciency means a balance that must be achieved between the demand and the generation of electric load. In order to achieve this balance, there must be a forecasting system which can predict the load requirement in future or any time for that matter. The efcient operation of a power system needs proper fuel scheduling and maintenance which can be achieved by using a sophisticated forecasting system that determines the electric load demand at any time in future. There are a number of forecast techniques developed for electric load forecasting. Some of these techniques are more conventional and hence less favored. ANN based forecasting techniques are amongst the modern approaches towards load forecast. It is indeed a tedious job to nd an optimal neural network to efciently predict the load requirements. Manual methods are being used hitherto, to obtain optimized ANNs for electric load forecasting. Using evolutionary techniques would make possible the generation of the optimized networks in an automated way with better accuracy. Load forecasting is categorized into three types: long term, medium term and short term load forecasting. The long term load forecasting deals with the forecast extending for duration longer than a year. For short term the forecast time ranges from an hour to a week. In this paper ANN based model has been developed by using the evolutionary technique known as Cartesian

S/N 1 2 3 4 5 6 7 8

Method Locally Linear Model Tree Support Vector MachinesGenetic Algorithm (SVM-GA ) Autonomous ANN Floating Search + SVM Prediction Framework + SVR Feed-forward ANN SOFNN + Bi-level Opt. MLP-NN + Leve-Marq.

MAPE 1.98% 1.93% 1.75% 1.70% 1.52% 1.42% 1.40% 1.60%

Reference [10] [1] [6] [21] [5] [17] [12] [3]

Table I C OMPARISON TABLE FOR P REVIOUS R ESULTS

which have the capability of learning along with the capability to deal with the nonlinear relationships between load and the factors affecting it directly from the historical data and without the selection of a given model in advance. Senjyu et al. [2] proposed short-term load forecasting using a neural network that would produce a simple correction value. They would add that value to the data associated with a similar day; a day which exhibits the same load conditions as of the forecast day. Pang Qingle et al. [19] proposed load forecasting based on rough set and neural network. Ghomi, Mohammad et al. [7] used ANN for peak load forecasting without weather information. They evaluated two popular algorithms of Multi Layer Perceptron (MLP) and Radial Basis Function (RBF) and found that MLP bears more favorable results in their forecasting system. Matsui et al. [13] proposed peak load forecasting by using analyzable structure ANN. Koushki et al. [10] presented neuro-fuzzy model for short-term load forecasting. Ferreira et al. [6] proposed two different procedures in a coupled way for the solution of Neural Network structure and input selection for short-term load forecasting. Daniel Ortiz-Arroyo et al. [17] used a simple Articial Neural Networks approach for load forecasting with a well populated set of attributes. Huina Mao et al. [12] presented a system for load forecasting that combined both self-organizing fuzzy neural network (SOFNN) and a bi-level optimization method. Amjaday et al. [3] presented a hybrid model that used ANN, preforecast mechanism and evolutionary algorithm to predict the midterm peak load. Table I shows the forecasting results for most of the previous methods aforementioned with most of the methods forecasting the load only for a month. In order to evaluate the performance of the forecasting algorithm, Mean Absolute Percentage Error (MAPE) is used to evaluate the prediction accuracy. MAPE is widely used standard for measuring load forecast performance. MAPE is given by: |LF LA | 1 x 100 MAPE(%) = N i=1toN LA Where LF is the forecasted load, LA is the actual load and N is the number of season days. B. Articial Neural Networks Articial Neural network, a bio-inspired approach which builds up from highly interconnected processing elementsmimics a real neural network naturally present in any living

organism. Self training and learning are the key aspects of an ANN. ANN is a layered model consisting of three neural layers; input layer, hidden layer and output layer. A connection between two neurons carries a weight; a specic activation function belonging to each neuron generating outputs based on given inputs. Each unit in articial neural network performs an elementary task i.e. to receive input from adjoining unit or external sources and generate output based on the activation function to other units. The network intrinsically behaves in a parallel fashion meaning that many nodes do the computation simultaneously. A unit in the network can fall in any of the three categories: an input unit receiving data from outside the network, an output unit sending data outside ANN and nally the hidden unit connected to the units inside the network. ANN can be classied into three types each of which exhibit different behavior: Radial Basis Function Network (FBFN), Multi-Layered Perceptron Network (MLP), and Recurrent Neural Network (RNN). MLPs are the most popular neural networks. It is mandatory to perform a sequence of steps before an ANN based model is available for functioning. Among those steps training (or learning) and testing are two vital steps essentially required before making an ANN operational. Because of the fact that ANN doesnt rely on human experience and it is a self-adapted and self-learnt system, it can be most certainly used to obtain promising results in a changing environment. C. Cartesian Genetic Programming(CGP) CGP is a specic type of genetic programming developed by Thomson and Miller [15]. It is meant for the evolution based design of the feed forward digital circuits. In CGP, A genotype consists of genes representing nodes. Each node has its inputs and a function that it performs on its inputs. The outputs from the intermediary nodes can be the inputs for the other nodes. The output of the system can be the intermediary input i.e. the output from any other node or even a system input itself. The genotype or also called chromosome has a constant length and it is represented by a stream or array of integers which represents the nodes (their functions and inputs) as well as the outputs. The phenotype is obtained by following the referenced links in the directed graph meaning that those genes are selected which make the path between the program inputs and outputs. The tness of each CGP program determines whether it should stay in population and produce offspring or should be removed. Offspring are obtained by mutating either node function or the connection between nodes, or the output genes. D. Neuro-Evolution The use of articial evolution with ANNs is known as Neuro Evolution (NE). In NE every aspect of an ANN is evolved including network topology, node function and connection weights. A genetic algorithm is used with the ANN

where the algorithm acts as the genotype and the network as the corresponding phenotype. The genotypic evolution is continued until the desired phenotypic behavior is obtained. The encoded attributes in the genotype can affect the search space of the solution. Sometimes only few parameters of the system are evolved and at times, it is possible to evolve a group of them. When only weights of the system are evolved by keeping the topology xed, the network solution space is restricted. Hence, a new solution to the problem cant be achieved because of the fact that evolution has to work in a more conservative environment. Topology and Weight Evolved Articial Neural Network (TWEANN) on the other hand can evolve both weights and topology and hence can boost the network efciency [23]. A number of neuroevolutionary techniques are presented in the past with some evolving weights, some topology and some evolving both [23]. Xin, Yao further elaborated through experimentation the fact that evolution of both topology and weight produces much better results than evolution of weights or topology alone. Symbiotic, Adaptive Neural Evolution (SANE) uses the simultaneous evolution of neuron population along with the network topologies representing the blue prints [16]. An extension of SANE called Enforced Sub-Population (ESP) evolves a subpopulation of hidden layer neurons rather one population of neurons [8]. Neuroevolution of Augmenting Topologies (NEAT) developed by Stanley solved three major problems in neuroevolution including the tracking of genes with historical markings that allow easy crossover between different topologies, protecting innovation via speciation and nally the starting from basic simple structure and complexifying over time as the generations pass. NEAT had a performance advantage over many previous NE algorithms [20]. Cooperative Synapse Neuroevolution (CoSyNE) coevolves the synaptic weight. They have used multipoint crossover and probabilistic mutation based on Cauchy distributed noise [8]. III. CARTESIAN GENETIC PROGRAMMING EVOLVED ARTIFICIAL NEURAL NETWORK (CGPANN) CGPANN [9], a new Neuroevolutionary algorithm introduced recently encode all the attributes of the neural networks including weights, topology and node functions in the genotype. It evolves all these parameters of the network until best combination of them is achieved. This paper implements the feed-forward CGPANN also called FCGPANN. The results achieved in [9] reveal that this algorithm learning rate is dramatically faster in contrast to its contemporary NE techniques. CGPANN and hence FCGPANN uses (1+ ) evolutionary strategy to generate the population of next generation. The developed network in this case has neurons that are not fully connected. Also, another fact about CGPANN is that the program inputs are not connected to every neuron in the input layer which makes

W1X I1 X

I2 W2X X

OX

Figure 1.

Representation of FCGPANN Node

I0W01I2W21F1

I2W22I0W02F1 (a)

I4W43I1W13F2

4 5

I0 O0
W01 W02 W43

I1
W21

F1
W22

F1
W13

F2

I2 (b)

O1

Figure 2.

(a) FCGPANN Genotype (b) Architecture of the Genotype

it peculiar in comparison to a conventional ANN. Such features in this algorithm makes possible for it to generate topologies that are more efcient in terms of hardware implementation and time. An FCGPANNs neuron (node) includes inputs, weights, connections and function. Inputs can be from the program or an intermediary input that is an output from a node. Randomly generated weights are used ranging from -1 to +1. Outputs can be from nodes or directly from program inputs. For each node, inputs and weights are multiplied and summed up. The sum is then forwarded to an activation function. The output of a node can either be an intermediary input or output of the program or it is possible that it is never used leading to the formation of a junk node. Figure 1 shows a FCGPANN based node with I1 and I2 as its inputs and W1X and W2X being weights of the connections. Whereas X represents the node number. Figure 2(a) shows the genotype of a FCGPANN with three nodes having corresponding inputs, weights and functions. The nal square represents the output of the system. The dotted boundary of the rst node reveals that it doesnt take part in the circuit and hence it is a junk node. Figure 2(b) shows the phenotype of the genotype in Figure 2 (a).

IV. APPLICATION OF CGPANN TO LOAD FORECASTING A. Experimental Setup The proposed load forecasting model is based on the past daily peak loads (historical consumption data) as the only input variable. The training is performed by dividing the training data into four types, one for each of the four seasons i.e. winter, spring, summer and autumn. The whole year data has also been used for training the corresponding model. The load at a particular instant can get affected by many factors but here we are using only the loads of previous ten days as the inputs to the model. This data is used to predict the peak load of the eleventh day and onwards. A random population of ten CGPANN networks is generated at the start of the process. The activation functions used is log-sigmoid. The number of inputs per node is 5. Mutation rate used in this case is 10%. The number of CGPANN rows in this case is one, thus the number of columns are equal to the number of nodes. Input(s) are applied and the MAPE value is measured for each of the ten Networks. These MAPE values are compared with each other and the network with best MAPE value is selected for promotion to the next generation. This very network is then used to produce nine other networks by mutation of parents genotype. This process continues until the maximum number of generations has arrived or the MAPE has reduced to zero. In this case, all experiments are run for two million generations during its training phase. B. Results And Analysis In order to evaluate the performance of algorithm the data is obtained from the United Kingdom national grid in the form of hourly load from which peak load data was extracted on daily bases1 . Each year data has been divided into four seasons along with the whole year data for a period of four years. Out of four years data only the rst year data (2006) is used for training of algorithm and the rest of the data is used for testing and validation. MAPE is used as a performance criterion. We have trained the network for every individual season and for the whole year. A separate model is obtained for each seasonal and annual data. At the end the best obtained genotype is transformed to phenotype and tested on the data of 2007, 2008 and 2009 for all the seasons and complete year. For both training and testing purposes, various morphological alterations have been made for different experiments in order to obtain an optimum network that can achieve the best of forecasting results. The morphology of the network is altered by changing the principle aspect of the network; the number of nodes constituting the whole network itself. By varying the number of nodes in the network, the corresponding size of network varies and thus affecting the possible types of network morphologies that can be developed. A number of runs
1 http://www.tutiempo.net

Model Winter Spring Summer Autumn Annual

MAPE for Different Number of Nodes 50 100 150 200 250 3.11 3.11 3.14 3.13 3.13 3.44 3.31 3.43 3.44 3.50 2.37 2.26 2.53 2.50 2.36 1.44 1.38 1.45 1.54 1.55 3.55 3.38 3.58 3.54 3.58

300 3.11 3.47 2.57 1.52 3.38

Table II T RAINING RESULTS FOR DIFFERENT COMBINATIONS OF NODES

Model Winter

Spring

Summer

Autumn

Annual

Year 2007 2008 2009 2007 2008 2009 2007 2008 2009 2007 2008 2009 2007 2008 2009

MAPE for Different Number of Nodes 50 100 150 200 3.32 3.34 3.39 3.48 4.43 4.43 4.47 4.50 3.74 3.77 3.81 3.94 3.96 4.00 3.77 3.96 3.24 3.18 3.28 3.24 4.05 4.14 3.69 4.06 2.23 2.19 2.39 2.35 2.82 2.72 2.99 2.94 3.39 2.95 3.86 3.59 1.78 1.71 1.80 1.83 2.31 2.16 2.35 2.46 1.78 1.49 1.83 1.93 3.36 3.16 3.40 3.38 3.85 3.61 3.91 3.85 3.74 3.52 3.82 3.76

250 3.48 4.50 3.94 3.79 3.33 3.74 2.26 2.78 3.37 1.84 2.46 1.97 3.40 3.92 3.81

300 3.28 4.39 3.49 4.03 3.30 4.16 2.47 3.02 3.63 1.82 2.43 1.91 3.16 3.63 3.54

Table III T ESTING RESULTS FOR DIFFERENT COMBINATION OF NODES

of the training and testing experiments are performed on the network sizes ranging from 50 to 300 nodes i.e. 50, 100, 150, 200, 250, 300. It is worth mentioning that the ultimate phenotype will not necessarily use all the nodes of the network, and statistics showed that only 5 to 10 percent of nodes are used to produce ultimate phenotype [9]. Table II and III summarize the results for both training and testing respectively. The MAPE has been calculated for different experiments in each of which the number of nodes in the network is varying. As the weather in UK is relatively stable in autumn, the forecasting accuracy is far better as clear from the table III. As the results suggest, with 100 nodes constituting the network the MAPE values are favorable. The forecasting error for 200 and 250 node networks reveal the poor performance of the networks. Figure 3 shows the variation in the best estimated and actual load for all the combinations, clearly demonstrating the capabilities of CGPANN to predict peak load accurately. Figure 3 depict the best cases of forecasting for a number of different combinations of nodes respectively for autumn seasons. In winters, and particularly in 2008, the forecasting accuracy is the poorest and hence MAPE values higher. Table III shows the seasonal and annual MAPE values obtained by testing the corresponding models. From the table it is evident that the results are best for the autumn 2009 being less variation in peak load. Figure 4 represents the corresponding curves for both actual and estimated peak load for autumn season, 2009. Both the curves exhibits close resemblance which shows the index to which the prediction has been successful. Figure 5 and 6 shows corresponding frequency plots of the peak load for the estimated and actual load respectively. The frequency spectrum of both the graphs reveals the extent to which

Figure 3. Peak load variation graphs showing the best estimations against the Actual peak load for various number of nodes (a) 50, (b) 100, (c) 150, (d) 200, (e) 250, (f) 300

0.9

Autumn 2009
Estimated 40 35 30 Spectrum Amplitude 25 20 15 10 5 Actual

Actual Autmn 2009

0.85

0.8 Peak Load

0.75

0.7

0.65

0.6 0 0.55 0 10 20 30 40 50 1 Number of Days 11 21 31 41 51

Peak Load Frequency

Figure 4. 2009

Comparison curve for actual and estimated load for autumn

Figure 6. 2009

Frequency distribution of the estimated peak load for autumn

Estimated Autumn 2009

40 35 30 Spectrum Amplitude

1.1

Winter 2008
Estimated Actual

0.9 Peak Load 1 11 21 31 41 51 25 20 15 0.7 10 5 0 0.5 0 Peak Load Frequency 20 40 60 Number of Days 80 100 0.6

0.8

Figure 5. 2009

Frequency distribution of the estimated peak load for autumn

Figure 7. Comparison curve for actual and estimated load through the winter 2008

the forecasting has been successful. Another important fact about the data in Table III is that the forecasting in this case is based on 10 days that is the daily forecast is made for next 50 days based on the previous 10 days actual load. Because load forecasting intrinsically depends on the geography and hence the weather conditions of an area, the results are better for autumn because the weather is steadier in that season in United Kingdom. A straight forward comparison isnt

possible with Table I because the results in Table III follow a different approach yet the performance can be analyzed. Figure 7 shows both actual and forecasted peak load for winter 2008 (4 months). The results are based on the load data from UK national grid and as obvious from the curve, the load prediction is poor in winter due to the whimsical and unpredictable nature of weather in Great Britain in that particular season. Winter 2008 shows the worst case

of results-MAPE obtained is 4.50% -as shown in Table II. V. CONCLUSION AND FUTURE WORK In this paper a novel Neuro-evolutionary technique called CGPANN has been used for the daily peak load forecasting. Electric Load data of different durations is used to train the ANN on both quarterly and annual basis and then the testing is done for the peak load forecasting. The results obtained in all autumn seasons are generally better. The results, as from Table III show the efciency of this approach in steadier weather conditions as weather parameters arent taken into account in this research. The various curves reveals the forecasting accuracy in both time and frequency domains. As this approach only use the peak load as an input to the articial neural network and doesnt take into account other relevant parameters like atmospheric temperature, humidity, rain and pressure, thus, there is a potential research to be conducted highlighting the effect of these parameters on load forecasting systems in future. R EFERENCES
[1] S. Abbas and A. M. Electric load forecasting using support vector machines optimzed by genetic algorithm. In Proc. CIMCA 05, pages 395399, 2006. [2] A. J. Al-Shareef, E. A. Mohamed, and E. Al-Judaibi. One hour ahead load forecasting using articial neural network for the western area of saudi arabia. 2008. [3] N. Amjady and F. Keynia. Mid-term load forecasting of power systems by a new prediction method. Energy Conversion and Management, 49(10):2678 2687, 2008. [4] E. Barakat, M. Qayyum, M. Hamed, and S. A. Al-Rashed. Short-term peak demand forecasting in fast developing utility with inherent dynamic load characteristics. IEEE Transactions on Power Systems, 5:813824, 1990. [5] E. E. El-Attar, J. Goulemas, and Q. Wu. Forecasting electric daily peak load based on local prediction. In Power and Energy Society General Meeting, pages 16, 2009. [6] V. H. Ferreira and A. P. Alves da Silva. Toward estimating autonomous neural network-based electric load forecasters. IEEE Trans. Power Systems, 22(4):15541562, 2007. [7] M. Ghomi, M. Goodarzi, and M. Goodarzi. Peak load forecasting of electric utilities for west province of iran by using neural network without weather information. In Proc. UKSIM 10, pages 2832, 2010. [8] F. Gomez, J. Schmidhuber, and R. Miikkulainen. Accelerated neural evolution through cooperatively coevolved synapses. J. Mach. Learn. Res., 9:937965, 2008. [9] M. M. Khan, G. M. Khan, and J. F. Miller. Evolution of neural networks using cartesian genetic programming. In IEEE CEC10, pages 18, 2010.

[10] A. Koushki, M. maralloo, B. Hashemitabar, and C. Lucas. Load forecasting with the aid of neuro-fuzzy modeling. In Proceeding of Seventh International Conference on CSIT, 2009. [11] K. Liu, S. Subbarayan, R. R. Shoults, M. T. Manry, C. Kwan, F. I. Lewis, and J. Naccarino. Comparison of very short-term load forecasting techniques. IEEE Transactions on Power Systems, 11(2):877882, 1996. [12] H. Mao, X.-J. Zeng, G. Leng, Y. Zhai, and J. A. Keane. Short and mid-term load forecasting using a bilevel optimization model. IEEE Transactions on Power Systems, 24(2):1080 1090, 2009. [13] T. Matsui, T. Iizaka, and Y. Fukuyama. Peak load forecasting using analyzable structured neural network. IEEE Power Engineering Society Winter Meeting, 2:405, 2002. [14] J. R. Mcdonald, K. L. Lo, and P. M. Sherwood. Application of short-term adaptive forecasting techniques in energy management for the control of electric load. Transactions of the Institute of Measurement and Control, 11:7991, 1989. [15] J. F. Miller and P. Thomson. Cartesian genetic programming. In Proc. of the 3rd European Conf. on Genetic Programming, volume 1802, pages 121132, 2000. [16] D. Moriarty. Symbiotic Evolution of Neural Networks in Sequential Decision Tasks. PhD thesis, University of Texas at Austin, 1997. [17] D. Ortiz-Arroyo, M. K. Skov, and Q. Huynh. Accurate electricity load forecasting with articial neural networks. In Proc. CIMCA 05, pages 9499, 2005. [18] T. Peng, N. Hubele, and G. Karady. Advancement in the application of neural networks for short-term load forecasting. IEEE Transactions on Power Systems, 7:250 257, 1992. [19] P. Qingle and Z. Min. Very short-term load forecasting based on neural network and rough set. In Proc.ICICTA 10, pages 11321135, 2010. [20] K. O. Stanley and R. Miikkulainen. Evolving neural network through augmenting topologies. Evolutionary Computation, 10(2):99127, 2002. [21] X. Tao, H. Renmu, W. Peng, and D. Xu. Input dimension reduction for load forecasting based on support vector machine. In IEEE DRPT Conference, pages 510513, 2004. [22] H.-T. Yang, C.-M. Huang, and C.-L. Huang. Identication of armax model for short term load forecasting: an evolutionary programming approach. IEEE Transactions on Power Systems, 11:403408, 1996. [23] X. Yao. Evolving articial neural networks. In Proceedings of the IEEE, volume 87(9), pages 14231447, 1999.