Performance of Artificial Neural Networks and Fuzzy Inference

Systems for Predicting Concrete Compressive Strength


Desempeño de las Redes Neuronales Artificiales y los Sistemas de

Inferencia Lógicos para Predecir el Esfuerzo de Compresión del

Hormigón


JORGE LUIS SANTAMARIA CARRERA
Universidad Central del Ecuador, Ciudadela Universitaria, Quito - Ecuador
jsantamaria@uce.edu.ec
DOI: HTTP://DX.DOI.ORG/10.24133/VINCULOSESPE.V3I2.1549


ABSTRACT

Artificial neural networks (ANN) have been traditionally utilized for developing prediction models based on experimental data; however, fuzzy logic theory is a novel tool that could also be applied as well in this cases. The present study compares the performance of two different approaches; namely, ANN and fuzzy inference systems (FIS) for predicting concrete compressive strength. Adaptive neuro fuzzy inference systems (ANFIS) was the technique based on input – output experimental data that was used to create two Sugeno type fuzzy models for estimating concrete compressive strength and then compared with several ANN prediction models created through different methods. Subtractive clustering method was the clustering procedure for establishing the number of membership functions and fuzzy rules. Predicted data resulting from all models were presented in a comparative manner and validation analyses were conducted in order to observe the performance of ANN and ANFIS. The results indicate that both fuzzy models performed very well when estimating concrete compressive strength (i.e., all R2 values are greater than 90%) while for ANN, only one ANN method had an R2 value greater than 90%.

Keywords: Compressive strength, Concrete, Fuzzy logic, Neural networks, Prediction models

RESUMEN

Las redes neuronales artificiales (ANN) han sido tradicionalmente utilizadas para desarrollar modelos de predicción basados en datos experimentales; sin embargo, la teoría de lógica difusa es una herramienta que podría ser aplicada en estos casos. El presente estudio compara el desempeño de los dos enfoques; ANN y los sistemas de inferencia lógicos (FISs) para predecir el esfuerzo de compresión del hormigón. ANFIS (“Adaptive neuro fuzzy inference systems”) fue la técnica basada en datos experimentales entrada – salida que fue utilizada para crear dos FISs del tipo Sugeno para estimar el esfuerzo de compresión del hormigón y luego compararlos con varios modelos de ANN creados usando diferentes métodos. Los datos estimados resultantes de todos los modelos fueron presentados de una manera comparativa y análisis de validación fueron realizados para observar el desempeño de ANN y ANFIS. Los resultados indican que ambos modelos difusos tienen un muy buen desempeño cuando predicen el esfuerzo de compresión del hormigón (valores de R2 mayores a 90%) mientras para ANN, solamente un modelo tuvo un valor de R2 mayor a 90%.

Palabras Clave: Esfuerzo de compresión, Hormigón, Lógica difusa, Redes neuronales, Modelos de predicción





INTRODUCTION

Several studies have been done to develop prediction models for estimating concrete compressive strength using experimental data through artificial neural networks (ANN). For instance, Kim et al. (2004) applied ANN for estimating concrete compressive strength based on mix proportions by using the results of 28-day uniaxial concrete compressive test results, concluding that ANN are very effective. Also, Kostic & Vasovic (2015) succeeded in creating three ANN models with different learning algorithms and six number of hidden nodes for estimating concrete compressive strength by using different water – cement ratios, age of testing and freeze and thaw cycles. However, fuzzy set theory, a novel theory, is being explored for predicting new observations based on experimental input – output data. This study utilizes fuzzy set theory to develop prediction models for estimating concrete compressive strength using experimental data through adaptive neuro fuzzy inference systems (ANFIS) and compares the performance of neural network-based models with fuzzy inference systems (FISs).

An experimenter or analyst usually uses experimental data obtained through testing to model or predict results when dealing with complex systems, and FISs may be applied in these cases (Ross, 2010). Some methods are available to develop both membership functions (MFs) and if – then rules while others work with predetermined rules in order to model a system. Passino & Yurkovich (1998) mentioned several techniques for fuzzy identification and estimation including Batch Least Squares (BLS), Recursive Least Squares (RLS), Learning from Example (LFE), Gradient Method (GM), Modified Learning from Example (MLFE), and Clustering Method (CM).

Nevertheless, the aforementioned methods or techniques for creating MFs and fuzzy rules (i.e., if-then rules) are not the only ones available nowadays. Jang (1993) proposed a method called Adapted-Network-Based Fuzzy Inference System (ANFIS) for constructing a FIS by developing if – then rules and MFs based on input – output data tuples. ANFIS is a neuro-fuzzy model that utilizes the advantages

of artificial neural networks by allowing fuzzy systems to learn through a hybrid learning algorithm (Jang et al.,1997).

Subtractive clustering method (Chiu, 1994) is a very fast and efficient clustering one-pass procedure algorithm that does not involve any iterative nonlinear optimization. It works well for partitioning the input space (i.e., experimental data) in order to find the initial structure of a fuzzy inference system that will be used by ANFIS. The number of cluster centers identified became the number of MFs and if – then rules.

Kostic & Vasovic (2015) developed three neural network-based models based on experimental data for predicting concrete compressive strength by using different learning algorithms (i.e., Levenberg–Marquardt, scaled conjugate gradient, one-step-secant back-propagation), and such data and results will be used for developing two Sugeno type fuzzy models and studying their performance when predicting new observations. In other words, two different approaches will be evaluated: (1) ANN and (2) FISs developed through ANFIS. The main goal of this study is to analyze and compare the performance of ANN models and FISs for estimating concrete compressive strength based on experimental data.

FUZZY MODELING

Yager & Filev (1994a) pointed out that there are two approaches for developing fuzzy models; namely, direct approach and system identification. The first one consists of creating fuzzy inference systems based on experts’ knowledge. For instance, an expert is in charge of partitioning the data, creating if – then rules, choosing an appropriate reasoning method and evaluating the model.

On the other hand, system identification is a method for developing a fuzzy model based on input – output data (e.g., experimental data), and this approach results in developing a Sugeno FIS. System identification can be divided into: (1) structure identification and (2) parameter identification (Sugeno & Yasukawa, 1993).





The main goal of structure identification is to determine the partitions of the input and output data points, if – then rules and the number of rules. Parameter identification involves adjusting the parameters of the model so that output errors are minimized. In this case ANFIS will be used for parameter identification which takes advantage of neural networks and fuzzy set theory.

System identification
Structure Identification

There are several methods for clustering (i.e., classifying data) such as fuzzy C means. This method is a very popular method proposed by Bezdek (1981) and is based on iterative optimization. The objective function is intended to minimize Euclidean distances between a data point a its cluster center and to maximize the Euclidean distance between cluster centers (Ross, 2010). Mountain method, a simple and effective clustering algorithm, is another procedure, and it was proposed by Yager and Filev (1994b). This method is based on gridding the data space of each input and output variable. A grid point with many surrounding points has a high potential value and is chosen as a cluster center. The main drawback is that it is very computational intensive when the number of inputs increase.

Subtractive Clustering

Subtractive clustering was proposed by Chiu (1994) and it is a variation of the mountain method. The main difference is that any data point is a potential cluster center instead of a grid point and the number of grid points is equal to the number of data points, making this method reduce computational load significantly even for a big number of input variables. Also, this method is fast because it does not involve iterative nonlinear optimization.

Subtractive clustering method will be used to determine the number of if – then rules and membership functions and then linear least squares to compute each

rule’s equation. As mentioned before, each point is considered as a potential cluster center and the potential of a data point xi is defined by Eq. (1).

Where alpha (α) is defined by Eq. (2):

The radius of influence of a cluster center (ra) is a positive constant. This parameter is specified by the user and a large ra produces fewer clusters and vice versa. This radius can be adjusted based on the results of the model. It can be inferred from Eq. (1) that a data point with many neighbors has a high potential value (Pi). After computing the potential of each point, the point with the highest potential value is assigned to be the first cluster center. Then the potential values of all remaining data points are updated with respect to the first cluster according to Eq. (3)

Where beta (ß) is defined by Eq. (4), x1* is the first cluster center, P1* is the corresponding potential value and rb is a positive constant defined as the radius of the neighborhood having measurable reductions. This parameter can be computed by using Eq. (5) where η is called the squash factor. Typically, a good choice for rb is when η =1.5 to ensure that cluster centers are not too close to each other





Once all potential values of the remaining data points are calculated using equation (3), the data point with the highest potential become the second cluster center. Then the potentials of the remaining data points are reduced with respect to the second cluster center and so forth according to Eq. (6), where xk* is the kth cluster center and Pk* is the corresponding potential value.

All cluster centers identified by subtractive clustering method determines the number of rules and antecedent membership functions that will be utilized by ANFIS during the parameter identification process.

Parameter Identification

Matlab platform was used for performing the parameter identification process through ANFIS, and the resulting FIS has the following characteristics. It is a Sugeno-type FIS with Gaussian MFs, the number of MFs is equal to the number of fuzzy rules and weighted average defuzzification method is used to compute a crisp result.

Adaptive Network Fuzzy Inference System (ANFIS)

ANFIS uses data available from a system such as experimental data to tune membership functions and create rules for a Sugeno FIS. A characteristic of this model is that it gives crisp outputs or functions for each rule. Figure 1 shows how a Sugeno system and an ANFIS model with two rules and two inputs (e.g., water – cement ratio (W/C) and age of concrete (A)), and one output (e.g., compressive strength (S)) work. There are five layers in ANFIS architecture where specific operations are carried out. Layer 1 is an input layer where the degrees of membership are calculated from Gaussian MFs. Layer 2 is a product layer where firing strengths (wi) are calculated by multiplying all degrees of membership that arrive to a specific node. Layer 3 is a normalization layer for firing strengths. Layer 4 is the layer where the

outputs of each rule are evaluated, and layer 5 is the layer where the total output is computed by using weighted average method. Finally, linear least squares estimation is used to determine each rule’s consequent equations.

METHODOLOGY

Experimental Data

Table 1 shows the experimental input – output data obtained from Kostic & Vasovic (2015) that will be used to create two Sugeno (FIS) by





subtractive clustering method and ANFIS. The letter A denotes the age of concrete, F/T is the number of freeze/thaw cycles, and S is concrete compressive strength. Training data is the data used to generate the ANFIS model while testing and checking data are used for verifying the performance of the model.

Table1
Concrete compressive strength

Subtractive clustering parameters

The following are the parameters used to partition the data: accept ratio (e) = 0.5, reject ratio (ε) = 0.15, range of influence (ra) = 0.4 and a squash factor (n) = 2.0; where ε is a threshold above which a data





point is accepted to be a cluster center and ε is a threshold below which a data point will be rejected as a cluster center. Once such parameters are applied, cluster centers (ci) and sigmas (σi) are calculated by the subtractive clustering method. They become the parameters of each Gaussian MF (see Figure 2) and the number of rules for ANFIS. In this case 11 cluster centers were found and thus 11 MFs and rules were generated.

Figure 2. Parameters of a Gaussian membership function

ANFIS model

As mentioned before, ANFIS consists of five layers and the membership values of each input are computed in layer 1. Figure 2 shows a typical Gaussian MF used by ANFIS and Table 2 contains all Gaussian parameters for the 11 MFs. When all parameters of Gaussian MFs are defined, it is possible to compute the corresponding membership value for each input by using Eq. (7). Table 3 comprises all calculated membership values of each input. Table 4 summarizes the results of the consequent parameters of each function f_i while Table 5 contains the values of firing strengths, normalized firing strengths and weighted output of each rule (i.e., the operations performed in layers 2,3, and 4).

Table 2.
Gaussian membership function parameters

Table 3.
Membership values





Table 4.
Consequent parameters

Table 5.
Results of layers 2, 3, and 4

RESULTS

Kostic & Vasovic (2015) utilized artificial neural networks ANN to develop models for predicting compressive strength of basic concrete mixture by using three different learning algorithms with six hidden nodes: Levenberg–Marquardt, scaled conjugate gradient, and onestep-secant OSS as mentioned before and the results of strength are comprised in columns (1), (2), and (3) of Table 6 respectively. Similarly, the results of applying FISs with 11 and 27 rules through subtractive clustering method and ANFIS are depicted in column (4) and (5) of Table 6 respectively. The values in column (5) correspond to another ANFIS model created by using the following parameters for clustering input data: accept ratio (ε) = 0.525, reject ratio (ε) = 0.15, range of influence (ra) = 0.35 and a squash factor (η) = 1.25. These parameters produced 27 clusters and hence 27 Gaussian MFs and rules, and this model was created to study the effect of the number of MFs and rules on FIS outputs. The rows that are in gray are the data tuples that will be used for validating FIS models.

Table 6.
Results of ANN and FISs for concrete compressive strengt





MODEL VALIDATION

Predicted versus experimental data plots were used for model validation. Training data should not be used to see how well a FIS model performs; instead, checking or testing data should be used (Tesfamariam, 2007). Thus in order to test the performance of the two different methods (i.e., ANN and FISs) checking data will be used as validation data for the two FISs to compare with the results obtained by Kostic & Vasovic (2015). Table 7 summarizes the results of R squared (R2) values and the corresponding standard error (SE) for each of method.

Table 7
R squared values and Standard Errors





CONCLUSIONS

The results of the study indicate that FISs perform slightly better than ANN when estimating concrete compressive strength. It can be inferred from Table 7 that all R2 values are high, meaning that all models perform well when predicting responses for new observations (i.e., concrete compressive strength).

All FIS models have R2 values greater than 90% while only the ANN model using Levenberg–Marquardt learning algorithm has an R2 greater than 90% (i.e., 91.5%), suggesting that FISs works very well when mapping input – output data even with a few number of rules (i.e., 11 rules). Regarding errors, FIS models produce less SE compared with the best ANN model (i.e., 2.795 and 2.816 are lower than 2.818).

A FIS is a very efficient technique when mapping input – output data. In this project, 45 observations were used to train the model and only 11 rules produced an acceptable model with a high R2 value of 90.1%. Increasing the number of rules to try to minimize the standard error was not effective. The results show that there is not a significant improvement when using more rules (i.e., 27 rules) are utilized. Therefore, the FIS model with 11 rules should be maintained for estimating new data.

Another conclusion that can be derived from this study is that FISs are not “black box” systems like ANN where complex calculations are performed inside to generate outputs. Instead, FISs such as Sugeno type FISs allow the experimenter or researcher to gain some knowledge of the system due to use of the deductive form. This is accomplished by using natural language expressions such as fuzzy rules (i.e., IF premise THEN conclusion).

Moreover, the number of rules is an important parameter of any FIS and depends on the complexity of the system. In addition, the number of rules should be paid special attention and minimized by trial and error so that output errors stay within desired ranges. In other words, the parameters for the clustering method can be adjusted accordingly in order to keep the number of rules small. Thus, as a rule of thumb, the number of rules should be as few as possible, meaning that the less number of rules, the better.

REFERENCES

Bezdek, J. C. (1981). Pattern Recognition with Fuzzy Objective Function Algorithms. In Advanced Applications in Pattern Recognition; Advanced applications in pattern recognition. Retrieved from Springer Link http://dx.doi.org/10.1007/978-1-4757-0450-1

Chiu, S. L. (1994). Fuzzy model identification based on cluster estimation. Journal of Intelligent & Fuzzy Systems, 2(3), 267-278.

Jang, J.-S. R., Sun, C.-T., & Mizutani, E. (1997). Neuro-fuzzy and soft computing : a computational approach to learning and machine intelligence. Upper Saddle River, NJ :: Prentice Hall.

Jang, J. S. R. (1993). ANFIS: Adaptive-Network-Based Fuzzy Inference System. IEEE transactions on systems, man, and cybernetics., 23(3), 665-685.

Kim, J. I., Kim, D. K., Feng, M. Q., & Yazdani, F. (2004). Application of neural networks for estimation of concrete strength. Journal of Materials in Civil Engineering, 16(3), 257-264.





Kostic, S., & Vasovic, D. (2015). Prediction model for compressive strength of basic concrete mixture using artificial neural networks. Neural Computing and Applications, 26(5), 1005-1024.

Passino, K. M., & Yurkovich, S. (1998). Fuzzy control. Menlo Park, Calif. :: Addison-Wesley.

Ross, T. J. (2010). Fuzzy logic with engineering applications (3rd ed. ed.). Chichester, U.K. :: John Wiley.

Sugeno, M., & Yasukawa, T. (1993). A fuzzy-logic-based approach to qualitative modeling. IEEE Transactions on fuzzy systems, 1(1), 7-31.

Tesfamariam, S. (2007). Adaptive Network–Fuzzy Inferencing to Estimate Concrete Strength Using Mix Design. Journal of Materials in Civil Engineering, 19(7), 550-560. Retrieved from

Yager, R. R., & Filev, D. P. (1994a). Essentials of fuzzy modeling and control. New York :: Wiley, 408 pp.

Yager, R. R., & Filev, D. P. (1994b). Generation of fuzzy rules by mountain clustering. Journal of Intelligent & Fuzzy Systems, 2(3), 209-219.

JORGE LUIS SANTAMARIA C.
jsantamaria@uce.edu.ecu



Biography: