Optimization of biodiesel production from oil using a novel green catalyst via development of a predictive model

1 Introduction

Biodiesel production is a very important issue due to the environmental advantages, biodegradability, and renewability (Ahmia et al., 2014; Jamil et al., 2018; Živković and Veljković, 2018). This renewable fuel is mainly manufactured using biomass sources such as palm oils in chemical esterification reactions (Kumar, 2021; Kumar et al., 2021). In transesterification reaction triglycerides transform into fatty acid alkyl esters, in the presence of a catalyst and alcohol. One of fatty acid ester which is named Fatty acid methyl esters (FAME) are produced from transesterification of fats with methanol (Inam et al., 2019; Bemani et al., 2020; Pullen and Saeed, 2014). In this regard, it is very important to select the correct operating factors to achieve high quality biofuels with improved physico-chemical properties. Therefore, providing a correct relation between the input parameters and their desired output properties is quite difficult and, in some cases, impossible. The complexity of biodiesel production systems makes it a challenging problem to optimize the production process and prediction of process efficiency (Kumar et al., 2020). New opportunities may arise as a result of recent advancements in machine learning (ML) and data science. ML is a hopeful approach for dealing with the complex, nonlinear, and multivariate biofuel production system (Aghbashlo et al., 2021; Liu et al., 2019; Kumar and Deswal, 2022).

An ensemble in machine learning is a collection of models whose combined prediction aims to enhance a single model's performance (accuracy). Ensembles have resulted in powerful predictive algorithms with large generalization capacity without relinquishing more local or specialized knowledge due to this mixing of various predictions (Ribeiro and dos Santos Coelho, 2020). Some ML methods, such as Decision Tree (DT) and Neural Network (NNs) algorithm are intrinsically unstable, meaning that any change to the training data points results in a drastically different predictor (Pandey and Sharma, 2013; Andrade Cruz et al., 2022; Pérez-Ortiz et al., 2016). Low bias and high variation are characteristics of unstable estimators. Ensemble techniques have been proposed to reduce generalization error, i.e., bias, variance, or both. These methods modify the training dataset and provide an ensemble of various base estimators. A single estimator is then created by combining these estimators (Izenman, 2008). As these results were not sufficiently generic to be used as the basis for a robust model, it was decided to use reiterative models instead. Bagging and boosting are two of the best ways to strengthen the Decision Tree. Breiman's (Breiman, 1996) Bagging (Bootstrap Aggregating) is a basic and straightforward ensemble method, demonstrating excellent performance while reducing variance and preventing overfitting. The bootstrap approach, which creates training data subsets by replicating training data sets, adds diversity to the Bagging process. The entire data set is split into several subsets, and each is used to fit a different type of basic estimator; the combined prediction results are decided upon by the majority.

Boosting is another ensemble technique developed from the research of Freund and Schapire (Freund and Schapire, 1996; Ferreira and Figueiredo, 2012). By gradually reweighting the training data, it produces a variety of fundamental learners, in contrast to Bagging. In the following training step, each sample that was poorly estimated by the previous estimator will be given more weight. As a result, incorrectly estimated training samples by predecessors are more likely to appear in the succeeding bootstrap sample, and bias can be successfully eradicated. All of the base estimators are combined into one final model using the Boosting algorithm, and their weights are determined by how well they predict. The DT algorithm's basic principle is to break down big problems into many smaller sub-problems (Divide-and-conquer), which may result in an easier-to-interpret answer (Xu et al., 2005). A DT depicts a group of conditional queries that are ordered hierarchically (tree architecture) and applied progressively from the tree's root to the leaf (Breiman et al., 2017). DTs are easy to use and understand, with a clear structure. DTs generate a trained predictor that can express rules, which can subsequently be employed to forecast fresh datasets using the repeating process of splitting (Ahmad et al., 2017; Dumitrescu et al., 2022).

In this study, for the first time in order to predict the FAME production from palm oil different decision tree-based machine learning modeling methods were used. The models employed in this study include AdaBoost (Boosting of Regression Trees, ADA + RT), Extra Trees (A bagging method for Regression Trees), and Gradient Boosting (A boosting method for Regression Trees, GBRT) which are based on DT Regressors. The obtained results of these models were compared with different performance indicators (R², RMSE, MAE) and the optimal conditions for highest FAME production yield were acquired.

2 Data set of biodiesel production

Table 1 shows the dataset that is used in this study for biodiesel production optimization, which was collected from a published source, and more details about the experimental methods can be found elsewhere (Zhang et al., 2020). This issue contains three decimal number input features and one target output, for a total of 17 rows. Indeed, the target of optimization is the production yield of biodiesel (percentage).

Additionally, Fig. 1 was created to explore the distribution of inputs and outputs and their relationship.

Close

Fig. 1

Scatter plot for Data Set of biodiesel modeling.

Export to PPT

3 Computational methodology

3.1

3.1 CART (Classification and regression tree)

Because the decision tree (DT) estimator of the CART type is employed as a weak or basic estimator in all of the models in this study, we will first introduce the concept of this method. The decision tree method is a learning model often utilized for both regression and classification tasks. The partitioning method results in a tree-like architecture so the model is called decision tree (DT). A DT contains a root node and child (sub) nodes that include all the cases and non-cases necessary for training the model (Cheraghlou et al., 2021). Fig. 2 depicts this process graphically as a binary decision tree.

Close

Fig. 2

A straightforward binary decision tree structure with a three-level depth.

Export to PPT

The splitting criterion is critical in the construction of a DT. Numerous DT algorithms employing diverse splitting criteria have been developed over the years, including the C4.5, ID3, and CART decision tree algorithms. (Quinlan, 1986, 2014). A sampling technique called CART was developed by Breiman et al. (Breiman et al., 2017; Saha et al., 2021). Impurity in a dataset can be quantified using the Gini index (GI). Using Eq. (1), we can calculate the Gini index of the node's sample set S. The GI for a node is more representative of the quality of its data if it is lower. This means that when using attribute splitting, the weighted mean of GI on each branch (sub-node) is optimised, resulting in the maximum possible value of $\mathrm{\Delta }GI\left(S\right)$ as defined in Eq. (2) (Liu et al., 2018).

(1)

$$\mathit{GI}\left(S\right)=1-\sum _{k=1}^{K_s}{p}_k^2$$

(2)

$$\mathrm{\Delta }\mathit{GI}\left(S,a\right)=\mathit{GI}\left(S\right)-\left[W_L·\mathit{GI}\left(S_L\right)+W_R·\mathit{GI}\left(S_R\right)\right]$$

In these equations, K_s reflects the count of outputs in the node and p_k denotes the proportion of the k^th category in the node. $\mathrm{\Delta }GI(S,a)$ denotes the change in impurity levels between the sample set S pre-split and post-split with feature an as the split attribute (Alobaida and Huwaimel, 2023). Sample sets of left and right child nodes are denoted by S_L and S_R, while W_L and W_R represent their respective proportions. Like other DT induction techniques, CART finds the most significant variables when choosing the optimum splitting features for each root or internal node, hence defining independent variables is unnecessary (Menze et al., 2009; Shang et al., 2007). As a result, CART seems to be appropriate for learning situations in which the relationship between inputs and output is uncertain (Safavian and Landgrebe, 1991).

4 Results and discussions

In this step, after looking at the tuned hyper-parameters, the final models will be created and compared using tree metrics to evaluate and analyze the results of the suggested models:

With the help of Eq. (11), the Mean Absolute Error metric (MAE) (Botchkarev, 2018), that is the horizontal distance between two continuous variables, namely the arrays of measured and model projected Yield of FAME, is calculated.

Here, n reflects the count of input samples and y_i is the observed value. Also, ${\widehat{y}}_i$ denotes model predicted values (Wang and Lu, 2018; Kumar et al., 2020).

The Root Mean Squared Error (RMSE) is the second indicator that was employed in the comparison (RMSE). In Eq. (12), it is stated as the standard deviation of model estimated outputs from observed values on a dataset (Kumar et al., 2020; Fajar and Sugiarto, 2012):

R²-Score, is the last metric. In regression tasks, the R²-Score (Botchkarev, 2018) is utilized to gauge how closely the model's predictions match the actual values. Equation (13) provides a relationship that can be used to compute it.

In this equation, μ stands for the average of the observed data.

For the ADA-RT model, we need to decide on nine essential hyper-parameters. Although other parameters can affect the output of our model, we have optimized these nine more essential parameters.

Table 2 lists some of the best combinations. The final optimized values for the parameters are:

• Criterion: The function's quality is determined by this parameter. As an alternative to using mean squared error, Friedman MSE takes into account both the error and the improvement made by the split, and is therefore used for future partitions.

• Splitter method: The mechanism used to find the split at each node. The chosen strategy is “best” for choosing the best split for identifying the appropriate random split.

• Max depth: The maximum tree depth has been determined to be 14.

• Minimum samples split: 6

• Minimum samples leaf: 2

• Maximum features: 3

• Learning rate: 1.0

• Number of estimators: 170

• Loss Function: Linear

Based on all combinations tests (top results are presented in Table 3), the final hyper-parameters for the Extra tree regressor are as follows:

• Number of estimators: 126

• Criterion: Absolute error was chosen as the function to improve the impact of a split.

• Max Depth: 8

• Max Features: 3

Finally, the top results for Gradient Tree Boosting may be found in Table 4. Based on these experiments, we chose the following hype-parameters:

• Learning Rate: 1.25

• Number of Estimators: 30

• Tolerance for the early stopping: 2.25E−4

• Criterion: This is the function that determines the quality of a split is set to squared error for gradient boosting.

• Loss Function: We used the Huber Loss function to optimize the gradient tree boosting.

The observed and predicted FAME yield production according to the ADA + RT model are presented in Figs. 3 and 4 for train and test values, respectively. As can be seen a good accuracy was obtained from this model in the training phase. However, according to Fig. 4, in ADA + RT model testing phase the distance between the actual and predicted values was increased in some values.

Close

Fig. 3

Train results of ADA + RT.

Export to PPT

Close

Fig. 4

Test results of ADA + RT.

Export to PPT

The result of Extra Tree model for the train and test data are presented in Figs. 5 and 6, respectively. By comparing Fig. 3 and Fig. 5 it can be said that both models had a large extent in the training phase. Same as ADA + RT model, the Extra Tree model had better performance and more accuracy in learning step however, the ADA + RT Extra Tree model (Fig. 6) performance in test phase was better than ADA + RT model in the same phase (Fig. 4).

Close

Fig. 5

Train results of Extra Tree.

Export to PPT

Close

Fig. 6

Test results of Extra Tree.

Export to PPT

Similarly, the observed and predicted FAME yield production by GBRT model in the train and test phase are presented in Figs. 7 and 8, respectively. Compared to ADA + RT and Extra Tree models, the GBRT model performance was better in both train and test phase. As can be seen in Figs. 7 and 8, the observed and the predicted values in the GBRT model were more accurate. model in this phase can be observed compared to the previous two models (Figs. 3 and 5).

Close

Fig. 7

Train results of GBRT.

Export to PPT

Close

Fig. 8

Test results of GBRT.

Export to PPT

For evaluation of models MAE, R-square, and RMSE were calculated for each model. These coefficients represent how well the model fits compared to the real values. The obtained results are illustrated in Table 5. As is obvious in this Table, the R² values of 0.979, 0.997, and 0.997 were obtained for ADA + RT, Extra Tree, and GBRT models, respectively. The higher R² value specifies a better fit in model results, therefore, it can be concluded that the Extra Tree, and GBRT models can fit the data better than ADA + RT model. The RMSE error rates for the ADA + RT, Extra Tree, and GBRT models were obtained as 1.549, 1.524, and 0.396, respectively. Also, lower MAE value was attained for GBRT model (0.333) while ADA + RT and Extra Tree models had higher MAE values (1.224 and 1.186, respectively). According to the obtained results it can be said that the GBRT model had more accurate prediction and more properly model the FAME production yield data compared to the ADA + RT and Extra Tree models. As a result, the following section will delve deeper into this particular model. This can also be confirmed by Fig. 9, which shows the residuals of this model.

Close

Fig. 9

Residuals of prediction using GBRT model.

Export to PPT

Further information in the form of 3D diagrams about the impact of operational factors on the FAME production yield is provided in Figs. 10–12. These two-way relationships between operating factors and FAME yield are visualized here with the help of GBRT model output. Fig. 10 represents the dual effect of reaction time and methanol to oil ratio on the biodiesel production yield while the third factor (catalyst amount) was fixed at the value of 2 wt%. Increasing the molar ratio increases the content of methanol in the reaction media and leads to higher production of biodiesel (Abdelbasset et al., 2022). However, as can be seen when the amount of methanol increased too much in the reaction media the production yield decreased. The higher amount of methanol lead to fact consumption of oil content in the system which decrease the FAME production (Yang et al., 2009). Therefore, the optimum amount of methanol to oil molar ration should be obtained.

Close

Fig. 10

Projection of X1 and X2 in final GBRT model. X3 = 2 kept Constant. Optimum value is y = 98.72 for x1 = 45, x2 = 12.

Export to PPT

Close

Fig. 11

Projection of X1 and X3 in final GBRT model. X2 = 12 kept Constant. Optimum value is y = 98.72 for x1 = 45, x3 = 2.

Export to PPT

Close

Fig. 12

Projection of X2 and X3 in final GBRT model. X1 = 45 kept Constant. Optimum value is y = 98.73 for x2 = 12, x3 = 1.95.

Export to PPT

Keeping the molar ratio of methanol to oil at 12 and varying the reaction time and amount of catalyst, Fig. 11 displays the combined effect on FAME production yield. As can be seen, the production of FAME went up as the amount of catalyst went up. However, biodiesel production is hampered after a certain threshold of catalyst higher increment. There can be a transesterification reaction if enough basic sites are provided when the catalyst amount is increased to a certain value. However, the biodiesel production yield is reduced when the catalyst amount is increased to a point where there is too much resistance in the reaction flow (Zhang et al., 2020; Li et al., 2013). Transesterification is a reversible reaction and at the equilibrium point the maximum yield of biodiesel could be obtained. In long period of reaction times the transesterification can be performed in the reverse direction and reduce the production yield.

Fig. 12 describes the dual influence of methanol to oil molar ratio and catalyst amount on the FAME generation rate while the reaction time was kept constant at 45 h. As can be seen at lower amount of catalyst and methanol to oil molar ratio the FAME production yields were low. This can be because of fast consumption of methanol and catalyst during the reaction. By increasing the amount of catalyst and the methanol-to-oil molar ratio, FAME production yields can be increased; however, biodiesel production reaches a maximum and then declines. This can be explained by the possibility of side reactions that consume methanol and catalyst (Abdelbasset et al., 2022; Chopade et al., 2012; Vicente et al., 2007). In order to obtain the maximum quantity of biodiesel production, it is evident that the ideal amount of catalyst and methanol to oil molar ratio should be calculated.

The impact of each variable on the yield of FAME was shown in Figs. 13–15. By keeping the other factors constant, the one unique impact of each variable on biodiesel production can be obtained as 2D diagrams. These findings are consistent with those found in Figs. 10–12.

Close

Fig. 13

Response tendency for x1.

Export to PPT

Close

Fig. 14

Response tendency for x2.

Export to PPT

Close

Fig. 15

Response tendency for x3.

Export to PPT

In general, these results suggest an increase in reaction to values that are near to those shown in Table 6 for each of the input variables. Thus, the pattern in these three data supports the validity of the optimizations in Table 6 as a whole. As can be seen, the maximum biodiesel production yield of 99.729% is achievable under the following optimized conditions: reaction time of 45 h, methanol to oil molar ratio of 12.0, and catalyst amount of 2.0 wt%.

5 Conclusion

Biodiesel production from oil was optimized in this work using a number of machine learning models. The models for the optimization included AdaBoost (DT base), Extra Trees, and Gradient boosting regression tree (GBRT). The R², RMSE, and MAE are metrics used to evaluate the developed regression models. According to the R-square criterion, all three models selected for this study had R² value greater than 0.9, which demonstrate the accuracy of assembled models. However, in terms of MAE and RMSE the GBRT had the lowest values (0.333 and 0.396, respectively) compared to ADA + RT (1.224 and 1.549, respectively) and Extra Trees (1.186 and 1.524, respectively) models. Although the three developed models were accurate and properly predict biodiesel production, the GBRT model was determined as the most appropriate model for further investigations. The results indicated that a maximum biodiesel production yield of 99.729% may be achieved under the ideal conditions of 45 h of reaction time, a methanol to oil molar ratio of 12.0, and a catalyst quantity of 2.0 wt%. The promising results of this study confirmed that ML techniques can provide new opportunities to enhance biodiesel production efficiency and reduce the overall time and cost of processes.