Translate this page into:
Predictive modeling and computational machine learning simulation of adsorption separation using advanced nanocomposite materials
⁎Corresponding authors at: Research and Development Department, Kimia Gostar Noyan Company, Karaj, Alborz, Iran (A. Sadeghi). fjlycthxf@163.com (Xuefang Hu), Arash.sadeghi198249@gmail.com (Arash Sadeghi)
-
Received: ,
Accepted: ,
This article was originally published by Elsevier and was migrated to Scientific Scholar after the change of Publisher.
Peer review under responsibility of King Saud University.
Abstract
Adsorption process was simulated in this study for removal of Hg and Ni from water using nanocomposite materials. The used nanostructured material for the adsorption study was a combined MOF and layered double hydroxide, which is considered as MOF-LDH in this work. The data were obtained from resources and different machine learning models were trained. We selected three different regression models, including elastic net, decision tree, and Gradient boosting, to make regression on the small data set with two inputs and two outputs. Inputs are Ion type (Hg or Ni) and initial ion concentration in the feed solution (C0), and outputs are equilibrium concentration (Ce) and equilibrium capacity of the adsorbent (Qe) in this dataset. After tuning their hyper-parameters, final models were implemented and assessed using different metrics. In terms of the R2-score metric, all models have more than 0.97 for Ce and more than 0.88 for Qe. The Gradient Boosting has an R2-score of 0.994 for Qe. Also, considering RMSE and MAE, Gradient Boosting shows acceptable errors and best models. Finally, the optimal values with the GB model are identical to dataset optimal: (Ion = Ni, C0 = 250, Ce = 206.0). However, for Qe, it is different and is equal to (Ion = Hg, C0 = 121.12, Ce = 606.15). The results revealed that the developed methods of simulation are of high capacity in prediction of adsorption for removal of heavy metals using nanostructure materials.
Keywords
Adsorption
Nanocomposites
Separation
Machine learning
Heavy metals
1 Introduction
Separation of heavy metals from water is a subject of great interest in order to develop efficient processes for production of potable water as well as treatment of wastewater (Liu et al., 2020; Ahmadi et al., 2020; Ge et al., 2019; Huang et al., 2022; Liu et al., 2022). Heavy metals such as Hg and Ni are present in a variety of sources especially industrial effluents, and they must be removed in order to achieve standard water. Different processing methods have been utilized and developed for treatment of water and removal of impurities such as membrane separation, biological processes, adsorption, etc. The design of process depends on a number of parameters such as the quality of feed water and the treated water as well as the operational conditions (Jalali Sarvestani and Ahmadi, 2020; Qaderi, 2020; Latif et al., 2021; Makiabadi and Zakarianezhad, 2020; Xu et al., 2021; Hu et al., 2022; Vahid and Jyotirmoy, 2021; Lin et al., 2021; Albadarin et al., 2017; Rezakazemi et al., 2019; Soltani et al., 2020; Rezakazemi and Shirazian, 2019).
Membrane separation has been recently extensively utilized for water treatment and desalination. Basically, reverse osmosis process is suggested for the case of ion removal from water. Also, nanofiltration is suggested for ion removal, however these processes require high energy for the separation due to the operation at high pressure. Another attractive membrane process which can be used for water treatment and desalination is the membrane contactors which can be used in various forms for application in water and wastewater treatment. In these membrane contactors, two phases including feed and the treatment agent are brought into contact for removal of the target components. The process is mainly driven by mass transfer through the boundary layers as well as the membrane pores.
Another attractive process which can be used for water treatment is adsorption which is driven by utilization of adsorbent for mass transfer between the liquid and the solid phase. The adsorbent plays crucial role in water treatment using adsorption process, and the high surface area is preferred to provide high separation efficiency (Syah et al., 2021). Indeed, the porous materials at nano size are usually applied for removal of species from liquid phase (Huang et al., 2021; Zhang et al., 2021; Zhang et al., 2016; Wang et al., 2018; Bai et al., 2021). The design of novel adsorbents for separation of impurities such as heavy metals need experimental design and extensive measurements and analytical effort. The best point in this method can be determined where the adsorption occurs at the shortest time. However, computational techniques can be alternatively employed for simulation of adsorption process and optimization of impurity removal from liquid phase (Jia et al., 2012; Zhu et al., 2021; Liu et al., 2021; He et al., 2020; Yin et al., 2022). Various method including mechanistic and machine learning models can be employed for simulation of processes such as adsorption separation where the optimum point is desirable to be determined (Rezakazemi et al., 2019; Chen et al., 2021; Gholami et al., 2015; Chen et al., 2021; Mohammadzadeh, 2021; Shang et al., 2021; Annapurna and Yesaswini et al., 2021; Mengting et al., 2019; Razavi et al., 2015; Shirazian and Ashrafizadeh, 2011; Khansary et al., 2017; Keshavarz et al., 2015; Marjani et al., 2011).
The method of machine learning (ML) has been recently employed in various areas such as adsorption modeling for removal of water impurities (Syah et al., 2021; Yang et al., 2021; Syah et al., 2021). The method has shown superior performance in terms of fitting for adsorption process compared to other traditional empirical models such as Langmuir model for description of adsorptive removal of pollutants from water. Machine learning techniques are now widely used to tackle classification, clustering, and regression problems across a wide range of disciplines. The tools in this field of artificial intelligence are classified into different groups used for different types of problems (Alpaydin, 2020; Goodfellow et al., 2016; Murphy, 2012). This study selected three different methods that fit the problem's nature to make predictions on the available dataset for description of a separation process based on adsorption for removal of two heavy metals including Hg and Ni from water.
One of the used computational techniques here is the elastic net model (ENET), which is an extension of the lasso model and resistant to extreme correlations between estimators. Elastic networks combine the L1 and L2 norms. It can not only obtain sparse parameter solutions to realize the selection of significant features, but it can also maintain a certain degree of regularity to avoid parameter overfitting in description of processes (Yu and Zhao, 2019; Sanejouand, 2013; Heiss et al., 2021).
A decision tree, which is based on machine learning theory, is a solution to resolve classification and regression problems easily. The decision tree operates on a tree-like (hierarchical) model. The tree is made up of a root node that contains all of the data, multiple internal nodes to divide the data into two or more subsets, and several terminal nodes (leaf nodes). One or more classes are cut off from the rest of the classes at each node in the decision tree structure. The processing is usually done by moving down of trees till the leaf node is reached, then moving up to the next node in the tree. This is called a “top-down” method (Xu et al., 2005; Breiman et al., 2017; Safavian and Landgrebe, 1991). Decision tree regression (that we used in this research) is a variant of a decision tree that can be used to approximate real-valued outputs like class proportions (Safavian and Landgrebe, 1991; Mathuria, 2013; Rokach and Maimon, 2007).
Gradient Boosted Trees is a DT-based method. This ensemble method uses Decision Trees as weak or base predictors for estimation problems. The Gradient boosting algorithm, which is developed on the concept of functional Gradient descent, optimizes a certain loss by charging a new predictor to residual errors found by a prior predictor (Amar et al., 2019; Friedman, 2001).
2 Data set for the computations
We used a dataset in this work to run the simulations for the process. The dataset is collected for removal of Hg and Ni from water at different conditions (Soltani et al., 2021). In the simulations, we considered two outputs including Ce and Qe which are the equilibrium concentration of solute, and the equilibrium capacity of the adsorbent, respectively. Ion is a categorized input and C0 is a real-valued one in this set. This dataset consists of a measly 18 entries for the adsorption process. The detailed procedures of the measured data can be found elsewhere (Soltani et al., 2021). There is a full representation of the data in Table 1. In addition, the distribution of all considered variables is represented in Fig. 1, and Pearson correlation is represented in Fig. 2.
No.
X1 = Ion
X2 = C0
Y1 = Qe
Y2 = Ce
1
Ni
0.5
5
0
2
Ni
2
20
0
3
Ni
5
50
0
4
Ni
10
95
0.5
5
Ni
25
230
2
6
Ni
50
380
12
7
Ni
100
426
57
8
Ni
160
433
116
9
Ni
250
439
206
10
Hg
0.5
5
0
11
Hg
2
20
0
12
Hg
5
50
0
13
Hg
10
99
0.1
14
Hg
25
247
0.3
15
Hg
50
445
5
16
Hg
100
500
50
17
Hg
160
518
108
18
Hg
250
527
197
Distribution of inputs/outputs.
Pearson Correlation of inputs/outputs.
3 Methodology
3.1 Elastic net technique
As a lasso extension, the elastic net model (ENET) is resistant to extreme correlations between estimators (Friedman, 2010). The elastic net was initially proposed for analyzing big data collections to avoid the instability of the lasso solution paths when estimators are highly correlated (Zou and Hastie, 2005). This model employs a combination of the ℓ1 (lasso) and ℓ2 (ridge regression) penalties and can be declared as (Jha, 2017):
On the condition of α = λ2/(λ1 + λ2), the elastic net estimator (Above Equation) is seen to be equal to the minimizer of:
Here, Pα(β) stands for the elastic net penalty. This method simplifies the lasso when α = 0 and the simple ridge regression when α = 1. The elastic net's ℓ1 component is utilized for feature selection. Furthermore, the ℓ2 component supports clustered selection and stabilizes solution routes based on random sampling, hence boosting prediction. When the groups of correlated input features are unknown in advance, the elastic net can pick them by creating a grouping effect through feature selection, so that a group of strongly correlated characteristics tends to have coefficients of comparable magnitude. Unlike the lasso, if p ≫ n, ENET selects more than n features (Chennubhotla et al., 2005; Leioatts et al., 2012).
3.2 The method of decision tree
Recently, the decision tree predictive model (DT) is a broadly used ML method for prediction of processes and events. In particular, this method is widely usable in problems that contain some categorical data like the current problem. A decision tree consists of several terminal nodes and several intermediate nodes (decision-makers). Each decision node divides the data into two parts based on one or more input features, and this continues hierarchically in the child nodes to reach the terminal nodes. Each terminal node specifies the final predicate value (regression and classification) (Mathuria, 2013; Rokach and Maimon, 2007). Fig. 3 depicts a simplified decision tree schematic (Wu, 2022).Schematics of a decision tree.
3.3 Gradient boosting
Gradient Boosted Trees is an ensemble approach, with the working mechanism that fresh weak estimators learn from prior weak estimators. As the number of shaky estimators rises, so does the model's inaccuracy. In the meantime, the GBRT algorithm establishes a ratio for weak learners, known as the learning rate, in order to minimize overfitting. Predictions made by weak estimators are added together and then multiplied by the learning rate in the GBRT model to arrive at the final forecast (Friedman, 2001; Elith et al., 2008; Mason et al., 1999; Truong et al., 2020; Xu et al., 2017). Following Algorithm shows the Gradient boosting method steps:
Initialize
For
1. Calculate the negative Gradient:
2. Create a tree model:
3. Select a Gradient descent step size as:
4. Recalculate the approximation of F(x):
Output: the aggregated regression function
4 Results and discussion
The configurations of selected models are tuned, and final models are implemented using these configurations to compare and analyze the final results for the considered adsorption process. In model regression analysis, the model error is defined as the difference between the observed data points and the best fit line produced by the algorithm. The model's error will be calculated using the following metrics when there are several data points:
-
MAE: This is the mean of the absolute values of the mistakes, which represent the deviation from real probability. This is mathematically expressed as:
-
RMSE: RMSE is a popular model performance evaluation statistic since it may be regarded as the standard deviation of the estimation errors. This is how it is written:
-
R2-Score
The final results of fitting for comparison of the models are represented in Tables 2 and 3, for equilibrium concentration and equilibrium capacity of the adsorbent, respectively. Based on Tables 2 and 3, we can choose the Gradient boosting (GB) model as the most general and accurate model for description of the adsorption for capture of ions. Figs. 4 and 5 compare the actual and predicted values for both outputs. In these figures, blue points are trained, and red points are test predicted values alongside a green line that stands for actual values. These figures show that test data are predicted accurately in the Gradient Boosting model which confirms the validity of the model and lack of overfitting issue in the prediction of the data points.
Technique
MAE
R2
RMSE
ELASTIC NET
1.605
0.976
0.168
DECISION TREE
0.127
0.996
0.134
GRADIENT BOOSTING
0.086
0.998
0.103
Technique
MAE
R2
RMSE
ELASTIC NET
3.201
0.887
3.550
DECISION TREE
2.331
0.889
3.004
GRADIENT BOOSTING
1.605
0.994
2.169
Fitting chart for Ce using GB model.
Fitting chart for Qe using GB model.
Finally, based on the obtained results, the GB model is selected, which is shown in Figs. 6 and 7 of the model surface. The optimal values are shown in Table 4 separately for both outputs. These values are confirmed by 2D charts in Figs. 8–11. It is clearly observed that Ce is increased with enhancing the initial concentration of ion in the solution (C0) for both ions and follow a non-linear pattern. This observation could be related to the accumulation of solutes in the bulk of feed with enhancing its initial concentration, as the adsorbent has limited capacity for adsorption of the solute onto its surface (Syah et al., 2021). Indeed, it can be said that adsorption is appropriate process for removal of impurities at low content as the adsorbent would be saturated at high dosage of solutes in the bulk of solution.Predicted 3D plot of Hg ion.
Predicted 3D plot of Ni ion.
Output
Ion
C0
Output Value
Ce
Ni
250
206.0
Qe
Hg
121.12
606.15
2D graph of Ce using Gradient boosting with Hg Ion.
2D graph of Ce using Gradient boosting with Ni Ion.
2D graph of Qe using Gradient boosting with Hg Ion.
2D graph of Qe using Gradient boosting with Ni Ion.
5 Conclusion
This study adopts three potential estimation techniques, including elastic net, decision tree, and gradient boosting, to make regression on the tiny data set with two inputs and two outputs. The dataset are for adsorption of Ni and Hg onto the surface of a nanocomposite structure. Inputs are Ion type and C0 (initial ion concentration in the solution), and outputs are Ce and Qe in this dataset. After adjusting their hyper-parameters, final models were implemented and assessed using several metrics. In terms of the R2-score measure, all models have a score of greater than 0.97 for Ce and more than 0.88 for Qe (Gradient Boosting has an R2-score of 0.994 for Qe). Also, considering RMSE and MAE, Gradient Boosting gives tolerable mistakes and best models. Therefore, GB model was chosen as the most accurate model among other models for description of the adsorption process.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References
- Evaluating Adsorption of Proline Amino Acid on the Surface of Fullerene (C60) and Carbon Nanocone by Density Functional Theory. Chem. Methodol.. 2020;4(1):68-79.
- [Google Scholar]
- Activated lignin-chitosan extruded blends for efficient adsorption of methylene blue. Chem. Eng. J.. 2017;307:264-272.
- [Google Scholar]
- Introduction to Machine Learning. MIT Press; 2020.
- Modeling oil-brine interfacial tension at high pressure and high salinity conditions. J. Petrol. Sci. Eng.. 2019;183:106413.
- [Google Scholar]
- Improved Hungarian Algorithm for Unbalanced Assignment Problems. Int. J. Commun. Comput. Technol.. 2021;9(1):27-33.
- [Google Scholar]
- The transport of silica powders and lead ions under unsteady flow and variable injection concentrations. Powder Technol.. 2021;387:22-30.
- [Google Scholar]
- Classification and Regression Trees. Routledge; 2017.
- Application of machine learning in rapid analysis of solder joint geometry and type on thermomechanical useful lifetime of electronic components. Mech. Adv. Mater. Struct. 2021:1-9.
- [Google Scholar]
- Engineering of Novel Fe-Based Bulk Metallic Glasses Using a Machine Learning-Based Approach. Arab. J. Sci. Eng.. 2021;46(12):12417-12425.
- [Google Scholar]
- Elastic network models for understanding biomolecular machinery: from enzymes to supramolecular assemblies. Phys. Biol.. 2005;2(4):S173.
- [Google Scholar]
- Greedy function approximation: a gradient boosting machine. Ann. Stat. 2001:1189-1232.
- [Google Scholar]
- Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw.. 2010;33(1):1.
- [Google Scholar]
- Insight into the enhanced sludge dewaterability by tannic acid conditioning and pH regulation. Sci. Total Environ.. 2019;679:298-306.
- [Google Scholar]
- Artificial neural network technique for rainfall temporal distribu-tion simulation (Case study: Kechik region). Caspian. J. Environ. Sci.. 2015;13(1):53-60.
- [Google Scholar]
- MRMD2.0: A Python Tool for Machine Learning with Feature Ranking and Reduction. Curr. Bioinform.. 2020;15(10):1213-1221.
- [Google Scholar]
- Nonparametric estimation of the random coefficients model: An elastic net approach. J. Economet. 2021
- [Google Scholar]
- Recent Technologies for the Extraction and Separation of Polyphenols in Different Plants: A Review. J. Renewable Mater.. 2022;10(6):1471-1490.
- [Google Scholar]
- Gold Nanoparticles-Loaded Polyvinylpyrrolidone/Ethylcellulose Coaxial Electrospun Nanofibers with Enhanced Osteogenic Capability for Bone Tissue Regeneration. Mater. Des.. 2021;212:110240.
- [Google Scholar]
- Lignin-enzyme interaction: A roadblock for efficient enzymatic hydrolysis of lignocellulosics. Renew. Sustain. Energy Rev.. 2022;154:111822.
- [Google Scholar]
- Adsorption of Tetryl on the Surface of B12N12: A Comprehensive DFT Study. Chem. Methodol.. 2020;4(1):40-54.
- [Google Scholar]
- High-Dimensional Linear and Functional Analysis of Multivariate Grapevine Data. Rochester Institute of Technology; 2017.
- Application of data mining in intelligent power consumption. In: International Conference on Automatic Control and Artificial Intelligence (ACAI 2012). 2012.
- [Google Scholar]
- Phase diagram of ternary polymeric solutions containing nonsolvent/solvent/polymer: Theoretical calculation and experimental validation. Polymer. 2015;73:1-8.
- [Google Scholar]
- On the search of rigorous thermo-kinetic model for wet phase inversion technique. J. Membr. Sci.. 2017;538:18-33.
- [Google Scholar]
- Flowing atmospheric-pressure afterglow drift tube ion mobility spectrometry. Anal. Chim. Acta. 2021;1163:338507.
- [Google Scholar]
- Elastic network models are robust to variations in formalism. J. Chem. Theory Comput.. 2012;8(7):2424-2434.
- [Google Scholar]
- Understanding the effects of different residual lignin fractions in acid-pretreated bamboo residues on its enzymatic digestibility. Biotechnol. Biofuels. 2021;14(1):143.
- [Google Scholar]
- Different Pathways for Cr(III) Oxidation: Implications for Cr(VI) Reoccurrence in Reduced Chromite Ore Processing Residue. Environ. Sci. Technol.. 2020;54(19):11971-11979.
- [Google Scholar]
- DeepBAN: A Temporal Convolution-Based Communication Framework for Dynamic WBANs. IEEE Trans. Commun.. 2021;69(10):6675-6690.
- [Google Scholar]
- Spatial and temporal distribution characteristics of haze and pollution particles in China based on spatial statistics. Urban Clim.. 2022;41:101031.
- [Google Scholar]
- Investigation of Adsorption of the Nitrosamine Molecule as a Carcinogen Agent on the AlN Nanotubes: A DFT Study. Chem. Methodol.. 2020;4(2):191-202.
- [Google Scholar]
- Vapor pressure prediction using group contribution method. Orient. J. Chem.. 2011;27(4):1331-1335.
- [Google Scholar]
- Decision tree analysis on j48 algorithm for data mining. Int. J. Adv. Res. Comput. Sci. Software Eng.. 2013;3(6)
- [Google Scholar]
- Applicability of BaTiO3/graphene oxide (GO) composite for enhanced photodegradation of methylene blue (MB) in synthetic wastewater under UV–vis irradiation. Environ. Pollut.. 2019;255:113182.
- [Google Scholar]
- Feedback Decoupling of Magnetically Coupled Actuators. In: 2021 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM). 2021.
- [Google Scholar]
- Machine Learning: A Probabilistic Perspective. MIT Press; 2012.
- A brief review on the reaction mechanisms of CO2 hydrogenation into methanol. Int. J. Innov. Res. Sci. Stud.. 2020;3(2):33-40.
- [Google Scholar]
- Investigations on the Ability of Di-Isopropanol Amine Solution for Removal of CO2 From Natural Gas in Porous Polymeric Membranes. Polym. Eng. Sci.. 2015;55(3):598-603.
- [Google Scholar]
- ANFIS pattern for molecular membranes separation optimization. J. Mol. Liq.. 2019;274:470-476.
- [Google Scholar]
- Lignin-chitosan blend for methylene blue removal: Adsorption modeling. J. Mol. Liq.. 2019;274:778-791.
- [Google Scholar]
- Data mining with decision trees: theory and applications. Vol vol. 69. World Scientific; 2007.
- A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybernet.. 1991;21(3):660-674.
- [Google Scholar]
- Elastic network models: theoretical and empirical foundations. Biomol. Simulat. 2013:601-616.
- [Google Scholar]
- Haze Prediction Model Using Deep Recurrent Neural Network. Atmosphere. 2021;12(12):1625.
- [Google Scholar]
- Near-Critical Extraction of the Fermentation Products by Membrane Contactors: A Mass Transfer Simulation. Ind. Eng. Chem. Res.. 2011;50(4):2245-2253.
- [Google Scholar]
- A water-stable functionalized NiCo-LDH/MOF nanocomposite: green synthesis, characterization, and its environmental application for heavy metals adsorption. Arabian J. Chem.. 2021;14(4):103052.
- [Google Scholar]
- A hierarchical LDH/MOF nanocomposite: single, simultaneous and consecutive adsorption of a reactive dye and Cr(vi) Dalton Trans.. 2020;49(16):5323-5335.
- [Google Scholar]
- Machine learning based simulation of water treatment using LDH/MOF nanocomposites. Environ. Technol. Innov.. 2021;23:101805.
- [Google Scholar]
- Artificial Intelligence simulation of water treatment using nanostructure composite ordered materials. J. Mol. Liq. 2021:117046.
- [Google Scholar]
- A robust method for safety evaluation of steel trusses using Gradient Tree Boosting algorithm. Adv. Eng. Softw.. 2020;147:102825.
- [Google Scholar]
- A Repairable System Supported by Two Spare Units and Serviced by Two Types of Repairers. J. Statis. Theory Appl.. 2021;20(2):180-192.
- [Google Scholar]
- Reversible calcium alloying enables a practical room-temperature rechargeable calcium-ion battery with a high discharge voltage. Nat. Chem.. 2018;10(6):667-672.
- [Google Scholar]
- Using Machine Learning Approach to Evaluate the Excessive Financialization Risks of Trading Enterprises. Comput. Econ.. 2022;59(4):1607-1625.
- [Google Scholar]
- Decision tree regression for soft classification of remote sensing data. Remote Sens. Environ.. 2005;97(3):322-336.
- [Google Scholar]
- PDC-SGB: Prediction of effective drug combinations using a stochastic gradient boosting algorithm. J. Theor. Biol.. 2017;417:1-7.
- [Google Scholar]
- Quantum chemical study on the adsorption of megazol drug on the pristine BC3 nanosheet. Supramol. Chem.. 2021;33(3):63-69.
- [Google Scholar]
- Artificial intelligence simulation of water treatment using a novel bimodal micromesoporous nanocomposite. J. Mol. Liq.. 2021;340:117296.
- [Google Scholar]
- Multiple machine learning models for prediction of CO2 solubility in potassium and sodium based amino acid salt solutions. Arabian J. Chem.. 2022;15(3):103608.
- [Google Scholar]
- Robust monitoring and fault isolation of nonlinear industrial processes using denoising autoencoder and elastic net. IEEE Trans. Control Syst. Technol.. 2019;28(3):1083-1091.
- [Google Scholar]
- A Novel Aluminum-Graphite Dual-Ion Battery. Adv. Energy Mater.. 2016;6(11):1502588.
- [Google Scholar]
- Instantaneous Differentiation of Functional Isomers via Reactive Flowing Atmospheric Pressure Afterglow Mass Spectrometry. Anal. Chem.. 2021;93(29):9986-9994.
- [Google Scholar]
- Crossover from Linear Chains to a Honeycomb Network for the Nucleation of Hexagonal Boron Nitride Grown on the Ni(111) Surface. J. Phys. Chem. C. 2021;125(48):26542-26551.
- [Google Scholar]
- Regularization and variable selection via the elastic net. J. R. Stat. Soc.: Ser. B (Stat. Methodol.). 2005;67(2):301-320.
- [Google Scholar]