Skip to main content
Open access
12 June 2020

Quantitative Structure–Activity Relationship Models for Predicting Inflammatory Potential of Metal Oxide Nanoparticles

Publication: Environmental Health Perspectives
Volume 128, Issue 6
CID: 067010



Although substantial concerns about the inflammatory effects of engineered nanomaterial (ENM) have been raised, experimentally assessing toxicity of various ENMs is challenging and time-consuming. Alternatively, quantitative structure–activity relationship (QSAR) models have been employed to assess nanosafety. However, no previous attempt has been made to predict the inflammatory potential of ENMs.


By employing metal oxide nanoparticles (MeONPs) as a model ENM, we aimed to develop QSAR models for prediction of the inflammatory potential by their physicochemical properties.


We built a comprehensive data set of 30 MeONPs to screen a proinflammatory cytokine interleukin (IL)-1 beta (IL-1β) release in THP-1 cell line. The in vitro hazard ranking was validated in mouse lungs by oropharyngeal instillation of six randomly selected MeONPs. We established QSAR models for prediction of MeONP-induced inflammatory potential via machine learning. The models were further validated against seven new MeONPs. Density functional theory (DFT) computations were exploited to decipher the key mechanisms driving inflammatory responses of MeONPs.


Seventeen out of 30 MeONPs induced excess IL-1β production in THP-1 cells. In vivo disease outcomes were highly relevant to the in vitro data. QSAR models were developed for inflammatory potential, with predictive accuracy (ACC) exceeding 90%. The models were further validated experimentally against seven independent MeONPs (ACC=86%). DFT computations and experimental results further revealed the underlying mechanisms: MeONPs with metal electronegativity lower than 1.55 and positive ζ-potential were more likely to cause lysosomal damage and inflammation.


IL-1β released in THP-1 cells can be an index to rank the inflammatory potential of MeONPs. QSAR models based on IL-1β were able to predict the inflammatory potential of MeONPs. Our approach overcame the challenge of time- and labor-consuming biological experiments and allowed for computational assessment of MeONP inflammatory potential by characterization of their physicochemical properties.


With mounting investment in nanotechnology research and development, the global nanotechnology market value is estimated to exceed $121.8 billion USD by 2025 (Industry ARC 2019). The nanotechnology industry is expected to generate more than 10% of the gross domestic product by 2025 in several developed countries (Bursten et al. 2016). Due to their unique physicochemical properties, engineered nanomaterials (ENMs) are used extensively as catalysts, drug carriers, pigments, foods, cosmetic additives, and more. However, some ENMs induce extremely hazardous effects in mammals (Dreher 2004; Nel et al. 2009; Niels et al. 2019). The unique properties of ENMs were found to dictate the nano–bio interactions (Dreher 2004; Nel et al. 2009). Therefore, substantial nanosafety concerns have been raised by governments and international organizations. For instance, REACH (Registration, Evaluation, Authorization and Restriction of Chemicals) requires manufacturers to evaluate the safety of nanomaterials before a product enters the European market (REACH 2006). The U.S. Food and Drug Administration (U.S. FDA) has released nanotechnology safety guidance and emphasized a thorough description of ENM physiochemical properties and biological effects (U.S. FDA 2014). A limitless number of ENMs can be produced in various conformations (Le and Winkler 2016). Considering the complexity of biosystems, the number of ENMs interacting with biosystems may reach several millions. Experimental assessments of nanosafety are extremely time-, labor-, animal-, and cost-consuming (Nel et al. 2013; Liu et al. 2018). Therefore, in silico models and high-throughput assays are attractive for ENM safety evaluations and risk assessments.
Recent efforts sought to develop in silico models for predicting ENM biological activities and for assisting biocompatible material design (Fang et al. 2018; Oh et al. 2016; Puzyn et al. 2011; Wang et al. 2017). For instance, Puzyn et al. (2011) developed a nano-QSAR (Quantitative Structure-Activity Relationship) model to describe the cytotoxicity of 16 different types of metal oxide nanoparticles (MeONPs) in Escherichia coli, using two quantum chemical descriptors (enthalpy of formation of a gaseous cation and energy of the lowest unoccupied molecular orbital). Oh et al. (2016) established cellular toxicity models to predict the toxicity of 17 quantum dots. The models demonstrated that toxicity is closely correlated with surface properties and diameters of quantum dots, assay types, and exposure times. However, most of the published models were developed for a traditional toxicological end point: cell death. Nanoparticles have exhibited subtle effects beyond merely killing cells, such as interfering with immune cell functions and communications (Wan-Seob et al. 2010; Zhu et al. 2012). Although the toxicities of nanoparticles in immune cells have been underlined in recent nanobiology studies (Li et al. 2014a, 2014b, 2013), the interactions between ENMs and immune systems (our primary defense systems against foreign invasion) have rarely been predicted in silico.
Due to their novel optical, magnetic, and electronic properties (Taeho and Taeghwan 2014), MeONPs were employed in many nanotechnology-based fields, including electronics (Li et al. 2015), catalysis (Zhao et al. 2015), and pharmaceuticals and medicines (Taeho and Taeghwan 2014), etc. The exploration and use of MeONPs both caused exposure and risks (Li et al. 2014a). A case study showed that high rare earth metal concentrations were detected in lung biopsy specimens of a male subject with severe pulmonary fibrosis after many years of exposure to fumes and dusts containing rare earth elements (Vocaturo et al. 1983). Some MeONPs were found to induce strong immune responses via inflammasome and Toll-like receptor activation in vitro (Li et al. 2014a) and in vivo (Wan-Seob et al. 2010), as evidenced by the substantial release of the proinflammatory cytokine interleukin-1 beta (IL-1β). From the perspective of nanosafety assessments, models for predicting the inflammatory potential of MeONPs are in demand for advancing the toxicity assessments of MeONPs.
QSAR models were proposed herein to predict the inflammatory potential of MeONPs. Figure 1 shows the workflow of this project, including four steps: construction of a MeONP library, development of predictive models, mechanism interpretation, and experimental validation. The objectives of this study were a) to construct a library of 30 MeONPs for characterization of their physicochemical properties as well as screening of IL-β production in THP-1 cells; b) to develop interpretable QSAR models for prediction of inflammatory potential; c) to validate the models’ predictions using independent MeONPs; and d) to identify the mechanisms of MeONP-induced inflammation.
Figure 1. Schematic workflow of metal oxide nanomaterials (MeONPs) library construction, predictive modeling, mechanism interpretation, and experimental validation.



MeONPs were in-house synthetized, donated, or purchased from commercial sources as outlined in Table S1; the in-house synthesis of TiO2 (George et al. 2011) and ZnO (Tani et al. 2002) was conducted by a flame spray pyrolysis reactor as previously described; deionized (DI) water was obtained from a Milli-Q® water purification system (Millipore); THP-1 cells were purchased from ATCC; Corning® RPMI 1640 (CAT 10-041-CV) were purchased from Corning Inc.; 10% fetal bovine serum (CAT 100-500) was purchased from GeminiBio. Phorbol 12-myristate acetate (PMA) and lipopolysaccharide (LPS) were purchased from Sigma-Aldrich. MTS kit was purchased from Promega. Magic RedTM cathepsin B assay kit was purchased from Immunochemistry Technologies; IL-1β, IL-6, TNF-α, and MCP-1 ELISA kits were purchased from BD Biosciences.

Characterization of the MeONPs

The MeONPs were subjected to physicochemical analyses for examinations of primary particle sizes, hydrodynamic sizes, dissolution rates, and ζ-potentials in deionized water (DI H2O). The primary sizes of MeONPs were measured by a transmission electron microscopy (TEM). TEM samples were prepared by suspending 50μg/mL of MeONPs in DI H2O. A drop of MeONP suspension was placed on 200-mesh, carbon-coated formvar support films mounted on copper grids (Ted Pella, Inc.), and was left to air dry at room temperature (RT). Grids were examined on a JEOL 1200 EX TEM with accelerating voltage at 80 kV. We randomly counted 100 particles for each MeONPs to calculate the average sizes by TEM images. In addition, the MeONP aqueous suspensions (1mL) at 50μg/mL were added into cuvettes to measure average values of hydrodynamic sizes and ζ-potentials after 10 repeated tests using a Nano-ZS90 analyzer (Brookhaven Instruments Corp.). Dissolution capabilities of MeONPs were examined in phagolysosomal simulated fluid (PSF; pH 4.5), which was prepared by a previously described recipe (Li et al. 2014a). MeONPs were dispersed in PSF at 50μg/mL by probe sonication (Sonics & Materials, Inc.) at 32W for 10 s. After 24 h incubation at RT, the supernatants of MeONP suspensions were collected by centrifugation at 15,000×g for 30 min. The concentrations of metal ions in supernatants were measured using an inductively coupled plasma-atomic emission spectrometry (ICP-OES, ICPE-9000, Shimadzu Corp.). The percentages of MeONP dissolution were calculated using the following equation:
where C (μg/mL) is the concentration of metal ions in supernatants, V (mL) is the volume of supernatants; R is the mass ratio of metal elements in each MeONPs.

Cell Culture

THP-1 cells were cultured in Corning® RPMI 1640 media supplemented with 10% fetal bovine serum, namely complete RPMI 1640 (c-RPMI 1640) medium at 5% CO2 and 37°C. Before exposure to MeONPs, THP-1 cells were primed by suspension in c-RPMI 1640 medium containing 1μg/mL PMA and seeded in 96-well plates or 8-well chambers (Corning Inc.) at a density of 3×105 cells/mL for overnight incubation.

Preparation of MeONP Suspensions for Cell Exposure

MeONPs were added in a c-RPMI 1640 medium supplemented with 10 ng/mL LPS at a concentration of 200μg/mL. Then the mixture was dispersed by probe sonication at 32W for 10 s. The MeONP suspensions were diluted to the desired concentrations (3.1, 6.2, 12.5, 25, 50, and 100μg/mL) for cell experiments.

Examination of Cell Viability and Cytokine Production

The supernatants of primed THP-1 cells seeded in 96-well plates (100μL/well, 3×105 cells/mL) were aspirated and replaced by 120μL serial MeONPs dilutions (0, 3.1, 6.2, 13, 25, 50, 100, and 200μg/mL). Three replicates were performed at each concentration. After 24 h incubation, while the supernatants (100μL/well) were collected for measurement of cytokine (IL-1β, TNF-α, and IL-6) production by ELISA according to the manufacturers’ protocols, the residual supernatants were aspirated and replaced by MTS working solution (120μL/well). After 3 h incubation, aliquots of 100μL MTS solutions were transferred into new 96-well plates to record the absorbance of MTS solutions on an SpectraMax M5 microplate reader (Molecular Devices, LLC) at 490 nm. Cell viabilities were calculated using the following equation:
where ANPS, ACTR, and ABL represent the absorbance of MTS solutions acquired from MeONP-treated cells, control cells, and blanks, respectively.

Confocal Imaging of Cathepsin B Release

Primed THP-1 cells were seeded in an 8-well chamber (300μL/well, 3×105 cells/mL) for overnight incubation. Then, the supernatants were replaced by 300μL MeONP suspensions (25μg/mL) for 12 h. The supernatants were aspirated and washed thrice by PBS. The cells in each chamber were stained in 100μL c-RPMI 1640 media containing 260× diluted Magic RedTM cathepsin B assay kit and 1μg/mL Hoechst 33342 for 1 h. After thrice washing by PBS, the cell images were captured by a confocal laser scanning microscope (FV 1200, Olympus Corp.) with a 60×oil immersion objective lens at excitation wavelengths of 405 and 592 nm.

Calculation of FCIL-1β

Fold changes of IL-1β production (FCIL-1β) were used to express the levels of MeONP-induced IL-1β in THP-1 cells. It was calculated using the following equation:
where avg(Sample) is the average level (pg/mL) of IL-1β induced by a specific MeONP at each dose; avg(Control) is the average level (pg/mL) of IL-1β in untreated THP-1 cells. A nanoparticle was classified to the group of MeONPs with inflammatory potential (ip-MeONPs) if its log2 (FCIL-1β) 1; otherwise, the MeONP was classified to the group of MeONPs with noninflammatory potential (nip-MeONPs).

Animal Experiments

C57Bl/6 female mice at 8 wk were purchased from Nanjing Peng Sheng Biological Technology (Nanjing) and were housed under standard laboratory conditions (25°C; 60% relative humidity; 12 h light, 12 h dark cycle) and hygiene status (autoclaved food and acidified water) according to Soochow University guidelines for the care and treatment of laboratory animals. Our animal treatment protocols were approved as following Soochow University Laboratory Animal Center protocols. Animals were exposed to MeONPs by an oropharyngeal instillation method. Briefly, MeONPs were suspended in PBS at 1mg/mL by a probe sonication (32W) for 10 s. The animals were anesthetized by intraperitoneal injection of sodium pentobarbital (200mg/kg) in a total volume of 100μL. The anesthetized animals were held in a vertical position to allow the instillation of MeONP suspension (2mg/kg) at the back of the tongue for pulmonary aspiration.
Vehicle and positive controls were included by exposure of animals to 50μL of PBS and 5mg/kg crystalline silica (Min-U-Sil®), respectively. Each group included six mice. After 40-h exposure, the mice were fully anesthetized by overdose of sodium pentobarbital (400mg/kg) and sacrificed by bilateral thoracotomy to collect bronchoalveolar lavage fluid (BALF) following a reported protocol (Sun et al. 2017). After that, the left lung was collected and stored in liquid nitrogen for future analyses. The right lung was expanded by injection of 0.5mL 10% formalin in PBS at a constant pressure and removed into a 2mL Eppendorf tube containing 1mL 10% formalin. After 24-h fixation at RT, lung tissues were transferred into 70% ethanol for further fixation (2 d). The fixed tissues were cut into 5-µm-thick paraffin sections for hematoxylin and eosin (H&E) staining according to a standard protocol (Cardiff et al. 2014). Three sections were prepared for each animal. Aliquots of 50μm BALF from each animal were added in 96-well half area plates precoated by the capture antibodies in MCP-1 and IL-1β ELISA kits. Quantification of IL-1β and MCP-1 in BALF was conducted according to the manufacturers’ instructions of each ELISA kit. The level of MeONP-induced IL-1β and MCP-1 production was expressed as FCIL-1β/MCP-1 (fold change of IL-1β/MCP-1 production) and calculated using Equation 3.

Descriptor Generation

There were three types of descriptors resulting from quantum-mechanical computations, experimental characterization, and the periodic table. In detail, the quantum-mechanical descriptors were selected if they met one of the following two criteria: a) they directly reflected toxicology-relevant properties, such as cluster size, shape, redox potential, and band structure; b) they were derived from optimized geometric structures that can reflect structural, electronic, and energetic properties of crystals. The crystalline files in common intermediate format (CIF) were obtained using FindIt software (FindIt 2009) and then transformed by Open Babel 2.4.1 (O’Boyle et al. 2011) into standard input files (POSCAR files). Molecular geometries of each cluster reflecting all characteristics of MeONP crystal structures were optimized by the general gradient approximate (GGA) of the Vienna Ab initio Simulation Package (VASP 5.4.1). Furthermore, 43 VASP descriptors were obtained via crystalline structure optimization, and 16 quantum-mechanical descriptors were obtained using the PM7 method implemented in the MOPAC 2016 software package (Stewart 2016). Four experimental descriptors (primary size, hydrodynamic diameter, ζ-potential in water, dissolution in PSF solution) were determined in our laboratory. Five periodic table–based descriptors (electronegativity of metal atoms and metal oxides, cations charge, periodic number of elements, and the atomic ratio of metal and oxygen atoms) were derived from the publicly available periodic table (

Model Development

Classification models were built using the C4.5 decision tree, support vector machine (SVM), and logistic regression. A C4.5 decision tree (Quinlan 1986) is generated from a set of training data, and each inner node contains a test on the original attributes. Prediction was accomplished by traversing a tree from the root to a leaf that directly classified a nanoparticle to the ip-MeONP group or nip-MeONP group. The logistic regression algorithm can predict dichotomous dependent variables by estimating the probability of the event’s occurrence (Hastie et al. 2001). The SVM algorithm is based on the statistical learning theory and the Vapnik-Chervonenkis dimension (Brereton and Lloyd 2010). A polynomial kernel was used to develop the SVM model. We exploited a 10-fold cross-validation procedure based on a grid search to determine the best parameters in three machine learning methods. In our case, log2(FCIL-1β) was adopted as the dependent variable. MeONPs were judged as ip-MeONPs if log2(FCIL-1β) 1. Otherwise, the particles were classified to nip-MeONPs. For validation purposes, the data set was randomly split into training (176 data points generated from 22 MeONPs tested at eight serial dilutions) and test (64 data points generated from eight MeONPs tested at eight serial dilutions) data sets. The data points from eight serial dilutions of one specific MeONP were either fully divided into the training set or test set. All procedures were performed using the Weka software (version
Partial least square (PLS) regression (Wold 1982) was used to develop the continuous model for FCIL-1β in THP-1 cells. The SIMCA 13.0 software package (Bylesjö et al. 2006) was used to conduct the PLS and subsequent analyses. The descriptors of the model were selected based on variable influence on projection values (VIP) (Roy P and Roy K 2008). Briefly, descriptors with the lowest VIP values were deleted from the model until the model showed the best performance in cross-validation.

Model Performance Evaluation

The performance of classification models were evaluated by four metrics: sensitivity (SE=TP/[TP + FN]), specificity (SP=TN/[TN + FP]), overall predictive accuracy (ACC=[TP + TN]/[TP + FP + TN + FN]), and Matthews correlation coefficient (MCC):
where TN, TP, FN, and FP represent true negatives, true positives, false negatives, and false positives. MCC (Matthews 1975) is a binary classification rate that generates a high score only if the binary predictor was able to correctly predict the majority of positive data instances and the majority of negative data instances. It ranges in the interval of 1 to +1. The extreme values at +1 and 1 represent perfect classification and perfect misclassification, respectively, whereas MCC=0 is the expected value for a coin-tossing classifier. In addition, the receiver operating characteristic (ROC) (Osei-Bryson 2004) was used in model evaluation. The ROC curves are constructed by plotting the true positive rate vs. the false positive rate at various threshold settings. A larger area under curve (AUC) indicates higher model predictivity. The learning procedure on the training set was executed in 10-fold cross-validation.
The fitting performance of the continuous model was measured by the squared correlation coefficient (R2) between the predicted and observed FCIL-1β. The performances of internal cross-validation and external validation were assessed by the root mean square error (RMSE) and the predictive squared correlation coefficient (Q2). RMSE and Q2 were calculated using Equations 5 and 6, respectively.
where yi and ŷi are the measured and predicted FCIL-1β values, respectively; y¯i is the average value of FCIL-1β for the training set; test indicates number of predictions. Q2 is a value between 1 and 1. The larger the Q2 is, the better the performance of the model. To estimate the possibility of chance correlation and the predictive capability of the continuous model, we performed a permutation test by scrambling the dependent variable values of samples. We built 500 models based on permutations using the same set of descriptors and algorithm.

Applicability Domain

The applicability domain of the classification model was characterized by a descriptor standardization approach (Roy K et al. 2015). Briefly, the descriptor values in the training set were normalized. If all of the normalized descriptors for a MeONP are larger than three, the MeONP is an outlier; otherwise, it is nonoutlier.
For the continuous model, a Williams plot of standardized residuals vs. leverage values (hi) was drawn to investigate applicability domain. hi is a metric for measuring the distance of a given value from the mean of all values and was used for identification of outliers in a data set and calculated using the following formula (Jaworska et al. 2005):
where X means the descriptor matrix of a data set; superscript T is transpose of a matrix or a vector; and xi is the descriptor vector of ith MeONP. The prediction value of the ith MeONP is considered highly reliable if its hi is lower than the warning leverage (h*) calculated by Equation 8.
Here, n is the number of MeONPs, and p is the number of descriptors used by the model.

Computation for Adsorption Energies

Density functional theory (DFT) computations were performed to investigate the adsorption of protons onto MeONPs. The adsorption energy (Ead) was calculated by Equation 9.
where TE represents the total energy consisting of the kinetic energy, static potential energy, Coulomb energy, exchange energies, and correlation energies. The subscript MH+ stands for the metal oxide (M) and proton (H+) complex system. Because the total energy is calculated based on the energy of electrons in quantum chemical calculations, the total energy of a single H+ (without electrons) is neglected in this case. For clarity, we used the absolute values of the adsorption energy (|Ead|) for discussion. According to our definition, larger |Ead| value means stronger interactions between the proton and metal oxides.
DFT computations were performed using a DMol program (Yang and Xing 2010). The exchange–correlation term was previously described by the Perdew-Burke-Ernzerhof generalized gradient approximation (GGA-PBE) (Perdew et al. 1996). The double numerical basis with polarization functions (DNP) (Inada and Orita 2008) was adopted. This basis is comparable to Pople’s 6-31G** basis set (Liu and Rodriguez 2005). The PBE+D2 method with the Grimme van der Waals (vdW) correction (Grimme 2006) was exploited to accurately describe the long-range electrostatic interactions. A 6×1×6 k-point sampling was used for MeONPs, and a Methfessel-Paxton smearing of 0.01 Ha was applied for the Brillouin-zone integration.

Statistical Analysis

All the cell samples were randomly allocated into experimental groups by drawing lots. All the experiments were repeated at least thrice with three to six replicates. Results were expressed as mean±standard deviation of multiple determinations from at least three repeated tests. Results were statistically analyzed using one-way ANOVA or Student t-test. The difference was regarded as statistically significant with p0.05 in analysis. Correlation was considered to be statistically significant if the R2 was 0.7, and the significance (p) of F test was 0.05.


Construction of MeONP Library to Assess Inflammatory Potential

Characterization of MeONPs.

We acquired 30 MeONPs to characterize their primary sizes, dissolution percentages, ζ-potential, and hydrodynamic sizes (Table S2). Most nanoparticles showed spherical morphologies with primary sizes ranging from 10 to 71 nm, except for Cr2O3, Ni2O3, Nd2O3, Sm2O3, and Er2O3, which ranged from 108 to 193 nm. In aqueous media, most MeONPs formed aggregates with hydrodynamic diameters of 202862 nm except for WO3 (111 nm). NiO, Cr2O3, In2O3, and Fe2O3 exhibited near neutral ζ-potential at 5.6 to 5.4 mV in DI H2O. Although Fe3O4, WO3, Hf2O3, Sb2O3, Sn2O3, ZnO, Zr2O3, β-MnO2, and Mn2O3 showed negative ζ-potentials at 35.8 to 14.3 mV, the other 17 MeONPs had a positive surface charge with ζ-potentials at 13.5 to 47.3 mV. Dissolution capabilities of MeONPs were examined in PSF (pH 4.5). Cr2O3, NiO, and ZnO quickly dissolved in PSF with >90% metal ion release, and Ni2O3, CoO, Mn2O3, CeO2, Sb2O3, and WO3 showed moderate dissolution effects. The other 21 MeONPs were almost insoluble, displaying <10% metal ion release.

In vitro toxicity caused by MeONPs.

THP-1 cells were exposed to a serial dilution of MeONPs (0, 3.1, 6.2, 13, 25, 50, 100, and 200μg/mL) for 24 h to assess cytotoxicity, including cell viability and inflammatory responses. Cell viability was examined by colorimetric detection of metabolic activity via MTS assay. Although a majority of MeONPs had no effect on the viability of THP-1 cells, CoO, CuO, ZnO, Co3O4, Cr2O3, MnO2, Mn2O3, and Ni2O3 induced considerable cytotoxic effect evidenced by the MTS results (Figure S1; Table S3). Proinflammatory cytokine IL-1β was examined in the supernatants of THP-1 cells exposed to MeONPs by ELISA. Then we used Equation 3 to convert the IL-1β values into fold changes (FCIL-1β) in comparison with the basal level of IL-1β in control. Notably, 17 of 30 MeONPs induced significantly higher IL-1β production than the controls in a dose-dependent manner over an extensive concentration range (Figure 2). Among the 17 ip-MeONPs, nine rare earth oxides (Sm2O3, Eu2O3, La2O3, Yb2O3, Gd2O3, Dy2O3, Y2O3, Nd2O3, and Er2O3) ranked in the top class and displayed FCIL-1β>24 at 200μg/mL exposure dose; two manganese oxides (Mn2O3 and MnO2) also elicited substantial cytokine release (24>FCIL-1β>10); six transition metal oxides (Fe3O4, In2O3, Al2O3, Fe2O3, ZrO2, and HfO2) induced relatively low IL-1β (Table S4).
Figure 2. Fold changes of IL-1β production (FCIL-1β) in metal oxide nanomaterials (MeONPs)-treated THP-1 cells. THP-1 cells were exposed to 0, 3.1, 6.2, 13, 25, 50, 100, and 200μg/mL MeONPs for 24 h. IL-1β levels in supernatants were quantified by ELISA. FCIL-1β was calculated by Equation 3. The FCIL-1β was expressed as the mean of three replicates and added in the heatmap.
We examined the impacts of MeONPs on the release of two additional cytokines (TNF-α and IL-6) in THP-1 cells. First, the MTS result and IL-1β release were exploited to assess the impacts of all 30 MeONPs on cell proliferation and proinflammatory effects, respectively. As a result, the MeONPs were divided into three groups, including the proinflammatory, cytotoxic, and noninflammatory/noncytotoxic (Table S5). Notably, the MeONPs in three groups were mutually exclusive. We randomly selected two materials from each group. As a result, six MeONPs were acquired: La2O3, Gd2O3, Co3O4, ZnO, WO3, and TiO2. As shown in Figure S2A, La2O3, Gd2O3 induced dose-dependent TNF-α production, and Co3O4 at high exposure doses induced slight increments of TNF-α, whereas ZnO, WO3, and TiO2 failed to elicit this cytokine. This trend is broadly consistent with their IL-1β levels, further supporting the inflammatory rankings of MeONPs determined by IL-1β production. However, all six nanoparticles failed to elicit IL-6 release in THP-1 cells (Figure S2B).

In vivo validation of the pulmonary inflammatory potential of MeONPs.

To test the validity of the in vitro hazard ranking to toxicity outcomes in animals, C57Bl/6 mice were exposed to the six selected MeONPs by oropharyngeal instillation for 40 h. Animals receiving La2O3 and Gd2O3 exposure showed significant increases of IL-1β and MCP-1 in BALF, whereas Co3O4, ZnO, TiO2, and WO3 showed levels of cytokine release similar to those of vehicle control (Figure 3A). The cytokine release profile of all six MeONPs was consistent with the fold changes of IL-1β production in THP-1 cells. In addition, we examined the inflammation in lung sections by H&E staining as inflammatory cytokines often contribute to immune cell recruitment. As expected, La2O3 and Gd2O3 elicited increases in focal areas of inflammation around small airways in lung sections, whereas ZnO, TiO2, and WO3 did not elicit discernable cell recruitment (Figure 3B). These images further supported the inflammatory rankings of MeONPs in THP-1 cells. However, Co3O4-treated lungs showed only a little inflammation, which is inconsistent with its effect on IL-1β release in vitro.
Figure 3. Pulmonary inflammation of six selected metal oxide nanomaterials (MeONPs) in mice. (A) Cytokine production (IL-1β and MCP-1) in BALF, and (B) H&E staining of lung tissues after 40 h exposure to MeONPs. C57Bl/6 mice (n=6) were exposed to La2O3, Gd2O3, Co3O4, ZnO, TiO2, and WO3 at 2mg/kg by oropharyngeal instillation. Quartz was used as positive control to treat animals (5mg/kg). After 40 h, animals were sacrificed to measure IL-1β and MCP-1 production in BALF by ELISA. The lung tissues were fixed for H&E staining (three sections for each mouse). Normal distribution was confirmed by Kolmogorov-Smirnov test (significance>0.05). *p<0.05 compared with vehicle control by two-tailed Student’s t-test.

Classification Models for Inflammatory Potential

To determine the optimal model for predicting inflammatory potential, we tried three different methods: logistic regression, SVM, and C4.5 decision tree. The optimized parameters of these three machine learning models are provided in Table S6. The performances of the three models on the training set and test set are summarized in Table 1. The C4.5 decision tree model was the top performer of the three models. The ACC values in the training and test set reached 95% and 92%, respectively; the MCC values of this model in training and test set were 86% and 83%, respectively; the AUC reached 95%. The C4.5 decision tree model used three descriptors for discriminating ip-MeONPs and nip-MeONPs. The descriptors were electronegativity (χme, metal atom electronegativity), cation charge, and concentration (the MeONP exposure concentration in μg/mL); the descriptor values are detailed in Table S7. Figure 4 shows the classification rules of the C4.5 decision tree model. The root and interior nodes were drawn with splitting descriptors and the splitting criteria. If a MeONP at a specific data point was predicted to be ip-MeONPs, the leaf node was marked as “YES.” Otherwise, the leaf node was marked as “NO.” The classification rules indicated that MeONPs with lower χme values (χme1.55) tended to induce more inflammatory potential than did those with higher χme values. As shown in Table S8, there were no outliers in the modeling sets of the classification model.
Table 1 Performance of the C4.5 decision tree, logistic regression, and support vector machine models on the training set and test set.
C4.5 Decision treeTraining set43/1331760.950.900.960.950.86
Test set20/44640.920.830.980.950.83
Logistic regressionTraining set43/1331760.860.730.900.770.56
Test set20/44640.800.730.820.770.58
Support vector machine
(Polynomial kernel)
Training set43/1331760.850.740.870.760.61
Test set20/44640.830.800.840.770.50
Note: Predictive accuracy was reflected by four indices: sensitivity (SE=TP/[TP + FN]), specificity (SP=TN/[TN + FP]), overall predictive accuracy (ACC=[TP +TN]/[TP + FP + TN + FN]), and Matthews correlation coefficient (MCC=TP×TN FP×FN(TP + FP)(TP + FN)(TN + FP)(TN + FN)). The area under the receiver operating characteristic curve (AUC) is a measure of how well a model distinguishes positive and negative data points; the >95% model AUC illustrated a high classification power. FN, false negatives; FP, false positives; n, number of data points in the data set; P/N, ratio of positive/negative data points; TN, true negatives; TP, true positives.
Figure 4. C4.5 decision tree for predicting the inflammatory potential of metal oxide nanomaterials (MeONPs). The root and interior nodes are drawn in rectangles with splitting descriptors inside, and the splitting criteria are under the rectangles. The leaf nodes are presented in oval circles, which are marked as “YES” and “NO,” indicating the nanomaterials in the leaf nodes are predicted to be inflammatory potential and noninflammatory potential, respectively.

Continuous Model for Inflammatory Potential

To comparatively evaluate MeONP hazard ranking, we developed the continuous model to assess the inflammatory potential of MeONPs. First, the FCIL-1β values of 30 MeONPs were exploited to develop the PLS regression model. As shown in Figure S3, four MeONPs that were largely distant from the majority (90%) of the data in the training set were identified as outliers based on σ* and h values (σ*>1.5 or hi>0.35) and removed from the 30 MeONPs. Second, the remaining 26 MeONPs were randomly split into a training set (20 MeONPs) and a test set (six MeONPs). A PLS model (Equation 10) were built to rank the inflammatory potential of MeONPs, using only three descriptors (χme – the electronegativity of metal atoms, ζ-potential, and Dwater – hydrodynamic diameter). In the PLS model, these three descriptors were combined to generate one principle component that accounted for 85% of the total variance.
The model was comprehensively validated according to the Organization for Economic Cooperation and Development QSAR validation principles (OECD 2004). The R2 of the 20 training MeONPs was 0.892, indicating an excellent goodness-of-fit. The leave-group-out cross-validation on the 20 training MeONPs yielded a cross-validated correlation coefficient (QCV2) of 0.875 and a RMSE of cross-validation (RMSECV) of 5.5, indicating a good robustness of the model. The RY-scrambling2 (0.119) and QY-scrambling2 (0.0361) average values yielded from the 500 models based on Y-scrambling permutation were much lower (Figure S4). This result confirmed that the model was not obtained by chance. The externally validated correlation coefficient (QEXT2) and the RMSE of external prediction (RMSEP) resulted from the predictions of the six MeONPs in test set. The QEXT2 and RMSEP values were 0.800 and 7.39, respectively, indicating a good predictive ability of the developed PLS model. Figure 5A and Table S9 display the plot of experimental FCIL-1β values vs. the predicted FCIL-1β values using Equation 10. The agreement between the observed and predicted values was satisfactory for MeONPs from the training set (squares) and the test set (triangles). As characterized by the Williams plot (Figure 5B), 25 MeONPs were located in the continuous model’s applicability domain with hi<h*=0.60 and 3<σ*<3, whereas WO3 was out of this region (hi=0.66>h*).
Figure 5. Performance of the continuous model. (A) Plot of experimentally determined data (x-axis) vs. predicted fold change of IL-1β production (FCIL1β) values (y-axis). The straight solid line represents perfect agreement between experimental and calculated values. Squares represent values predicted for the metal oxides (MeONPs) from the training set; triangles represent MeONPs from the test set. The distance of each symbol from the solid line corresponds to its deviation from the related experimental value. The dotted lines showing the range encompassing 90% of the predictions. (B) Model applicability domain: Williams plot of standardized residuals (σ*) vs. leverage values (hi) for FCIL1β. MeONPs having hi>h* or |σ|>3 should be identified as outliers.

Experimental Evaluation of Model Predictive Ability

To further evaluate the predictive performance of the established classification model, new MeONPs were selected for validation based on following two criteria: a) they were not included by the models in training or test sets; and b) the positive:negative ratio in this validation set fell into the range of 1:3 (the ratio in training set) to 1:2.2 (the ratio in test set). As a result of applying these criteria, seven MeONPs (Ho2O3, Pr6O11, MoO3, Ta2O5, Nb2O5, Mn3O4, and MoO2) were selected and exposed to THP-1 cells to determine IL-1β production. The seven new MeONPs at eight serial dilutions in this external data set generated 56 experimental data points. The experimental data were compared with the predictions from the classification model (Table S10). The model showed a high predictivity for this independent data set with ACC at 0.86 and MCC at 0.74. The 56 data points were all located in the model’s applicability domain (Table S11).
The validated classification model was applied to predict the inflammatory potential of 29 MeONPs that lacked inflammatory response experimental data. Among these MeONPs, six MeONPs (CaO, SrO, BaO, Sc2O3, Tm2O3, and Tb2O3) displayed inflammatory potential at a very low concentration of 6.2μg/mL, six MeONPs (BeO, V2O3, CdO, Tl2O, Tl2O3, and Ga2O3) showed inflammatory potential at concentrations >62.5μg/mL, whereas other MeONPs had no inflammatory effect (Table S12). The predicted results of 66 MeONPs were integrated into a color-coded periodic representation, which allowed us to check the safe exposure dose of a given MeONP. Different colors represent the minimal doses eliciting inflammation (MDEI) by MeONPs. As shown in Figure 6, 19 MeONPs displayed strong inflammatory potential with MDEI12.5μg/mL; 15 MeONPs had moderate inflammatory potential with 12.5 < MDEI<200μg/mL; 32 MeONPs indexed by blue color were relatively safe with MDEI200μg/mL. All these predicted MeONPs were located in the model’s applicability domain (Table S13).
Figure 6. Periodic table of predicted inflammatory potential of metal oxide nanomaterials (MeONPs). Sixty-six MeONPs were plotted according to the metal element position in the periodic table. The number in each color square represents MDEI. *, experimental data is unavailable. The predicted value is not shown when the experimental value is available.
The external data set (seven new MeONPs) was also used to verify the performance of the continuous model. As shown in Figure S5, most predictions by our models could be validated by the experimental data. The model predictions correlated well with the experimental results, as evidenced by the R2 and p values (R2=0.90, p<0.05). As shown in Table S14, all the new MeONPs could be found in the continuous model’s applicability domain except Ho2O3 (hi=0.97>h*=0.6).

Potential Mechanisms Involved in MeONP-Induced Inflammation

The six MeONPs for animal experiment were selected to examine potential cellular events involved in inflammatory effect. First, the cellular uptake of these MeONPs was examined by detection of the concentrations of metal elements in particle-treated THP-1 cells. As shown in Figure S6, the ip-MeONPs (La2O3 and Gd2O3)showed higher cellular internalization at 6469μg per mg protein, whereas the four nip-MeONPs nanoparticle showed relatively lower levels (2637μg/mg protein) of cellular uptake. We further examined lysosomal damage by confocal imaging of cathepsin B release from lysosomes by a Magic Red™ kit consisting of nonfluorecent substrate, which is able to emit strong red fluorescence after enzymatic cleavage by cathepsin B. We exploited this kit to stain cathepsin B in particle-treated cells. As shown in Figure S7, ZnO, Co3O4, TiO2 and WO3, showed red punctates of cathepsin B confined in lysosomes, which is consistent with the distribution pattern of cathepsin B in untreated cells. These images indicated that ZnO, Co3O4, TiO2, and WO3 had limited effects on lysosomal compartments. In contrast, La2O3 and Gd2O3 induced massive cathepsin B release with diffused red color in cytoplasm, suggesting severe lysosomal damage. Taken together, these results indicated that cellular internalization and lysosome impairment may be involved in MeONP-induced inflammatory response.
Because the QSAR analysis suggested that the χme value was closely related to inflammatory effects, we speculated that the ip-MeONPs with χme1.55 may have a sufficiently strong interaction with protons to trigger proton sponge effects and elicit lysosomal damages. To test this feasibility, DFT computations were performed to investigate the capabilities of six MeONPs to absorb protons. Interestingly, we found the χme values of MeONPs had impacts on proton adsorption capabilities. As shown in Figure 7, Gd2O3, Y2O3, and Nd2O3 with χme1.55 showed significantly stronger proton adsorption than SnO2, CuO, and CoO χme>1.55 showed. This result was in agreement with our QSAR model predictions that MeONPs with lower χme values would tend to induce more inflammatory potential than those with higher χme values.
Figure 7. Adsorption energy (|Ead|) of protons onto metal oxide nanomaterials (MeONPs) with different metal atom electronegativity (χme). Bar heights represent calculated values of |Ead|. MeONPs with higher χme (black) have much weaker proton adsorption than those with lower χme (red). Normal distribution was confirmed by Kolmogorov-Smirnov test (significance>0.05). *p<0.05 compared with ip-MeONPs by two-tailed Student’s t-test.


Selection of FCIL-1β in THP-1 Cells for Predicting Inflammatory Potential of MeONPs

To explore the hazard effects of MeONPs, THP-1 cells (a macrophage-like myeloid cell line) were exploited to examine the impact of MeONPs on cell viability and inflammatory responses. There are three reasons for the selection of this cell line: a) phagocytic uptake is the main mechanism to remove insoluble fine particulates (Geiser 2010); b) the behavior of differentiated THP-1 cells is more like that of monocyte-derived macrophages (Auwerx 1991); c) a correlation of nanotoxic effects between THP-1 cells and alveolar macrophages has been discovered in our previous studies (Cai et al. 2018; Li et al. 2018).
THP-1 cells were exposed to a library of 30 MeONPs for assessment of proinflammatory cytokine production, and 17 nanoparticles showed significant IL-1β production. Among them, nine rare earth oxides elicited massive IL-1β release, which is consistent with a previous report (Li et al. 2014b). This report indicated that La2O3, Eu2O3, Sm2O3, Er2O3, Nd2O3, Dy2O3, Y2O3, Gd2O3, and Yb2O3 were able to induce NLRP3 inflammasome activation and IL-1β production in THP-1 cells.
FC values and p values are two ways to determine the inflammatory potential of MeONPs. Significance analysis by p values may generate false-positive results for the MeONPs inducing limited IL-1β production with small standard deviations. Calculation of the FC values directly reflects the cytokine levels increased by MeONPs in comparison with control. However, FC values depend largely on the number of replicates. In the cases of limited replicates with large standard deviations, a very large FC value may not be significant. In this study, FC values of IL-1β were more appropriate to determine the inflammatory potential of MeONPs because the standard deviations of replicates in IL-1β assay was small (Table S2).

Validation of Inflammatory Potential of MeONPs in Mouse Lungs

To test the validity of the in vitro hazard ranking to toxicity outcomes in animals, mice were exposed to MeONPs by oropharyngeal instillation. We deliberately selected female mice for lung toxicity test because they are reported to be more sensitive than male mice in pulmonary nanotoxicity studies (Zhang et al. 2015; Ray and Holian 2019). We therefore randomly selected six MeONPs for animal experiments, including Gd2O3, La2O3, Co3O4, ZnO, TiO2, and WO3. Although animal studies on the six selected MeONPs have been conducted in literature, there is no comparative study of all six nanoparticles in one single animal experiment (Li et al. 2014a; Larsen et al. 2016; Agarwal et al. 2019). Moreover, the physicochemical properties of MeONPs with identical purported composition from different sources may elicit different hazard effects in animals (Nel et al. 2006; Gajewicz et al. 2012). Inconsistent material source as well as deviations resulting from animal experiment batches may significantly affect the validation of our in vitro predictions. Therefore, we systematically tested the in vivo inflammatory potentials of six MeONPs and leveraged our animal data rather than literature results to validate the in vitro predictions.
It has been demonstrated that the pulmonary inflammation induced by 2mg/kg as-prepared multiwalled carbon nanotubes (Ap-MWCNTs) is on the steep part of the dose–response curve of pulmonary inflammation in C57Bl/6 mice (Wang et al. 2011). Interestingly, the dispersed Ap-MWCNTs elicited dose-dependent IL-1β production in THP-1 cells, which is similar to our findings on 17 MeONPs. Moreover, the in vitro doses of Ap-MWCNTs at 12.5–100 μg/mL are close to the doses of MeONPs in our study. We therefore compared the lung toxicity of MeONPs at 2mg/kg. Although we are not able to include real-life human exposures in our study, a report by Cai et al. (2017) indicated that an in vivo dose of 2mg/kg is relevant to the levels of released fine particulates during welding at the welders’ breathing zone.
The inflammatory potentials of La2O3, Gd2O3, ZnO, WO3, and TiO2 were well-validated in animal lungs. Co3O4 induced slight inflammation in lung tissues although it showed low potential to induce IL-1β production in vitro. The inconsistent results may be attributed to the capability of Co3O4 to elicit other inflammatory cytokines, such as TNF-α (Figure S2). These results indicate that although IL-1β is a prominent factor to predict the inflammatory potential of MeONPs, we cannot fully rule out the contribution of other proinflammatory cytokines. More cytokine indexes should be considered in future studies to enhance the predictive accuracy of QSAR models for assessment of nano-immunotoxicity.
Moreover, the inflammatory potential of some MeONPs were validated by others’ lung toxicity studies. In a 24-h post-instillation acute pulmonary toxicity experiment in female mice, La2O3, Eu2O3, Sm2O3, Er2O3, Dy2O3, Y2O3, and Nd2O3 also induced much higher inflammogenic potential than ZnO and NiO induced, as evidenced by the increased number of granulocytes and IL-1β (Han et al. 2018). Moreover, pulmonary inflammation by La2O3 was confirmed in rats that received longer-term-exposure MeONP. Kim et al. (2017) found that La2O3 particles induced pulmonary inflammation in rats after 0.5mg/m3 inhalation exposure for 28 d (5 d/wk). These findings were consistent with our experimental results and suggested that IL-1β could be exploited as a predictive index to rank the inflammatory potential of MeONPs.

Reliability and Applicability of the QSAR Models

Successful development of QSAR models depends on the quality of biological data and reduction of uncertainty in modeling. Despite the vast array of MeONP toxicity publications to date, there are limited reliable data that can be used for hazard ranking or QSAR modeling. Moreover, variations in the data published by different laboratories hinder the development of reliable QSAR models (Gajewicz et al. 2012). The MeONP data set developed in this study was suitable for hazard ranking and QSAR modeling. This suitability is because, to the best of our knowledge to date, it was the largest data set and was obtained entirely in the same laboratory, complete with a full experimental protocol description and nanomaterial characterization (concentration, ζ-potential, dissolution, size, etc.). We tried to reduce uncertainty by building transparent models. The prediction rules in both classification and continuous model were clear. The descriptors used in the models were well-defined and can be derived quickly from the chemical composition information (χme and cation charge) and characterization of basic physicochemical properties (ζ-potential and Dwater). Our results indicated that these descriptors could encode the inflammatory potential of MeONPs efficiently in comparison with classical quantum chemical descriptors involving time-consuming computations and extensive prior expertise. We also tried to reduce uncertainty in the models by performing statistical assessments, assessing the applicability domain, and conducting experimental validation. Taken together, these steps suggest increased confidence in our models.

Virtual Screening of MeONP Inflammatory Potential by Classification Model

The high predictivity (ACC>90%, MCC>80%, AUC>95%) of classification model allowed us to predict the MDEI for a set of MeONPs for which the experimental data have not been available (Figure 6). The virtual screening results were expected to provide guidance for inflammatory potential evaluation: MeONPs with a predicted MDEI<50μg/mL could potentially induce inflammation above this concentration; MeONPs with predicted MDEI<12.5μg/mL (BaO, SrO, Sc2O3, Tb2O3, and Tm2O3) were predicted to have high inflammatory potential. Because all these predicted MeONPs were located in the classification model’s applicability domain, we considered the predictions to be reliable. It should be noted that the model may have limitations for the prediction of MeONPs with exposure doses close to MDEI. This limitation is probably because the variations of FCIL-1β at doses close to MDEI may affect the statistical significances between MeONP treated cells and control cells.

Continuous Model is Capable of MeONP Hazard Ranking

The number of MeONPs used to build the continuous model, from a QSAR viewpoint, was small but sufficient for construction of a predictive model. Classic QSAR studies have been performed and published elsewhere using even smaller sets of nanoparticles (Puzyn et al. 2011). Kubinyi (1993) had recommended R20.81 for in vitro data and R20.64 for in vivo data as cutoffs for picking a good model. Moreover, a criterion (<0.3) of differences between R2 and Q2 was proposed for reliability assessment and applicability evaluation of QSAR models (Eriksson et al. 2003). Taken together, fulfilment of the above criteria (R2=0.892, QCV2=0.875, QEXT2=0.800) suggested that our continuous model can be applied to evaluate the inflammatory potential of new, untested MeONPs. Due to the individual differences between subject organisms/cells, it is unrealistic to expect perfect repeatability and stability of toxicity data generated across different batches of experiments. Therefore, perfect agreement between quantitatively predicted values vs. experimental toxicity data cannot be anticipated (Gajewicz et al. 2015), and the continuous model was built only to comparatively evaluate MeONP hazard ranking. There were two clear outliers in the continuous model, WO3 (Figure 5B) and Ho2O3 (Table S14), which belonged to the test set and the external validation set, respectively. Ho2O3 was considered an outlier because its hydrodynamic diameter (Dwater, 1145 nm) was much bigger than most MeONPs (202800 nm) in the training set. Dwater is a measure of sedimentation capacity of nanoparticles. MeONPs with large Dwater tended to settle at the bottom of the container during cell culture and consequently had high cellular internalization. In contrast, the Dwater of the outlier WO3 (112 nm) was much smaller than that of most MeONPs in training set. As a result, WO3 was more likely to remain suspended in culture media, which considerably reduces cellular internalization. Therefore, the inflammatory potential of Ho2O3 was underestimated, and the inflammatory potential of WO3 was slightly overestimated by the continuous model.

Physicochemical Properties Responsible for Inflammatory Effects of MeONPs

We present here a schematic representation of three hypothesized mechanisms that collectively determine the inflammatory potential of MeONPs (Figure 8).
Figure 8. Proposed schematic image of inflammatory mechanisms by metal oxide nanomaterials (MeONPs). (A) Endocytosis: MeONPs with a positive ζ-potential were most internalized by THP-1 cells and lysosomes. (B) Proton sponge effect. MeONPs with metal atom electronegativity 1.55 tend to trigger a proton sponge effect, followed by lysosome damages, leakage of lysosomal contents and excess IL-1β production. (C) Release of toxic ions.
The classification model pinpointed χme and cation charge are the most efficient factors for predicting inflammatory potential of MeONPs in THP-1 cells. The first factor χme describes the tendency of metal atoms to attract electrons. In other words, metal atoms with lower χme are more favorable for donating electrons to oxygen atoms. Oxygen atoms with more electrons tend to trap positively charged protons (H+). As a result, the adsorption of protons was more likely to occur in MeONPs with lower χme than in those with higher χme. According to the proton sponge effect (Nel et al. 2009), some nanoparticles could enter the acidifying lysosomal compartment and sequester protons that are supplied by the v-ATPase (proton pump). This process keeps the pump functioning and leads to the retention of one Cl ion and one water molecule per proton. Subsequently, lysosomal swelling and rupture cause lysosomal content spillage, triggering excess IL-1β production and inflammation. This mechanistic hypothesis was supported by our experimental results and DFT calculation: MeONPs with χme1.55 exhibited stronger proton adsorption and more inflammatory potential than those with higher χme values.
The second important factor, cation charge, describes the detachment of metal cations from the surface of metal oxides. It has been demonstrated that the release of cations with a smaller charge was more energetically favorable than the release of cations with a larger charge (Kar et al. 2016; Puzyn et al. 2011). Obviously, much more energy is required to detach four electrons than three (or two) to form the appropriate cations (Kar et al. 2016; Puzyn et al. 2011). Thus, it can be inferred that metal oxides having lower cation charge value usually exhibit stronger reductive properties, and thus the ions are easier to detach from the metal oxides, increasing toxicity. The above mechanism explained the toxicity order of the studied MeONPs (Me2+>Me3+>Me4+), which was consistent with our experimental observations and model predictions.
In the continuous model, the descriptors χme and ζ-potential were used to predict inflammatory potential of MeONPs in THP-1 cells. Our results demonstrated that inflammatory potential of MeONPs depended on the cellular internalization and biocompatibility, which were highly dependent on the surface charge (ζ-potential). Cho et al. (2009) found that gold nanoparticles with a positive charge showed higher cellular internalization than negative nanoparticles showed. Moreover, the positively charged MeONPs may actively interact with the negatively charged lysosomal membrane, leading to lysosomal destabilization (Nel et al. 2009). Rupture of the lysosomal membrane can trigger excess IL-1β production and inflammation (Hornung et al. 2008). In concordance with our results, the majority of previous work attributed toxicity to positive ζ-potential (Cho et al. 2009; Pang et al. 2016; Sendra et al. 2018). Experimental results provided strong support to our QSAR analysis and mechanistic interpretation that cellular internalization and lysosomal damage are the key events involved in MeONP-induced inflammation. The ip-MeONPs (La2O3 and Gd2O3) showed slightly higher cellular internalization than other four nanoparticles (ZnO, Co3O4, TiO2 and WO3) showed (Figure S6). Furthermore, confocal imaging of cathepsin B indicated that La2O3 and Gd2O3 induced massive cathepsin B release (Figure S7). The released cathepsin B was able to trigger the assemble of NLRP3 inflammasomes (Li et al. 2013), which would further assist the maturation of IL-1β by caspase-1 activation.
Previous studies (Kumari et al. 2012; Prabhu et al. 2010) revealed that MeONP toxicity is inversely proportional to size. It is interesting to note that, in our investigation, the descriptors that characterize a nanoparticle’s primary size were not selected by the models. A probable explanation was the agglomeration of nanoparticles in cell culture media. Thus, the influence of primary size on MeONP toxicity was insignificant. Moreover, nanoparticle toxicity likely depends on shape. For instance, recent studies suggest that macrophages may struggle to incorporate relatively long (>20μm) and stiff nanoparticles into phagosomes (Nel et al. 2009). However, because the MeONPs used here were in spherical or approximate spherical shape, such phenomenon was not considered in our models. Except for size and shape, MeONPs with different crystal structure and chemical modifications on their surface should be considered in future nano-QSAR model development.
Protein corona consisting of hard and soft layers is also found to affect the biological effects of nanoparticles, including thrombocyte activation, hemolysis, cellular uptake, biodistribution, and cytotoxicity (Cai and Chen 2018). Although the hard layer of corona is long-lived and relatively stable as a monolayer of protein molecules binding directly and tightly on particle surfaces, the soft layer often undergoes dynamic changes as the particles interact with different biological media and cellular compartments. Due to the complexity and variability of corona in physiological conditions, it is challenging to use corona structure as a descriptor for QSAR analysis. In contrast, our study focused more on the intrinsic properties of nanoparticles, such as hydrodynamic diameter, ζ-potential, and electronegativity as descriptors for QSAR analysis because evidence increasingly indicates that the corona structures of nanoparticles are dictated by intrinsic physicochemical properties (Cai and Chen 2018), including hydrodynamic diameter (Carril et al. 2017), ζ-potential (Wang et al. 2013), hydrophobicity (Saha et al. 2014), etc.


A proinflammatory cytokine of IL-1β in THP-1 cells could be used as an index to rank the inflammatory potential of MeONPs. QSAR models were developed and validated for predicting the inflammatory potential of MeONPs. χme, ζ-potential, and cation charge were three key properties responsible for inflammatory effects of MeONPs. DFT computations revealed that a proton adsorption effect of MeONPs with χme1.55 may be responsible for lysosomal damages. We speculate that the QSAR models could be used to predict the inflammatory potential of other untested MeONPs.


This study was supported by the National Natural Science Foundation (21777019, 21477016, and 31671032) of China, the Key Projects for International Science and Technology Cooperation Program, Ministry of Science and Technology of China (2018YFE0120400), and the Programme of Introducing Talents of Discipline to Universities (B13012) of China.
The views presented in this article do not necessarily reflect those of the U.S. Food and Drug Administration.

Article Notes

R.K. is an employee of Immuneering Corporation. All other authors declare they have no actual or potential competing financial interest.

Supplementary Material

File (ehp6508.smcontents.508.pdf)
File (ehp6508.s001.acco.pdf)
File (


Agarwal H, Nakara A, Shanmugam VK. 2019. Anti-inflammatory mechanism of various metal and metal oxide nanoparticles synthesized using plant extracts: a review. Biomed Pharmacother 109:2561–2572.,
Auwerx J. 1991. The human leukemia cell line, THP-1: a multifacetted model for the study of monocyte-macrophage differentiation. Experientia 47(1):22–31.,
Brereton RG, Lloyd GR. 2010. Support vector machines for classification and regression. Analyst 135(2):230–267.,
Bursten JR, Roco MC, Yang W. 2016. Nano on reflection: a number of experts from different areas of nanotechnology describe how the field has evolved in the last ten years. Nat Nanotechnol 11:828–834.,
Bylesjö M, Rantalainen M, Cloarec O, Nicholson JK, Holmes E, Trygg J. 2006. OPLS discriminant analysis: combining the strengths of PLS-DA and SIMCA classification. J Chemometrics 20(8–10):341–351,
Cai R, Chen CY. 2018. The crown and the scepter: roles of the protein corona in nanomedicine. Adv Mater 31(45):1805740.,
Cai XM, Dong J, Liu J, Zheng HZ, Kaweeteerawat C, Wang FJ, et al. 2018. Multi-hierarchical profiling the structure-activity relationships of engineered nanomaterials at nano-bio interfaces. Nat Commun 9(1):4416.,
Cai X, Lee A, Ji Z, Huang C, Chang CH, Wang X, et al. 2017. Reduction of pulmonary toxicity of metal oxide nanoparticles by phosphonate-based surface passivation. Part Fibre Toxicol 14(1):13.,
Cardiff RD, Miller CH, Munn RJ. 2014. Manual hematoxylin and eosin staining of mouse tissue sections. Cold Spring Harb Protoc 2014(6):655–658.,
Carril M, Padro D, Del Pino P, Carrillo-Carrion C, Gallego M, Parak WJ. 2017. In situ detection of the protein corona in complex environments. Nat Commun 8(1):1542.,
Cho EC, Xie J, Wurm PA, Xia Y. 2009. Understanding the role of surface charges in cellular adsorption versus internalization by selectively removing gold nanoparticles on the cell surface with a I-2/KI etchant. Nano Lett 9(3):1080–1084.,
Dreher KL. 2004. Health and environmental impact of nanotechnology: toxicological assessment of manufactured nanoparticles. Toxicol Sci 77(1):3–5.,
Eriksson L, Jaworska J, Worth AP, Cronin MTD, McDowell RM, Gramatica P. 2003. Methods for reliability and uncertainty assessment and for applicability evaluations of classification- and regression-based QSARs. Environ Health Perspect 111(10):1361–1375.,
Fang G, Li W, Shen X, Perez-Aguilar JM, Chong Y, Gao X, et al. 2018. Differential Pd-nanocrystal facets demonstrate distinct antibacterial activity against Gram-positive and Gram-negative bacteria. Nat Commun 9(1):129.,
FindIt. 2009. Inorganic crystal structure database. [accessed 3 June 2020].
Gajewicz A, Rasulev B, Dinadayalane TC, Urbaszek P, Puzyn T, Leszczynska D, et al. 2012. Advancing risk assessment of engineered nanomaterials: application of computational approaches. Adv Drug Deliv Rev 64(15):1663–1693.,
Gajewicz A, Schaeublin N, Rasulev B, Hussain S, Leszczynska D, Puzyn T, et al. 2015. Towards understanding mechanisms governing cytotoxicity of metal oxides nanoparticles: hints from nano-QSAR studies. Nanotoxicology 9(3):313–325.,
Geiser M. 2010. Update on macrophage clearance of inhaled micro- and nanoparticles. J Aerosol Med Pulm Drug Deliv 23(4):207–217.,
George S, Pokhrel S, Ji Z, Henderson BL, Xia T, Li L, et al. 2011. Role of Fe doping in tuning the band gap of TiO2 for the photo-oxidation-induced cytotoxicity paradigm. J Am Chem Soc 133(29):11270–11278.,
Grimme S. 2006. Semiempirical GGA-type density functional constructed with a long-range dispersion correction. J Comput Chem 27(15):1787–1799.,
Han Y, Lee D-K, Kim S-H, Lee S, Jeon S, Cho W-S. 2018. High inflammogenic potential of rare earth oxide nanoparticles: the new hazardous entity. Nanotoxicology 12(7):712–728.,
Hastie T, Tibshirani R, 2001. and Friedman J. The Elements of Statistical Learning, Data Mining, Inference, and Prediction, 1st ed. New York: Springer.
Hornung V, Bauernfeind F, Halle A, Samstad EO, Kono H, Rock KL, et al. 2008. Silica crystals and aluminum salts activate the NALP3 inflammasome through phagosomal destabilization. Nat Immunol 9(8):847–856.,
Inada Y, Orita H. 2008. Efficiency of numerical basis sets for predicting the binding energies of hydrogen bonded complexes: evidence of small basis set superposition error compared to Gaussian basis sets. J Comput Chem 29(2):225–232.,
Industry ARC. 2019. Nanotechnology Market by Type (Nano Composites, Nano Devices, Nano Tools, Nano Materials and Others), by Applications (Medicine & Healthcare, Environment, ICT, Energy, Nano EHS and Others), by End-user Industries (Biotechnology, Cosmetics, Textile, Electronics & Semi-conductor, Automobile and Others)—Forecast (2020–2025). [accessed 3 June 2020].
Jaworska J, Nikolova-Jeliazkova N, Aldenberg T. 2005. QSAR applicability domain estimation by projection of the training set in descriptor space: a review. Altern Lab Anim 33(5):445–459.,
Kar S, Gajewicz A, Roy K, Leszczynski J, Puzyn T. 2016. Extrapolating between toxicity endpoints of metal oxide nanoparticles: predicting toxicity to Escherichia coli and human keratinocyte cell line (HaCaT) with nano-QTTR. Ecotoxicol Environ Saf 126:238–244.,
Kim Y-S, Lim C-H, Shin S-H, Kim J-C. 2017. Twenty-eight-day repeated inhalation toxicity study of nano-sized neodymium oxide in male Sprague-Dawley rats. Toxicol Res 33(3):239–253.,
Kubinyi H. 1993. Methods and Principles in Medicinal Chemistry, vol. 1. New York: VCH Publishers.
Kumari M, Rajak S, Singh SP, Kumari SI, Kumar PU, Murty USN, et al. 2012. Repeated oral dose toxicity of iron oxide nanoparticles: biochemical and histopathological alterations in different tissues of rats. J Nanosci Nanotechnol 12(3):2149–2159.,
Larsen ST, Jackson P, Poulsen SS, Levin M, Jensen KA, Wallin H, et al. 2016. Airway irritation, inflammation, and toxicity in mice following inhalation of metal oxide nanoparticles. Nanotoxicology 10(9):1254–1236.,
Le TC, Winkler DA. 2016. Discovery and optimization of materials using evolutionary approaches. Chem Rev 116(10):6107–6132.,
Li R, Guiney ML, Chang HC, Mansukhani DN, Ji Z, Wang X, et al. 2018. Surface oxidation of graphene oxide determines membrane damage, lipid peroxidation, and cytotoxicity in macrophages in a pulmonary toxicity model. ACS Nano 12(2):1390–1402.,
Li R, Ji Z, Chang CH, Dunphy DR, Cai X, Meng H, et al. 2014a. Surface interactions with compartmentalized cellular phosphates explain rare earth oxide nanoparticle hazard and provide opportunities for safer design. ACS Nano 8(2):1771–1783.,
Li R, Ji Z, Qin H, Kang X, Sun B, Wang M, et al. 2014b. Interference in autophagosome fusion by rare earth nanoparticles disrupts autophagic flux and regulation of an interleukin-1β producing inflammasome. ACS Nano 8(10):10280–10292.,
Li R, Wang X, Ji Z, Sun B, Zhang H, Chang CH, et al. 2013. Surface charge and cellular processing of covalently functionalized multiwall carbon nanotubes determine pulmonary toxicity. ACS Nano 7(3):2352–2368.,
Li Y, Yu S, Yuan T, Yan M, Jiang Y. 2015. Rational design of metal oxide nanocomposite anodes for advanced lithium ion batteries. J Power Sources 282:1–8,
Liu SJ, Lu YL, Chen W. 2018. Bridge knowledge gaps in environmental health and safety for sustainable development of nano-industries. Nano Today 23:11–15,
Liu P, Rodriguez JA. 2005. Catalysts for hydrogen evolution from the [NiFe] hydrogenase to the Ni2P(001) surface: the importance of ensemble effect. J Am Chem Soc 127(42):14871–14878.,
Matthews BW. 1975. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim Biophys Acta 405(2):442–451.,
Nel AE, Lutz MD, Darrell V, Tian X, Hoek EMV, Ponisseril S, et al. 2009. Understanding biophysicochemical interactions at the nano-bio interface. Nat Mater 8(7):543–557.,
Nel AE, Xia T, Mädler L, Li N. 2006. Toxic potential of materials at the nanolevel. Science 311(5761):622–627.,
Nel AE, Xia T, Meng H, Wang X, Lin S, Ji Z, et al. 2013. Nanomaterial toxicity testing in the 21st century: use of a predictive toxicological approach and high-throughput screening. Acc Chem Res 46(3):607–621.,
Niels H, Feriel R, Nicklas RJ, Anne TS, Petra J, Stefan B, et al. 2019. Acute phase response and inflammation following pulmonary exposure to low doses of zinc oxide nanoparticles in mice. Nanotoxicology 23:1–18.,
O’Boyle NM, Banck M, James CA, Morley C, Vandermeersch T, Hutchison GR. 2011. Open Babel: an open chemical toolbox. J Cheminform 3:33.,
OECD (Organization for Economic Cooperation and Development). 2004. Principles for the Validation, for Regulatory Purposes, of (Quantitative) Structure Activity Relationship Models. 37th Joint Meeting of The Chemicals Committee and Working Party on Chemicals, Pesticides and Biotechnology. Paris, France: Organisation for Economic Co-Operation and Development.
Oh E, Liu R, Nel A, Gemill KB, Bilal M, Cohen Y, et al. 2016. Meta-analysis of cellular toxicity for cadmium-containing quantum dots. Nat Nanotechnol 11(5):479–486.,
Osei-Bryson KM. 2004. Evaluation of decision trees: a multi-criteria approach. Comput Oper Res 31(11):1933–1945,
Pang C, Brunelli A, Zhu C, Hristozov D, Liu Y, Semenzin E, et al. 2016. Demonstrating approaches to chemically modify the surface of Ag nanoparticles in order to influence their cytotoxicity and biodistribution after single dose acute intravenous administration. Nanotoxicology 10(2):129–139.,
Perdew JP, Burke K, Ernzerhof M. 1996. Generalized gradient approximation made simple. Phys Rev Lett 77(18):3865–3868.,
Prabhu BM, Ali SF, Murdock RC, Hussain SM, Srivatsan M. 2010. Copper nanoparticles exert size and concentration dependent toxicity on somatosensory neurons of rat. Nanotoxicology 4(2):150–160.,
Puzyn T, Rasulev B, Gajewicz A, Hu XK, Dasari TP, Michalkova A, et al. 2011. Using nano-QSAR to predict the cytotoxicity of metal oxide nanoparticles. Nat Nanotechnol 6(3):175–178.,
Quinlan JR. 1986. Induction of decision trees. Mach Learn 1(1):81–106,
Ray JL, Holian A. 2019. Sex differences in the inflammatory immune response to multi-walled carbon nanotubes and crystalline silica. Inhal Toxicol 31(7):285–297.,
REACH (Registration, Evaluation, Authorisation and Restriction of Chemicals). 2006. European Commission Regulations (EC) NO 1907/2006 of the European Parliament and of the Council of 18 December 2006 Concerning the Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH), Establishing a European Chemical Agency, Amending Directive 1999/4 5/EC and repealing Council Regulation (EEC) No 793/93 and Commission Regulation (EC) No 1488/94 as well as Council Directive 76/769/EEC and Commission Directives 91/155/EEC, 93/67/EEC, 93/105/EC and 2000/21/EC. Luxembourg City, Luxembourg: Off J Eur Communities. [accessed 3 June 2020].
Roy K, Kar S, Ambure P. 2015. On a simple approach for determining applicability domain of QSAR models. Chemometrics Intell Lab Syst 145:22–29,
Roy P, Roy K. 2008. On some aspects of variable selection for partial least squares regression models. QSAR Comb Sci 27(3):302–313,
Saha K, Moyano DF, Rotello VM. 2014. Protein coronas suppress the hemolytic activity of hydrophilic and hydrophobic nanoparticles. Mater Horiz 1(1):102–105.,
Sendra M, Volland M, Balbi T, Fabbri R, Yeste MP, Gatica JM, et al. 2018. Cytotoxicity of CeO2 nanoparticles using in vitro assay with mytilus galloprovincialis hemocytes: relevance of zeta potential, shape and biocorona formation. Aquat Toxicol 200:13–20.,
Stewart JJ. 2016. Stewart computational chemistry—mopac home page. [accessed 3 June 2020].
Sun F, Xiao G, Qu Z. 2017. Murine bronchoalveolar lavage. Bio Protoc 7(10):e2287.,
Taeho K, Taeghwan H. 2014. Applications of inorganic nanoparticles as therapeutic agents. Nanotechnology 25:012001.,
Tani T, Mädler L, Pratsinis SE. 2002. Homogeneous ZnO nanoparticles by flame spray pyrolysis. J Nanopart Res 4(4):337–343,
U.S. FDA (U.S. Food and Drug Administration). 2014. Guidance for Industry. Safety of Nanomaterials in Cosmetic Products. [accessed 3 June 2020].
Vocaturo G, Colombo F, Zanoni M, Rodi F, Sabbioni E, Pietra R. 1983. Human exposure to heavy metals. Rare earth pneumoconiosis in occupational workers. Chest 83(5):780–783.,
Wang L, Li J, Pan J, Jiang X, Ji Y, Li Y, et al. 2013. Revealing the binding structure of the protein corona on gold nanorods using synchrotron radiation-based techniques: understanding the reduced damage in cell membranes. J Am Chem Soc 135(46):17359–17368.,
Wang W, Sedykh A, Sun H, Zhao L, Russo DP, Zhou H, et al. 2017. Predicting nano-bio interactions by integrating nanoparticle libraries and quantitative nanostructure activity relationship modeling. ACS Nano 11(12):12641–12649.,
Wang X, Xia T, Ntim SA, Ji Z, Lin S, Meng H, et al. 2011. Dispersal state of multiwalled carbon nanotubes elicits profibrogenic cellular responses that correlate with fibrogenesis biomarkers and fibrosis in the murine lung. ACS Nano 5(12):9772–9787.,
Wan-Seob C, Rodger D, Poland CA, Howie SEM, William MN, Mark B, et al. 2010. Metal oxide nanoparticles induce unique inflammatory footprints in the lung: important implications for nanoparticle testing. Environ Health Perspect 118(12):1699–1706.,
Wold H. 1982. Soft modeling: the basic design and some extensions. In Systems under Indirect Observation: Part 2. Joreskog KG, Wold H, eds. Amsterdam, Netherlands: North Holland Press, 343.
Yang K, Xing B. 2010. Adsorption of organic compounds by carbon nanomaterials in aqueous phase: polanyi theory and its application. Chem Rev 110(10):5989–6008.,
Zhang X, Xu Y, Zhou L, Zhang CC, Meng QT, Wu SS, et al. 2015. Sex-dependent depression-like behavior induced by respiratory administration of aluminum oxide nanoparticles. Int J Environ Res Public Health 12(12):15692–15705.,
Zhao Z, Tian J, Sang Y, Cabot A, Liu H. 2015. Structure, synthesis, and applications of TiO2 nanobelts. Adv Mater Weinheim 27(16):2557–2582.,
Zhu M, Li Y, Shi J, Feng W, Nie G, Zhao Y. 2012. eExosomes as extrapulmonary signaling conveyors for nanoparticle-induced systemic immune activation. Small 8(3):404–412.,

Information & Authors


Published In

Environmental Health Perspectives
Volume 128Issue 6June 2020
PubMed: 32692251


Received: 7 November 2019
Revision received: 14 May 2020
Accepted: 18 May 2020
Published online: 12 June 2020



Yang Huang
Key Laboratory of Industrial Ecology and Environmental Engineering, School of Environmental Science and Technology, Dalian University of Technology, Dalian, China
Xuehua Li
Key Laboratory of Industrial Ecology and Environmental Engineering, School of Environmental Science and Technology, Dalian University of Technology, Dalian, China
Shujuan Xu
State Key Laboratory of Radiation Medicine and Protection, Collaborative Innovation Center of Radiological Medicine of Jiangsu Higher Education Institutions, School for Radiological and Interdisciplinary Sciences (RAD-X), Soochow University, Suzhou, Jiangsu, China
Huizhen Zheng
State Key Laboratory of Radiation Medicine and Protection, Collaborative Innovation Center of Radiological Medicine of Jiangsu Higher Education Institutions, School for Radiological and Interdisciplinary Sciences (RAD-X), Soochow University, Suzhou, Jiangsu, China
Lili Zhang
Key Laboratory of Industrial Ecology and Environmental Engineering, School of Environmental Science and Technology, Dalian University of Technology, Dalian, China
Jingwen Chen
Key Laboratory of Industrial Ecology and Environmental Engineering, School of Environmental Science and Technology, Dalian University of Technology, Dalian, China
Huixiao Hong
Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, U.S. Food and Drug Administration, Jefferson, Arkansas, USA
Rebecca Kusko
Immuneering Corporation, One Broadway, 14th Floor, Cambridge, Massachusetts, USA
Ruibin Li
State Key Laboratory of Radiation Medicine and Protection, Collaborative Innovation Center of Radiological Medicine of Jiangsu Higher Education Institutions, School for Radiological and Interdisciplinary Sciences (RAD-X), Soochow University, Suzhou, Jiangsu, China


Address correspondence to X. Li, Dalian University of Technology, Linggong Rd. 2, Dalian 116024, PR China. Email: [email protected] or R. Li, 99 Ren’ai Rd., 401 Building, Suzhou 215123, PR China. Email: [email protected]

Metrics & Citations


About Article Metrics


Download citation

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click DOWNLOAD.

Cited by

  • Exploring the relationships between physiochemical properties of nanoparticles and cell damage to combat cancer growth using simple periodic table-based descriptors, Beilstein Journal of Nanotechnology, 10.3762/bjnano.15.27, 15, (297-309), (2024).
  • Computational Nanotoxicology Models for Environmental Risk Assessment of Engineered Nanomaterials, Nanomaterials, 10.3390/nano14020155, 14, 2, (155), (2024).
  • Fingerprinting Interactions between Proteins and Ligands for Facilitating Machine Learning in Drug Discovery, Biomolecules, 10.3390/biom14010072, 14, 1, (72), (2024).
  • BERT-based language model for accurate drug adverse event extraction from social media: implementation, evaluation, and contributions to pharmacovigilance practices, Frontiers in Public Health, 10.3389/fpubh.2024.1392180, 12, (2024).
  • Package delivered: folate receptor-mediated transporters in cancer therapy and diagnosis, Chemical Science, 10.1039/D3SC05539F, 15, 6, (1966-2006), (2024).
  • Predicting Bioaccumulation of Nanomaterials: Modeling Approaches with Challenges, Environment & Health, 10.1021/envhealth.3c00138, 2, 4, (189-201), (2024).
  • Harmonization Risks and Rewards: Nano-QSAR for Agricultural Nanomaterials, Journal of Agricultural and Food Chemistry, 10.1021/acs.jafc.3c06466, 72, 6, (2835-2852), (2024).
  • Ferroptosis Induced by Pollutants: An Emerging Mechanism in Environmental Toxicology, Environmental Science & Technology, 10.1021/acs.est.3c06127, 58, 5, (2166-2184), (2024).
  • Machine learning-based models to predict aquatic ecological risk for engineered nanoparticles: using hazard concentration for 5% of species as an endpoint, Environmental Science and Pollution Research, 10.1007/s11356-024-32723-1, 31, 17, (25114-25128), (2024).
  • Application of Computing as a High-Practicability and -Efficiency Auxiliary Tool in Nanodrugs Discovery, Pharmaceutics, 10.3390/pharmaceutics15041064, 15, 4, (1064), (2023).
  • See more

View Options

View options


View PDF

Get Access

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.







Copy the content Link

Share on social media