Demonstration of a diagnostic clinical laboratory neural network agent applied to three laboratory data conditioning problems
Izaak Mayzlin Larry Bernstein, MD
Principal Scientist, MayNet Technical Director
Boston, MA Methodist Hospital Laboratory, Brooklyn, NY
Our clinical chemistry section services a hospital emergency room seeing 15,000 patients with chest pain annually. We have used a neural network agent, MayNet, for data conditioning. Three applications are – troponin, CKMB, EKG for chest pain; B-type natriuretic peptide (BNP), EKG for congestive heart failure (CHF); and red cell count (RBC), mean corpuscular volume (MCV), hemoglobin A2 (Hgb A2) for beta thalassemia. Three data sets have been extensively validated prior to neural network analysis using receiver-operator curve (ROC analysis), Latent Class Analysis, and a multinomial regression approach. Optimum decision points for classifying using these data were determined using ROC (SYSTAT, 11.0), LCM (Latent Gold), and ordinal regression (GOLDminer). The ACS and CHF studies both had over 700 patients, and had a different validation sample than the initial exploratory population. The MayNet incorporates prior clustering, and sample extraction features in its application. Maynet results are in agreement with the other methods.
Introduction: A clinical laboratory servicing a hospital with an emergency room seeing 15,000 patients with chest pain to produce over 2 million quality controlled chemistry accessions annually. We have used a neural network agent, MayNet, to tackle the quality control of the information product. The agent combines a statistical tool that first performs clustering of input variables by Euclidean distances in multi-dimensional space. The clusters are trained on output variables by the artificial neural network performing non-linear discrimination on clusters’ averages. In applying this new agent system to diagnosis of acute myocardial infarction (AMI) we demonstrated that at an optimum clustering distance the number of classes is minimized with efficient training on the neural network. The software agent also performs a random partitioning of the patients’ data into training and testing sets, one time neural network training, and an accuracy estimate on the testing data set. Three examples to illustrate this are – troponin, CKMB, EKG for acute coronary syndrome (ACS); B-type natriuretic peptide (BNP), EKG for the estimation of ejection fraction in congestive heart failure (CHF); and red cell count (RBC), mean corpuscular volume (MCV), hemoglobin A2 (Hgb A2) for identifying beta thalassemia. We use three data sets that have been extensively validated prior to neural network analysis using receiver-operator curve (ROC analysis), Latent Class Analysis, and a multinomial regression approach.
In previous studies1,2 CK-MB and LD1 sampled at 12 and 18 hours postadmission were near-optimum times used to form a classification by the analysis of information in the data set. The population consisted of 101 patients with and 41 patients without AMI based on review of the medical records, clinical presentation, electrocardiography, serial enzyme and isoenzyme assays, and other tests. The clinical or EKG data, and other enzymes or sampling times were not used to form a classification but could be handled by the program developed. All diagnoses were established by cardiologist review. An important methodological problem is the assignment of a correct diagnosis by a “gold standard” that is independent of the method being tested so that the method tested can be suitably validated. This solution is not satisfactory in the case of myocardial infarction because of the dependence of diagnosis on a constellation of observations with different sensitivities and specificities. We have argued that the accuracy of diagnosis is associated with the classes formed by combined features and has greatest uncertainty associated with a single measure.
Methods: Neural network analysis is by MayNet, developed by one of the authors. Optimum decision points for classifying using these data were determined using ROC (SYSTAT, 11.0), LCM (Latent Gold)3, and ordinal regression (GOLDminer)4. Validation of the ACS and CHF study sets both had over 700 patients, and all studies had a different validation sample than the initial exploratory population. The MayNet incorporates prior clustering, and sample extraction features in its application. We now report on a new classification method and its application to diagnosis of acute myocardial infarction (AMI). This method is based on the combination of clustering by Euclidean distances in multi-dimensional space and non-linear discrimination fulfilled by the Artificial Neural Network (ANN) trained on clusters’ averages. These studies indicate that at an optimum clustering distance the number of classes is minimized with efficient training on the ANN. This novel approach to ANN reduces the number of patterns used for ANN learning and works also as an effective tool for smoothing data, removing singularities, and increasing the accuracy of classification by the ANN. The studies conducted involve training and testing on separate clinical data sets, which subsequently achieves a high accuracy of diagnosis (97%).
Unlike classification, which assumes the prior definition of borders between classes5,6, clustering procedure includes establishing these borders as a result of processing statistical information and using a given criteria for difference (distance) between classes. We perform clustering using the geometrical (Euclidean) distance between two points in n-dimensional space, formed by n variables, including both input and output variables. Since this distance assumes compatibility of different variables, the values of all input variables are linearly transformed (scaled) to the range from 0 to 1.
The ANN technique for readers accustomed to classical statistics can be viewed as an extension of multivariate regression analyses with such new features as non-linearity and ability to process categorical data. Categorical (not continuous) variables represent two or more levels, groups, or classes of correspondent feature, and in our case this concept is used to signify patient condition, for example existence or not of AMI.
The ANN is an acyclic directed graph with input and output nodes corresponding respectively to input and output variables. There are also “intermediate” nodes, comprising so called “hidden” layers. Each node nj is assigned the value xj that has been evaluated by the node’s “processing” element, as a non-linear function of the weighted sum of values xi of nodes ni, connected with nj by directed edges (ni, nj).
xj = f(wi(1),jxi(1) + wi(2),jxi(2) + … + wi(l),jxi(l)),
where xk is the value in node nk and wk,j is the “weight” of the edge (nk, nj). In our research we used the standard function f(x), “sigmoid”, defined as f(x)=1/(1+exp(-x)). This function is suitable for categorical output and allows for using an efficient back-propagation algorithm7 for calculating the optimal values of weights, providing the best fit for learning set of data, and eventually the most accurate classification.
Process description: We implemented the proposed algorithm for diagnosis of AMI. All the calculations were performed on PC with Pentium 3 Processor applying the authors’ unique Software Agent Maynet. First, using the automatic random extraction procedure, the initial data set (139 patients) was partitioned into two sets — training and testing. This randomization also determined the size of these sets (96 and 43, respectively) since the program was instructed to assign approximately 70 % of data to the training set.
The main process consists of three successive steps: (1) clustering performed on training data set, (2) neural network’s training on clusters from previous step, and (3) classifier’s accuracy evaluation on testing data.
The classifier in this research will be the ANN, created on step 2, with output in the range [0,1], that provides binary result (1 – AMI, 0 – not AMI), using decision point 0.5.
In this demonstartion we used the data of two previous studies1,2 with three patients, potential outliers, removed (n = 139). The data contains three input variables, CK-MB, LD-1, LD-1/total LD, and one output variable, diagnoses, coded as 1 (for AMI) or 0 (non-AMI).
Results: The application of this software intelligent agent is first demonstrated here using the initial model. Figures 1-2 illustrate the history of training process. One function is the maximum (among training patterns) and lower function shows the average error. The latter defines duration of training process. Training terminates when the average error achieves 5%.
There was slow convergence of back-propagation algorithm applied to the training set of 96 patients. We needed 6800 iterations to achieve the sufficiently small (5%) average error.
Figure 1 shows the process of training on stage 2. It illustrates rapid convergence because we deal only with 9 patterns representing the 9 classes, formed on step 1.
Table 1 illustrates the effect of selection of maximum distance on the number of classes formed and on the production of errors. The number of classes increased with decreasing distance, but accuracy of classification does not decreased.
The rate of learning is inversely related to the number of classes. The use of the back-propagation to train on the entire data set without prior processing is slower than for the training on patterns.
Figures 2 is a two-dimensional projection of three-dimensional space of input variables CKMB and LD1 with small dots corresponding to the patterns and rectangular as cluster centroids (black – AMI, white – not AMI).
We carried out a larger study using troponin I (instead of LD1) and CKMB for the diagnosis of myocardial infarction (MI). The probabilities and odds-ratios for the TnI scaled into intervals near the entropy decision point are shown in Table 2 (N = 782). The cross-table shows the frequencies for scaled TnI results versus the observed MI, the percent of values within MI, and the predicted probabilities and odds-ratios for MI within TnI intervals. The optimum decision point is at or near 0.61 mg/L (the probability of MI at 0.46-0.6 mg/L is 3% and the odds ratio is at 13, while the probability of MI at 0.61-0.75 mg/L is 26% at an odds ratio of 174) by regressing the scaled values.
The RBC, MCV criteria used were applied to a series of 40 patients different than that used in deriving the cutoffs. A latent class cluster analysis is shown in Table 3. MayNet is carried out on all 3 data sets for MI, CHF, and for beta thalassemia for comparison and will be shown.
Discussion: CKMB has been heavily used for a long time to determine heart attacks. It is used in conjunction with a troponin test and the EKG to identify MI but, it isn’t as sensitive as is needed. A joint committee of the AmericanCollege of Cardiology and European Society of Cardiology (ACC/ESC) has established the criteria for acute, recent or evolving AMI predicated on a typical increase in troponin in the clinical setting of myocardial ischemia (1), which includes the 99th percentile of a healthy normal population. The improper selection of a troponin decision value is, however, likely to increase over use of hospital resources. A study by Zarich8 showed that using an MI cutoff concentration for TnT from a non-acute coronary syndrome (ACS) reference improves risk stratification, but fails to detect a positive TnT in 11.7% of subjects with an ACS syndrome8. The specificity of the test increased from 88.4% to 96.7% with corresponding negative predictive values of 99.7% and 96.2%. Lin et al.9 recently reported that the use of low reference cutoffs suggested by the new guidelines results in markedly increased TnI-positive cases overall. Associated with a positive TnI and a negative CKMB, these cases are most likely false positive for MI. Maynet relieves this and the following problem effectively.
Monitoring BNP levels is a new and highly efficient way of diagnosing CHF as well as excluding non-cardiac causes of shortness of breath. Listening to breath sounds is only accurate when the disease is advanced to the stage in which the pumping function of the heart is impaired. The pumping of the heart is impaired when the circulation pressure increases above the osmotic pressure of the blood proteins that keep fluid in the circulation, causing fluid to pass into the lung’s airspaces. Our studies combine the BNP with the EKG measurement of QRS duration to predict whether a patient has a high or low ejection fraction, a measure to stage the severity of CHF.
We also had to integrate the information from the hemogram (RBC, MCV) with the hemoglobin A2 quantitation (BioRad Variant II) for the diagnosis of beta thalassemia. We chose an approach to the data that requires no assumption about the distribution of test values or the variances. Our detailed analyses validates an approach to thalassemia screening that has been widely used, the Mentzer index10, and in addition uses critical decision values for the tests that are used in the Mentzer index. We also showed that Hgb S has an effect on both Hgb A2 and Hgb F. This study is adequately powered to assess the usefulness of the Hgb A2 criteria but not adequately powered to assess thalassemias with elevated Hgb F.
References:
1. Adan J, Bernstein LH, Babb J. Lactate dehydrogenase isoenzyme-1/total ratio: accurate for determining the existence of myocardial infarction. Clin Chem 1986;32:624-8.
2. Rudolph RA, Bernstein LH, Babb J. Information induction for predicting acute myocardial infarction. Clin Chem 1988;34:2031- 2038.
3. Magidson J. “Maximum Likelihood Assessment of Clinical Trials Based on an Ordered Categorical Response.” Drug Information Journal, Maple Glen, PA: Drug Information Association 1996;309[1]: 143-170.
4. Magidson J and Vermoent J. Latent Class Cluster Analysis. in J. A. Hagenaars and A. L. McCutcheon (eds.), Applied Latent Class Analysis. Cambridge: CambridgeUniversity Press, 2002, pp. 89-106.
5. Mkhitarian VS, Mayzlin IE, Troshin LI, Borisenko LV. Classification of the base objects upon integral parameters of the attached network. Applied Mathematics and Computers. Moscow, USSR: Statistika, 1976: 118-24.
6.Mayzlin IE, Mkhitarian VS. Determining the optimal bounds for objects of different classes. In: Dubrow AM, ed. Computational Mathematics and Applications. MoscowUSSR: Economics and Statistics Institute. 1976: 102-105.
7. RumelhartDE, Hinton GE, Williams RJ. Learning internal representations by error propagation. In:
RumelhartDE, Mc Clelland JL, eds. Parallel distributed processing. Cambridge, Mass: MIT Press, 1986; 1: 318-62.
8. Zarich SW, Bradley K, Mayall ID, Bernstein, LH. Minor Elevations in Troponin T Values Enhance Risk Assessment in Emergency Department Patients with Suspected Myocardial Ischemia: Analysis of Novel Troponin T Cut-off Values. Clin Chim Acta 2004 (in press).
9. Lin JC, Apple FS, Murakami MM, Luepker RV. Rates of positive cardiac troponin I and creatine kinase MB mass among patients hospitalized for suspected acute coronary syndromes. Clin Chem 2004;50:333-338.
10.Makris PE. Utilization of a new index to distinguish heterozygous thalassemic syndromes: comparison of its specificity to five other discriminants.Blood Cells. 1989;15(3):497-506.
Acknowledgements: Jerard Kneifati-Hayek and Madeleine Schlefer, Midwood High School, Brooklyn, and Salman Haq, Cardiology Fellow, Methodist Hospital.
Table 1. Effect of selection of maximum distance on the number of classes formed and on the accuracy of recognition by ANN
| ClusteringDistanceFactor F(D = F * R) | Number ofClasses | Number of Nodes inThe HiddenLayers | Number ofMisrecognizedPatterns inThe TestingSet of 43 | Percent ofMisrecognized |
| 10.90.80.7 | 2414135 | 1, 02, 03, 01, 02, 03, 0
3, 2 3, 2 |
121121
1 1 |
2.34.62.32.34.62.3
2.3 2.3 |
Figure 1.
Figure 2.
Table 2. Frequency cross-table, probabilities and odds-ratios for scaled TnI versus expected diagnosis
| Range | Not MI | MI | N | Pct in MI | Prob by TnI | Odds Ratio |
| < 0.45 | 655 | 2 | 657 | 2 | 0 | 1 |
| 0.46-0.6 | 7 | 0 | 7 | 0 | 0.03 | 13 |
| 0.61-0.75 | 4 | 0 | 4 | 0. | 0.26 | 175 |
| 0.76-0.9 | 13 | 59 | 72 | 57.3 | 0.82 | 2307 |
| > 0.9 | 0 | 42 | 42 | 40.8 | 0.98 | 30482 |
| 679 | 103 | 782 | 100 |
2012pharmaceutical
Amandeep Kaur
Aashir Awan, Phd
Abhisar Anand
Adina Hazan
Alan F. Kaul, PharmD., MS, MBA, FCCP
alexcrystal6
anamikasarkar
apreconasia
aptubman
Arav Gandhi
aviralvatsa
David Orchard-Webb, PhD
danutdaagmailcom
Demet Sag, Ph.D., CRA, GCP
Dror Nir
dsmolyar
Ethan Coomber
evelinacohn
Gail S Thornton
Irina Robu
jayzmit48
jdpmd
jshertok
kellyperlman
Ed Kislauskis
larryhbern
Madison Davis
marzankhan
megbaker58
ofermar2020
Dr. Pati
pkandala
ritusaxena
Rick Mandahl
sjwilliamspa
Srinivas Sriram
stuartlpbi
Dr. Sudipta Saha
tildabarliya
zraviv06
zs22