Comparison of CCI, BISAP and APACHE II Scoring Systems to Predict Severe Disease in Patients with Mild Acute Pancreatitis: A Retrospective Observational Study

ABSTRACT


Introduction
Acute Pancreatitis (AP) is one of the most common acute gastrointestinal disorders requiring hospitalization worldwide. AP imposes a significant burden on the health system with an incidence of 13-45/100,000 person-years 2 . The clinical spectrum of AP ranges from mild cases to severe cases that can result in death 1, 2 . AP is divided into 3 groups as mild, moderate, and severe according to the revised Atlanta classification (RAC) 3 . Patients with mild AP (MAP) usually recover within a few days to a week, furthermore, the mortality rate in this group is less than 1% 3 . Moderate acute pancreatitis (MSAP) is characterized by the presence of transient organ failure or local/systemic complications 3,4 . Severe AP (SAP) is defined as permanent organ failure, and patients in this group usually have one or more local and/or systemic complications 3 . SAP may develop in 10% to 20% of all AP patients, however, high mortality rates of up to 50% can be seen in this group 5,6 .
The ability to predict its severity can help identify patients at increased risk for morbidity and mortality, thereby assisting in appropriate early triage to intensive care units and the selection of patients for specific interventions 7 . Many scoring systems have been developed to assess the prognosis and disease severity in AP [8][9][10] . The Bedside Index of Severity in Acute Pancreatitis (BISAP) and the Acute Physiology and Chronic Health Evaluation (APACHE II) scores are gaining popularity among clinicians as assessment tools that provide information about disease severity 10,11 . However, when the APACHE II score and the BİSAP score indicate severe disease, the patient's condition is apparent regardless of the score. The ability of these two scores to predict the risk of severe illness in the subgroup of AP patients who have not yet developed organ failure remains unclear. Moreover, individual patient response in AP patients is highly variable, so scores that do not adequately include comorbidities in the calculations may be difficult to predict clinical prognosis 12 .
This study focused on a subgroup of AP patients classified as MAP by the RAC. We aimed to compare the effectiveness of the Charlson Comorbidity Index (CCI), BİSAP, and APACHE II scores in predicting 30day mortality and SAP development in patients with MAP.

Study design
We conducted a retrospective, observational and cross-sectional study based on the review of the clinical documentation databases at our hospital. We analyzed the clinical records of patients diagnosed with AP between January 01, 2018, and April 01, 2022.

Definitions and classification
According to the current guidelines of the American College of Gastroenterology, those who met two or more criteria were considered AP: 1) typical upper abdominal pain with acute onset and usually radiating to the back, 2) elevation of lipase or amylase levels at least three times the normal upper range, and 3) findings on abdominal imaging consistent with AP 2 . AP was classified as MAP, MSAP, and SAP according to the 2012 revision of the RAC 3 . Accordingly, those without organ failure and local or systemic complications were considered MAP; those with temporary organ failure and/or local or systemic complications that resolved within 48 hours were considered MSAP, and those with permanent organ failure that did not resolve within 48 hours were considered SAP 3 . Local and systemic complications were defined according to the RAC. Accordingly, acute peripancreatic fluid collections, pancreatic pseudocysts, acute necrotic collection, well-circumscribed wall necrosis, gastric outlet dysfunction, splenic and portal vein thrombosis, and colonic necrosis were considered local complications. As a systemic complication, the definition of exacerbation of existing chronic diseases due to AP was used 3 . Organ failure was defined as a score of 2 or more using the modified Marshall scoring system for the renal, cardiovascular, or respiratory organ system 13 . SIRS was defined as the presence of at least two of the following: heart rate >90 bpm, respiratory rate >20/min, or arterial paCO2 <32 mmHg, body temperature <36°C or 38°C, leukocyte count >12000/ml or <4000/ml.

Inclusion and exclusion criteria
Each patient's first episode of mild interstitial AP in the study period was included. During the study period, patients who visited the emergency department (ED) and were hospitalized with the diagnosis of AP according to the International Classification of Diseases (ICD 10th revision, code K 85) were scanned from the hospital's electronic medical record system (EMRS). The following patients were excluded from the study: those classified as SAP or MSAP within the first 48 hours of arrival at the ED, pregnant, those with recurrent AP or pancreatic surgery, and those with missing data.

Study variables
Demographic characteristics (age, gender, comorbidities), vital signs, and laboratory and radiological data at the time of arrival to the ED were recorded in the study form and analyzed. AP patients were categorized as MAP, MSAP, and SAP based on their data for the first 48 hours after symptom onset. The clinical outcomes of patients with MAP, including progression to SAP, death, and hospitalization time, were recorded and analyzed. The CCI was calculated based on physician and nurse notes from the hospital's EMRS (Table 1) 14 . The APACHE II and BISAP scores were calculated using data from the first 24 hours after the ED visit (Table 1). No points were added to the scores for missing data.

Endpoints of the study
The primary endpoint was SAP development in patients with MAP. The secondary endpoint was survival within 30 days of ED visits in patients with MAP.

Statistical analysis
Statistical analyses were performed with SPSS 23.0 (IBM Corp., Chicago, IL, USA) and MedCalc 12.3.0.0 (MedCalc Software, Mariakerke, Belgium). Normality analyses of the data were conducted using histograms and the Kolmogorov-Smirnov test. As appropriate, continuous variables were presented as the mean±SD or median (25% to 75% interquartile range [IQR]), and categorical variables were presented as counts and percentages. Normally distributed data were analyzed using Student's t-test. Data with abnormal distribution were analyzed using Mann-Whitney U tests. Intragroup comparisons of the categorical variables were made using the chi-square test and Fisher's exact test. To evaluate the ability of CCI, APACHE II, and BISAP scores to predict SAP development and 30-day mortality, receiver operating characteristic (ROC) curve analysis was performed, and the area under the curve (AUC) was compared ( Figure 2). AUCs derived were further compared using the De Long test 15 . Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were also calculated based on the cutoff values (CCI ≥2, BISAP ≥2, APACHE II ≥8) determined according to previous studies (Table 4) 16,17 . The significance level was taken as P < 0.05.

Ethical aspects
This study was approved by the ethics committee of the research institution (Protocol No: 2022/157 dated: 09.05.2022) and conducted following the Declaration of Helsinki. Due to the study's retrospective nature, the requirement for informed consent was waived; however, informed consent about the risks of AP and all treatment modalities was obtained from all patients at their first visit. Finally, we used the STROBE checklist for cross-sectional studies to design the research and write this original article 18 .

Results
A total of 1419 patients with MAP were included in this study (Figure 1). The mean age of the patients was 55.2±18.7 years, and 57.4% (n=815) were female. The rate of alcohol use in the patients was 5.4% (n=82). The most common comorbidities were diabetes mellitus (27.8%, n=395), chronic lung disease (16.9%, n=240), and liver diseases (15.8%, n=224). None of the patients had AIDS, Leukemia, and Lymphoma.
All patients included in the study were hospitalized and treated in our hospital, and the median length of stay was six days (IQR 5-7). In this study, 14.4% (n=205) of patients with MAP developed SAP. The 30-day mortality rate was 1.8% (n=25) in all patients and 11.2% in patients with SAP. The median age in patients with SAP was statistically higher than that in patients without SAP (69 [IQR: 39-69] vs. 54 [IQR: 57-79], p<0.001). There was no statistically significant difference between the gender regarding SAP development (p=0.458). The incidence of many comorbid diseases (solid tumor, moderate or severe liver disease, hemi or paraplegia, moderate or severe renal disease, diabetes with end-organ damage, congestive heart failure, cerebrovascular accident or transient ischemic attack, dementia, and chronic pulmonary disease) in patients with SAP was higher than that in patients without SAP. The mean sodium (131.0±5.8 vs. 135.7±4.5; p<0.001) and potassium (4.17 ±0.76 vs. 4.48±0.57; p<0.001) concentrations were lower in the patient group with SAP than in the patient group without SAP. The mean heart rate was higher (86.8±12.5 vs. 81.2±9.6; p<0.001) in the SAP group than in the non-SAP group. The demographic characteristics, laboratory data, and vital signs of patients with and without developed SAP are summarized in Table 2.
ROC curves were plotted for CCI, APACHE II, and BISAP scores for predicting the SAP and 30-day mortality in patients with MAP (Fig. 2). In predicting SAP development, CCI had the highest accuracy (AUC= 0.797 ± 0.015), followed by APACHE II (AUC= 0.755 ± 0.017) and BISAP (AUC= 0.736 ± 0.019). In predicting 30-day mortality, the CCI had the highest accuracy (AUC= 0.797 ± 0.040), followed by the BISAP (AUC= 0.790 ± 0.041) and APACHE II (AUC= 0.762 ± 0.042). AUCs derived were further compared using the De Long test. The accuracy of CCI in predicting SAP development was significantly higher than BISAP (p<0.001) and APACHE II (p=0.028) scores. The accuracy of CCI in predicting 30-day mortality was not significantly different from the BISAP score (p=0.844) and APACHE II (p=0.417) scores. There was no significant difference between the accuracy of BISAP and APACHE II scores in predicting SAP (p= 0.196) development and 30-day mortality (p=0.462). Sensitivity, specificity, PPV, and NPV for cutoff values (CCI ≥2, BISAP ≥3, APACHE II ≥8) determined according to previous studies are given in table 3 and table 4.   (Fig. A) and 30-day mortality (Fig. B).

Discussion
In this retrospective study, we focused on a subset of AP patients classified as MAP by the RAC. We showed that CCI, consisting of demographic parameters, can successfully predict SAP risk and 30-day mortality in MAP patients. ROC curve analyses performed in our study showed that CCI had a higher AUC value than APACHE II and BISAP in predicting the development of SAP in MAP patients. In addition, CCI ≥2 had higher sensitivity values than APACHE II ≥8 and BISAP ≥3 in predicting both SAP development and 30-day mortality. Sensitivity in identifying high-risk patients is critical because it is essential to avoid misclassifying high-risk as low-risk when making decisions about early discharge.
The APACHE II score can successfully predict severity in AP patients; therefore, it has been one of the methods of selecting patients for treatment in AP studies 3,19 . However, there are some controversial cases for the APACHE II score. One of the most contentious aspects of APACHE II is the complexity of the 12 parameters, plus the limited utility of the 24-hour score 20 . In addition, the fact that it was obtained for the follow-up of intensive care patients makes its use in the ED controversial. Some limitations in the ability of the APACHE II score to classify AP patients for disease severity have been reported [21][22][23] . Also, some of the parameters it contains may be ineffective in predicting the severity of AP. This study observed a statistically significant difference in sodium, potassium, and pH concentrations between patients with and without SAP; however, no significant difference was observed for other APACHE II parameters such as WBC, hematocrit, or creatinine. Unfortunately, more parameters are not universally accepted criteria due to their low sensitivity and complexity for rapid evaluation. Besides good sensitivity and specificity, an ideal prognostic score should be simple, use readily available parameters, and not expose the patient to significant discomfort. For this reason, more uncomplicated prediction scores may be needed, especially in the clinical practice of emergency physicians. The BİSAP score performs similarly to the APACHE II score in predicting the outcome in AP. Moreover, its calculation is much easier than the APACHE II scoring system. The European Society for Gastrointestinal Endoscopy recommends using the BİSAP score within the first 24 hours of presentation as an early indicator of severity and mortality in AP, with moderate quality evidence and weak recommendation 24 . It has been reported that the BİSAP score performs well in predicting SAP in different patient populations 25,26 . Determining the severity of the disease at the time of admission of AP patients is an essential factor to be considered. Therefore, BİSAP and APACHE II scores are valuable for clinicians as assessment tools that provide information about organ failure 5,12 . However, these scores had not previously been evaluated for predicting SAP risk and mortality in a cohort that included only MAP patients. This study included MAP patients who did not develop organ failure in the first 48 hours of admission to the ED. In our study, cutoff scores ≥8 for APACHE II and cutoff ≥2 for BISAP had low NPV and sensitivity in estimating SAP risk and 30-day mortality in this patient group.
CCI has several strengths. It is primarily a score based on readily available demographic data. It is also easy to calculate. One of the weaknesses of CCI for the AP patient group may be the comorbidities included or not included in the score. In our study, no significant difference was observed between the groups with and without SAP in terms of the incidence of diabetes without end-organ damage, myocardial infarction, peripheral tissue disease, connective tissue disease, and peptic ulcer disease. CCI may need some revisions to increase its effectiveness in predicting prognosis in AP patients. Obesity can be included as an additional parameter 27

Study Limitations
There are some limitations to our study. Firstly, this was a retrospective analysis; therefore, the possibility of selection bias cannot be excluded. Second, this study only included AP patients from a single center, which may prevent us from spreading the results to a broader population. Third, since the data on which we base this study are from an observational database, the results reported in our study should be viewed as a reference only and should be further validated externally.

Conclusion
In this study, the CCI score was found to be effective in predicting the risk of SAP development and 30-day mortality among patients with MAP. The CCI score can help clinicians in the decision-making process for the management of AP patients.

Availability of data and material
All data is fully anonymized and freely available through the corresponding author upon request.

Ethics approval
The authors confirm that the study was approved by the University of Health Sciences, Bakırköy Dr. Sadi Konuk Training and Research Hospital research ethics committee (Protocol No: 2022/157 dated: 09.05.2022) and certify that the study was performed by the ethical standards as laid down in the 1964 Declaration of Helsinki.

Conflict of interest
The authors declare no competing interests.