External validation of Finnish Diabetes Risk Score (FINDRISC) and Latin American FINDRISC for screening of undiagnosed dysglycemia: analysis in a Peruvian hospital health care workers sample
1Grupo de Investigación en Neurociencias, Efectividad Clínica y Salud Pública, Universidad Científica del Sur.. Lima, Perú
2Centro de Excelencia en Investigaciones Económicas y Sociales en Salud, Universidad San Ignacio de Loyola,, Lima, Perú
3Epi-gnosis Solutions, Piura, Peru
4Facultad de Medicina,Universidad Nacional Federico Villarreal, Lima, Perú
5Servicio de Endocrinología, Hospital María Auxiliadora, Lima, Perú
6Red de Eficacia Clinica y Sanitaria, Lima, Perú
7Centro de Tecnologías Aplicadas a la diabetes, CAVIMEDIC, Lima, Perú
8Instituto de Evaluación en Tecnologías en Salud e Investigación (IETSI), Lima, Perú
9Universidad Continental, Huancayo, Peru
10Escuela de Posgrado, Universidad Peruana Cayetano Heredia, Lima, Perú
*Corresponding author: Marlon Yovera-Aldana. E-mail: myovera@cientifica.edu.pe (MYA)ABSTRACT
Aims
To evaluate the external validity of Finnish diabetes risk score (FINDRISC) and Latin American FINDRISC (LAFINDRISC) for undiagnosed dysglycemia in hospital health care workers.
Methods
We carried out a cross-sectional study on health workers without a prior history of diabetes mellitus (DM). Undiagnosed dysglycemia (prediabetes or diabetes mellitus) was defined using fasting glucose and two-hour oral glucose tolerance test. LAFINDRISC is an adapted version of FINDRISC with different waist circumference cut-off points. We calculated the area under the receptor operational characteristic curve (AUROC) and explored the best cut-off point.
Results
We included 549 participants in the analysis. The frequency of undiagnosed dysglycemia was 17.8%. The AUROC of LAFINDRISC and FINDRISC were 71.5% and 69.2%; p=0.007, respectively. The optimal cut-off for undiagnosed dysglycemiaaccording to Index Youden was ≥ 11 in LAFINDRISC (Sensitivity: 78.6%; Specificity: 51.7%) and ≥12 in FINDRISC (Sensitivity: 70.4%; Specificity: 53.9%)
Conclusion
The discriminative capacity of both questionnaires is good for the diagnosis of dysglycemia in the healthcare personnel of the María Auxiliadora hospital. The LAFINDRISC presented a small statistical difference, nontheless clinically similar, since there was no difference by age or sex. Further studies in the general population are required to validate these results.
Article notes
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
The author(s) received no specific funding for this work.
INTRODUCTION
Only half of the people with diabetes mellitus (DM) in the world know they have this disease. The delay in the diagnosis of diabetes mellitus affects the costs for treatment, management of macro and microvascular complications, and quality of life.[1] South America and the Caribbean have the lowest global prevalence after Africa, but it will increase by 55% in 2045.[2] In Peru, 19.5 new cases are detected every 1000 person-years; this rate is one of the highest reported globally [3]. The systematic screening of diabetes and the application of lifestyles will prevent complications and their prices and reduce the incidence in the medium term [4].
The diagnosis of dysglycemia, DM or prediabetes requires a laboratory test. In order to further improve the performance of the screening approach, we must apply them to the population at risk. [5] In addition, the clinical practice guidelines from the US, Europe, and certain countries from Latin America (LATAM) promote DM screening in the general population as a health policy through clinical practice rules (CPR)[6–8]. There are several CPR for DM, but the FINDRISC is the most common tool used in LATAM [9], where certain countries use it through an adapted or simplified version. [10].
Adaptation of CPR is necessary and highly relevant, especially when the characteristics of the population to be diagnosed are different from the participants of the original validation study [11]. Abdominal obesity in LATAM presents a different pattern than in Europe; based on this consideration, the Latinamerican Group for the Study of Metabolic Syndrome and obesity proposed a new cut-off point for waist circumference in women (90 cm) and men (94cm), which is correlated with a visceral fat area value >100 cm2 obtained by dual X-ray absorptiometry [12]. These cut-off points correlate better to insulin resistance than Adult Treatment Panel III cut-off points based on the body mass index. Based on these findings and the original FINDRISC, they developed the Latin American FINDRISC (LAFINDRISC), which was validated in Colombia and Venezuela in the general population using this updated criteria [13,14].
On the other hand, health care workers show a higher risk of DM than the general population. A condition influenced by shift work, loss of the circadian rhythm of eating, mental health impairment, and sleep disturbances [15]. Prediabetes has pathophysiological alterations as diabetes, and there are microvascular complications in the early stages. [16] Therefore, our study aimed to evaluate the external validity of FINDRISC and LAFINDRISC for undiagnosed dysglycemia in health care workers at a high complexity general hospital from Peru. Our results constitute a piece of substantial primary evidence to address the DM research in a high-risk occupational health group.
Material and methods
Design and setting
We carried out a cross-sectional study from 20/06/2017 to 30/09/2017 in the María Auxiliadora General Hospital (MAGH), a national health facility of the Ministry of Health, located in southern Lima’s suburban area capital city of Peru. The MAGH has 1,839 workers, of which 70% are health care personnel. It has a health network that involves around one million users affiliated to Comprehensive Health Insurance (SIS from Spanish Acronym) with subsidised public health insurance.
Population, sample, and sampling
We included adults, residence in Lima for more than six months, and a minimum working time of three months in the MAGH. We excluded subjects with DM, pregnancy, under corticosteroid therapy (at least one month in the last year), a history of antiretroviral or oncological treatment, people disabilities to walk, personnel with medical leave due to illness, vacations, or suspension from work during the selection process.
We estimated a minimum sample size in 549 participants using Epidat 4.2 (Xunta de Galicia, Santiago de Compostela, Spain) based on a prevalence of dysglycemia (diabetes mellitus and impaired fasting glucose) in Peru of 29.4% [17], expected sensitivity values of 66% and 80% for FINDRISC and LAFINDRISC, respectively, with 95% confidence level and 5% precision. In addition, we added 10% in case of refusal to participate or absence from work. The selection process was through random sampling from a list of 1839 employees.
Dysglycemia
The diagnosis of dysglycemia included prediabetes or diabetes. Prediabetes had fasting glycemia between 100 and 125 mg/dl (impaired fasting glucose) or two hour blood glucose after a 75 g load between 140-199 mg/dl (glucose intolerance). Diabetes mellitus was diagnosed by fasting blood glucose ≥126 mg/dl or blood glucose two hours after 75 g loading ≥ 200 mg/dl. [6]
Findrisc and Lafindrisc
Both questionnaires present eight items: age, body mass index, abdominal circumference, personal history of physical activity, frequency of consumption of fruits and vegetables, history of antihypertensive medication, history of high blood glucose, and family history of diabetes. The difference between the two scores lies in the cut-off point for waist circumference to define abdominal obesity; in the LAFINDRISC, these values changed from 88 to 90 cm in women and 102 to 94 cm in men. Likewise, the modified questionnaire has only two categories, while the FINDRISC has three.[10](Tab S1)
Procedures
During the break in the working day, we assessed the eligibility criteria and requested the signing of the informed consent. Then, we scheduled a maximum of eight people per time. Trained nursing staff administered the questionnaires, the oral glucose tolerance test and collected the blood samples. In the case of night work, OGTT was performed after 48 hours.
We used a digital weight scale SECA ® (USA), calibrated daily, with an accuracy of 0.5kg and a height rod attached to the wall using the standard measurement technique. According to WHO, we place an inelastic tape measure in the middle of the distance of the coastal ridge and the anterior superior iliac spine for the abdominal circumference.[18]. We prepared the glucose load in 300 ml of water containing 75 g of glucose and 1.6 g of citric acid (1 squeezed lemon). We requested a minimum fasting time of 8 hours on the appointment day. We collected the basal venous samples 2 hours after glucose loading in dry tubes and centrifuged them in the next 30 minutes.[19] We used a COBAS 6000 (c501 module) automated analyser (ROCHE, USA), according to the Center for Disease Control and Prevention. [20]
Statistical analysis
We described categorical data by frequencies and proportions. Using generalized linear models, Poisson family, logarithmic link function, and robust variance, we calculated prevalence ratios (PR) with its 95% confidence interval for each component of the FINDRISC or LAFINDRISC. In addition, we estimated the sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio for both indices. Also, we effectuated the comparison between both areas under the Receiver Operating Characteristics (ROC) curve. We used the Youden Index to identify the score with the best discriminative capacity based on estimates of specificity and sensitivity [21]. Finally, we performed a simulation with 1000 patients to calculate those correctly diagnosed considering a dysglycemia prevalence of 29.4% based on the reports from a sizeable Peruvian cohort study (PERUDIAB). We used STATA version 17.0 (Stata Corp, College Station, Texas, USA).
Ethics
All participants signed an written informed consent based on the principles of the Declaration of Helsinki. Subjects were free to refuse to participate at any time. The Institutional Ethics Committee for Research of the Universidad Peruana Cayetano Heredia approved the study protocol, under the code CONSTANCIA 382-13-17. We kept the data confidential through codes only the principal investigator had access to the data. We communicate the results to all patients in writing. People with dysglycemia or high LAFINDRISC scores were referred to an endocrinology outpatient clinic.
RESULTS
From 1,835 health care workers, we randomly selected 589 subjects and 561 workers for the oral glucose tolerance test. Finally, we included 549 participants in the analysis. (Fig 1).
Seventy-seven per cent of the study subjects were female; the age range ranged from 20 to 70 years, with a median of 51 years. Seventy-five per cent presented a body mass index greater than 25 kg/m2, and 65.4% showed abdominal obesity according to Latin American criteria (90 cm for women and 94 cm for men) (Table 1).
Prevalence of Undiagnosed dysglycemia
We found that 17.9% (95% 14.7 – 21.3) had undiagnosed dysglycemia, 2.6 % (CI 95% 1.4 – 4.2) DM and 15.2% (CI 95% 12.4 – 18.6) prediabetes (Table 1). Likewise, its prevalence was significative higher in people aged 65 years or more (44.1%), B.M.I.≥ 30 (30.5%), higher value of circumference (22.8%), with hypertension (28.2%) and history of hyperglycemia (42.9%) (Table 2).
Regression models
In both adjusted regression models, we found that age (≥ 65), BMI (≥ 30), and history of hyperglycemia increased the probability of undiagnosed dysglycemia. Hypertension medication was associated only in crude analysis Regarding waist circumference, the European and Latin American cut-off points were not associated with this outcome in adjusted models, although the LA was associated in the crude model. The FINDRISC presented a pseudo R2 of 0.1045 (p <0.001), while LAFINDRISC had a Pseudo R2 of 0.1034 (p <0.001) (Table 2).
Comparison between Scores
The discriminatory diagnostic capacity of the LAFIDNRISC was statistically greater than the FINDRISC, AUROC 71.5% (95% CI 65.8 – 77.2) vs 69.2% (95% CI 63.2 – 75.2); p = 0.007.
When stratifying them by sex, there was similar discrimination of both questionnaires in men (61.6 vs 62.6%; p = 0.130) as in women (73.9% vs 74.8%; p = 0.338). LAFINDRISC also shows better performance for diabetes mellitus and prediabetes. (Table 3)
In the LAFINDRISC, the score with the best Youden index was 11, showing sensitivity and specificity of 78.6% (95% CI 69.1 – 86.2) and 51.7% (95% CI 46.9 – 56.4%) respectively, and a negative likelihood ratio of 0.41 (CI95% 0.28 – 0.61) (Table 4). The supplementary material describes the complete analysis and additional comparisons (Table S2 y Table S3).
Simulations according to different scenarios of prevalence
We performed a simulation in 1000 patients with dysglycemia prevalence reported by the PERUDIAB (29.4%). The negative predictive value decreased from 91.7% to 85.3%, losing 6% of the test’s ability to detect people without dysglycemia. (Table 5).
DISCUSSION
Our research found that one out of six healthcare subjects had undiagnosed dysglycemia. One out of seven had prediabetes, and one out of fifty had diabetes mellitus. We showed a greater discriminative capacity of LAFINDRISC than FINDRISC for screening undiagnosed dysglycemia in healthcare workers. The best cut-off points for LAFINDRISC and FINDRISC were 11 and 12, respectively. The change of the cut-off point from 102 cm to 94 cm in men and from 88 to 90 cm in women better validated the results.
FINDRISC obtained an area under the ROC curve between 85 and 87 % to predict drug-treated diabetes mellitus at ten years of follow-up.[22] In a captive population of northern Colombia, LAFINDRISC obtained an area under the ROC curve of 73% for undiagnosed dysglycemia. [13]. LAFINDRISC received an area under the ROC curve of 68% for undiagnosed DM [23]. In both studies, there were no differences between LAFINDRISC and original FINDRISC. In our work, the area under the ROC curve was also lower than the original, with a difference of 2.3%. in favor of LAFINDRISC (71.5% vs. 69.2%). When a clinical prediction rule is validated in a population different from the original one or when a different outcome is evaluated, the discriminatory capacity tends to decrease.
We chose the Youden index to define the best score. Due to its screening purpose, it should have a higher sensitivity than specificity. [24]. In our study, a score of 14 obtained the highest Youden Index with specificity greater than sensitivity. However, we chose score 11, which presented the second-best Youden Index and the requirement of having a higher sensitivity than specificity. In Colombia, a score ≥ 8 showed the highest Youden Index with a sensitivity of 78% and specificity of 50% for dysglycemia [13]. In Peru, a cut-off point of 10 of the LAFINDRISC presented a sensitivity of 70.4% and specificity of 59.1% for undiagnosed diabetes mellitus. [23] However, the original FINDRISC validation study chose the best cut-off point if it presented a negative predictive value of 99%.This criterion ensures that 1% or less of those discarded would be false negatives. If we apply this last criterion, the cut-off point would be five, and it would imply performing a second confirmatory examination on 85.1% of the population. This policy will require a higher investment and be challenging to carry out in developing economies. [22]
Both questionnaires presented the same performance for dysglycemia when separately analysed in men or women in our study. Nevertheless, regardless of the questionnaire used, performance in women was 12% higher than that of men. In Latin America, the area under the ROC curve of LAFINDRISC for dysglycemia in Bogotá was 76.9% in men and 77.9% in women. In Barquisimeto, the area under the ROC curve was 91.2% in men and 92.0% in women. Performance was slightly higher in women than in men in both cities.[25]. In a nationwide Venezuelan study, there were no differences between FINDRISC and LAFINDRISC for dysglycemia when analyzing men and women separately.[14].
Plausibility and explanation of results
Isolated fasting hyperglycemia implies insulin deficiency and hepatic insulin resistance but with normal muscle insulin sensitivity. This is executed by counterregulatory hormones in a context with increased lipolysis of adipose tissue and fatty esterification of liver cells that exaggerate fasting gluconeogenesis. In contrast, postprandial hyperglycemia implies a failure of secretion plus a decrease in hepatic sensitivity and moderate or high muscular resistance, preventing the internalization of glucose through GLUT4 receptors in muscle and liver. Fasting hyperglycemia could be considered an earlier failure and would predominate above all in subjects with abdominal obesity, acanthosis nigricans, skin tags or metabolic syndrome. [26]
The better performance in women may be due to the high percentage of excess weight. In our study, 75% had a body mass index > 25%. Likewise, the high frequency of excess weight and abdominal obesity in our healthcare workers exceeds the national average in the general population. [27] This risk is due to workgroups that perform shift work. Highlighting the nursing staff’s risk of obesity and other metabolic problems represents a large percentage of healthcare workers. [28]
Despite this high excess weight and abdominal obesity, our study only found 17.8% of undiagnosed dysglycemia. A difference of 11.6% concerning the national prevalence[17]. Annual occupational controls could explain this lower prevalence to detect metabolic disorders that decreased their frequency in our sample.
Limitations and strengths
Our study has limitations. In the first place, the chosen score cannot be used in the general population since the findings would only apply to healthcare personnel of the María Auxiliadora hospital. A complimentary evaluation in a more representative population was not performed. But a simulation was performed by changing the prevalence of dysglycemia to observe the variation in performance. We do not use glycosylated hemoglobin as a confirmatory method for dysglycemia, as we do not have methods validated by the National Gycohemoglobin Standardization Program. However, the ADA guideline recommends OGTT as a sufficient criterion for dysglycemia. Although the best discriminatory capacity of LAFINDRISC is clinically small, an instrument adapted to local characteristics is always desirable. Despite these concerns, our study has important strengths, such as using a modified questionnaire with cut-off points for the Latin American obesity phenotype. In addition, subjects were randomly selected based on the sample frame of workers’ payroll with minimal subject loss. OGTT was performed on all participants, regardless of the questionnaire result, avoiding selection bias.
Implications, recommendations and future research
Quantifying the risk of diabetes or dysglycemia is a cost-effective activity recommended by the Clinical Practice Guidelines. Applying the clinical prediction rules outside the original context requires a validation process to check if the discriminative capacity is maintained. External validations in Colombia and Peru found no differences in performance between FINDRISC and LAFINDRISC. Both studies were carried out in private insurance people and the general population, respectively [23] [31]. Despite these results, transculturation of Clinical Prediction Rules according to the local characteristics should be the standard before applications. [10] Governments or funders will require complementary cost-effectiveness analysis and decision tree analysis for potential outcomes to apply the early diagnosis in public health.[29] It will impact the costs of screening, confirmatory diagnosis, follow-up, and treatment. As well as evaluation of potential benefits in reducing years of life gained and greater survival.[30,31]
Each country, institution, or funder chooses the cut-off point and establishes the strategy that best suits their reality. The original FINDRISC validation study determined that the best score for diabetes mellitus screening was 11. However, the Finnish Diabetes Prevention Program recommends performing OGTT from a score of 15 and initiating lifestyle changes from 7.[32] The clinical guideline of the Colombian Ministry of Health [8] recommends performing fasting blood glucose as a confirmatory test for a score > 15 and initiating lifestyle changes if score ≥ 12. These actions are derived from decision analysis and may vary according to economic and administrative conditions.
Conclusion
The discriminative capacity of both questionnaires is good for the diagnosis of dysglycemia in the health care personnel of the María Auxiliadora hospital. With LAFRINDRISC presenting a small statistical difference, but clinically similar since there was no difference by age or sex. Further studies in the general population are required to validate these results.
Supporting information
S1 Table. Scores of FINDRISC and LAFINDRISC
S2 Table. Performance of LAFINDRISC regarding different cut-off points
S3 Table. Performance of FINDRISC regarding different cut-off points
Data Availability
All relevant data are within the manuscript and its Supporting Information files.