Diagnosing the impacts of health policy

A new statistical technique offers a better way to gauge the effectiveness of complex healthcare interventions. 

The method, developed by KAUST biostatistician Hernando Ombao and his colleagues Maricela Cruz and Miriam Bender from the University of California, allows health policy researchers to determine if and when a particular intervention has led to a changed specific outcome.

It is based on the interrupted time series (ITS) model, which has been used to study the health benefits of smoking bans, changes in drug packaging and other public-health initiatives. But the ITS model has had a major limitation: it required researchers to specify exactly when the intervention had an effect rather than estimating that time point from the data.

“You had to assume so many things in order to use the analysis,” Bender says. Yet in nursing care delivery, the interventions are not precise. New systems are often introduced gradually with a lag before changes in health metrics become evident.

Bender’s particular interest lay in the Clinical Nurse Leader (CNL) program, which has been rolled out at hospitals across the United States in recent years. This program introduced a new nursing role focused specifically on improving quality and safety outcomes for patients. This involved a years-long training period during which the nursing system underwent a slow and steady redesign. 

Bender wanted to know whether the program improved patient outcomes.

She put the question to Ombao, a time-series expert. He and Cruz adapted the ITS model, making it robust so that it “allows the data to tell us when exactly the change is experienced on the desired outcome,” Ombao says. “It’s really the science that drove the statistical innovation,” he adds.

Ombao and his colleagues applied the new model to study the effect of the CNL program on patient satisfaction. Although they didn’t see a bump overall, the robust-ITS analysis did show that satisfaction indicators were more consistent from patient to patient after CNLs were introduced to the hospital. 

Next, the researchers plan to look at the impact of the new nursing program on other health outcomes, including infection rates, lengths of hospital stays and numbers of medical errors.

Thinking beyond nursing, Ombao says that the robust-ITS model could also prove valuable for judging several public health initiatives in Saudi Arabia, including whether school exercise programs are bringing down rates of childhood obesity and whether vaccination campaigns are lowering the incidence of communicable diseases.

Ombao presented the work to government officials from the Saudi General Authority for Statistics in Riyadh in November 2017.

References

1. Cruz, M., Bender, M. & Ombao, H. A robust interrupted time series model for analyzing complex health care intervention data. Statistics in Medicine advance online publication 29 August 2017.| article