April 17, 2015

Making the most of operational performance dashboards

By: OR Manager
Safety  
Share

Most hospitals use some form of performance dashboard to communicate with surgeons and hospital senior leadership. The dashboard is intended to show the efficiency of processes within the surgical suite.

However, to be truly effective, the dashboard must provide accurate, reliable, and useful information. Does your dashboard do what you need it to do, or could it use some improvement?

 

Tune in and tune up

The performance dashboard is built on data gathered in the operating room information system. Depending on how data are entered into the dashboard, errors such as wrong surgeon names or incorrect starting and ending case times may be introduced.

“Many data entry errors can be avoided through some basic scrutiny,” says Barbara Ripollone, MBA, BSN, RN, director and practice leader of perioperative service at Navigant Consulting, a global expert services firm headquartered in Chicago. For example, at one of her client hospitals, the dashboard showed low block utilization for one of the high-volume surgeons.

“While going through the block utilization report, I spotted another name that was spelled slightly differently from that of the surgeon,” Ripollone says. Further investigation revealed that the surgery minutes of this particular surgeon had been split between two names.

It may be easy to find some errors and communicate with staff about them, but some data errors aren’t easily detected, she notes. Ripollone once led an assessment to evaluate the perioperative services in a multifacility healthcare system. Each facility collected OR operations data and sent the performance report to a central office. The data did not reflect what Ripollone’s team had observed throughout their assessment.

One of the facilities where surgeons complained of long wait times between cases reported a 19-minute turnover time (TOT), compared with an average 40-minute TOT for the rest of the system. A deeper dive into the details of the data calculation revealed that any intervals longer than 30 minutes were considered “extended wait time” and taken out of the calculation, whereas other facilities used 90 minutes as the cutoff, a widely accepted benchmark. After adopting the universal cutoff time and standard calculation methods, facility personnel got a truer picture of the situation.

OR business managers and analysts need a dashboard tune-up once in a while to ensure data entry is clear and accurate.

“Common data entry mistakes need to be documented. The data analyst or nurse educators should address recurring data errors with staff either individually or at a staff meeting. Some groundwork has to be laid for reliable reports,” says Ripollone, who currently serves as interim AVP of perioperative services at Georgia Regents Hospital, Augusta.

 

Standardize KPIs

Most ORs collect first case on-time starts (FCOTS), turnover times, cancellation rates, room utilization, and block utilization as key performance indicators (KPIs). However, definitions and calculation methods for reporting KPIs differ among facilities.

For instance, FCOTS can be defined as sharp on-time start, start within 5 minutes of scheduled start time, or start within 15 minutes of scheduled start time. Thus, a case starting 8 minutes after the scheduled start time can be considered either on time or delayed, depending on the definition.

Misinterpreted KPI definitions and dashboard information may lead to distrust in data accuracy and report credibility.

Soon after Hazel Garcia, MS, RN, CNOR, became the perioperative director of Mount Sinai Hospital in Chicago, one of the surgeons questioned her about the TOT on their performance dashboard: “Why is it that we wait on average over 1 hour between cases, but your report says turnover time is only 28 minutes?” That surgeon defined TOT as “closure to incision” instead of “wheels out to wheels in.”

This is just one example of the kind of clarifications that were needed, she says.

“It seems there is always a need to refresh one’s understanding of KPI data whenever it gets presented, and there is always a chance that people will misinterpret the data,” says Diane Skorupski, MS, RN, CNOR, NE-BC, associate vice president of perioperative services at Shands Hospital, University of Florida Health, Gainesville. “The conditions are footnoted in the dashboard distributed to surgeons. The same calculation methodologies and conditions are consistently applied to all the reports.”

 

Engage physicians

An OR operational performance dashboard works well only when physicians are involved in the process. At Garcia’s facility, she says, “It was not in the culture to quantify the OR’s operational performance.” She began promoting the practice as soon as she started working there.

“I attend monthly surgeon meetings to clarify the definition of KPIs and address their questions to get everybody on the same page,” she notes. In addition, Garcia and her chief of surgery send surgeons monthly reports about the department’s efficiency, including relevant performance indicators such as FCOTS, cancellations, and block utilization for each surgeon.

“The use of data analysis helps the surgeons realize the value of collecting performance metrics,” Garcia says.

One surgeon suspected that his block utilization was being affected by the frequent case extension of the preceding block holders. By analyzing the case duration and block usage pattern, Garcia and an analyst were able to validate the surgeon’s assertion.

Garcia and the chief of surgery then modified the block starting and ending times. With an OR committee, they reviewed the metrics currently in use and redesigned the performance dashboard to add out-of-block surgery minutes and overall block utilization. As a result, the dashboard presented a more complete picture.

The surgeons universally endorsed the decision. Detailed data analysis has gradually been accepted as an effective means to demonstrate the real picture and a useful tool for making decisions. “Surgeons are scientists. They want to know the data and what these data can do for them,” Garcia says.

Engaging senior executives is also an effective way to improve performance.

“Our CEO receives the monthly and quarterly performance reports and also attends our monthly governance meeting. We take it very seriously,” says Skorupski, who oversees four sites with 47 OR suites. “By involving the senior leadership in the process, we are effectively communicating to the physicians that we are one team, and the leadership is here to support them while helping to drive best practice.”

 

Make the dashboard work for you

OR leaders need to identify the most effective means of using the data. Both Georgia Regents and Shands send daily emails to surgeons and key stakeholders to report FCOTS and cancellations before 10 am.

“The near real-time report makes it possible for the managers to follow up with staff, and to immediately understand the reasons for delay. Any possible issues are identified and fixed on the spot,” Skorupski says.

“We post certain KPIs on the bulletin board every day. The information is open to everyone,” Ripollone says. At Georgia Regents, the report is broken down by service lines and sites. “We want to spark some friendly competition,” Ripollone notes.

By taking advantage of the accessible data, both organizations have achieved significant results. Shands has consistently realized on-time starts and prime time room utilization of more than 85%; Georgia Regents has reduced its case cancellation rate to 4.5%.

When collecting data, sometimes a small additional step can reveal information that may otherwise be ignored. Two ORs with similar average overall room utilization may have very different utilization patterns throughout a day (sidebar).

Source: Lucy C. Lee, MBA, BSN, RN, PMP

Source: Lucy C. Lee, MBA, BSN, RN, PMP

Hospital A’s OR has low prime time utilization but very high utilization between 15:00 (3 pm) and 17:00 (5 pm). Hospital B has acceptable prime time utilization but a consistently low rate beyond 15:00. Although overall utilizations are close, the OR managers at the two hospitals may need to use different tactics to tackle the issues. Hospital A should aim to improve prime time utilization and reduce overtime, whereas Hospital B might seek ways to allocate resources more efficiently.

Drilling down on the room utilization can reveal a more accurate picture and help management diagnose the problems causing the inefficiencies.

By observing KPI trends, OR directors can detect potential issues and proactively address them. “My analyst and I compare the data with those of previous months and historical records. We can identify issues such as extended turnover time of certain rooms and decreasing block utilization,” Garcia explains. The chief of surgery, chair of anesthesia, and I review the report together to figure out the solution before we have to take action. It is always better to give a warning than to take away blocks.”

With advanced technology and data analysis, it is feasible to create customized performance metrics and generate timely reports for each surgeon or circulator, thus driving greater efficiency by highlighting individual performance.

 

In a nutshell

Comprehensive performance measurement is the first step in continuous process improvement. To make your dashboard really work for you, the components should be based on the following:

• quality data entry and maintenance

• consistent KPI definitions and calculation methods

• physician buy-in

• flexible and in-depth use of data

• proactive data interpretation and action planning.

A well-managed OR performance dashboard can lead to changes in processes and help unite perioperative teams in their efforts to increase efficiencies. ✥

 

Lucy C. Lee, MBA, BSN, RN, PMP

Lucy C. Lee, MBA, BSN, RN, PMP, is a principal and the practice leader of Clinical & Business Intelligence service at HOPE Management based in Lewes, Delaware.

 

References

Patterson P. OR dashboards: A useful tool for telling a story through data. OR Manager. 2011;27(2):1, 6-8. https://www.ormanager.com/2011/02/01/or-dashboards-a-useful-tool-for-telling-a-story-through-data/#.VImluXvYh2A.

World Health Organization. How can hospital performance be measured and monitored? WHO Regional Office for Europe’s Health Evidence Network (HEN), August 2003.

Comprehensive dashboards paint a fuller picture of OR performance. OR Manager. 2013;29(10):18-20. https://www.ormanager.com/2013/10/01/comprehensive-dashboards-paint-a-fuller-picture-of-or-performance/#.VImrGnvYh2A.

Litt A, Bleustein, C, Swarup S, et al. Healthcare analytics: The way forward. (White paper.) November 2013.

Lucy Lee will be a presenter at the annual OR Manager Conference, October 7-9, in Nashville. Visit www.ormanagerconference.com.

Join our community

Learn More
Video Spotlight
Live chat by BoldChat