March 16, 2017

Checklist compliance soars to 100% after ‘red rule’ implementation

By: David Kashmer, MD, MBA
Share

Most OR leaders believe the World Health Organization (WHO) Surgical Safety Checklist can help prevent surgical errors and complications. But although many hospitals have formally adopted the WHO checklist, fewer ORs use it consistently.

At Signature Healthcare Brockton Hospital, a nonprofit teaching facility south of Boston, we addressed this issue head on in 2015. Through a quality improvement project, we achieved 100% checklist compliance.

Our experience shows one possible way to overcome the obstacles to full checklist adoption. The project also illustrates a set of principles that are important to every quality improvement initiative.

 

Incomplete implementation

The OR at Brockton Hospital implemented the WHO checklist for nonemergent surgical cases about 4 years ago, but adoption was partial at best. In 2015 we began to look more closely at checklist utilization, especially compliance with the preoperative debrief.

Staff were divided on how well we were doing. Some felt checklist compliance was fairly high. Others held the opposite view. In the end, we decided to follow the advice of quality visionary W. Edwards Deming: “Without data, you’re just another person with an opinion.”

 

Getting started

To get the data we needed, we developed a simple tracking system. At the end of every day, the OR manager checked procedure documentation for each case and noted whether or not the checklist was marked complete. After tracking compliance during the first few weeks of May 2015, we had some hard numbers to work with. The data showed that our checklist compliance rate for nonemergent cases was 44%, meaning that only 44% of all checklists (>480 per month) were completely filled out.

We began exploring ways to increase checklist compliance. As is often the case in quality improvement efforts, suggestions included incident reports and formal reprimands. However, we wanted to avoid creating a punitive environment.

As we sought to find best practices, one member of the project team suggested an approach developed by a tertiary center in Boston. OR leaders there found the main obstacle to performing the preoperative checklist was securing the presence of the surgeon.

Was this also the case at our hospital? To find out, we spent time observing preoperative processes in our facility. Our observations verified that surgeon presence was an obstacle to checklist completion. We also found that there was no set time for performing the preoperative debrief.

 

Adding the ‘red rule’

Based on the experience at the tertiary facility and our observations, we established a single inviolable “red rule” for the OR: For nonemergent cases, the patient cannot be moved from the stretcher to the operating table if the surgeon is not present in the room.

This rule addressed the major obstacle—getting the entire team together with the surgeon to review the checklist. It also created a reliable point within the preoperative process where the initial steps of the checklist could take place. Extending this idea, we decided that the postprocedure debrief portion of the checklist would be initiated by staff before any team member left the OR.

To support checklist compliance, we also created a visual control—a slider board (photo below). The board displays each element of the preoperative checklist. As the team completes each element, a nurse slides the indicator from red to green on the board.

 

A slider board at Signature Healthcare Brockton Hospital serves as a visual reminder and process aid during the preoperative checklist timeout.
Source: David Kashmer, MD, MBA. Used with permission from Signature Healthcare, Brockton, MA.

Rapid improvement

We implemented the new rule at the end of May 2015. Within a matter of weeks, we began to see dramatic improvement. From our baseline rate of 44%, checklist compliance rose to 65% in June, 78% in July, and 85% in August. By September 2015, it reached 100%.

Compliance has been at 100% ever since, and we continue to monitor the process on an ongoing basis (including checklist observations and reviewing the checklist sheets).

Has our checklist project increased patient safety? Anecdotally, yes. Since we modified our preoperative process, staff have caught several potential safety issues before they affected the patient, including problems with equipment and blood products as well as two potential wrong-site procedures.

Five quality lessons

Every quality project presents its own challenges. However, most successful initiatives share some common characteristics. Our team’s checklist project illustrates five key principles in healthcare quality improvement.

1. Focus on the system, not the person. In many hospitals, performance improvement is a pejorative process, although often not intentionally. In these cases, teams focus on devising new punishments for noncompliance.

There are three problems with this approach. First, a punitive atmosphere damages morale and undermines teamwork. Second, punitive systems are time-consuming for leaders. As chief of surgical services during this period, I did not have time to confront colleagues every time they failed to follow a given rule. What’s more, I know it isn’t a great way to influence behavior. Third, the punitive approach ignores the fact that in most cases, mistakes occur because the system sets people up to make them.

In every quality initiative, efforts should focus on the process flaws that are leading to errors. In other words, don’t make it personal. In our checklist improvement project, we resisted the temptation to blame and punish. Instead, we tried to understand why our system made it hard for staff to complete the safety checklist.

2. Make it easy to do the right thing. As you think about what makes it hard for staff to do what they should, do not forget the flip side—what can you do to make it easier for staff to do the right thing? Healthcare providers often attempt to improve quality by creating complex rules that cover every eventuality. This approach fails to solve the underlying problem and makes it harder to achieve quality.

A key concept of the Toyota Production System is poka-yoke, which is Japanese for “making it easier to do the right thing.” Our checklist improvement project was built around a single rule that is easy to remember and follow. It also incorporates an automatic “stop the line” command. Staff members are not allowed to move the patient to the table if the surgeon is not present. No one has to speak up to defend the checklist. The process halts itself when necessary.

3. “Go to the gemba.” At the start of our project, we observed staff in the preoperative holding area to understand whether our proposed solution could work and how it might fit into our existing organization. In Lean Six Sigma, this is known as going to the gemba, another Japanese word loosely translated as “the process itself.” The basic idea is that to improve a process, you have to go physically to where it takes place and watch it unfold.

Going to the gemba is especially important when the underlying issues are unknown. In our case, we thought we had already found a potential solution from a nearby facility. But taking the time to observe our process directly helped us adapt the proposed rule to our unique situation. We did not implement a solution prior to looking at our own data and experience. Don’t put the solution ahead of data about the problem, no matter how often you see others take that approach in healthcare quality improvement.

4. Make data collection simple.No one in healthcare has time to collect data. But without data, no quality improvement is possible. That’s why it is important to make data collection as simple and fast as possible.

For our checklist project, we could have created a complex scoring system that measured degree of completion. That would have allowed us to characterize checklist compliance more precisely, and it might have helped us identify the checklist elements that were giving us the most trouble. Unfortunately, it would also increase the risk that our project would never get out of the starting gate. At that time, our system was not able to collect data that was at all complex.

For this reason, we decided to collect discrete data—the checklist either was completed or it wasn’t. This not only simplified data collection, it reflected the philosophical point that every element of the checklist was important. Nothing could be skipped.

There are many strategies for simplifying data collection. Our solution was to use discrete data, which is a type of data that comes in small “packets” like yes or no; red, green, or yellow; or a percentage reflecting correct or not correct.

Another approach is to use continuous data, ie, data that can be divided into smaller units yet still make sense. For example, time is continuous data—an hour may be divided into minutes, minutes divided into seconds, and onward. The advantage of using continuous data endpoints is that they typically require a smaller sample to demonstrate meaningful change.

For both continuous and discrete data, there are data collection techniques that make it very simple and rapid to collect meaningful data right from the process.

5. Design an alarm into the system. A perennial challenge of healthcare quality is maintaining long-term improvements. Anyone familiar with the define-measure-analyze-improve-control (DMAIC) methodology knows that every quality improvement project must transition into an ongoing control phase. Again, the key is to make it manageable. One solution is to build a system that will signal whenever a process backslides.

Because we framed our checklist project in simple terms (binary “yes/no” performance), our control phase is also straightforward. The OR manager runs an electronic medical record report on checklist compliance every month and reports it at the monthly meeting of the Surgical Services Executive Committee. If for some reason the report does not work, we can always check the electronic records individually for any outliers.

More complex process improvements should be monitored with a control chart. Many quality leaders use an Individual-Moving Range (I-MR) chart to monitor processes for variation. Used correctly, an I-MR chart can show you that a process is out of control even before an adverse event occurs.

However, two cautions are in order. First, you can only use a typical I-MR chart if your data are normally distributed. (At the risk of oversimplifying this concept, normally distributed data will produce a familiar bell-shaped curve.) Second, I recommend you only implement an I-MR chart after your target process has been improved and is performing at a high level when you move into the control phase. Otherwise, the only thing an I-MR chart can show you is whether your process is in statistical control, even if it’s completely unacceptable in terms of its performance.

We in healthcare are often led to believe that quality must be complicated, whether it is a checklist implementation or any other improvement initiative. Although it is true that many quality goals require complex interventions, we should always be open to looking for a straightforward solution.

A simple solution you think will improve your system by 80% is preferable to one that could perfect your system yet is very complex. Simple interventions—and the quality principles that underlie them—are often the best way to achieve dramatic gains in quality and safety. ✥

 

David Kashmer, MD, MBA, is a trauma and acute care surgeon as well as Lean Six Sigma Master Black Belt. He writes about data-driven healthcare quality improvement with a focus on how best to measure value. He is the author of the recent Amazon bestseller, Volume to Value: Proven Methods for Achieving High Quality in Healthcare.

Join our community

Learn More
Video Spotlight
Live chat by BoldChat