Continuous Quality Improvement (CQI)

1y ago
938.28 KB
20 Pages
Last View : 1m ago
Last Download : 1m ago
Upload by : Hayden Brunner

Step 9Continuous Quality Improvement (CQI)ContentsMaterials . 9-2Step 9 checklist . 9-3FYN tackles Step 9 . 9-3Reasons for conducting CQI . 9-4Information to get you started . 9-6How to implement a CQI review . 9-8Prepare the review . 9-8Complete the review . 9-9Applying Step 9 when you already have a program . 9-17Sustainability at this stage . 9-18Lessons learned. 9-18Getting ready for Step 10 . 9-20**********Focus questionHow will we continuously improve the quality of the program?Step 9 CQI takes you through an intentional process of discovering what canbe improved so that things work even better the next time you offer this oranother teen pregnancy prevention program. Of course, we hope that mostthings turned out the way you thought they would, demonstrating goodprocesses and outcomes. Even so, you probably discovered that some thingsdidn’t turn out as you’d hoped.We’ve based the tasks in this step on a common business strategy calledcontinuous quality improvement (CQI). CQI takes what you’ve learned as youevaluated your planning, implementation, and outcomes and then applies it tothe continuous improvement and growth of your programming. It’s a simple,Continuous Quality Improvement9-1

but systematic work review to see what changes you could make to improveyour program the next time around.It means, simply, that you constantly collect and use information to makeadaptations to your program so that it works better over time. Although CQI canbe used to identify areas in need of improvement, it’s more than just a process offinding and fixing problems. It shows you what’s working well so that you canbuild on it and repeat your successes. Asking questions, discussing the possibleanswers, reaching agreement, and communicating decisions are all part of acyclical, healthy, and useful CQI process.CQI also helps build the sustainability of your program. The more you canrepeat your successes and improve the planning, implementation, andevaluation of your program at every stage, the more you increase your chancesof sustaining your work on a number of levels. Engaging in an on-going CQIprocess keeps you current and responsive to changes going on around you andwithin your organization. Being responsive and adaptable is important tokeeping your good work going.MaterialsYou’ve probably already compiled all of the materials you will need to do a CQIreview in your three-ring binder: Completed Step 1 Resource Assessment and Priority Needs Filter tools Completed Step 2 BDI Logic Model and SMART Desired Outcomes tools Completed Step 4 Program Fit Assessment tool Completed Step 5 Capacity Assessment tool Completed Step 6 work plan, Work Plan tool, and Culturally AppropriatePrograms Checklist Completed Step 7 Process Evaluation showing successes and challenges ofdelivering your program Summaries of staff and participant satisfaction surveys (if performed) Completed Step 8 Outcome Evaluation and data summary from fidelitymonitoringTo complete this step, you’ll need: 9-2Copies of the Step 9 CQI Results tool on the CDC Teen Pregnancy websiteContinuous Quality Improvement

Step 9 checklistUpon completing Step 9, your organization will have conducted CQIassessments of your work and come up with an associated plan forimprovement. Document successful program activities Assess program activities that did not work well overall or for specific groups Identify areas for improvement Create strategies for improvement Increase buy-in within your organization by soliciting and acting on thesuggestions of program staffFYN tackles Step 9Before FYN implemented and evaluated MPC for the first time, the work groupalready knew they wanted to conduct a formal continuous quality improvementreview of everything they had done so far. They hoped that a CQI review wouldshow what had worked well, what had not worked out as planned, and where FYNcould make improvements the next time they ran MPC.Led by the FYN program director, the adult and youth facilitators, FYN staff andadministrators, and work group members met to review how things had gone. Theydrew upon the materials saved in their three-ring binder, including programinformation and evaluation results, and walked through the CQI Results tooltogether.Continuous Quality Improvement9-3

Reasons for conducting CQIPrograms that are implemented multiple times are frequently subject to somekind of review to ensure that the intended benefits are still being achieved.Funders want evaluation results to demonstrate that their funds are well spent,and those involved with implementing programs want to know they are havingthe desired impact. Even if adequate resources exist to thoroughly study theimpact of programs, however, very few get all components correct the first try.Continuous quality improvement (CQI) is one important aspect of evaluationthat can be used to improve a repeated program’s effectiveness. You havealready done a short form of CQI by performing needed midcourse correctionsduring the implementation. You now have an opportunity to make moresubstantial improvements in program performance before you repeat it. Wesometimes call this strategic CQI because it involves a broader approach toimprovement. Being conducted after a round of your program has beenimplemented, strategic CQI is also a great opportunity to learn from previousimplementation efforts in which program processes and outcomes were welldocumented (Wilson et al. 2009).Benefits to the programDocument program components that worked well. Tracking success helps to ensurethat future implementations also will be successful.Assess program components for elements that did not work well. Recognizingweaknesses and failures helps to clarify improvements you can make.Support program staff by obtaining and using feedback. Useful critique promotesgrowth and an increasing ability to produce more effective programs.Benefits to the organizationSupport staff and organizational values. CQI is the right thing to do for thoseinterested in offering the most effective services possible for their clients.Participating in CQI activities can also energize people by reflecting on theirwork and having input into programming moving forward.Polish the organization reputation. Funding and sustaining an organization maydepend on how much an organization demonstrates to the community that it canprovide quality services consistently over time.9-4Continuous Quality Improvement

Boost service delivery and help maintain consistently high quality. Organizationshave to invest time and money to satisfy funder and regulatory requirements fordata collection. Systematically studying and using data makes good businesssense and provides important information on which to base serviceimprovements, thus helping to increase the return on the investment required tocollect the data in the first place.Promote an understanding of program and organization dynamics. CQI participantsbecome more skilled in anticipating and responding to changes in demand forservices or resources.Show ways to use data for smarter, targeted choices. Learning to collect, study, anduse data helps organizations better understand the link between internal goalsand outcomes. CQI thus informs organizational processes for reaching thosegoals and outcomes.Key pointCQI offers an opportunity to make more substantial improvements inthe program’s performance before you repeat it.Continuous Quality Improvement9-5

Information to get you startedNow that you’ve implemented your program, it’s unlikely that everythingworked exactly as planned. You may not have obtained all the outcomes you hadhoped for. You may have run into barriers and challenges along the way that youdidn’t anticipate. This is all normal. CQI can help you surmount those issues sothat your program can grow and improve.You can use what you learn to adjust and improve your program. Program staffthat learn from evaluations and feedback will implement increasingly moreeffective programs. For example, there may have been challenges withimplementation, participant retention, or issues related to fit. CQI tasks can helpyou decide how to adjust your plan and its implementation so that you continueto move closer to your goals and desired outcomes. They will help you articulate: What worked What didn’t What should be changedThe process takes you through a review of your results by having you answer afew key questions associated with the eight preceding steps.Step 1 Needs & ResourcesAre our data still relevant?Are the needs we’re addressing still the most important ones?If not, do we need to perform another assessment?Have there been significant shifts in our resources?Step 2 Goals & OutcomesWhich of our goals and desired outcomes have we achieved and which oneshave we not achieved?Do we need to adjust or change our desired outcomes or reconsider whichparticipants we’re trying to reach?Step 3 Best PracticesWhich strategies are producing our desired outcomes and which ones aren’t?9-6Continuous Quality Improvement

Do we need to find new strategies to achieve our unmet desired outcomes?Step 4 FitDo we need to change our assessment of fit with regard to our participants?Does the program still fit with our organization and stakeholders?Step 5 CapacityAs we review what’s working and what isn’t, do we still have adequatefiscal capacities to support the program?Do we still have adequate staff capacity (e.g., training, qualifications, time)to maintain program fidelity?Step 6 PlanWas our implementation plan adequate?Were our process and outcome evaluation plans adequate?How well did we follow our work plan?Step 7 Process EvaluationDid we implement the program with fidelity?What mid-course corrections did we make, if any?Were our process and outcome evaluation plans adequate?Step 8 Outcome EvaluationDid we reach our desired outcomes?Were our process and outcome evaluation plans adequate?NoteIf you’re interested in delving even more deeply into the CQIprocess, look on the CD for CQI description.pdf, an extract from theHealthy Teen Network CQI and Program Sustainability Training.Continuous Quality Improvement9-7

How to implement a CQI reviewThe CQI review process is a straightforward, systematic review of your work tosee what changes you could make to improve your program the next timearound. When you look at those questions for each step, it sounds more complexthan it really is. Actually, when you sit down to look over all that you’ve learnedand accomplished, you’re asking yourself one basic question:What can we do better?Keep in mind that adjustments to improve the functioning of your program neednot be major. You may find, for example, that enhancing staff training in theiruse of some evaluation tools will help you capture more useful detail. Suchadjustments can be made as you move ahead with other successful elements ofyour program.Prepare the reviewStart by establishing a CQI review group from members of your program workgroup. To ensure that all the tasks get done and changes get implemented,designate one person to lead the group.Next, engage your program staff in discussions about the CQI process. Helpthem understand what’s coming so that they’ll be ready to offer input, whichyou’ll incorporate into your CQI documentation.NoteBe thoughtful as you plan CQI meetings in which you discusschallenges. Limit participation to those individuals whose input isessential. Emphasize the team approach and a balanced point of viewin CQI discussions so that no one feels blamed for something thatdidn’t work. Critique the program based on data; don’t critique thepeople.The last bit of preparation before you begin is to gather up all the informationyou want to review (see Materials, p. 9-2). If you don’t already have everythingyou need assembled in a three-ring binder, now is the time to do it.9-8Continuous Quality Improvement

Memory flashCQI starts with actions taken in Step 1 Needs & Resources:Establish a work group.Collect data. You probably have already gathered most of thematerial you need.Make sure everyone involved in the CQI review obtains the relevant materialbefore they arrive at the review session. That way they can get up to speed inadvance, which will save time in the meeting. You could also distribute copies ofthe CQI Results tool ahead of time so that people know which issues to thinkabout.ToolThe CQI Results tool is located . The fictionalFYN illustrates its use.Complete the reviewIt may take more than one session to complete the CQI review. Depending on theamount of advance preparation you’ve done, you might be able to complete yourreview in a long meeting, or you might want to use several meetings or even adaylong retreat to digest everything.Let the CQI Results tool guide you through your materials and select data. Thetool is divided into three parts. Because CQI uses a stepwise process, we ask thatyou carefully follow the tool instructions to accomplish the following:Needs & Desired Outcomes1. Match needs identified in Step 1 Needs & Resources with correspondingMeasurable Evidence Statements from the SMART Desired Outcomes tool inStep 2 Goals & Outcomes.2. Rate the success and impact (e.g., indicate whether any perceived changewas Positive, Negative, or None [didn’t occur at all]) with each measureusing data in the Step 8 Outcome Evaluation.3. Determine whether or not further action is required to revisit the programto address negative impacts or missed targets.Continuous Quality Improvement9-9

Outcome Evaluation4. Use information in the Interpretation column of the Step 8 OutcomeEvaluation to indicate the impact of each evidence measure with respect tothe desired outcome and the amount of change.Key pointIf you find that there are new needs in the community, you’ll have tocome up with new goals and desired outcomes targeting those needsas well as different programming, fit and capacity assessments, plansand evaluations.Process Evaluation5. Copy data from the Step 7 Process Evaluation into the Dates andParticipation Targets and Attendance sections, calculating the percent ofyour priority population that actually attended and the percent of actualparticipants included in the evaluation.Tip sheetYour Fidelity Tracking tip sheet from Step 7 may come in handy here.Planning Program Improvements6. Make decisions about changes the group needs to institute beforeimplementing the program again. Even if you don’t present the exactsame program, the information here can inform further programmingdecisions.Save itSave the completed CQI Results tool with all your other programdocumentation. Whether you recreate the current program or choosesomething else, this information will help you achieve optimumresults in the future.9-10Continuous Quality Improvement

FYN completes the CQI processUsing the CQI Results tool, the FYN work group documented its results.Generally, the group agreed that the need for the program still existed and thattheir goals and outcomes remained the same. MPC, as adapted andimplemented, suited the 8th grade participants in the middle school, but FYNwants to expand the use of the curriculum to other grades in the middle school.FYN knows that adapting MPC to fit with younger students will require somework, however, and expansion may take more resources than they currentlyhave.The more immediate concern with regard to resources was discovering that ittook more time and effort than the staff had anticipated to recruit and train peerfacilitators. FYN remained committed to having peer facilitators and planned touse the lessons learned to include more time and support for recruitment andtraining.Everyone agreed that the time spent developing a detailed, phased in work planthat included process and outcome evaluation plans was time well spent. It paidoff in high quality program implementation and good data collection. Next time,FYN wants to concentrate on improving the outcomes that were not met whilemaintaining those that were.Although FYN anticipated that involving peer facilitators could be positive forthe youth involved, the peer facilitators themselves reported a significantlyincreased confidence level and improved leadership skills. The FYN programdirector wants to enlist these peer facilitators next year to help train new peerfacilitators. She also wants to see if there is some way to measure the impact oftheir leadership on the participant experience.FYN concluded its CQI process by incorporating several changes into the workplan for the next cycle of MPC implementation.Continuous Quality Improvement9-11

CQI ResultsProgram: Making Proud Choices Compiled by: Alisha Date: June11, 2016Needs & Desired OutcomesNeed AddressedMeasurable Evidence StatementOutcome Evaluation Result1.By the end of the program, 15% of MPCstudent participants will decrease thefrequency of sexual intercourse, as reportedon post-tests. Reached Missed ExceededChange: Positive Negative 2MPC participants will increase knowledge ofHIV/STI prevention, transmission, andprotection by 20%, as reported on post-tests. Reached Missed ExceededChange: Positive Negative MPC participants will increase knowledge ofproblem-solving/ negotiation by 20%, asreported on post-tests. Reached Missed ExceededChange: Positive Negative ActionYes/NoNoNoneYesNone 8YesNone 22MPC participants will increase goals,dreams, and beliefs (belief that sexualinvolvement might interfere with one’s goalsand dreams for education and career), by30%, as reported on post-tests. Reached Missed ExceededChange: Positive Negative YesNone 30MPC participants will increase condom useskills and self-efficacy, by 25%, as reportedon post-tests. Reached Missed ExceededChange: Positive Negative NoneNo

Process EvaluationDates and Participation TargetsProgram datesA. Total priority population: 252Age/Grade: 8thOther characteristics:AttendanceB. Total program participants who attended every session 22Did you maintain fidelity (offer activitiesaccording to program requirements)? Yes NoProcess evaluation resultsDivide the total at C. by the total at A.C. Total participants in evaluation: 25Evaluation participants (check all that apply):facilitatorsIf No, what percent of the program was actually delivered?% of target: 10% (C A x100)Divide the total at C. by the total at B.% of actual: 88 (C B x 100) Facilitators or staff Participants (all) Participants (some) OthersHow well does the evaluation represent the population served? (check on): Not at all wellpeer Somewhat well Very well

Planning Program ImprovementsCQI QuestionsResponse1. Do we need to do another needs assessment?How relevant and current to our participants is the data?No, we just need to be able toaccommodate the whole cohort of8th graders.2. Do we need to change goals & desired outcomes or potentialparticipants?Target different behaviors?Address other determinants (see BDI logic model)?No, the original overall goal ofdecreasing frequency o

9-2 Continuous Quality Improvement but systematic work review to see what changes you could make to improve your program the next time around. It means, simply, that you constantly collect and use information to make adaptations to your program so that it works better over time. Although CQI can