Using Outcomes to Inform Our Practices- Results Based Accountability
In January 2015, Washington County Mental Health and Developmental Services (WCMHS), published its FY14 Outcomes Report, as it has done for the last two years. The report is a compilation of data across our agency that answers the three main tenants of the Results Based Accountability™ (RBA) model, developed by Mark Friedman, “How Much We Do, How Well We Do It and Is Anyone Better Off?”. The report provides a wide lens across all Divisions and highlights some of the unique and specialized programs that we offer through education, support, and treatment of individuals who live with mental health challenges, substance abuse issues, or intellectual disabilities to the community members of the Washington County region.
This outcomes framework, better known as RBA, is a tool we embraced as an agency a few years ago and just recently in June 2014, the State of Vermont adopted RBA and now mandates that all State governmental programs use it to assess and report on the three main questions as outlined above. The RBA structure helped us organize our report in a way that succinctly and effectively informs our community about how our services impact the lives of those we serve. Just as importantly, we also use RBA to inform our programmatic practices and evaluate the day-to-day work we are accomplishing and track the differences we are making in the lives of the clients we support.
As a mental health agency, we are always striving to do the best work we can and make the most positive impact in a myriad of ways for our community. We use the RBA model to help us assess our accomplishments and areas that need improvement.
How Much Do We Do?
Answering the “How Much Do We Do?” question helps us evaluate one piece of the picture in assessing effectiveness. We essentially count the number of services we provide and the number of clients we support. In FY14, WCMHS served approximately 7,055 clients, unduplicated by program, or 3,067 distinct individuals. These clients received 277,492 services and there were 80,220 Residential/Crisis Bed Days in FY14. Below is a pictograph showing who the people are who received our services. We also estimate that we served an additional 2,000 community members through non-billable education and service provision.
We also look at the number of clients served on a programmatic level. For instance, in Fiscal Year (FY) 2014 our Green Mountain Workforce Program, that exists in our Community Support Program Division, worked with 137 clients and provided 3,732 services.
In the Supported Employment Program, within our Community Developmental Services Program, we answer the “How Much Do We Do?” question by tracking the percentage of clients employed. In FY14, over 73% of our CDS clients were employed.
We apply this quantitative question to every program and Division within WCMHS. It helps us assess the amount of support we provide to our community, across the agency. More importantly though, is answering the question, “How Well Do We Do It?”.
Not only do we use client satisfaction surveys to assess the quality of our work, but we also evaluate our programs on some specific internal measures. One example is analyzing the wait time for our clients to receive services. In our Children’s Integrated Services Programs, from January-June 2014, 80% of children and families were seen within 5 days of their initial contact at WCMHS (VT Agency of Human Services). This is comparable to the state-wide average. This is a measure that we find essential to track as it is clinically important to initiate contact with families as soon as possible after their initial contact with an agency. The quicker we see our clients in need, is one way we measure how well we are doing.
Another measure we implement across WCMHS, is to track our employee turnover rate to see how long staff are staying employed with us, to help us internally see how well we are doing and what improvements we can implement to keep this rate as low as possible. We also administer a staff satisfaction survey, in order to discern if employees are happy and enjoy working at WCMHS. In FY14, 94%of the employees said they enjoy what they do. We additionally gleaned from the staff satisfaction surveys ways in which we can improve some facets of the agency to then improve upon the bottom line of how well do we do it.
How Well Do We Do It?
In June 2014, WCMHS surveyed its clients to see how well we are doing our job according to our customers’ opinions. The following graph shows very favorable feedback from the community we serve.
Another equally important question we ask and answer is, “Is Anyone Better Off?”
Is Anyone Better Off?
WCMHS administers a variety of standard assessments to monitor clients’ progress toward recovery. These assessments are implemented across different agency programs and over time are evaluated to see how our clients are better off. One example is in our Community Living Program. Clients are evaluated every six months to asses if they are maintaining or improving their community integration skills. In FY14, 60% of the clients maintained or improved these specific skills. This assessment is an important tool to help us determine if anyone is better off.
Our Client Satisfaction Survey also helps us determine if the community we serve is better off. In our June 2014 survey,
87% of clients said that our services made a difference in their lives. We will continue to survey our clients each year across our agency and within our programs to assess how their lives are better off.
WCMHS is consistently implementing the RBA outcomes model in more and more of our programs. We have found it to be an effective tool to evaluate our programmatic goals and assess how our clients’ lives are better off because of the support we provide. We will also continue to think critically about what our results show us and the best way to share our outcomes with our stakeholders through our annual Outcomes Report.
To download the FY14 WCMHS Outcomes Report and to view additional data on our Results Scorecard website – click here. To learn more about Results Based Accountability, you can visit their website – click here.