For any parents out there or for anyone who’s ever spent time with a three-year-old, you are familiar with the question “why”? It seems like the stream of why’s can be endless. This inquisitive nature is lost for many in adulthood. This lost art of asking why can be very beneficial when you’re looking at issues within your organization. Sometimes it’s uncomfortable, just like those toddler questions! But, I offer you this: it can be a great way to discover the root cause of problems, needs, or performance gaps you and your organization face. While the repeated asking can be somewhat monotonous, there are great insights and answers to be found in the process. I invite you to learn more about the 5-why’s and get to the root of your problem or questions…just stop short of shouting “because I said so!”
What’s the Problem?
Whether you are for-profit, non-profit, or government you are in business to solve a problem. Being able to state the problem effectively is a giant step toward offering your customers a solution to the problem you aim to solve.
“We need money for a counselor. That’s why we’re looking for grants”
“We need to cut costs to be profitable”
You are trying to solve a problem: your organization needs funding. But is that the real problem you are trying to solve?
Defining the true problem will set you up for success.
A word of caution: the problem is not the solution. The problem isn’t the need for the solution; there’s a problem that’s driving you to look at potential solutions.
Here’s a couple of examples of solutions in the problem statement:
“The problem is that we need a new school in our community.”
Is the problem that you need a new school, or is it that the current schools are overcrowded or falling apart?
“The problem is that there isn’t any research that supports my theory.”
Is the problem that there isn’t any research that supports the theory, is it that the research has found the opposite, or that the research isn’t the problem at all – it’s a problem in the community that you are exploring?
To revisit the example above:
“We need a counselor to address the growing substance use problems in our school before students get into real trouble.”
Is the problem/need for a school counselor, or that there is a growing substance use problem and students are facing life-changing consequences because of the substance use?
“We have a financial shortfall and need to cut expenses to below our revenue”
Is the financial shortfall because your expenses are too high? Or did something else happen that shifted the balance of income/expense?
If you have a solution, intervention, or research question in your problem statement, you need to dig a little deeper.
How do you get to the real problem? The 5-Why’s exercise, while newly energized as a go-to business tool, has been around for decades as a method for digging in to find the source of an issue.
This simple yet effective method will take a concern and drive the group to analyze the reasons for problems and thinking patterns.
The exercise goes like this:
State the problem at hand
Ask “Why” (or “so that,” or “because…”, whatever phrasing will get the dialogue going)
Capture the thought(s) that come next
Ask “Why” again and capture the thoughts produced until you’ve asked at least 5 times.
The result is a root-cause of the problem.
When you do this in a group, it’s best to have everyone do the asking and answering on their own and then come back together to discuss what everyone’s chain of statements. You will likely come up with several root causes that (1) are all right answers, and (2) can be addressed in a comprehensive programmatic solution that will address the problem from multiple vantage points.
Here’s an example of the 5-Why’s digging deeper to get to the root cause of a problem.
We need money for a counselor.
There’s a growing substance use problem in our school, and students are getting into real trouble.
Students are showing up to events high or drunk, and we are obligated to report them.
Their parents think it’s ok for them to drink or use marijuana at home.
They don’t think it’s a big deal.
Alcohol and marijuana are legal in some places for adults, so they think it’s ok as long as they don’t drive.
Do you need a counselor at the school? Probably. Will that solve the problem? Probably not. The issue extends to the adults in the community, not just the students. You’ll likely want to have a community education/outreach initiative as well as the counselor.
Using the 5-Why’s approach can help you clarify what you are really trying to solve. This method is especially useful when you feel you are at an impasse, the solution doesn’t feel right, the problem seems too big, or what you are doing does not seem to resonate with your customers/participants.
Template for Reference
Here’s an example of a template to work through your 5-Why’s. After your first draft in the table, revise the statement to be a complete thought/sentence.
<<This is what is happening now>>
<<this is why that’s a problem>>
Why… or So that… or Because…
<<Rewrite in a statement>>
What’s the problem you need to run through the 5-Why’s? Drop a comment to let me know how you might use this approach.
Most of us associate evaluation with our annual employee performance review. No one likes the way performance evaluation is generally done. Not the manager. Not the employee. It’s probably the most dreaded, feared, heart-wrenching, ulcer-causing activity done in the business world.
So why would we want to put our entire program under that kind of microscope? Could anything be more repulsive?
I’ve really only encountered two types of people when it comes to evaluation: those that avoid it like the plague, and those that are so passionate about it that they will shout about its virtues from the rooftops.
Can you guess which I am?
Okay, so I am one that would shout it from the rooftops. Why, might you ask?
The simplest explanation is, I like to know where I stand, and what people are thinking. I like to know what I am doing really well, and what I need to work on in order to be the best “me” I can be. These concepts transcend my work and personal life.
For those like me, we can get geeked out pretty quickly when we start talking data, outcomes, process, budgets, the benefits of measurements, and what can be measured versus what should be measured. I am certain my pupils just dilated and my heart skipped a beat or two writing that sentence, and I didn’t even go into the difference between an output and an outcome!
For those whom evaluation is the plague, you may have just experienced sweaty palms, a blood pressure spike, hives breaking out across every limb, and may even feel the start of a migraine coming on just from reading that sentence!
So for those of you who think evaluation is like the plaque, know this – it really doesn’t have to be that bad. Breathe deeply and repeat that sentence again if you need to. It doesn’t have to, and frankly shouldn’t be that bad.
I know, I come from a very special place when I say that, but…really…as business professionals (yes, nonprofits, universities, and governments are businesses too!), and I’d stretch to even say as humans, we need to know if what we are doing is working. Otherwise, why are we even doing it (other than job security)? Don’t you want to know if your program is as good as you think it is?
In an environment of “proving your worth” evaluation can be the best prescription: it can show that what you do is worthwhile and you can also show that you identified what was wrong and made efforts to correct course. That’s pretty admirable.
So, next time you hear the word “evaluation” take a deep breath and start the process: 1) plan the evaluation, 2) conduct the evaluation, 3) determine the results of the evaluation, and 4) create and implement a game plan for addressing the results. Yes, this is an over-simplified view of the process, but it’s the framework that can get you to a better place mentally to face the project.
Also remember, that you don’t go though evaluation alone. It’s an all-hands-on-deck type of project. Even better, the best evaluations are those done by neutral third parties (hence you don’t do the bulk of the work!).
So again, take that deep breath. Now, here’s a brief overview of what you can expect when conducting an evaluation project.
What can you expect?
In general, an evaluation project will include the following components.
A literature review:
What are the industry standards for the program?
Research pertinent to your industry and particular program/activity is used to inform the final evaluation design and the outcomes evaluation indicators.
An evaluation design:
What is the focus of the evaluation and how will it be conducted?
The design will be finalized as part of the proposed evaluation project. All outcomes, indicators, and the methodology most appropriate for your organization and target program are reviewed or created. During the evaluation design, interview instruments, process evaluation objectives, outcome indicators with operational definitions, and project charter will be created using appropriate industry frameworks for evaluation (e.g. CDC Framework for Program Evaluation in Public Health and Program Evaluation Standards). The SNF Writing Solutions methodology also includes the Lean Six-Sigma DMAIC model.
What do people think about the program?
Interviews with key stakeholders (leadership, employees, management, board members, possible clients, and funders) are conducted to assess program impetus, implementation, and interaction experience. The stakeholder pool and interview schedule is coordinated with the evaluation champion in your organization. These interviews will inform the remainder of the evaluation schedule and serve as a progress gate for any evaluation design modifications before moving forward.
What data do we have versus what we need?
Information. Excel Sheets. Databases. Calendars. Websites. Marketing literature. Budgets. Tally sheets. Meeting notes. The list goes on.
Depending on the focus of your evaluation, different types of data are gathered to feed the evaluation process. Some will be readily available and some will need to be culled from various sources. With the evaluation design completed, the data needed to compete the evaluation has already been determined and must be pulled together. The SNF Writing Solutions team includes data-gathering and data-entry experts that aid organizations through this process of gathering data and even creating new data sets.
How well is the program executed in relation to the intended model?
The process evaluation assesses the extent to which the target program/department/ service is delivered as planned as well as the facilitators and barriers to implementation. The process evaluation will clarify how, why, and for whom the program works and which components are most/least effective. Through the process evaluation, an understanding of the opportunities for quality improvement and/or corrective feedback is established as well as determining the feasibility for replication and/or expansion of the program operations (remember, funders love programs that are replicable). Consideration of the reach of the program, the intervention dose delivered and the dose received, as well as the implementation and model fidelity within the context of the program environment will be included in the evaluation design.
The process evaluation with SNF Writing Solutions will generally include a Lean Six Sigma review of the program with a process documentation review, analysis of client and organizational service expectations, observation of all organization practices (with regard to HIPAA or other legal requirements as appropriate), documentation of current state, and statistical analysis of current state against service expectations.
How are costs balanced with benefits?
A cost evaluation looks at the impact of program to the cost of the program/service for the target population or stakeholder (can be organization as a whole). Methodologies for the cost evaluation can vary. The SNF Writing Solutions evaluation includes a review of dollars spent versus income (or funds available in the case of a grant-funded program) and population served and a cost-benefit analysis (CBA) of providing services. As part of the cost evaluation, the return on investment (ROI) is generally calculated for the program. In reviewing the financials and CBA for the program, the feasibility for replication and/or expansion of the program in achieving outcomes can be considered. A high cost program can still be a sound investment for a funder if the results are impressive and long-lasting.
Are we meeting the needs of our customer?
When thinking of evaluating a program, this is the evaluation that most think of: assessing the impact of a program on the societal improvements sought based on existing or determined short-term and long-term activities/services targeted in the evaluation. The literature review and stakeholder interviews will be used for qualitative analysis of the program. SNF Writing Solutions includes text theme determination. Quantitative data analysis of service provision versus expected outcomes will include a review of all outputs and outcome indicators from the program using a correlational analysis. The outcomes evaluation can include the feasibility for replication and/or expansion of the program in meeting societal needs. If needed, a baseline for indicators can be retroactively established from organizational data prior to the implementation of the project/program.
What did we learn?
The final deliverable of a project is largely determined by the client-side champion of the evaluation project. A report will generally include the interpretation of all evaluations with conclusions regarding the program. The report will include statements regarding the merit, worth, and significance of the program/service and interpretations of the findings against the standards identified in the literature review, evaluation design, and stakeholder interviews. Recommendations for specific actions to take into consideration for improvement, replication and/or expansion of the program will also be included in the report.
At the conclusion of the evaluation, SNF Writing Solutions shares the methodology and results with organizational stakeholders, and, at the request of the organization, will assist in presenting to funders, staff, or other groups, and can prepare the findings for dissemination as a potential white paper, journal article, or for conference presentations as opportunities are available.
What will we do with what we learned?
The evaluation process is only as good as the follow-through on the findings. Many things will prove to be working and should be left alone – don’t fix what isn’t broken. That leaves those items that weren’t quite what you were expecting, or had not gone planned. Take a good look at those items and figure out the root cause (this is why SNF Writing Solutions uses the Lean Six Sigma approach – you will already know root causes) and develop potential fixes for the root cause. Create an action plan to implement the change (if a big change, you may want assistance with change management) and monitor the effects of the changes you implement. This ongoing measurement and evaluation can be an extension of your relationship with the evaluator. As you see improvements, acknowledge and celebrate them!
Evaluation is not a one-and-done experience. Those organizations that are successful are always measuring (the right things) and evolving to meet the demands of their customers within the scope of the organizational expertise and mission. This is accomplished with rigorous and continuous evaluation of the organization and individual programs. SNF Writing Solutions generally includes post-evaluation follow-up and implementation support as part of any evaluation project.
Hopefully, with better overall evaluation experiences, those employee performance evaluations are also a little less stressful for you – everyone will know, on a regular basis, where everyone stands!
Here’s to the evaluation plague – turn it into your passion!