ECUR 809 Assignment #2
The following is an assessment of possible evaluation strategies regarding Alberta Education’s ECS Programming for Children with Severe Disabilities.
Given the ECS program’s description, I believe the primary evaluation strategy would be an outcome-based or goal oriented, summative evaluation. The ultimate purpose of this program is that it “must meet the child’s needs”; a very broad objective but one that would be paramount in a program such as this. Michael Scriven’s model would be appropriate in that the reasons for the program need to be identified. The effectiveness of the program would be measured based on how well the primary goals have been achieved. Ideally, this evaluation would start by identifying just what some of the specific needs of these children are, obviously followed by an assessment of how well they were being met.
This program has children with severe/profound disabilities so, ideally, a pre-test and post-test would reveal just how well their needs have been met. If this is unavailable, I believe much could be learned by a post-test only. Measurement tools might be changes in skills, behaviours, and learning while the children were in the program. Since the program is three years long maximum, children could also be monitored for changes once leaving the program to give further indications of its effectiveness. In addition to the children themselves, much information may be gathered through interviews with other care providers like social workers or those in health services who might be involved in the child’s care. The primary source of information as to whether or not the needs of these children are being met would ultimately come from the parents or principal care givers. Knowing the child the best, these people would be in the best position to reflect on whether the program was indeed providing the benefits (i.e. meeting the child’s needs) that they require.
I would not rule out a formative or process evaluation but I think the summative one would be valuable to do first to help guide the direction of the formative one. If interviews with primary care givers reveal that some facets of the program are inadequate, it would help immensely to direct the issues focused on in the formative evaluation. For example, a process evaluation could look at things like the allocation of in-home vs center-based services, time of average home visits (1.5 hours), number of home visits per year (minimum of four), age criteria for eligibility, and criteria for assessing the child’s current level of functioning. In fact, this type of program would lend itself quite well to a combination of summative and formative evaluation where the outcomes are assessed and, at the same time, possible improvements in the process are identified as well. This evaluation would be time consuming as much of the information would be attained through personal interviews but it would be all the richer for the insights gleaned from this approach. Practical solutions to any potential problems could result directly from the involvement of the people in the best position to analyze how the program is working for, or worked for, the child in their care.
Friday, September 18, 2009
Thursday, September 10, 2009
ECUR 809 Assignment #1
ECUR 809: Assignment #1
Program Evaluation Summary
The following summary pertains to the North Carolina General Assembly’s program evaluation regarding controlling the cost of Medicaid private duty nursing services, Dec. 2008
http://www.ncleg.net/PED/Reports/documents/PDN/PDN_Report.pdf
The Program Evaluation Division’s report on controlling private duty nursing services costs in North Carolina is, as the name suggests, a cost-effectiveness evaluation. The impetus for the evaluation was a response to the fact that the number of recipients receiving private duty nursing, and the costs of their care, had outpaced the growth of Medicaid from 2003-04 to 2006-07. North Carolina Medicaid funds private duty nursing benefits and the goal of this program evaluation was to determine the cost savings of alternatives.
The evaluators used extensive sources for data in their study including Medicaid expense records, both federal and from other states, as well as interviews with private duty nursing recipients. The former gave the study a solid quantitative grounding. For example, the cost of Medicaid spent on recipients of private duty nursing was compared with the amount spent on residents of nursing facilities. The total costs, costs per recipient, and number of recipients were compared from 2004 to 2007. Trends were used to create projections for future years, illustrating the need for cost controls. Interviews provided a qualitative dimension, although too few were conducted (ten recipients and/or their families). Interviewees provided reasons why they chose to receive private duty nursing services.
Twenty states pay for private duty nursing for adults under their state care and North Carolina is one of only two that do not set limits on benefits. Therefore, the evaluators used cost-containment mechanisms of these other states to help guide them in their recommendations for North Carolina. Furthermore, the Division of Medical Assistance had created suggestions for cutting costs, including establishing clearer and more objective criteria for evaluating recipient need which the North Carolina evaluators used in their recommendations.
While the North Carolina evaluation is comprehensive, looking at past results, projections, and possible solutions, it certainly has its limitations. Some of the cost-containment recommendations, although intuitively reasonable, had no quantitative justification. Since private duty nursing costs were combined with other health care expenditures, there was no way to tease out the actual cost savings of several mechanisms. Similarly, the evaluation noted two potential conflict of interest reasons why recipients might receive private duty nursing services longer than they might need, or perhaps not need at all. First, physician’s care for their patients may bias them to recommend licensed nursing services more frequently than necessary. Secondly, the recipient’s home care agency, which has a financial stake in renewing care, may influence decisions on continued care. While the authors of the evaluation are likely correct that these forms of patient assessment (the Private Duty Nursing Team is dependent on these external assessments) are problematic and some form of independent assessment would be better, they show no evidence that the decisions of physicians and home care agencies are biased. Because this would be extremely difficult to measure, the potential cost savings of hiring independent assessors is impossible for these evaluators to measure.
A couple final problems are evident. Although it is understandable that a Medicaid program would look to other Medicaid programs in the United States for guidance on cost savings, potential solutions to the problem will be missed by not looking at other medical systems worldwide. American health issues are not unique and perhaps the evaluation could have been more insightful by looking at health care systems in a few other countries with similar structures. Finally, the recommendations essentially boil down to reducing the number of recipients using private duty nursing services. The sparse qualitative data mentioned in the evaluation notes that recipients chose to receive private duty nursing services due to better quality of life at home and better care than in nursing facilities. This indicates that perhaps decreasing the number of recipients might not be the best tack. Where human health is concerned, the bottom-line approach can be dehumanizing and, in this case, narrows the focus on possible solutions to reducing costs. If quality of life was given a more prominent status, perhaps more attention could have been directed toward finding ways to reduce costs elsewhere in the medical system.
Program Evaluation Summary
The following summary pertains to the North Carolina General Assembly’s program evaluation regarding controlling the cost of Medicaid private duty nursing services, Dec. 2008
http://www.ncleg.net/PED/Reports/documents/PDN/PDN_Report.pdf
The Program Evaluation Division’s report on controlling private duty nursing services costs in North Carolina is, as the name suggests, a cost-effectiveness evaluation. The impetus for the evaluation was a response to the fact that the number of recipients receiving private duty nursing, and the costs of their care, had outpaced the growth of Medicaid from 2003-04 to 2006-07. North Carolina Medicaid funds private duty nursing benefits and the goal of this program evaluation was to determine the cost savings of alternatives.
The evaluators used extensive sources for data in their study including Medicaid expense records, both federal and from other states, as well as interviews with private duty nursing recipients. The former gave the study a solid quantitative grounding. For example, the cost of Medicaid spent on recipients of private duty nursing was compared with the amount spent on residents of nursing facilities. The total costs, costs per recipient, and number of recipients were compared from 2004 to 2007. Trends were used to create projections for future years, illustrating the need for cost controls. Interviews provided a qualitative dimension, although too few were conducted (ten recipients and/or their families). Interviewees provided reasons why they chose to receive private duty nursing services.
Twenty states pay for private duty nursing for adults under their state care and North Carolina is one of only two that do not set limits on benefits. Therefore, the evaluators used cost-containment mechanisms of these other states to help guide them in their recommendations for North Carolina. Furthermore, the Division of Medical Assistance had created suggestions for cutting costs, including establishing clearer and more objective criteria for evaluating recipient need which the North Carolina evaluators used in their recommendations.
While the North Carolina evaluation is comprehensive, looking at past results, projections, and possible solutions, it certainly has its limitations. Some of the cost-containment recommendations, although intuitively reasonable, had no quantitative justification. Since private duty nursing costs were combined with other health care expenditures, there was no way to tease out the actual cost savings of several mechanisms. Similarly, the evaluation noted two potential conflict of interest reasons why recipients might receive private duty nursing services longer than they might need, or perhaps not need at all. First, physician’s care for their patients may bias them to recommend licensed nursing services more frequently than necessary. Secondly, the recipient’s home care agency, which has a financial stake in renewing care, may influence decisions on continued care. While the authors of the evaluation are likely correct that these forms of patient assessment (the Private Duty Nursing Team is dependent on these external assessments) are problematic and some form of independent assessment would be better, they show no evidence that the decisions of physicians and home care agencies are biased. Because this would be extremely difficult to measure, the potential cost savings of hiring independent assessors is impossible for these evaluators to measure.
A couple final problems are evident. Although it is understandable that a Medicaid program would look to other Medicaid programs in the United States for guidance on cost savings, potential solutions to the problem will be missed by not looking at other medical systems worldwide. American health issues are not unique and perhaps the evaluation could have been more insightful by looking at health care systems in a few other countries with similar structures. Finally, the recommendations essentially boil down to reducing the number of recipients using private duty nursing services. The sparse qualitative data mentioned in the evaluation notes that recipients chose to receive private duty nursing services due to better quality of life at home and better care than in nursing facilities. This indicates that perhaps decreasing the number of recipients might not be the best tack. Where human health is concerned, the bottom-line approach can be dehumanizing and, in this case, narrows the focus on possible solutions to reducing costs. If quality of life was given a more prominent status, perhaps more attention could have been directed toward finding ways to reduce costs elsewhere in the medical system.
Subscribe to:
Posts (Atom)