|
Step 10: Evaluate the Program
Although it is the last topic in the program development process, program evaluation is essential and occurs throughout program implementation.
What To Evaluate
Both process and outcome evaluation measures are vital. Process evaluation examines the implementation of the program, while outcome evaluation focuses on the impact of the program. For example, questions that might be answered through a process evaluation include:
- Are the intended youth being tested at the appropriate frequency?
- Are staff correctly following chain-of-custody procedures?
- How is drug testing affecting the staff?
Questions that might be answered through outcome evaluation include:
- How many youth are using drugs, and what kinds of substances are they using?
- Are youth who test positive receiving needed assessment and treatment services?
- Are tested youth abstaining from using drugs?
- Is recidivism reduced for youth who are tested?
Many other important process and outcome questions can be answered through evaluation. The specific areas to be measured should correspond with the agency mission and the purpose and goals of the testing program.
TOP
When To Evaluate
Evaluation should be built into the testing program and should begin when the program begins. It is helpful if baseline data can be collected against which later findings can be compared.
Data Collection
The purpose of the program determines the data to be collected. Data can be collected through traditional paper forms and filing systems, or they can be entered, stored, and retrieved using computers. Selection of a method will depend, in part, on the availability of technology, the amount of data to be collected, and the expertise and time required by staff to use the method. Whichever method is selected, it is vital that a systematic process be established for collecting and using the data. Some of the types of data that should be collected on individual youth include the following:
- General identifying information about the youth (e.g., name, age, legal status).
- Substance use history (e.g., types, amount, and frequency of drugs used).
- Treatment history and current status (e.g., types of treatment received, attendance at present treatment program).
- Drug-testing results (e.g., dates tested, types of drugs tested for, results).
- Responses given to drug-testing results.
TOP
Evaluation Findings
Collected data can be used in several ways. This information may be used to track the services provided and the progress of an individual youth. Data such as those listed above can be used to show trends in drug use and youth responses to treatment and juvenile justice interventions. This is useful in assessing the effectiveness of the current case plan and modifying it if necessary.
Aggregate data also can be used to assess the drug-testing program. For example, cumulative data can reveal trends in drug use among youth over time, the effectiveness of treatment services and case plans, and any problems in program implementation.
Evaluation results should be used to make ongoing decisions about program operations. Evaluation findings can help to answer these questions:
- Are program goals appropriate?
- Are the program's processes adequate and are they being implemented accurately?
- What are the outcomes?
- Are the outcomes found through the evaluation consistent with the program goals?
- Are there any unintended outcomes?
- Is the cost of the program appropriate for the results received?
- Are the data collected appropriate and adequate for effective evaluation and decisionmaking?
- Are the findings consistent with local community data?
A management information system is necessary for evaluating a substance-testing program. The system may be either manual or automated, but it should be capable of providing information about the effectiveness of the program. It should be easy to use, and results should be easily retrievable. The agency should establish standard procedures for staff responsibilities and operation of the management information system. Confidentiality must also be considered in designing the input, storage, and retrieval of information.
TOP
|
