Replication of Blueprints Programs

The selection of a Blueprints program eliminates the need for an outcome evaluation in the short-term (4 or 5 years), because these programs have already been evaluated rigorously and continue to be evaluated, typically in clinical trials funded by the Federal Government. The critical issue for a Blueprints program is the quality of the implementation. To ensure quality implementation, training and technical assistance and monitoring of the implementation (a process evaluation) are essential.

OJJDP began supporting the Blueprints Initiative in April 1998 by providing 2-year training and technical assistance grants (3 years for Life Skills Training) through CSPV to interested sites. CSPV has contracted with the designers of the Blueprints programs to provide training and technical assistance and is conducting a process evaluation to ensure program fidelity at each site. Technical assistance includes help with choosing a Blueprints program, planning for program implementation, and carrying out the program, which includes training, onsite visits to troubleshoot problems, and regular telephone consultation. A total of 112 sites (including 290 schools) were selected to receive the assistance provided by these grants. Another 35 school/district sites currently are being assessed for selection.

OJJDP supports four grant programs (Formula Grants, Title V Community Prevention Grants, Juvenile Accountability Incentive Block Grants, and Enforcing the Underage Drinking Laws) that award money to State governments. States then determine which programs they will support with the OJJDP funds. States and localities wishing to implement Blueprints programs may look to these four grant programs as sources for funding.

Typically, success depends on implementing the program as designed. Little is known, however, about the implementation problems that cause many programs to fail. In response, CSPV is accumulating data on the Blueprints replication sites regarding problems encountered, attempted solutions, efficacy of the solutions tried, and the reasons for their success or failure. To screen potential replication sites effectively, CSPV has also collected data on the organizational capacity, funding stability, commitment, and resources required for a high probability of success. CSPV is gathering this information through a process evaluation conducted at each site. The evaluation includes two onsite visits and one telephone interview per year with program implementers. Although it is too early to determine the extent of implementation integrity and all of the qualities relevant for success at each site, several themes have surfaced regarding the factors that may impede or enhance a successful program. A few notable issues are discussed below.

Assessment and Planning

Four general conditions are important in the earliest stages of planning and development to achieve successful implementation: needs assessment, strong buy-in, adequate resources, and commitment. While this list is not exhaustive, these conditions can increase the probability of selecting and carrying out successful programs.

Needs assessment. Program initiators must begin by assessing the unique problems of the community or school setting in which the program will be implemented, clearly identifying the target population (e.g., age, gender, socioeconomic status, geographic area, level of risk), and choosing a program that will address the site’s problems and target population. Failure to choose a program that matches the site’s problems and population can lead to unsuccessful outcomes, even if the program is carried out with fidelity. For example, a drug prevention program meant for youth who have not yet initiated drug use may be ineffective if implemented in a treatment setting with substance-abusing youth.

Buy-in. Building a strong base of support early in the process among key decisionmakers and individuals responsible for program implementation is extremely important and requires time and effort. Achieving buy-in involves building consensus at the site around the program’s philosophy and core components. Although it is preferable to have a team of staff with a strong leader, sites need to have at least one staff member who is given ample time to plan for successful implementation. This includes convening meetings with other staff and organizations to educate and garner support from all levels of the applying agency and to establish other pertinent interagency support. Most of the Blueprints sites had only one person who championed the program and worked on all planning and development issues. These sites could have benefited by organizing a team of individuals to manage the initial planning and development tasks, which are generally more than one person can handle. A team approach can improve communication among all levels of management and staff and can help build a strong base of support among many individuals within the organization so that, even if key individuals leave, others can carry the program forward. (In fact, sites should plan on turnover among key staff.) Failure to enlist from the start the active involvement and support of all key players, including implementing staff, can seriously jeopardize program success.

Adequate resources. A third important condition is obtaining all the required resources—tangible (e.g., monetary) and nontangible (e.g., staff support, client referral base)—for program planning and implementation. Sites should ensure that they have the appropriate funding for program assessment, planning, startup, and maintenance of all required program components. Failure to understand all of the resources necessary to implement a program (e.g., funding to cover costs of curriculums or supplemental materials, assessments, travel costs for training, and other expenses) and failure to commit to their acquisition can result in weak implementation and possible program failure.

Commitment. Related to buy-in, the last major condition is a commitment to high-quality implementation. All programs have key elements that must be adhered to in order to ensure success. Program goals, principles, and core components must be identified during the planning stages and clearly communicated to all relevant persons involved in the implementation. Failure to understand and commit to the program’s philosophical underpinnings and key components can undermine success.

Implementation Challenges

As of the publication of this Bulletin, CSPV’s process evaluation remains under way. Therefore, formal data regarding the types of organizational, staff, financial, and other factors that contribute to implementation failure or success are not yet available. However, many themes regarding common challenges have emerged and offer a general picture of what communities will encounter when adopting a Blueprints program. One common theme is that problems are unavoidable. All sites, even those with the best planned implementations and most rigorous assessment processes, have confronted numerous hurdles. Another common theme is that, regardless of the unique qualities of each site and each Blueprints program, sites tended to face similar challenges. The most complex and time-consuming challenges occurred during the startup phases (first 4–8 months) of program development. Most sites overcame initial hurdles only to run into a new set of challenges during later developmental stages.

Early challenges. Faced with hiring and training staff, accomplishing an array of administrative tasks, and raising funds, many practitioners reported feeling overwhelmed during the first 4 to 8 months of program development. Their frustrations included but were not necessarily limited to the following:

• Administrative tensions. Many staff faced difficulties tackling new work systems (e.g., paperwork, communication models, management styles) within preexisting, and sometimes conflicting, agency structures.

• Professional insecurities. Staff commonly felt that they did not have sufficient experience, training, or skills to implement new program models properly. Initial trainings were designed to introduce a new set of skills to staff, but many felt insecure about their new and developing talents.

• Intra-agency and interagency links. The introduction of Blueprints programs into communities required agencies to hire new staff or assign existing staff to new workgroups. In many cases, agencies had to reach out to and build collaborative relationships with other institutions in the community. In most cases, developing relationships and lines of communication proved a complicated and sometimes frustrating task.

Support from onsite supervisors and administrators and regular contact with technical assistance providers (through training, telephone consultation, and onsite visits) and CSPV staff generally eased many of these early problems.

Later challenges. Process evaluation visits with sites that had implemented the program for at least 1 year revealed several common trends. After the first year, many of the early implementation challenges subsided as staff members’ confidence increased and relationships within and among multiple agencies became more secure. However, implementation obstacles did not disappear entirely.

• Persistent challenges were the ever-present, low-level tensions (e.g., frustration with paperwork, the ongoing stress of providing time-intensive interventions) involved in providing services. While these challenges seemed daunting in the early stages of program development, site staff became used to paperwork demands and new work styles over time. Such tensions, although ongoing, rarely threatened program development.

• Episodic problems included short-term changes at a site (e.g., staff turnover, personality conflicts, administrative shifts within and between agencies). Such challenges required time and energy to manage but were often solved once new staff were hired, relationships within and between agencies were repaired, and staff members became accustomed to new working arrangements.

• Threatening problems occurred when sites experienced dramatic changes (e.g., loss of program champions, funding cuts, severed relationships with key agencies in the community) or confronted prominent individuals who opposed the philosophies behind and implementation of particular programs. Such problems posed serious challenges to program integrity.

During startup, such problems were more likely to result in site failure. Sites that had completed their first year of implementation often found that organizational and community investment in and motivation for their program helped them overcome such threatening problems.

Implementation Strengths

Sites had varying implementation strengths and weaknesses. Some had hired expert staff members whose commitment to the program and keen skills helped sites overcome problems with little disruption of implementation. Other sites had administrators whose unwavering support helped carry programs through myriad challenges. Although sites varied in their strengths and weaknesses, preliminary data suggest that four factors played a role in how programs overcame common problems: strong administrative support, presence of powerful program champions, staff expertise, and ongoing help from technical assistance providers. All of these factors not only determined whether and how sites overcame challenges—they also seemed to determine the overall quality of implementation.

Administrative support. Strong administrative support for programs was crucial. Administrators ensured that staff had the necessary resources to implement programs. In addition, they had the power to make organizational changes to accommodate new programs. As leaders, administrators also built staff support for programs and thus helped sites overcome problems with staff morale and motivation.

Program champions. These were the individuals who spearheaded implementation efforts. They helped maintain motivation, initiated necessary changes, and orchestrated completion of the many tasks necessary to make projects successful. In general, the most successful champions were those who commanded some power at the site. In some cases, champions were supervisors at a site and had good relations with both administrators and staff members. In other cases, champions were outsiders who had a long history of working well with site members.

Staff. A strong staff also contributed to program success. Staff members who supported or “bought in” to the philosophies of the programs, had the necessary experience and credentials, and were dedicated to learning new skills helped programs overcome a variety of obstacles. Staff members who were adequately reimbursed for their time were more effective implementers than those who were not.

Technical assistance providers. By recognizing the source of problems and suggesting solutions, technical assistance providers could help sites overcome weaknesses in many areas of implementation. During phone conferences and onsite meetings, technical assistance providers suggested ways to increase administrative support, identified appropriate program champions, and provided ongoing training to staff members. In a number of cases, technical assistance providers gave periodic presentations to schools, agencies, and community members to increase general awareness of and support for the Blueprints programs.

Blueprints for Violence Prevention	Juvenile Justice Bulletin • July 2001