MENU TITLE: Trial Court Performance Standards and Measurement System Implementation Manual. Series: BJA Monograph Published: July 1997 Author: Bureau of Justice Assistance 248 pages 550,135 bytes U.S. Department of Justice Office of Justice Programs Bureau of Justice Assistance To receive copies of the forms that accompany this text, please contact Jennifer Knobe, Bureau of Justice Assistance at (202) 616-3212. Trial Court Performance Standards and Measurement System Implementation Manual Monograph July 1997 NCJ 161567 ------------------------------ Foreword Developing a common language for describing, classifying, and measuring the performance of trial courts was the goal of an 8-year effort, the Trial Court Performance Standards Project, initiated in 1987 by the National Center for State Courts and the Bureau of Justice Assistance (BJA). The Trial Court Performance Standards and Measurement System is the result of that effort. Crafted by a commission of leading trial judges, court managers, and scholars and demonstrated in trial courts across the Nation, the measurement system is an invaluable resource for enhancing a court's ability to provide fair and efficient adjudication and disposition of cases. Because many trial courts lack the resources to create a mechanism for self-evaluation, the project is critical to improving the administration of justice on the basis of universally accepted performance standards. This publication is a detailed guide to implementing the Trial Court Performance Standards and Measurement System. It is intended for judges, court managers, lawyers, policymakers, court staff, and other professionals who will participate in the implementation process. It is our hope that every trial court in the Nation will use this guide and its companion publications to begin improving access to justice and its administration with equality, integrity, and timeliness. Nancy E. Gist Director ------------------------------ Preface The Trial Court Performance Standards and Measurement System is the culmination of an 8-year initiative begun in 1987 by the Commission on Trial Court Performance Standards to develop measurable performance standards for the Nation's State trial courts. The Commission first created the Trial Court Performance Standards, which set forth standards of performance for trial courts in five performance areas: o Access to Justice o Expedition and Timeliness o Equality, Fairness, and Integrity o Independence and Accountability o Public Trust and Confidence The Commission's next challenge was to provide trial courts with a systematic and sound means to examine how well they achieve these performance standards. To meet this challenge, the Commission and the Trial Court Performance Standards Project staff developed a set of measures for assessing trial court performance. Twelve trial courts in Ohio, New Jersey, Virginia, and Washington subsequently tested the measures during a 4-year demonstration. In addition to the Commission and project staff, more than 100 personnel of the demonstration courts, as well as program monitors of the Bureau of Justice Assistance and the State Justice Institute, contributed to the demonstration process. This extensive and collaborative undertaking was necessary to ensure that the measures, many of which are complex and novel, are operable in court settings, do not consume unreasonable amounts of resources, and produce information courts can use readily. The resulting measurement system is intended to be a versatile tool for self-assessment and improvement, and not a means for evaluating the performance of individuals or for drawing comparisons across courts. The measurement system attempts to balance practicality and economy with precision and scientific rigor. The 68 measures within the system accomplish this balance to varying degrees, but a workable measurement system should be viewed as evolutionary and subject to continuing development. The Commission expects that as trial courts implement the system, their experiences will inspire innovative approaches that improve both the individual measures and the system as a whole. Although implementing the measurement system requires dedication, perseverance, and flexibility from a trial court's staff, the undertaking is critical to effective court performance. The Commission commends the Trial Court Performance Standards and Measurement System to the court community as a useful set of tools for conducting self-evaluation and for engaging in the worthy pursuit of improving public service. ------------------------------ Acknowledgments The Bureau of Justice Assistance would like to thank the Commission on Trial Court Performance Standards for its dedication and vision in guiding the Trial Court Performance Standards to their fruition. Commission members include the following individuals: Honorable Robert C. Murphy, Chair Chief Judge (Retired) Court of Appeals of Maryland Towson, Maryland Honorable Rebecca A. Albrecht Associate Presiding Judge Superior Court of Arizona, Maricopa County Phoenix, Arizona Robert N. Baldwin State Court Administrator Supreme Court of Virginia Richmond, Virginia Carl F. Bianchi Director of Legislative Services Legislative Services Office Boise, Idaho Honorable Robert C. Broomfield Chief Judge, U.S. District Court District of Arizona Phoenix, Arizona John A. Clarke Executive Officer/Clerk Los Angeles Superior Court Los Angeles, California Judith A. Cramer Manager, Weed & Seed Neighborhood Revitalization Program Tampa, Florida Honorable Anne L. Ellington Assistant Presiding Judge King County Superior Court Seattle, Washington Howard Hanson County Clerk/Superior Court Administrator Marin County San Rafael, California Robert D. Lipscher Administrative Director (Retired) New Jersey Administrative Office of the Courts Trenton, New Jersey Edward B. McConnell (ex-officio) President Emeritus National Center for State Courts Williamsburg, Virginia Doris Marie Provine Chair, Department of Political Science Syracuse University Syracuse, New York Honorable Henry Ramsey, Jr. Dean, Howard University School of Law Washington, D.C. Honorable Leo M. Spellacy Judge, Ohio Court of Appeals Eighth District Cleveland, Ohio Whitfield Smith (1987-1991) Clerk of Court (Former) Superior Court for DeKalb County, Georgia Decatur, Georgia Honorable Fred B. Ugast Chief Judge (Retired) Superior Court of the District of Columbia Washington, D.C. Bureau of Justice Assistance, U.S. Department of Justice Marilyn Nejelski, Program Manager Charles Hollis, Chief, Adjudication Branch National Center for State Courts Sally T. Hillsman, Vice President (1992-1995) Geoff Gallas, Vice President (1987-1991) Trial Court Performance Standards Project Staff Pamela Casey, Director Ingo Keilitz,* Director Hillery Efkeman Margaret Fonner John Goerdt Thomas Hafemeister* Roger Hanson William Hewitt Brenda Jones* Susan Keilitz Fred Miller* Beatrice Monahan* Pamela Petrakis* David Rottman* *denotes former project staff In addition to those listed above, numerous individuals at the National Center for State Courts and elsewhere gave generously of their time to assist the development and initial testing of the Trial Court Performance Standards and Measurement System, including Stevalynn Adams, David Aday, Carl Baar, Kent Batty, Richard Berk, Chuck Campbell, Joy Chapper, George Cole, Hank Daley, Michael Dann, Tom Dibble, Chris Duncan, Bill Fishback, Gene Flango, Sandy Garcia, Debbie Gause, George Gish, Gordon Griller, Mary Hogan, Cindy Huffman, Michael Jeans, Lynn Jordaans, Carl Kessler, Kay Knapp, Gerald Kuban, Monica Lee, Chris Lomvardias, Kay Loveland, Jennifer Rae Lovko, Robert Lowe, James Lynch, Barry Mahoney, Mary McCall, Craig McEwen, Jan Michaels, Barbara Meierhoefer, Tom Munsterman, Raymond Nimmer, Jessica Pearson, Mike Planet, Maryann Rondeau, Jane Raynes, Teresa Risi, Dalton Roberson, Ronald Rosenberg, Jeffrey Roth, Fred Rusillo, Hisako Sayers, Bob Tobin, Anne Walker, Patricia Wall, Steven Wasby, Joan White, Matt Williams, and Robert Williams. We also are indebted to the many individuals in the 4 pilot States and the 12 courts for the countless hours and invaluable insights they contributed during the project's 4-year demonstration phase: in New Jersey, Robert D. Lipscher and Theodore Fetter, Administrative Office of the Courts-- Atlantic County Superior Court, Burlington County Superior Court, Morris County Superior Court, Ocean County Superior Court, and Somerset County Superior Court; in Ohio, Stephan W. Stover and Ruth Ann Elmer, Supreme Court of Ohio--Meigs County Court of Common Pleas, Stark County Court of Common Pleas, and Wayne County Court of Common Pleas; in Virginia, Robert N. Baldwin and Beatrice P. Monahan, Office of the Administrator for the Courts--Fairfax County Circuit Court; and in Washington, Mary Campbell McQueen and Yvonne Pettus, Office of the Administrator for the Courts--Spokane County Superior Court, Thurston County Superior Court, and Whatcom County Superior Court. We also gratefully acknowledge Richard Van Duizend, Deputy Director of the State Justice Institute (SJI), for his commitment to the project and SJI's financial support of many demonstration phase activities. The Commission on Trial Court Performance Standards, the National Center for State Courts, and the Bureau of Justice Assistance thank these individuals and the many individuals not named here who assisted in the Trial Court Performance Standards Project in its 8-year duration. ------------------------------ Table of Contents Introduction History of the Trial Court Performance Standards and Measurement System Developing the System's Measurement Component Using the Measurement System Organization of This Implementation Manual Planning to Use the Trial Court Performance Standards and Measurement System Performance Area 1: Access to Justice Standard 1.1: Public Proceedings Measure 1.1.1: Access to Open Hearings Measure 1.1.2: Tracking Court Proceedings Measure 1.1.3: Audibility of Participants During Open Court Proceedings Standard 1.2: Safety, Accessibility, and Convenience Measure 1.2.1: Courthouse Security Audit Measure 1.2.2: Law Enforcement Officer Test of Courthouse Security Measure 1.2.3: Perceptions of Courthouse Security Measure 1.2.4: Court Employees' Knowledge of Emergency Procedures Measure 1.2.5: Access to Information by Telephone Measure 1.2.6: Evaluation of Accessibility and Convenience by Court Users Measure 1.2.7: Evaluation of Accessibility and Convenience by Observers Standard 1.3: Effective Participation Measure 1.3.1: Effective Legal Representation of Children in Child Abuse and Neglect Proceedings Measure 1.3.2: Evaluation of Interpreted Events by Experts Measure 1.3.3: Test of Basic Knowledge Required of Interpreters Measure 1.3.4: Assessing Non-English Language Proficiency Through Back Interpretation Measure 1.3.5: Participation by Persons with Disabilities Standard 1.4: Courtesy, Responsiveness, and Respect Measure 1.4.1: Court Users' Assessment of Court Personnel's Courtesy and Responsiveness Measure 1.4.2: Observers' Assessment of Court Personnel's Courtesy and Responsiveness Measure 1.4.3: Treatment of Litigants in Court Standard 1.5: Affordable Costs of Access Measure 1.5.1: Inventory of Assistance Alternatives for the Financially Disadvantaged Measure 1.5.2: Access to Affordable Civil Legal Assistance Measure 1.5.3: Barriers to Accessing Needed Court Services Performance Area 2: Expedition and Timeliness Standard 2.1: Case Processing Measure 2.1.1: Time to Disposition Measure 2.1.2: Ratio of Case Dispositions to Case Filings Measure 2.1.3: Age of Pending Caseload Measure 2.1.4: Certainty of Trial Dates Standard 2.2: Compliance With Schedules Measure 2.2.1: Prompt Payment of Moneys Measure 2.2.2: Provision of Services Measure 2.2.3: Provision of Information Measure 2.2.4: Compliance With Reporting Schedules Standard 2.3: Prompt Implementation of Law and Procedure Measure 2.3.1: Implementation of Changes in Substantive and Procedural Law Measure 2.3.2: Implementation of Changes in Administrative Procedures Performance Area 3: Equality, Fairness, and Integrity Standard 3.1: Fair and Reliable Judicial Process Measure 3.1.1: Performance in Selected Areas of Law Measure 3.1.2: Assessment of Court Performance in Applying the Law Standard 3.2: Juries Measure 3.2.1: Inclusiveness of Jury Source List Measure 3.2.2: Random Jury Selection Procedures Measure 3.2.3: Representativeness of Final Juror Pool Standard 3.3: Court Decisions and Actions Measure 3.3.1: Evaluations of Equality and Fairness by the Practicing Bar Measure 3.3.2: Evaluations of Equality and Fairness by Court Users Measure 3.3.3: Equality and Fairness in Sentencing Measure 3.3.4: Equality and Fairness in Bail Decisions Measure 3.3.5: Integrity of Trial Court Outcomes Standard 3.4: Clarity Measure 3.4.1: Clarity of Judgment and Sentence Measure 3.4.2: Clarity of Civil Judgments Measure 3.4.3: Experience in Interpreting Orders and Judgments Standard 3.5: Responsibility for Enforcement Measure 3.5.1: Payment of Fines, Costs, Restitution, and Other Orders by Probationers Measure 3.5.2: Child Support Enforcement Measure 3.5.3: Civil Judgment Enforcement Measure 3.5.4: Enforcement of Case Processing Rules and Orders Standard 3.6: Production and Preservation of Records Measure 3.6.1: Reliability of the File Control System Measure 3.6.2: Adequate Storage and Preservation of Physical Records Measure 3.6.3: Accuracy, Consistency, and Utility of the Case Docket System Measure 3.6.4: Case File Integrity Measure 3.6.5: Reliability of Document Processing Measure 3.6.6: Verbatim Records of Proceedings Performance Area 4: Independence and Accountability Standard 4.1: Independence and Comity Measure 4.1.1: Perceptions of the Court's Independence and Comity Suggested Steering Committee Activities for Standard 4.1 Standard 4.2: Accountability for Public Resources Measure 4.2.1: Adequacy of Statistical Reporting Categories for Resource Allocation Measure 4.2.2: Evaluation of Personnel Resource Allocation Measure 4.2.3: Evaluation of the Court's Financial Auditing Practices Suggested Steering Committee Activities for Standard 4.2 Other Related Considerations for Standard 4.2 Standard 4.3: Personnel Practices and Decisions Measure 4.3.1: Assessment of Fairness in Working Conditions Measure 4.3.2: Personnel Practices and Employee Morale Measure 4.3.3: Equal Employment Opportunity Suggested Steering Committee Activities for Standard 4.3 Standard 4.4: Public Education Measure 4.4.1: Court and Media Relations Measure 4.4.2: Assessment of the Court's Media Policies and Practices Measure 4.4.3: Community Outreach Efforts Suggested Steering Committee Activities for Standard 4.4 Standard 4.5: Response to Change Measure 4.5.1: Responsiveness to Past Issues Suggested Steering Committee Activities for Standard 4.5 Performance Area 5: Public Trust and Confidence Standard 5.1: Accessibility Measure 5.1.1: Court Employees' Perceptions of Court Performance Measure 5.1.2: Justice System Representatives' Perceptions of Court Performance Measure 5.1.3: General Public's Perceptions of Court Performance Standard 5.2: Expeditious, Fair, and Reliable Court Functions Standard 5.3: Judicial Independence and Accountability Appendix A: Bibliography Appendix B: Sources for Further Information Appendix C: Forms for Implementing the Trial Court Performance Standards and Measurement System List of Tables and Figures Tables Table 1: Summary of Measures Table 2: Summary of Measures by Primary Data Collection Method Figures Figure 1: Case Disposition Time Standards Adopted by the Conference of State Court Administrators, the Conference of Chief Justices, and the American Bar Association Figure 2: List of Problems Affecting Judicial Independence That Might Be Produced Using Nominal Group Technique Figure 3: Relationship Between Records Staff Support, Judges, and Case Categories Figure 4: Resource Allocation Model List of Forms Performance Area 1: Access to Justice Standard 1.1: Public Proceedings Form for 1.1.1: Record of Access to Courtroom Form for 1.1.2: Tracking Court Proceedings Form for 1.1.3: Courtroom Audibility Evaluation Form Standard 1.2: Safety, Accessibility, and Convenience Form for 1.2.1: National Sheriffs' Association Physical Security Checklist Form for 1.2.3: Survey of Courthouse Security Form for 1.2.4: Interview Protocol on Emergency Procedures Form for 1.2.5: Access to Information by Telephone Directions and Recording Sheet Form for 1.2.6 and 1.2.7: Accessibility and Convenience of the Court Standard 1.3: Effective Participation Form for 1.3.1: Evaluation of Legal Representation of Children in Child Abuse and Neglect Proceedings (a) Case Data Collection (b) Judge (c) Guardian ad litem (d) Caseworker Form for 1.3.2: Evaluation of Interpreter Services Form for 1.3.3: Court Interpreter Terminology, Procedure, Protocol, and Ethics Fundamentals Test Form for 1.3.5: Access to Courthouse Facilities by Individuals With Disabilities Standard 1.4: Courtesy, Responsiveness, and Respect Form for 1.4.1 and 1.4.2: Questionnaire for Courteous and Responsive Treatment Form for 1.4.3: Recording Form for the Treatment of Litigants in Court Standard 1.5: Affordable Costs of Access Form for 1.5.1: A Checklist of Court Activities To Promote Affordable Access to Justice Form for 1.5.2: Access to Affordable Civil Legal Assistance Illustrative Data Collection Form Form for 1.5.3: Refer to Form for 5.1.3 Performance Area 2: Expedition and Timeliness Standard 2.1: Case Processing Form for 2.1.1: Generic Civil/Criminal Case Data Collection Form (a) Generic Civil Case Data Collection Form (b) Code Sheet--Civil Cases (c) Generic Criminal Case Data Collection Form (d) Code Sheet--Criminal Cases Form for 2.1.2: Ratio of Dispositions to Filings (Clearance Rate) Worksheet Form for 2.1.3: Display Tables--Age of Pending Caseload Form for 2.1.4: Civil Jury Trial Settings--Data Collection Form (a) Civil Jury Trial Settings--Data Collection Form (b) Civil Jury Trial Continuance Rate Worksheet Standard 2.2: Compliance With Schedules Form for 2.2.2: Provision of Services Forms (a) Provision of Services Data Collection Form (b) Checklist of Services Required in ABA Standards Form for 2.2.3: Information Request Data Collection Form Form for 2.2.4: Generic List of Court Activity Reporting (a) Generic List of Court Activity Reporting (b) Data Collection Form--Compliance With Reporting Schedules (c) Data Summary Report for Overall Court Compliance With Reporting Schedules Standard 2.3: Prompt Implementation of Law and Procedure No Forms Performance Area 3: Equality, Fairness, and Integrity Standard 3.1: Fair and Reliable Judicial Process Form for 3.1.2: Illustrative Questions for Measuring Court Employees' and Attorneys' Assessments of Fidelity to the Law Standard 3.2: Juries No Forms Standard 3.3: Court Decisions and Actions Form for 3.3.1: Illustrative Questionnaire Concerning the Practicing Bar's Views of the Court's Equality and Fairness Form for 3.3.2: Illustrative Questionnaire Concerning the Users' View of the Court's Equality and Fairness Form for 3.3.3: Illustrative Sentencing Data Collection Form Form for 3.3.4: Illustrative Bail Decision Data Collection Form Form for 3.3.5: Illustrative Data Collection Form for Outcomes of Criminal Appeals Standard 3.4: Clarity Form for 3.4.1: Illustrative Data Collection Form--Clarity of Judgment and Sentence Form for 3.4.2: Illustrative Data Collection Form--Clarity of Civil Judgments Form for 3.4.3: Illustrative Questionnaire Form--Experience in Interpreting Orders and Judgments Standard 3.5: Responsibility for Enforcement Form for 3.5.1: (a) Illustrative Data Elements for Measuring Enforcement of Probationary Orders (b) Illustrative Probationary Enforcement Data Collection Form Form for 3.5.2: Illustrative Data Elements for Measuring Enforcement of Child Support Orders/ Illustrative Child Support Enforcement Data Collection Form Standard 3.6: Production and Preservation of Records Form for 3.6.1: Illustrative Data Collection Form--The Reliability of the File System Form for 3.6.2: Illustrative Data Collection Form--Adequate Storage and Preservation of Physical Records Form for 3.6.3: Illustrative Data Collection Form--Accuracy, Consistency, and Utility of the Case Docket System Form for 3.6.4: Illustrative Data Collection Form--Case File Integrity Form for 3.6.5: Illustrative Data Collection Form--Reliability of Document Processing Form for 3.6.6: Illustrative Questionnaire-- Verbatim Records of Proceedings Performance Area 4: Independence and Accountability Standard 4.1: Independence and Comity Form for 4.1.1: Questionnaire Regarding the Independence of the Judiciary and Intergovernmental Relationships Standard 4.2: Accountability for Public Resources Form for 4.2.3: Auditing Practices Checklist and Performance Index Standard 4.3: Personnel Practices and Decisions Form for 4.3.1: Illustrative Position Groupings and Schedule Form for 4.3.2: Employee Survey on Personnel Practices and Employee Morale Form for 4.3.3: (a) Illustrative Data Collection Form for Personnel Information (b) Illustrative Summary Statistical Report on Race and Gender Mix among Employees Standard 4.4: Public Education Form for 4.4.1: Checklist for Court Policy Governing Response to Media Inquiries Form for 4.4.2: Illustrative Survey Form (a) For Media Representatives Regarding Court and Media Relations (b) For Court Employees Regarding Court and Media Relations Form for 4.4.3: Checklist of Potential Community Outreach Efforts (a) Organizational Efforts (b) Individual Efforts Standard 4.5: Response to Change No Forms Performance Area 5: Public Trust and Confidence Standard 5.1: Accessibility Form for 5.1.1: Court Employees' Perceptions of Court Performance Form for 5.1.3: Public Perceptions of Court Performance Standard 5.2: Expeditious, Fair, and Reliable Court Functions No Forms--refer to Forms for 5.1.1 and 5.1.3 Standard 5.3: Judicial Independence and Integrity No Forms--refer to Forms for 5.1.1 and 5.1.3 ------------------------------ Introduction This implementation manual provides trial courts with both the rationale and detailed instructions for implementing the Trial Court Performance Standards and Measurement System. This introduction provides an overview of the development and application of that system. It is intended to help courts translate the philosophy of the system into its practical application. History of the Trial Court Performance Standards and Measurement System The Trial Court Performance Standards and Measurement System expresses a new philosophy and framework for defining and understanding the effectiveness of trial courts by focusing attention on performance, self-assessment, and self- improvement. The 22 standards in the system establish goals for effective court performance in five areas: access to justice; expedition and timeliness; equality, fairness, and integrity; independence and accountability; and public trust and confidence. The measurement component consists of 68 field-tested measures for evaluating how well the court is meeting these performance standards. The Trial Court Performance Standards and Measurement System is an approach to self- assessment that courts can adapt to meet their individual needs; it is neither intended nor suited for comparing performance across courts. A hallmark of the system is its emphasis on the systematic assessment of a trial court's performance as a service organization and on the application of those findings to improve performance. This assessment applies to the court as a whole and does not include individual performance evaluations. The court is viewed as a system involving processes and tasks that are linked together and affect one another. The collective work of the court involves not only judges, but all who perform administrative court functions, including clerks of court, administrators, probation officers, and other court staff, as well as private lawyers, public defenders, prosecutors, and social service providers. The Trial Court Performance and Standards and Measurement System is the major product of the Trial Court Performance Standards Project. The National Center for State Courts (NCSC) and the Bureau of Justice Assistance (BJA), U.S. Department of Justice, initiated the project in August 1987, to develop measurable performance standards for State trial courts. The impetus for the enterprise was the recognition of the need for State trial courts to increase their capacity to provide fair and efficient adjudication and disposition of cases. To carry out the mission of the Trial Court Performance Standards Project, the NCSC established the Commission on Trial Court Performance Standards (Commission). Composed of trial judges, court managers, and scholars, the Commission formulated, deliberated, and generated the measurement system with assistance from the project staff. The initial work of the Commission progressed over 3 years, culminating in 1990 with the publication of Trial Court Performance Standards With Commentary, which has been endorsed by the Conference of Chief Justices, the Conference of State Court Administrators, and the National Association for Court Management, and adapted by the National College of Probate Judges. During the ensuing 4 years, trial courts in Ohio, New Jersey, Virginia, and Washington applied the standards and tested the utility and feasibility of the system's measures. BJA continued funding to NCSC to coordinate and assist the demonstration phase of the Standards Project, while the State Justice Institute supported the demonstrations in the four States. Developing the System's Measurement Component The Trial Court Performance Standards and Measurement System provides the tools for assessing the extent to which a court meets the performance criteria set forth in the 22 performance standards. These tools consist of procedures for systematically gathering and analyzing quantitative and qualitative data and for drawing conclusions from the data to identify areas in need of attention or improvement. Field Testing. As the measurement system evolved, 75 measures were developed, tested, and refined by the Standards Commission and Standards Project staff. Trial courts in Arizona, Michigan, and Ohio contributed to this process by serving as test sites for the draft measures. Following the research and development phase, a 4-year demonstration phase commenced in Ohio, New Jersey, Virginia, and Washington. The demonstration phase of the Standards Project has been crucial to the widespread acceptance and use of the system. Twelve trial courts in 4 States participated in the demonstration, and each of the 75 measures was tested in at least one court.[1] Most measures were tested in two or more of the courts. The 12 trial courts varied on a variety of factors including size, organization, jurisdiction, funding source, demographic and economic context, and, of course, State law and court rules. This variation across the courts provided the opportunity to test the measures under diverse conditions and produced a rich body of information relevant to the application of the measurement system in other trial courts throughout the country. As the demonstration proceeded, the Standards Commission and Standards Project staff reviewed and revised the measures to reflect the experiences of these trial courts in implementing the measurement system. Along the way, the original 75 measures were refined to a set of 68. The outcome of this comprehensive and cooperative undertaking is the system presented in this manual. Using the Measurement System Purposes of Measurement. The Trial Court Performance Standards and Measurement System defines a philosophy that encourages trial courts to conduct regular self-assessments and improvements, treating them as routine court administrative activities. To this end, the system's measurement component is designed to gather information that the court can use in a variety of ways, including budgeting, case management, implementing court improvement projects, and strategic planning. The initial application of the measures aids the court in identifying areas requiring attention or potentially in need of improvement. The measures also may be used to establish benchmarks with regard to court performance on each standard the court wishes to address. Subsequently, the court can use the measures to determine whether its performance with respect to a particular standard is better, about the same, or worse than when the measures were originally applied. The information gathered through the measures also is helpful in determining whether the court's prior improvement efforts have been successful or need to be altered in some way. Nature of the Measures. Some measures and their specific methods build on others and should be conducted in a particular sequence. For example, in Standard 2.1: Case Processing, Measures 2.1.1, Time to Disposition, and 2.1.2, Ratio of Case Dispositions to Case Filings, examine case processing times and case clearance rates. If these measures indicate that average case processing times exceed State or local standards or that clearance rates are not keeping pace with the incoming caseload, the court should proceed to Measure 2.1.3, Age of Pending Caseload, to determine whether a case backlog exists and, if so, to ascertain its nature and extent. Other measures and methods stand alone and can be applied independently. Furthermore, some measures, such as Measure 1.1.3, Audibility of Participants During Open Court Proceedings, are relatively easy to apply while others, such as Measure 3.3.3, Equality and Fairness in Sentencing, are more complex and time consuming. Measurement Methods. The measurement system employs numerous data-gathering methods and taps diverse data sources. The data sources and collection methods used include both familiar processes, such as court and case record reviews and tallies of case filings and dispositions, as well as other social science techniques used less commonly by courts, such as systematic observations, structured interviews, surveys of various reference groups, simulations, group techniques, and public opinion polls. Just as the measurement techniques vary, different types of evaluators are employed depending on the object of the measure. For example, volunteers conduct structured observations of court proceedings and simulations of public access to information, while court staff conduct many of the measures involving record reviews. A few measures are best carried out by consultants or court staff with expertise in areas such as data analysis. Following this introduction, two tables provide information designed to help the reader understand the measurement system. Table 1: Summary of Measures lists the specific measures associated with each standard, the primary data collection method (how the measure is applied), the primary evaluators (who should apply the measure), and the source of data (the subject of the measurement). The 3-digit number identifying each measure in Table 1 provides a key to the measure's place in the measurement system. The first digit denotes the performance area, the second denotes the standard within the area, and the third refers to the measure associated with the standard. Table 2: Summary of Measures by Primary Data Collection Method provides a different perspective of the measurement system. It displays the various measurement methods and lists the individual measures that employ each method. The reader can determine from Table 2 if two or more measures the court intends to implement use the same methods and sources of data. For example, Measure 1.2.3, Perceptions of Courthouse Security, and Measure 1.2.6, Evaluation of Accessibility and Convenience by Court Users, both survey court employees. Consequently, the court can economize by distributing one questionnaire to court employees to accomplish the data collection for both measures. The measurement methods most commonly recommended in the system are summarized below. They are described more fully in this guide in the instructions for conducting the individual measures. Court Record Reviews and Case Data Examination. Because a primary function of courts has been to make and preserve records of civil and criminal matters as well as court operations, court and case record reviews are the most traditional and familiar of the measurement methods. Thirty-two of the measures entail court case and record reviews. These reviews require staff to consult case files, docket sheets, case summary screens in automated systems, and administrative reports. Many of the record review measures are very time consuming, even for courts with automated systems (e.g., Measure 2.1.1, Time to Disposition), but only a few of the measures require knowledge of advanced analysis techniques (e.g., Measure 3.3.3, Equality and Fairness in Sentencing). Because these reviews provide primarily quantitative information, they are more objective in evaluating the court's performance than are surveys and interviews, which usually report the perceptions of the respondents. The results provide insight into areas such as caseflow and case file management practices, compliance with procedural reporting requirements, and timeliness in implementing changes in laws and procedures. Observations and Simulations. The measurement system incorporates several measures that involve observations of court proceedings or simulations of court activities and interactions with court staff. Court personnel can perform a few of these measures, such as rating the audibility of court proceedings (Measure 1.1.3). Many of the measures, however, attempt to simulate the experiences of people who have business in the court only occasionally. These measures require volunteer observers who are unfamiliar with the court system, court procedures, or courthouse facilities. Examples of the activities performed by volunteers include gaining entrance to court proceedings that should be open to the public (Measure 1.1.1), obtaining information about the status of scheduled proceedings (Measure 1.1.2), requesting information about the time and location of a court proceeding by telephone (Measure 1.2.5), and checking the accessibility of court facilities and services for persons with disabilities (Measure 1.3.5). Surveys and Questionnaires. Eighteen measures incorporate the use of surveys. The surveys seek a variety of information from different court constituencies including employees, attorneys, jurors, and the general public. Some of the information gathered is factual, such as demographic information about jurors (Measure 3.2.3). Most of the surveys, however, are designed to gauge opinions on topics such as the accessibility and convenience of court facilities and services (Measures 1.2.6 and 1.2.7) or the fairness and equality of court proceedings and actions (Measures 3.3.1 and 3.3.2). Many of the surveys seek basic demographic information from survey respondents, such as gender, age, and relationship to the court. For some measures, this information is critical in comparing the attitudes of different groups of court users responding to the survey. However, in some instances, requesting demographic data may result in a lower response rate if respondents believe their anonymity is threatened. This is particularly true in small jurisdictions. Consequently, the court needs to weigh the value of obtaining the information against the possibility of losing some respondents. Several of the surveys developed for the measurement system were adapted from instruments used by other organizations for similar assessment purposes. For instance, Measure 1.2.3 uses a survey for assessing courthouse security adapted from the National Crime Survey--Attitude Questionnaire and from the National Crime Survey--Basic Screening Questionnaire. Another example is Measure 5.1.3's survey for gauging public trust and confidence in the courts, which was drawn from other surveys of the public's perceptions of the justice system.[2] Interviews. In addition to surveys, the measurement system employs interviews to gather information and opinions from court staff and court users. In some measures, the surveys and interviews are offered as alternative approaches while in others the two are used in tandem, such as a survey followed by a focus group. Interviews are used as a primary method of data collection when a measure calls for more detailed responses than a written survey might yield, such as in assessing employees' familiarity with emergency procedures (Measure 1.2.4). More typically, interviews are used to collect background information when preparing to conduct a measure (e.g., interviewing the court records manager about case file storage procedures) or to gather followup or clarifying information after collecting data in a different format (e.g., after a records review or survey). Interviews with court employees may be the most efficient way to gather information when court policies are governed less by written documents than by unwritten practices and rules (e.g., Measure 4.4.2). Group Techniques. Group techniques are used in five measures. These techniques include review panels composed of knowledgeable practitioners (Measure 3.1.1, Performance in Selected Areas of Law) and more structured interactions that require a facilitator to guide the group through the activity. Examples of techniques in the latter group are Nominal Group Technique (NGT) and Ideawriting. NGT is used to generate and select among ideas and to make decisions. Ideawriting is a method for developing ideas and works well for groups that communicate well in writing. These techniques are used primarily in measures for Performance Area 4, such as Measure 4.2.1, Adequacy of Statistical Reporting Categories for Resource Allocation. Although the techniques are not difficult to implement, readers planning to use them should consult a text on group techniques such as Group Techniques for Idea Building by Carl Moore (volume 9 of the Applied Social Science Research Series from Sage Publications; see Appendix A for more information). Organization of This Implementation Manual The organization of this manual follows that of the measurement system. The measurement approaches associated with each standard in the five performance areas are described in separate sections. The titles, text, and commentary of the standards are followed by a brief overview of the measures, methods, and techniques associated with them. The overviews are designed to assist the reader in understanding the general approach and requirements for the measures without studying the detailed prescriptions for applying them. Specific measures and data collection forms follow each standard and measurement overview. Applying the Measures. The description of each measure has four parts. First, an introductory section explains the measure's purpose and how it relates to its associated standard. Next, a planning/preparation section details any preparatory work that is necessary to apply the measure. For example, a measure involving a survey identifies the individuals or groups who should be included in the survey sample; a record review measure designates which case files are to be examined. This section also indicates whether certain individuals in the court should be consulted before conducting a measure or if the services of an expert are recommended to assist court staff in applying a measure. A data collection section then outlines the particular steps necessary for actually gathering the data for the measure. For instance, a survey measure details how the surveys should be distributed, and a record review measure includes a description of the data elements that will be collected from court records. A final section on data analysis and report preparation describes how the gathered data should be analyzed and often recommends how the results can be presented to court officials, others who work in the court, or other relevant audiences. In some measures, this section indicates the optimum level of performance, while for other measures the level of satisfactory performance is left to the court to determine. Modifying the Measures. Although at first it may appear that the more complex measures cannot be implemented without some simplification, these measures should be modified only after careful deliberation. For example, an item of information should not be eliminated simply because it is hard to obtain. Instead, the court should consider how to overcome the perceived difficulties in obtaining the information, as well as the consequences of not including the information in the measurement process. On the other hand, strict adherence to every element of a measure might stifle the development of innovative approaches. Therefore, thoughtfully conceived modifications may be undertaken as long as efforts are made to balance a measure's feasibility and utility with its scientific merit. Planning to Use the Trial Court Performance Standards and Measurement System The manual that follows provides both the rationale and detailed instructions for conducting each measure in the measurement system. Before a court undertakes the measurement process, however, those who will conduct the evaluation are likely to have many questions about where and how to begin, how to proceed most efficiently, and where the measurement process might lead. Unlike the explicit directions presented in the measurement system, answers to important questions such as these generally cannot be prescribed for individual courts. Each court must identify its particular needs, set its own performance goals, and determine how it can best apply the Trial Court Performance Standards and Measurement System to both guide the evaluation process and achieve the improved performance it seeks. For guidance on these implementation issues, the reader is strongly urged to consult a companion publication, the Planning Guide for Using the Trial Performance Standards and Measurement System. (To order this publication, contact the BJA Clearinghouse at 1-800-688-4252.) The Planning Guide is based on the experiences of the 12 demonstration courts that tested the system and reflects the lessons they learned in the undertaking. Intending to serve as a conceptual bridge from the Trial Court Performance Standards to the measurement system, the planning guide presents an implementation model to help courts translate the application. The guide provides direction for using the system as a planning, evaluation and monitoring tool and addresses many of the questions and issues courts are likely to encounter when embarking on the process of self- evaluation and self-improvement. Endnotes: [1] In New Jersey, the five demonstration courts were the Superior Courts of Atlantic County, Burlington County, Morris County, Ocean County, and Somerset County. In Ohio, the three demonstration courts were the Common Pleas Courts of Meigs County, Stark County, and Wayne County. The demonstration court in Virginia was the Fairfax County Circuit Court. In Washington, the three demonstration courts were the Superior Courts of Spokane County, Thurston County, and Whatcom County. [2] See, for example, Citizen's Commission to Improve Michigan Courts, Final Report and Recommendations to Improve the Efficiency and Responsiveness of Michigan Courts (Lansing, MI: Michigan Supreme Court, 1986). See also Yankelovich, Skelly, and White, Inc., The Public Image of Courts: Highlights of a National Survey of the General Public, Judges, Lawyers, and Community Leaders (Williamsburg, VA: National Center for State Courts, 1978). ------------------------------ Performance Area 1: Access to Justice Trial courts should be open and accessible. Location, physical structure, procedures, and the responsiveness of personnel affect accessibility. Accordingly, the five standards grouped under Access to Justice require a trial court to eliminate unnecessary barriers to its services. Such barriers can be geographic, economic, and procedural. They can be caused by deficiencies in both language and knowledge of individuals participating in court proceedings. Additionally, psychological barriers can be created by mysterious, remote, unduly complicated, and intimidating court procedures. Overview of Standards. The intent of the first two standards is to bring the administration of justice into the open and to make it accessible. Standard 1.1 requires the trial court to conduct its business openly. To ensure that all persons with legitimate business before the court have access to its proceedings, Standard 1.2 requires the trial court to make its facilities safe, accessible, and convenient to use. Accessibility is required not only for those who are guided by an attorney but also for all litigants, jurors, victims, witnesses, and relatives of litigants. Access to trial courts is also required for many other individuals--for example, beneficiaries of decedents in probate matters, parents and guardians in juvenile cases, persons seeking information from public records held by the court, employees of agencies that regularly do business with the courts (e.g., investigators, mental health professionals, sheriff's deputies, and marshals), and the public. Because a trial court may be accessible to most and still hinder access to some, Standard 1.3 requires the court to provide opportunities for the effective participation of all who appear before the court, including persons with linguistic difficulties and handicaps. To promote access to justice and to enhance citizen confidence and trust in the court, Standard 1.4 urges that all court personnel accord respect, courtesy, and dignity to all with whom they come into contact. Standard 1.5 recognizes that there are financial and procedural barriers to access to justice. It requires that the fees imposed and procedures established by the court be fair and reasonable. Recognizing the importance of the relationship between public records and access to justice, the standard also requires that public records be preserved and made available at a reasonable cost. Overview of Measures. Twenty-one measures are associated with the five Access to Justice standards. Taken together these measures provide both breadth and depth of measurement of a court's performance in offering the public access to justice. Data obtained from the measure of one standard are often relevant to assessment of performance for another standard. This is especially true of Standards 1.3 and 1.5. Standard 1.3 requires that all who appear before the court be able to participate effectively, and Standard 1.5 calls for affordable costs of access to court proceedings. Effective representation by counsel is an important implicit factor for effective participation in court proceedings. Unfortunately, the cost of legal services makes access to justice impossible for many people. Thus, measurement for Standard 1.5 requires the collection of data that are relevant for this implicit requirement of Standard 1.3. The measures in this performance area rely on a variety of data collection methods: surveys, observations (in some measures combined with simulation), interviews, and reviews of court records and documents. Three measures call for administering surveys to individuals who are "regular users of the courthouse." The information sought from these people relates to safety and security, ease of doing business, and the courtesy and respect they experience in the courthouse. Although each survey measure is described separately in relation to a particular standard, it may be easier and less time consuming to combine the questions for each measure into one questionnaire and survey regular users once rather than three times. The method described most often for measuring access to justice is observation (sometimes combined with simulation). Observers systematically record what they see and hear. This structured information can then be examined quantitatively as well as qualitatively. These "see, hear, and record" measures range from concrete and objective (Was an observer able to gain entrance to a courtroom?) to subjective (Did activity taking place in a courtroom detract from the dignity of the proceedings?). There are 12 measures of this type. Although the observations could be carried out by almost anyone, the recommended approach is to use citizen volunteers who are relatively naive to the legal system and who are unfamiliar with the facilities and "customs" of the courthouse. This results in records of experiences that resemble those of ordinary citizens who have infrequent occasion to do business with the court. Furthermore, the observers chosen should optimally be representative of the jurisdictional community of the court. Representativeness is more important for some measures than others. However, because the same individuals could be asked to obtain data for all the observation measures, it may be helpful to recruit one pool of observers who vary on demographic factors. Observers may be recruited by contacting volunteer organizations, universities, senior citizen groups, and so forth. This "volunteer observer" method has other advantages, notably its relatively low cost. The court must invest staff time to recruit volunteers, orient them to their assignments, and evaluate results. Once the recruitment and orientation are completed, however, the observers may be used to collect data for many measures described throughout the measurement process. Because the observers are relatively few in number, they offer the added advantage of being able to provide court staff with additional information during interviews following their structured assignments. A much richer, qualitative analysis results when explanations, descriptions, and suggestions can be elicited from the observers to augment what is provided on written forms, questionnaires, and checklists. Two other measurement methods rely on data collected through interviews and examination of court records and written policy documents. Some of the measures of this type focus on case data. Measure 1.3.1, regarding effective legal representation of children in child abuse and neglect proceedings, is of this type. In this measure, court case records are examined and those involved in the cases are surveyed and interviewed to document how the guardian ad litem process actually has worked for several selected cases. Other measures focus on administrative documents. For example, Measure 1.5.1 relies on an examination of forms, brochures, and written policies to evaluate court efforts to facilitate affordable access alternatives for individuals with low incomes. Interviews with court staff also are conducted to identify and locate the relevant documents. Finally, measures addressing the issues of court security (Measure 1.2.1) and interpreter services (Measure 1.3.2) rely on evaluation by outside experts in the respective areas. Standard 1.1: Public Proceedings The trial court conducts its proceedings and other public business openly. Commentary. This standard requires the trial court to conduct all proceedings openly, contested or uncontested, that are public by law or custom. The court must specify proceedings to which the public is denied access and ensure that the restriction is in accordance with the law and reasonable public expectations. Further, the court must ensure that its proceedings are accessible and audible to all participants, including litigants, attorneys, court personnel, and other persons in the courtroom. Measurement Overview. The three measures for Standard 1.1 determine the degree to which a court openly conducts its business. The measures assume that a trial court meets Standard 1.1 if it (1) provides public access to its courtrooms, (2) ensures that information regarding the status of court proceedings is obtainable, and (3) ensures that judges and other court participants can be heard in open proceedings. All three measures rely on direct observations. The measures require court staff to compile some basic calendaring information. Once this information is available, each of the measures can be completed within a few days. Each of the measures can be accomplished separately, but it would be more efficient to conduct them simultaneously. Although almost anyone can serve as observers for these measures, as noted in the overview of measures for Access to Justice beginning on page 1, it is recommended that individuals who are unfamiliar with the court be recruited. The same individuals also may be used for obtaining observation data for measures related to other standards of access, particularly measures of the convenience of access, perceptions of safety, courtesy, and responsiveness of court personnel. Measure 1.1.1: Access to Open Hearings This measure verifies that the public has access to court proceedings that should be open to the public. The coordinator for the measure provides volunteer observers a list of scheduled court hearings and asks the observers to verify whether they can enter the courtroom in which the hearings take place. Planning/Preparation. Preparation for this measure involves identifying at least 30 court proceedings[1] for the volunteer observers to attend. The first step is to select several days during which the observations will take place. The number of days selected will depend on: o The court's daily volume of proceedings. If few proceedings are held each day, the observations will have to be conducted over many days or weeks. o The variety of proceedings conducted each day. If certain matters are heard only on certain days (e.g., all or most civil and criminal motions are heard only on Mondays), then several days will be needed to observe a cross-section of proceedings. o The number of volunteer observers available to conduct the measure. If a large number of observers are available, data could be collected across many days without asking observers to visit the courthouse repeatedly. Alternatively, if observers must collect data on a number of proceedings, it will be more convenient to do so on 1 or 2 days than to have them traveling to the courthouse across many days. o The observers' schedules. The court may have to collect data across several days (or in just a few days) in order to accommodate the various schedules of the observers. The measure provides an example in which five volunteers observe two proceedings each across 3 days. As noted above, the data collection process can be modified to accommodate a court's particular caseload and volunteers' schedules. Select more or fewer days as necessary. To select the 3 days, first ask court employees involved in scheduling court proceedings whether certain matters are heard only on certain days. If, for example, most short matters are heard only on Mondays, be sure to include at least one Monday in the sample.[2] The selected days should include a cross-section of the types of proceedings the court hears. If the court hears the same types of matters each day, randomly select 3 days. Next, review the list of proceedings scheduled for each day for nonpublic proceedings. Eliminate any matters specifically noted as closed to the public. (Eliminated proceedings may be examined in connection with Standard 3.1, Measure 3.1.1, to determine whether the court's practices for closing hearings are in compliance with Federal and State case law and applicable statutes.) Randomly select 10 proceedings scheduled for each day.[3] Because some proceedings (such as trials) may be canceled before their scheduled start times, it is advisable also to select several additional proceedings as backup. On the morning of the planned observation, give each of the five volunteers two proceedings to attend. Make sure that the two proceedings are not scheduled to take place at the same time in different courtrooms. Data Collection. An observer goes to each scheduled hearing at the designated location and time. For each event, the observer records (see Form 1.1.1, Record of Access to Courtroom) whether he or she was successful in gaining access to the proceeding. If the observer is excluded from any of the scheduled proceedings, he or she should talk with court officials and record the reasons for exclusion. If some of the proceedings with individually scheduled start times (such as trials) are canceled before the scheduled start time, additional proceedings should be chosen to replace them. Canceled proceedings that are part of a court session including many short matters do not need to be replaced. As long as the observer gains access to the courtroom where the matter was scheduled to be heard, the observer can record that the proceeding was accessible. Data Analysis and Report Preparation. Analyzing the data involves a two-step process. If all of the court proceedings were open to the public, the court is performing well on this measure and there is no need to undertake the second step of analysis. If, on the other hand, some of the court proceedings were closed, court officials should examine the legitimacy of the explanations that were given for closing the proceedings. Were the proceedings closed according to the standards enumerated by the Supreme Court in Press-Enterprise Co. v. Superior Court?[4] These standards include: o There is an overriding interest that would be prejudiced by open proceedings. o The closure order is no broader than necessary to protect that interest. o Reasonable alternatives to closure have been considered. o The trial court needs findings on the record adequate to support closure. The standards enumerated for closing a pretrial hearing in criminal cases are:[5] o There is substantial probability that the defendant's right to a fair trial will be prejudiced by publicity. o No reasonable alternatives to closure could protect the defendant's fair trial rights. If any of the proceedings were closed for reasons other than these, the court is not performing optimally on this measure. If proceedings were closed for illegitimate reasons, court officials should take steps to ensure that, in the future, the Supreme Court's standards for closing proceedings are followed. Measure 1.1.2: Tracking Court Proceedings This measure is a logical extension of Measure 1.1.1. If an observer has physical access to a courtroom but cannot identify which proceeding is underway, public access is compromised. The measure examines whether an observer can obtain information about the status of specific court proceedings on the court's calendar. Planning/Preparation. This measure can use the same sample of court proceedings that was drawn for Measure 1.1.1, Access to Open Hearings. The method for selecting the sample of court proceedings is described in the planning/preparation section of Measure 1.1.1. Data Collection. After following the data collection procedure described for Measure 1.1.1, the observer tries to determine when a specific court event will be heard. For each court event, the observer records (see Form 1.1.2, Tracking Court Proceedings) how he or she determined the status of the event (e.g., saw it take place or asked a court official) and any difficulties encountered during the process. Data Analysis and Report Preparation. If the observers were able to identify the status of each scheduled court event, the court is performing well on this measure. If the observers were unable to determine the status of one or more court events, court officials should review the types of court events that could not be tracked. Are there any patterns in the data? For example, did most of the problems occur with court events that did not have specific start times? In order to improve the court's performance on this measure, court officials should examine observers' reports of difficulties they encountered during the data collection process and suggestions for improving the dissemination of information regarding the status of specific proceedings (e.g., provide periodic reviews of the calendar or post the calendar in the courtroom and update it as matters are heard or rescheduled). Measure 1.1.3: Audibility of Participants During Open Court Proceedings This is a measure of the audibility of proceedings when court is in session. Observers collect qualitative data regarding the audibility of judges, attorneys, litigants, witnesses, and other court participants during court proceedings. Planning/Preparation. The first step in applying this measure is selecting a sample of court proceedings. A subset of the sample of court proceedings drawn for Measure 1.1.1, Access to Open Hearings, may be used for this measure. The method for selecting the sample for Measure 1.1.1 is described in the planning/preparation section of that measure. From this sample, at lease five court proceedings are selected to be observed for audibility.[6] The sample should be stratified by courtroom to ensure that proceedings are observed in several different courtrooms. (Smaller courts may prefer to test all of their courtrooms.) The second step is the identification and recruitment of observers with normal hearing. The observers may be volunteers or court employees. Data Collection. An observer with normal hearing attends each of the selected court proceedings and sits for approximately 5 minutes on each side of the courtroom's public seating. The time spent in the courtroom may need to be extended if the observer only had the opportunity to hear one or two of the court participants speak. After observing each court proceeding, the observer should answer the questions on Form 1.1.3, Courtroom Audibility Evaluation Form, and record any specific acoustic and human speech factors that seemed to affect audibility in the courtroom. This qualitative information will help court officials identify factors that may be contributing to poor audibility in the courtroom. Data Analysis and Report Preparation. A report should be prepared that compiles and synthesizes the results of the qualitative evaluations by the observers in the various courtrooms. If audibility is a problem, the report should address whether the problem is common across courtrooms and types of proceedings or generally limited to one courtroom or one type of proceeding. The report should be disseminated to the court administrator and other appropriate court officials. If a problem exists across all courtrooms, court officials should consider contacting a sound engineer for suggestions. Other problems may be alleviated by making minor changes in a courtroom's environment or by developing and enforcing administrative rules related to courtroom audibility. Standard 1.2: Safety, Accessibility, and Convenience Trial court facilities are safe, accessible, and convenient to use. Commentary. Standard 1.2 considers three distinct aspects of court performance: the security of persons and property within the courthouse and its facilities, access to the courthouse and its facilities, and the reasonable convenience and accommodation of those unfamiliar with court facilities and proceedings. It urges a trial court to be concerned about matters such as the centrality of its location in the community that it serves, adequate parking, the availability of public transportation, the degree to which the design of the court provides a secure setting, and the internal layout of court buildings (e.g., the signs that guide visitors to key locations). Because the attitudes and behavior of trial court personnel can make (or fail to make) the courthouse safer, more accessible, and more convenient to use, Standard 1.2 pertains to the conduct of trial court personnel as well. Unusual or unexpected conditions, such as bomb threats, records destruction, employee strikes, sting operations, mass arrests, and natural disasters, challenge the routine operations of the court. Mechanisms (both internal and operated in coordination with other justice system agencies) may be required to handle emergent situations that could impede the courts and disrupt daily routines. Measurement Overview. Measurement of performance for Standard 1.2 addresses three components: safety, accessibility, and convenience. The seven measures for Standard 1.2 utilize a variety of methods, including: (1) a formal audit of courthouse security measures carried out by an expert, (2) simulations by law enforcement personnel evaluating courthouse security, (3) facts and opinions collected from observers who role-play the occasional courthouse visitor, and (4) surveys of regular users of the courthouse and court employees. Courthouse security is defined as "the feeling of safety combined with the measures taken to provide that feeling of safety--against personal injury, property damage, and the loss of records housed in the courthouse." [7] Four measures examine both of these aspects of courthouse security. Measure 1.2.1 examines the physical security of the courthouse with a formal audit of security measures. Measure 1.2.2 requires that trained law enforcement officers conduct a test of courthouse security by observing and trying to breach the court's security measures. Measure 1.2.3 uses a survey to assess the general sense of safety perceived by regular users of the court. Measure 1.2.4 examines the training courthouse employees have received in responding to emergency situations. Accessibility and convenience are addressed together in Measures 1.2.5, 1.2.6, and 1.2.7, reflecting the close connection between the concepts. Measure 1.2.5, relating to access to information by telephone, and Measure 1.2.7, relating to accessibility and convenience of court facilities, rely on observers who simulate business transactions in the court. Measure 1.2.6 uses a survey method to obtain opinions of regular users of the courthouse about accessibility and convenience. The measures that utilize observers are relatively inexpensive compared to those using surveys. Through exit interviews, the "observation" measures also allow the court to gather more detailed information about the observers' experiences than would be possible through a mailed survey. In addition, as a result of their experiences, the observers may be able to provide the court with suggestions for improving the court's services to the public. As a consequence, the observers could serve as emissaries between the court and the community and provide an ongoing source of information and support for the court. Measure 1.2.1: Courthouse Security Audit "The general goal of a comprehensive court security policy should be to establish appropriate protection for court staff and facilities, the general public, and the judicial process as a whole." [8] Measure 1.2.1 considers the court's performance in taking precautions to reduce or eliminate threats to the public's safety in the courthouse. This measure addresses "the degree to which design features of the court provide a secure setting," mentioned in the commentary for Standard 1.2. Planning/Preparation. An expert in court security features should be retained to help conduct the security audit. The National Sheriffs' Association can help identify available consultants. In some jurisdictions, appropriate expertise may be available from the local sheriff's department or the U.S. Marshals Service. The security consultant and security officers from the court should be provided Form 1.2.1, National Sheriffs' Association Physical Security Checklist, as a resource for drafting an audit to fit the court's building(s) and grounds. For example, some audit items will vary depending on whether the court is located within a multipurpose government building or has its own facility. While developing the checklist, the consultant and officers also should consider what would constitute a positive response for each question. That is, in some cases, a "no" response on the National Sheriffs' Association Checklist may be positive (see, for example, question 10 under "Parking Areas.") Data analysis will be simpler if responses are consistent across items, i.e., all "yes" responses are positive. (See the section below on data analysis and report preparation.) Data Collection. The security consultant conducts an in-person security audit, using the modified security checklist described earlier. The court's security officers should assist the consultant in obtaining any information he or she needs in order to conduct the audit. Data Analysis and Report Preparation. Simple descriptive statistics are used to analyze the results of the security audit. The number of positive responses are summed and divided by the number of total responses possible on the court's version of the security checklist. (It is important to note that if some "no" responses are positive, the total number of positive responses cannot be obtained by adding only the responses in the "yes" column.) The court's performance on this measure increases as the percentage of positive responses on the checklist increases. If the security audit indicates problems, court security officials can examine the percentage of positive responses in each of the major areas of security (e.g., parking areas, courtrooms, elevators) to determine where added precautions may be necessary. Measure 1.2.2: Law Enforcement Officer Test of Courthouse Security This measure determines the adequacy of the court's security in protecting both the public and confidential court records. The measure should be conducted as a follow-up to Measure 1.2.1. Data are gathered by law enforcement experts through simulation exercises. The measure requires the cooperation of local law enforcement officials. Planning/Preparation. Local law enforcement officials should be contacted and asked to help court officials conduct a security audit of the courthouse. Law enforcement officials should be informed that the security audit will involve simulations in which one or two officers, dressed in plain clothes, will attempt to breach the court's security system. The officers who conduct the simulations should not be well known to court personnel. Court staff, in consultation with the security expert retained for Measure 1.2.1, should develop simulations to target security areas in which potential weaknesses (e.g., safety of parking areas, ease of gaining access to confidential files, or access to courtrooms and chambers during and after normal business hours) were identified during the security audit conducted for Measure 1.2.1. On a cautionary note, simulations should not be developed that place individuals in potentially dangerous situations (e.g., simulations that involve carrying a concealed weapon). Data Collection. The officers should visit the courthouse in plain clothes. Only the court manager and judge should be aware of the officers' presence in the courthouse. The officers should "wander" through the courthouse conducting the simulations developed by court staff and note any security problems encountered. If, after conducting the simulation exercise, the officers have any questions or need additional information on specific aspects of court security, they should conduct follow-up interviews with relevant court personnel. Court officials should ensure that the officers are introduced to the appropriate staff and should encourage staff to answer the officers' questions as accurately and thoroughly as possible. Data Analysis and Report Preparation. Once the officers have completed the simulations, they should prepare a report on the overall security status of the courthouse. The report should answer questions such as: Was the court's security system successful in protecting the public and in protecting confidential court files and records? Did the officers notice any specific security problems that the court should address? What recommendations do they have for improving court security? Measure 1.2.3: Perceptions of Courthouse Security The extent to which the courthouse is perceived as a safe environment is measured through the administration of a questionnaire to regular users of the court (e.g., court employees, attorneys, probation officers, and jurors). The measure requires the assistance of someone skilled in survey research methods. Planning/Preparation. Measure 1.2.6, Evaluation of Accessibility and Convenience by Court Users, and Measure 1.4.1, Court Users' Assessment of Court Personnel's Courtesy and Responsiveness, also involve surveying regular users of the court. If these measures are being applied in the court, coordinators for these measures may find it efficient to combine the three measures into a single survey instrument. The first step in applying the measure is to review Form 1.2.3, Survey on Courthouse Security. The survey form consists of three sections. The first section seeks to gauge the respondents' sense of threat to their person and property while visiting the courthouse. The second section asks about actual victimizations experienced in the courthouse. The third section seeks general background information such as gender, age, and relationship to the court. The questions are adapted from the National Crime Survey--Attitude Questionnaire and the National Crime Survey--Basic Screening Questionnaire.[9] The survey form can be used as is or modified to better fit the specific characteristics of the court. For example, smaller jurisdictions conducting this measure may want to eliminate some or all of the background information questions included in the third section of the survey. Small jurisdictions may have fewer respondents and thus responses to demographic questions could essentially reveal the identity of individual respondents. Courts in this situation will have to weigh the benefits of including the demographic information to allow for more detailed analysis against a possible low response rate because anonymity cannot be guaranteed. Data Collection. The survey form is administered to four groups of individuals who use the court on a regular basis: court employees, attorneys, probation officers,[10] and jurors. To ensure that each group is represented in the sample of survey recipients, stratified sampling should be used. A sample of at least 80 individuals[11] should be drawn for a total sample of 320 individuals.[12] Court employees should be selected from a list of employees maintained by the court's personnel office. Probation officers should be selected from personnel lists maintained by the probation department. Attorneys should be selected according to the procedure described in Measure 3.3.1. A list of individuals who served as jurors for the court during the previous 18 months should be prepared and used for obtaining the juror sample. A questionnaire is sent to each person in all four groups. For best results, a stamped envelope with the administrator's name and address on it should be included with each questionnaire. As a means to track which surveys have been returned while still preserving the confidentiality of the respondents, each questionnaire should be accompanied by an index card with a code number that corresponds to a master list of the survey recipients. The code number should not be included on the questionnaire itself. Recipients should be instructed to return the card with their completed survey or, if they prefer, in a separate envelope. They should also be informed that the code-numbered cards will not be used for identification of the surveys, but only to determine which surveys are still outstanding and thus require a "reminder" note. The card should be destroyed once the return of the survey is recorded on the master list. To increase the response rate, those who have not returned their surveys should be sent a reminder notice after 10 days. Data Analysis and Report Preparation. The responses for each item of the questionnaire are associated with a number code. For example, "male" is coded as 1 and "female" is coded as 2. The responses for each questionnaire are recorded using these number codes. These codes subsequently are entered into a computer file and tabulated using a statistical software program. In general, court performance depends on two factors: the number of courthouse areas rated safe by a majority of respondents and the number of crime incidents reported by respondents. As perceived safety increases and reported crime decreases, court performance on this measure improves. Two sets of analyses are conducted. The first set examines the frequency (or percentage) of responses for each category of each item. For example, an analysis of question 1 will indicate the number of survey respondents who thought crime in the courthouse had increased, the number who thought crime had decreased, and the number who thought crime had remained about the same. If the majority of respondents thought that crime had increased in the courthouse, then the court is not performing well on this measure. The second set of analyses compares the responses on one item with the responses on other items through the use of a cross-tabulation procedure and the gamma coefficient. These analyses will help explain some of the percentages derived from the first set of analyses. For example, of those respondents who thought that crime had increased, how many had actually been victimized in the courthouse? (See Part II of the survey form.) The result of this analysis will clarify whether a response of "increased crime" is based on actual incidents of crime. The needs of the court should dictate the dissemination and utilization of the results of this measure. Results will provide useful feedback to the Trial Court Administrator and Supervisor of Courthouse Security. If actual incidents of courthouse crime were reported, the court will need to examine its security more closely. If there is a perception that crime has increased but no incidents were reported, the court may have to better publicize its security efforts. Measure 1.2.4: Court Employees' Knowledge of Emergency Procedures When emergencies arise that threaten the safety of courthouse users, court employees must be knowledgeable about and prepared for correct responses. Their actions and decisions will have consequences for their safety, the safety of others, and the integrity of court records. This measure uses interviews to determine the extent to which court employees are familiar with emergency procedures. Planning/Preparation. The first step in applying the measure is to compile a list of employees. From this list, a sample of employees will be drawn to serve as interviewees. At least 15 supervisors or managers and 15 employees should be randomly selected.[13] Form 1.2.4, Interview Protocol on Emergency Procedures, should be reviewed and modified (e.g., change terminology, add specific questions, or modify particular questions) as necessary to better address local jurisdictional settings. For instance, questions referring to weather emergencies (see questions 3 and 14) could be specified to include those weather situations likely to occur in the locale (e.g., a flood or a blizzard). In addition, more questions may be added regarding power outages if they are a particular problem in the jurisdiction. Power outages may occur more often than some of the other emergency situations and may be particularly problematic given the widespread use of technology in both facility operations and court communications. Thus, it may be particularly important for employees to be aware of how to respond to them. Next, court procedures for responding to each emergency situation should be reviewed. If a court does not have written procedures regarding a particular emergency, questions about that emergency should be eliminated from the protocol. Before the interviews are conducted, each interviewer should be given an orientation to the court's security procedures. The data collection phase will be shorter if several individuals are available to conduct the interviews. However, care should be taken to ensure that interview responses are scored consistently across interviewers. One method for doing this is to have each interviewer complete an interview protocol for two or three "practice" interviews and then to compare the interviewers' protocols. If discrepancies exist, the instructions for the interview protocol should be modified to increase consistency among the raters. Data Collection. The interviews should be conducted in person with approximately 15 minutes allocated for each interview. The date and time of each interview should be recorded as part of the data for the measure. (Results of earlier interviews can be compared with results of later interviews. If employees interviewed at a later date have a higher level of familiarity with security measures than employees interviewed earlier, it is likely that the measurement process has prompted employees to become more informed.) Data Analysis and Report Preparation. If the court discovers during the planning/preparation stage that no written procedures exist regarding a certain type of emergency situation, the identified area requires immediate attention. For emergency situations that have written procedures, data analysis proceeds with an examination of the interview information. The interviews gather information on three topics: (1) the training provided to court personnel about security procedures, (2) the effectiveness of the training, and (3) the extent to which employees believe that improved security measures are needed. Summary statistics are used to analyze the results for each question in Part I and Part II. A benchmark of acceptable court performance is that 75 percent or more of all employees recall being briefed on emergency procedures. Court performance improves as the length of time since employees' last briefing decreases. In addition, 75 percent of employees should know what emergency procedures are in place. The responses to individual items in Parts I, II, and III can be examined to determine the areas in which the court is performing well and the areas in which the most improvement is needed. For example, the court may be very conscientious about preparing employees for a bomb threat but may be less conscientious about providing information on handling a hostage situation. A review of the individual items can help court officials determine which areas need the most attention. Measure 1.2.5: Access to Information by Telephone This measure involves simulating a request by a litigant or other interested person for information about the location and time of a court proceeding. A volunteer observer attempts to obtain information about the specific time and location of a court proceeding as well as the type of proceeding it is and its case number. The observer knows only the formal name of the court, the name of the litigant, and the day on which the proceeding in question is scheduled. He or she is not knowledgeable about routine court operations. Planning/Preparation. Five proceedings from the court events sampled for Measure 1.1.1, Access to Open Hearings, are selected and the name of the parties, date, time, and location (i.e., courthouse, floor, and courtroom) of the scheduled events are recorded. If Measure 1.1.1 has not been conducted, five scheduled court events will have to be selected from the court's calendar. A stopwatch or watch with a second hand will be needed during the data collection phase. Data Collection. The first step is for the observer to attempt to find the court's general telephone number from the local telephone directory using the court's official name (see Form 1.2.5, Access to Information by Telephone--Directions and Recording Sheet). If the number is not readily obtained from the local directory, the observer contacts the local directory information service. The observer notes the availability and difficulty of obtaining the court's telephone number and records the number(s) obtained on the data collection form. Using the telephone number obtained from the directory or directory assistance, the observer calls the court to obtain the time and location of each of the five events. To improve the simulation, the telephone contacts with the court should be distributed so that the frequency of the calls will not be noteworthy. Court officials should establish this distribution. For each event, the observer notes the elapsed time before the requested information is provided, using a standard stopwatch, and notes the number of individuals with whom he or she comes into contact. This information is recorded on Form 1.2.5. If the required information cannot be obtained within 1 hour (or if it cannot be obtained at all), the observer records a maximum of 60 minutes and six contacts for each event for purposes of the aggregate summary. He or she also makes notes as appropriate. Data Analysis and Report Preparation. Data obtained for the five events are aggregated. First, the observer summarizes the ease or difficulty of obtaining the court's telephone numbers from the telephone directories and notes the range of elapsed times for the five events. The elapsed time and number of contacts to acquire the information is then averaged across the five events (Telephone Information Accessibility Score). If the range of results from the calls varies widely, the court should separately evaluate, if possible, the circumstances of each simulation. The court may also wish to increase the number of simulations in order to achieve a more reliable average score and to better diagnose the patterns that explain extremes. Measure 1.2.6: Evaluation of Accessibility and Convenience by Court Users The ease and convenience of conducting business with the court is measured through a survey of regular court users (i.e., court employees, attorneys, probation officers, and jurors). Planning/Preparation. Measure 1.2.3, Perceptions of Courthouse Security, and Measure 1.4.1, Court Users' Assessment of Court Personnel's Courtesy and Responsiveness, also involve surveying regular users of the court. If these measures are also being conducted, the coordinators for these measures may find it efficient to combine the three measures into a single survey instrument. Review Form 1.2.6-1.2.7, Accessibility and Convenience of the Court. The survey form covers three subjects related to ease of conducting business: (1) convenience and cost of access to the building itself, (2) signs and other help for finding the right location or service in the building, and (3) the amenities that are available to those who are in the courthouse on business. The questionnaire should be adapted, as necessary, to local conditions and for each of the four groups. For instance, smaller jurisdictions conducting this measure may want to pay particular attention to the demographic questions included on the survey (Part IV, Background). In small jurisdictions responses to demographic questions might reveal the identity of individual respondents. If this is a potential problem, the court may find it best to eliminate some or all of the demographic questions. Data Collection. The questionnaire is administered following the same procedures described in the data collection section of Measure 1.2.3. Data Analysis and Report Preparation. The number and percentage of each response for each question is calculated. The percentages can then be compared across groups. For example, do jurors report more difficulty in getting to or conducting their business in the courthouse? If so, court officials should investigate methods for improving juror access to the court and its facilities. Specific problem areas may be examined and analyzed on a situation-by-situation basis. A review of the responses from all four groups also will highlight those areas in which the court generally is performing well and those areas in which improvement is needed. For example, do people tend to have more difficulty getting to the courthouse or in finding their way around the courthouse once they are there? Once a court has conducted this measure, court officials should establish benchmarks for "poor," "adequate," and "good" ratings. For example, an "adequate" rating might mean that less than 25 percent of the respondents report some difficulty finding parking, and a "good" rating might mean that less than 10 percent report difficulty. Courts should strive to meet the "good" benchmark when conducting the measure in the future. Because different groups may experience different problems, the benchmarks might differ for each group. Measure 1.2.7: Evaluation of Accessibility and Convenience by Observers Several measures in this document require information to be collected by volunteers who are unfamiliar with court facilities and procedures. For this measure, volunteers are given a survey questionnaire on the ease of conducting business with the court at the end of their first observation day in the courthouse. The survey questionnaire is basically the same as that used in Measure 1.2.6. Planning/Preparation. The questionnaire used for Measure 1.2.6 (Form 1.2.6-1.2.7, Accessibility and Convenience of the Court) should be reviewed and modified, as necessary, for the volunteer observers group. Data Collection. The observers are given a questionnaire at the conclusion of their first day of simulated business in the courthouse. They are asked to return it when they make their next observation for one of the measures. This procedure should apply to all observers who visit the courthouse during the evaluation process (not just those collecting data for measures in the Access to Justice performance area of the survey). Data Analysis and Report Preparation. The basic analyses are the same as those discussed in the data collection section of Measure 1.2.6. In addition, the responses of the volunteer observers can be compared to those of the "regular users" who were surveyed for Measure 1.2.6. How do the percentages differ? For example, do regular users tend to report that finding a restroom or telephone is easy, while the volunteer observers report that it is difficult? If so, perhaps the directional signs in the courthouse could be improved to better accommodate the needs of a stranger. The court may also get suggestions for how to improve its accessibility to strangers by conducting "debriefing" interviews with the volunteer observers once the observers have completed the questionnaire. Standard 1.3: Effective Participation The trial court gives all who appear before it the opportunity to participate effectively, without undue hardship or inconvenience. Commentary. Standard 1.3 focuses on how a trial court accommodates all participants in its proceedings--especially those who have language difficulties, mental impairments, or physical handicaps. Accommodations made by the court for impaired or handicapped individuals include the provision of interpreters for the deaf and special courtroom arrangements or equipment for blind and speech-impaired litigants. Measurement Overview. The measures for this standard focus on four groups of people with special needs: (1) children who require special treatment by counsel and the court in order to be represented effectively in court proceedings, (2) hearing or speech impaired individuals who require the services of interpreters in order to participate effectively in court proceedings, (3) non-English-speaking individuals who also require the services of interpreters, and (4) individuals with physical disabilities that impede their ability to get to and move around the courthouse with a reasonable degree of ease and autonomy. The five measures for this standard consider whether these four groups are given the opportunity for effective participation. Measure 1.3.1 examines the representation provided to children in child abuse and neglect proceedings. It relies on case record, survey, and interview data. Measures 1.3.2, 1.3.3, and 1.3.4 examine interpreter services. Measure 1.3.2 examines the quality of interpreting services and the conformity of those services with interpreter standards. It relies on observation data. Measures 1.3.3 and 1.3.4 evaluate interpreters on their knowledge of basic legal and justice system terminology and concepts and on the interpreter's knowledge of a language other than English. Both of these measures require administering tests to the interpreters. The final measure, 1.3.5, relies on observation data. Individuals with physical disabilities collect the data by conducting real or simulated business in the courthouse. Measure 1.3.1: Effective Legal Representation of Children in Child Abuse and Neglect Proceedings The Federal Child Abuse Prevention and Treatment Act of 1974 requires all States to appoint an individual to represent the interests of children involved in judicial proceedings regarding child abuse and neglect. The individual appointed for this purpose is usually called a guardian ad litem (GAL). The States employ various models for providing the services of a GAL. In some States the GAL must be an attorney, while in others a trained volunteer (most often a court-appointed special advocate) may serve as the GAL or may work in conjunction with an attorney. The model used by individual jurisdictions within States also may vary from one another. In addition, the roles and responsibilities of GALs vary across the States, and many State statutes offer little guidance on the GAL's specific duties. In most States, however, the GAL is expected at a minimum to act as an independent investigator of the facts related to the abuse or neglect, an advocate of the child's interests, and a case monitor.[14] Proponents of the rights of children and guidelines on GAL representation recommend that the GAL perform other duties as well.[15] This measure determines the effectiveness of legal representation of the child in child abuse and neglect proceedings. An evaluator (or court staff) reviews the state statutes and court rules relevant to the appointment and responsibilities of GALs in child abuse and neglect proceedings, compares the statutes or rules to recommended practices for GALs, and obtains data from court records and surveys or interviews with GALs, judges, and child protective services caseworkers. Planning/Preparation. Planning and preparation for conducting this measure includes four steps. First, court staff review the relevant statutes, court rules and policies, and case law on the appointment of guardians ad litem and their roles and responsibilities. Second, court staff modify the sample case data collection form (see Form 1.3.1a, Evaluation of Legal Representation of Child Abuse and Neglect Proceedings: Case Data Collection Form) and the survey forms (see Forms 1.3.1b, Judge Survey, 1.3.1c, Guardian ad litem Survey, and 1.3.1d, Caseworker Survey) to conform to the court's procedures, practice, and terminology. Forms 1.3.1a through 1.3.1d include items related to practices recommended in the literature on GAL representation and in guidelines developed in a few States.[16] Unless these items bear no relationship to local practice or are contrary to State law or court rule, they should not be eliminated because they are an important gauge of the effectiveness of legal representation. The third step is the selection of the case sample, which should include 20 current child abuse and neglect cases that have reached a disposition hearing and 20 current review cases that have had a review hearing. The measure requires current cases to ensure that the judges, GALs, and caseworkers have fresh memories of their experiences in the sample cases. The sample should include a broad representation of the pool of individuals who serve as GALs in the jurisdiction. Staff also should determine if they must have approval to access the case files and obtain any approval that is required. Fourth, as staff select the case sample, they create a list of judges, GALs, and child protective service caseworkers involved in the sample cases. The list should match the judges, GALs, and caseworkers to the specific case in which they were involved. These individuals will be surveyed to obtain information about GAL performance that is not available from the case record. In some instances, court staff may also need to interview the judges, GALs, and caseworkers to clarify their responses to the survey. If interviews become necessary, staff may need to request assistance in scheduling interviews with GALs and caseworkers. Data Collection. Data collection from the case records and from the judges, GALs, and caseworkers may proceed simultaneously to reduce the time required to complete this measure. The surveys should be distributed with a cover letter from the chief or presiding judge of the division of the court that has jurisdiction over child abuse and neglect cases. The letter explains the purpose of the survey and that all responses are and will remain confidential. Provide the name of the specific case on each of the survey forms distributed to the judge, GAL, and caseworker. In some jurisdictions, one GAL, judge, or caseworker may have been involved in several cases and therefore will receive multiple surveys. In those cases, only one of the surveys should include Part III, which calls for the respondent's general opinions about training and practice issues related to GAL representation rather than his or her views about GAL representation in a specific case. As the surveys are being prepared and distributed, court staff complete Form 1.3.1 for each of the sampled cases. As the surveys are returned, court staff should review them to determine if calls to the respondents will be needed to clarify responses. Data Analysis and Report Preparation. Case records: The analysis of case record data provides information on the timeliness of GAL appointments, the level of participation of GALs in court proceedings, and the degree to which GALs contribute to case dispositions. For each case, determine whether the appointment of the GAL was made within the time limit set by statute or court rule. Calculate the percentage of cases that fall within the time limit. Also calculate across all cases the average time (mean) in days between the appointment of the GAL and the filing of the petition, the emergency removal order, or other initial court action in the case. The quality of representation is likely to be higher when appointments are made within the time limit and in cases in which appointments are made shortly after the first court action taken because the GAL will have greater opportunity to assess the child's environment and the need for placement outside the home. Next, calculate the number and percentage of hearings in which the GAL participated. The higher the rate of GAL participation in hearings, the higher the effectiveness of representation is likely to be. To assess the level of GAL preparation, calculate the average number of required reports submitted by the GAL. To determine the extent to which GAL performance creates delays in child protection proceedings, calculate the number of continuances of hearings because the GAL was not prepared, the percentage of GAL reports filed on time, and the number of days past the deadlines reports were filed. The extent to which reports from involved agencies are in the case record indicates whether adequate information is available for the GAL to review, making preparation for the case more efficient and effective. Finally, calculate the percentage in both new and review cases in which the GAL made recommendations regarding the placement of the child. The higher the percentage of cases in which the GAL offers the court recommendations, the greater the likelihood that GALs are aggressively representing the child's interests. Surveys: For each GAL activity, calculate the percentage of judges, GALs, and caseworkers who reported that the activity was undertaken. To calculate these percentages, the number of "x's" for each activity are summed across all cases and divided by the total number of cases. If an activity was marked with a "0" or an "I", that case is not counted as part of the numerator or the denominator of the percentage. In addition, calculate the mean rating of each group surveyed (judges, GALs, and caseworkers) for all cases for the items in Parts II and III of Forms 1.3.1b through 1.3.1d. The results should be calculated separately for each group so that the perceptions of the different players can be compared. Court officials should review the average "overall ratings" in Part II first. The higher the average ratings, the better the court is performing on this measure. Is the quality of legal representation generally good (average rating of 4 or 5), or is it considered better in some areas than in others? To determine why the ratings of the quality of representation vary, court officials can examine the responses in Parts I and III. Do GALs undertake certain tasks more frequently than others? Do they demonstrate greater competence in fulfilling particular responsibilities than in completing others? Do review cases receive adequate attention? Are some important activities in representing a case neglected? Do judges, GALs, and caseworkers believe that GALs and judges receive sufficient training? Answers to questions such as these will help court officials determine whether children are being represented properly and, if not, what kinds of improvements are needed. These improvements might include additional GAL or judicial training, clearer definition of the roles and responsibilities of the GAL, implementation of compensation policies that encourage GALs to spend more time on the case, and the development of standards of practice. Measure 1.3.2: Evaluation of Interpreted Events by Experts This measure involves observation and evaluation of the work of court interpreters by individuals who are skilled in foreign language and sign language interpretation.[17] The experts observe interactions in which interpreters are involved, make an assessment of the interpreter's proficiency, and record interpretation problems or violations of interpreter standards. This measure is only appropriate when courts can predict with reasonable certainty that interpreters will be used in specific locations during predictable timeframes. Before arranging for this measure, court personnel should first inform themselves of the qualifications that "skilled individuals" used as observers should possess. For example, if the proposed observers are very proficient in both English and the other language but not familiar with the code of professional responsibility for court interpreters, they should not be used. "Certified" professional interpreters would make the best observers. However, they are not available for many languages nor are they available in many parts of the country.[18] Planning/Preparation. Individuals skilled in foreign languages and sign communication should be identified and recruited to evaluate the court's interpretation services. These individuals, serving as paid professionals or as volunteers, will provide an independent viewpoint of the quality of the court's interpreter services. It is essential for this measure that the language experts understand the requirements for interpreting in court settings. If the observers are not themselves certified court interpreters, they need to be thoroughly familiarized with the professional responsibilities of court interpreters. In addition to any State or local rules governing appropriate professional conduct, the observers should be provided with the following material from Court Interpretation: Model Guides for Policy and Practice in the State Courts:[19] o Chapter 2: Interpreting Terminology o Chapter 6: Judges' Guide to Standards for Interpreted Proceedings o Chapter 9: Model Code of Professional Responsibility for Interpreters in the Judiciary Experts may be located by contacting national and state interpreter associations;[20] the State's office of social services that is responsible for services to deaf or hearing impaired individuals; universities; or community agencies that serve foreign language or handicapped citizens. Experts should be informed that what they see or hear in open court should be discussed only with court officials and that they should not attempt to intervene in any way in the cases they observe. The next step is to select a sample of scheduled court proceedings to observe. Ideally, this sample includes both nonevidentiary and evidentiary hearings. High-volume calendars that likely will include interpreters are good choices for observation scheduling. Examples include traffic court sessions, misdemeanor arraignment and plea dockets, and child support calendars. Felony arraignment and plea calendars should be included if possible. Observations of evidentiary hearings in which interpreters are used for witness testimony are also important to include in the sample. Pending cases should be examined to obtain a list of cases in which interpreters will be needed. When arranging for these observations, identify several proceedings that observers could go to in the same day. The key to scheduling is to ensure that if some of the scheduled proceedings are continued or delayed, other observation opportunities are available. If a court uses interpreters infrequently, this measure should not be attempted. Data Collection. Evaluators observe short procedural hearings in their entirety, striving to achieve as much variety in languages as possible and as many different interpreters as possible. Observations of interpreters working during witness testimony should last at least 5 minutes but not longer than 30 minutes. Using Form 1.3.2, Evaluation of Interpreter Services, the evaluator records observations regarding the quality of interpreter services. The observer first identifies the session of court and the type of proceeding observed. The specific case number, date, and time should be noted, but this may not be possible in high-volume court sessions. If different interpreters are used during a session of court, a separate form should be used for each interpreter. If one interpreter is used for several different cases, a separate form should be completed for each case. For each interpreted session recorded on the form, the observers should rate the overall performance of the interpreter on three dimensions, as shown on the form. These dimensions are: o Language proficiency o Interpreting skills o Professional conduct If problems with the interpreter's performance are noted during the proceeding, these should be briefly recorded on the form. Data Analysis and Report Preparation. After the data collection is complete, the observer should prepare a brief report summarizing the observations. The report should include the following: (1) the number of individual cases that were observed, (2) the number of different interpreters that were observed, by language, and (3) a summary of the evaluation results for all of the cases observed, by language (e.g., the percentage of all cases observed where problems were noted). A summary qualitative assessment should also be provided informing the court of any problem areas that are severe in the observer's opinion, with examples included in the narrative. Measure 1.3.3: Test of Basic Knowledge Required of Interpreters Interpreters cannot adequately perform their job without knowledge of the principles of appropriate professional conduct and basic legal and justice system terminology and concepts. Research has shown, however, that many interpreters used by the courts have not mastered these fundamentals. This measure involves administering a written test that is used to determine whether interpreters have acquired this knowledge. Other essential job requirements--language proficiency and interpreting skills--must be measured independently. Measures 1.3.3 and 1.3.4 may be unnecessary if courts already have a valid and reliable testing process for interpreters used in their courts, including freelance interpreters. If freelance interpreters are not tested prior to employment, use of the measure should be considered. Planning/Preparation. Preparation for this measure involves reviewing and modifying the attached model written test (see Form 1.3.3, Court Interpreter Terminology, Procedure, Protocol, and Ethics Fundamentals Test) to ensure that it reflects local terminology and concepts. After the revisions are complete, the instrument should be pretested by giving it to at least three experienced local court personnel or practicing lawyers (legal and justice system terminology) and to at least two professional court interpreters (questions related to professional conduct). No time limit should be imposed during the pilot test. All pilot test takers should agree on which answer is correct for each test item and that there is only one correct answer. If there is disagreement, the question should be eliminated or replaced with a test item that is agreed upon by the test takers. Each test taker should also be asked to suggest cutoff scores for "excellent," "good," "acceptable," "poor," and "very poor" levels of performance on the exam. It is recommended that the test then be reviewed by at least one judge (preferably two) before setting the final criteria that will be used to evaluate individual test performance. Data Collection. Data collection involves administering the test to all or a majority of the individuals who the court uses as interpreters, and then scoring the test using a standardized scoring guide. It is useful to prepare a scoring template to add greater speed and reliability to the scoring process. Data Analysis and Report Preparation. Results should be analyzed using a standard statistical analysis and reporting software package, if possible. This method allows greater speed and flexibility of analysis. Every time the test is administered, the new scores should be added to the database. The analysis should include, at a minimum, a frequency report showing the number and percentage of test takers in each ranking group (i.e., "excellent," "good," etc.). It is recommended that the analysis also include frequency reports of score rankings by language, and, within each language group and overall, breakdowns by years of experience and educational level. Measure 1.3.4: Assessing Non-English Language Proficiency Through Back Interpretation This measure allows the court to make an assessment of a person's knowledge of a language other than English. The procedure can be used for virtually any language and can be applied by an examiner who speaks only English. Measures 1.3.3 and 1.3.4 are unnecessary if courts already have a valid and reliable testing process for interpreters for the language in question, including freelance interpreters. If freelance interpreters are not tested prior to employment, the measure should be used. Back translation is a technique in which a candidate interprets or translates English into the foreign language in question and, after the passage of time, interprets or translates her or his own foreign language version back into English. The interpreted or translated English version is then compared to the original English to determine how faithfully the original message has been preserved. Planning/Preparation. Before undertaking this measure the court should acquire the textbook Fundamentals of Court Interpretation: Theory, Policy and Practice.[21] The textbook includes a detailed description of the proper procedure for administering and scoring the back translation exercise, including 10 sample questions and statements with underlined scoring units. The measure also requires the use of two audiotape recorders, one for playing a recorded script and one into which the interpreter records her or his interpretation of the script. A written script in English is then prepared in a form identical or similar to the script suggested in Fundamentals of Court Interpretation. The written script is read aloud into a tape recorder in the same way that an attorney would pose a question to a witness or a witness would answer a question. Between each prerecorded question or statement there must be a pause long enough for the interpreter to complete the interpretation. To conduct the measure the court identifies all interpreters who work regularly in the court and plans a testing schedule. The schedule should require the interpreters to report to the testing room on two separate occasions. On the first occasion the interpreter listens to a tape-recorded passage in English and interprets it aloud in the foreign language, using a second tape recorder to record the foreign language rendition. On the second occasion the interpreter will listen to her or his own recorded foreign language rendition of the original script and interpret it back into English. The interval between the first occasion and the second occasion may be as little as one hour. However, separating the occasions by one or several days is not only acceptable but may result in a better test because the passage of time reduces the opportunity for the interpreter to rely on memory of the original English. Approximately 15 minutes should be allocated for each interpreter for each test session. The final preparation step is to select one or two individuals to score the test results. These individuals should have highly developed language skills in English and be able to discern the difference between substitution of words and distortion of meaning. Data Collection. Data collection consists of administering the test to the candidates as summarized above and as described in more detail in Fundamentals of Court Interpretation (pp. 196- 199). Test raters listen to each interpreter's back-translated English version of the script and compare it to the original. The script will contain approximately 40 underlined scoring units that are used to determine the individual's score. The resulting data sources are scoring sheets prepared for each interpreter by the test rater showing the number of scoring units on the back translation that match the meaning of the original English script. Data Analysis and Report Preparation. After all of the tests are scored, a listing of their scores should be prepared. The analysis should then report the summary results in terms of percentiles, as shown in the following table. Results of Back Translation for All Interpreters Report Illustration Score grouping (out of 40 possible correct responses) 36-40 correct (90% or better) 32-35 (80 to 89%) 28-31 (70 to 79%) 24-27 (60 to 69%) 20-23 (50 to 59%) 20 or less correct (49% or below) Number in the group 1 3 4 6 6 3 Percentage of test takers (n=23) 4 13 17 26 26 13 Research and experience with court interpreter testing suggests that analysis of test results should examine test scores of interpreters by language groups. One obvious way to do this in most States is to prepare a report that distinguishes the test results for Spanish language interpreters from other languages. In interpreting the results, the court's policymakers should draw their own conclusions about what is an acceptable level of performance. The mathematics speak for themselves in terms of performance: an interpreter who gets 20 correct items is only rendering one-half of the questions or testimony accurately; a score of 30 correct implies that 25 percent of the "message" is changed, distorted, or lost altogether in the process of being rendered from one language to another. Measure 1.3.5: Participation by Persons with Disabilities This measure examines access to courthouse facilities by persons with physical disabilities. The measure produces two kinds of information: general information about accessibility in the courthouse as a whole and more specific information on the ease or difficulty with which individuals with disabilities conduct business transactions with the court. (The volunteers who complete this measure may also be included in the samples for Measure 1.2.6, Evaluation of Accessibility and Convenience by Court Users, and Measure 1.4.2, Observers' Assessment of Court Personnel's Courtesy and Responsiveness.) Planning/Preparation. The first step is for court staff to complete Part I of Form 1.3.5, Access to Courthouse Facilities by Individuals with Disabilities. Part I is based on a checklist developed by the National Center for State Courts to assess court compliance with the Americans with Disabilities Act.[22] These questions will provide the court with general information about the accessibility and usability of courthouse facilities and services as a whole for persons with disabilities. Next, court officials should prepare a list of routine activities that citizens engage in while using the court's services. These should include: (1) transacting business in the clerk's office, (2) appearing for jury duty, (3) observing a domestic relations calendar, (4) observing a criminal arraignments calendar, (5) observing a trial or simulating the experience of being a litigant during a trial (e.g., visit the courtroom, sit in the litigation area), (6) accessing facilities for special services such as ADR program offices, child support complaint and payment offices, and bail payment windows, and (7) using general courthouse facilities such as cafeterias, restrooms, attorney- client conference rooms, and public telephone areas. Local service agencies or advocacy associations for individuals with physical disabilities should be contacted to obtain the names of individuals who may be willing to participate in a simulation exercise. At a minimum, two individuals confined to wheelchairs and two individuals with a visual impairment should be asked to visit the courthouse. Data Collection. A list of the simulation activities and a copy of Part II of Form 1.3.5 is given to each volunteer. (The information is provided verbally as well.) Each volunteer should attempt each activity and note the results of each simulation on Form 1.3.5. (Volunteers with visual impairments will need assistance in recording the results of the simulations.) Form 1.3.5 allows 15 simulations to be recorded. If more than 15 simulations are conducted, the form should be modified to accommodate the additional simulations. Each simulation should be started from outside the courthouse. Volunteers should record the length of time it takes to conduct each simulation, the ease with which the activities are accomplished, and any specific problems encountered. After the volunteers have completed the simulations, court officials should schedule a meeting with them. During the meeting, the volunteers can compare their experiences with one another and discuss possible improvements for making the court more accessible to individuals with disabilities. Problems encountered in obtaining the resources to make improvements should be described, and both court officials and the volunteers should discuss possible options for overcoming the problems. Court officials may also choose to interview regular users of the court who have disabilities (such as court employees or attorneys) to discuss problems they encounter while working in the courthouse, as well as suggestions they have for improvements. Data Analysis and Report Preparation. Court staff first review the answers to Part I of Form 1.3.5 and consider whether the court has adequately addressed the issue of access for persons with disabilities. To what extent are employees knowledgeable about policies and procedures related to accommodating persons with disabilities? Did the answers to these questions provide a favorable or unfavorable impression of the courthouse's accessibility for all persons? Next, court staff review the results of the simulation exercises. The average rating for "ease of conducting business" for all simulations is calculated. (If 4 volunteers each rated 15 simulations, the average would be based on 60 simulations.) The closer the average rating is to "1," the better the court is performing on this measure. The average length of time needed to conduct each simulation also is calculated. Did some activities take longer than others? If so, what specific problems were encountered? Does an examination of all the simulated activities reveal that some areas of the courthouse are less accessible than others? Using both the general information gathered in Part I and the more specific information gathered from the simulations in Part II, court officials should summarize where the most serious problems exist and develop an action plan (incorporating the volunteers' suggestions, if possible) for alleviating the problems. If court officials have not already done so, they should refer to the Americans with Disabilities Act for suggestions and requirements when developing their plan. Standard 1.4: Courtesy, Responsiveness, and Respect Judges and other trial court personnel are courteous and responsive to the public, and accord respect to all with whom they come into contact. Commentary. The intent of Standard 1.4 is to make the justice system more accommodating and less intimidating. A responsive court ensures that judicial officers and other court employees are available to meet both the routine and exceptional needs of those it serves. Requirements of the standard are particularly important in the understanding shown and assistance offered by court personnel to members of minority or disadvantaged groups and to those unfamiliar with the trial court and its procedures. In keeping with the public trust embodied in their positions, judges and other court employees should reflect by their conduct the law's respect for the dignity and value of all individuals who come before, or make inquiries of, the court. No court employee should by words or conduct demonstrate bias or prejudice based on race, religion, ethnicity, gender, sexual orientation, color, age, handicap, or political affiliation. These requirements extend to the manner in which the employees of the court treat each other. Measurement Overview. The three measures for Standard 1.4 determine whether court personnel are courteous, responsive and respectful to one another and to various members of the public. Measure 1.4.1 uses survey data, Measure 1.4.3 relies on observation data, and Measure 1.4.2 utilizes both methods. The survey for Measure 1.4.1 asks regular court users and court personnel about their treatment by court personnel in general. The survey respondents for this measure are from the same groups surveyed for Measure 1.2.3 and Measure 1.2.6: attorneys, probation officers, jurors, and court employees. The survey for Measure 1.4.2 is similar to that for Measure 1.4.1 but is directed at observers who are unfamiliar with the court recruited to collect data for one or more of the other measures. After collecting data for the other measures, the observers are asked to complete a questionnaire that summarizes their overall impressions of the courtesy and responsiveness of court employees. Measure 1.4.3 relies on observation data to determine the degree of courtesy and respect shown to litigants during court proceedings. The measure requires observers to watch several court proceedings and record information on interactions among the various parties involved in the proceedings. Because each measure is directed at different groups of court users, all three measures should be undertaken to obtain the best assessment of courtesy, responsiveness and respect. Although Measures 1.4.1 and 1.4.2 use survey data, they should not be considered interchangeable. Each measure has a different focus and methodological advantage. Measure 1.4.1 surveys a greater number of people and thus will yield more reliable quantitative results. Because the number of respondents surveyed for Measure 1.4.2 depends on the number of observers collecting data for the court, the number of respondents will be small for most courts. Given the small number of respondents, Measure 1.4.2 offers court officials the opportunity of collecting more indepth qualitative information (e.g., for clarifying problems and obtaining suggestions for improvements) through followup interviews with respondents. Measure 1.4.1: Court Users' Assessment of Court Personnel's Courtesy and Responsiveness The courtesy and responsiveness of court personnel is measured through a survey of regular court users, including court employees, attorneys, probation officers, and jurors. Planning/Preparation. Measures 1.2.3, Perceptions of Courthouse Security, and Measure 1.2.6, Evaluation of Accessibility and Convenience by Court Users, also involve surveying regular users of the court. If these measures are also being conducted, the coordinators for the measures may find it efficient to combine the three measures into a single survey instrument. Review Form 1.4.1-1.4.2, Questionnaire for Courteous and Responsive Treatment. It addresses four aspects of courteousness and responsiveness: (1) the courtesy of court employees, (2) the availability of staff to answer questions, (3) the knowledge of court staff, and (4) the willingness of court staff to explain court policies and procedures to the public. The questionnaire also asks respondents to rate the degree of respect with which judges treat the public. Adapt the survey form, as necessary, to local conditions and for each of the four groups receiving the survey. For example, the questionnaire administered to court employees should be modified to ask for employees' perceptions of the public's treatment by