A Police Guide to Surveying Citizens and Their Environment. MENU TITLE: A Police Guide to Surveying Citizens and Their Environment Series: BJA Published: October 1993 108 pages 227,045 bytes Bureau of Justice Assistance A Police Guide to Surveying Citizens and Their Environment Monograph Acknowledgments The Bureau of Justice Assistance wishes to thank the Police Executive Research Forum (PERF) for its help and guidance in compiling this document, in particular Nancy G. LaVigne and John E. Eck. Table of Contents EXECUTIVE SUMMARY INTRODUCTION PART I: SURVEYING CITIZENS: A GUIDE FOR POLICE OF WHAT VALUE ARE SURVEYS? The Uses of Surveys Types of Questions To Be Addressed Attitudes and Opinions Behavior and Experience Characteristics WHO WILL BE SURVEYED? Simple Random Sampling Nonrandom Sampling HOW MANY WILL BE SAMPLED? HOW WILL THE RESPONDENTS BE CONTACTED? Mail Surveys Telephone Surveys Inperson Interviews Summary WHAT QUESTIONS WILL BE ASKED? General Considerations How To Ask Questions Open-Ended Questions Designing the Questionnaire HOW TO MAKE SENSE OF THE DATA Characteristics of the Sample Representativeness of the Sample Making Inferences About the Population Estimating Relationships Significance Testing CONCLUSIONS PART II: ENVIRONMENTAL SURVEYS FOR PROBLEM SOLVING: A PRACTICAL GUIDE ENVIRONMENTAL SURVEYS AND PROBLEM SOLVING THE OFFENDER AND RATIONAL DECISIONMAKING Defensible Space Neighborhood Maintenance and Safety Physical Defense and Target Hardening ENVIRONMENTAL SURVEYS IN PRACTICE Large-Scale Analysis Assessing Situational Factors of Crime and Disorder DEVELOPING AND USING A SURVEY INSTRUMENT Designing an Approach Defining the Area Reliability and Validity Survey Design Layout of the Survey Instrument Numbering Questions Sampling and Pretesting Survey DRAWING ON OTHER DATA SOURCES Police Calls for Service Surveys of the Public Official Land Use Data CONCLUSIONS GLOSSARY SOURCES FOR FURTHER INFORMATION BIBLIOGRAPHY FURTHER READING APPENDIX A: EXAMPLE OF A HOUSING COMPLEX ENVIRONMENTAL SURVEY APPENDIX B: EXAMPLE OF A BLOCK ENVIRONMENTAL PROBLEM IDENTIFICATION SURVEY APPENDIX C: EXAMPLE OF A DRUG "HOT SPOT" SURVEY APPENDIX D: EXAMPLE OF A SITUATIONAL ENVIRONMENTAL SURVEY USED FOR ANALYZING CONVENIENCE STORE PROBLEMS Executive Summary "How are we doing?" How the public answers this question can determine police strategies and, ultimately, their success or failure. When a police department surveys public opinion it may well seek to measure much more than citizen perceptions of where police stand in public sentiment. Research teaches that many popular yardsticks of police performance-speedy response to calls for service, for example-may tell little about law enforcement's true effectiveness in reducing crime and the fear of crime. Police survey teams may be probing deeper questions: How do citizens feel about open-air drug markets? Did that last crackdown make people feel safer? How do the elderly feel about our efforts? What do minority groups think? school children? single female heads of households? Some evidence suggests that door-to-door surveys by police officers are enough by themselves to reduce crime and fear and enhance citizen attitudes toward police, independent of any information they gain or what police do with it. Surveys can also help measure the characteristics of neighborhood residents, the background of crime victims, or the background of offenders. They can supplement and help interpret other, more readily obtainable information such as census data, arrest reports, or complaints of crime. Selecting a Sample Those conducting a survey must first determine from what total group of individuals-called the "population"-they want to obtain data. If it were necessary to interview every member of this group, surveys would be so expensive that few would ever be conducted. Instead, one selects a group of individuals from within the population that seems statistically likely to reflect the views of the whole. This is called a "sample." For many reasons, the most popular manner of selecting a representative sample is random selection. It is mathematically simple to calculate the (usually small) odds that a random sample is not representative, and it is easy to "stratify" or "weight" a random sample to ensure that it fairly reflects the population on such matters as racial or economic makeup. Accident, coincidence, or a search for a particular type of respondent will yield a sample that is less likely to be unbiased. The next question is harder: How large must the sample be? There's no simple answer. The greater assurance of accuracy the surveyor requires (the technical term is "level of confidence"), the larger the sample must be-and thus the more expensive the survey. But unless the population is itself very small, the size of the population means little in terms of the size of the sample. If the population is so small as to require a larger sample, then one might as well interview the total population. How To Question Respondents Generally one can question members of the chosen sample in one of three ways: by sending a questionnaire in the mail or questioning by telephone or in person. Mail surveys tend to have a lower response rate. Great skill is required to avoid ambiguous questions, and great expense is needed to make the survey instrument so impressive looking that it's less likely to be thrown out with the junk mail. Telephone surveys have a much higher response rate, but automatically exclude citizens without telephones. Like inperson surveys, they require questioners trained to administer them. Face-to-face surveys, in person, are most expensive, but usually most accurate. Designing the questions to ask respondents requires precision in wording, clarity of thought, and care not to wear out the respondent's patience by taking too much of his or her time. The reader is given examples of good and bad questions and ways to improve them, together with instructions on how to code the questionnaire in advance for easy data entry and how to test it in advance so as to improve its clarity and ease of interpretation. Making Sense of the Results When all the results are in, some simple data analysis is in order; as the survey technician gains skill, he or she will learn of other works that teach more advanced techniques. The "central tendency" or typical response to a question can be determined, the range of answers from one extreme to the other, and even some good ideas of how well one succeeded in drawing a representative sample. All these together enable the analyst to make inferences about the population as a whole. Construction of numerical tables is one of the simplest basic ways of drawing such inferences. The reader also learns some simple ways of testing the results for statistical significance. How Surroundings Affect Behavior Just as opinion surveys may themselves help deter crime and quiet fears of crime, environmental surveys help police quantify the physical characteristics of neighborhoods and link them with specific neighborhood problems. These surveys help identify problems, determine what changes will help solve them, and measure the effectiveness of those efforts, once taken. Criminals may not be fully rational, but they can read the environment well enough to reach the conclusion that a crime target is likely to be a successful or unsuccessful one. A drug dealer recognizes a litter-strewn street with many abandoned houses and cars as a likely place in which to hide his wares and himself from police and from his business rivals. If police or others clean up that neighborhood and populate it more hours of the day with working residents, the drug dealer's haven disappears. A burglar, similarly, avoids crime targets that have many physical barriers because those barriers would increase the time required to escape. Other types of barriers, however, might work to the thief's advantage, making it harder for police or law-abiding citizens to interrupt him in the midst of his crime. Tipping the Balance to the Good Guys Through an environmental survey, police can determine what environmental cues work for or against the law-abiding resident. Then they try to shift the balance in the good guys' favor. One useful form of crime analysis is identifying "hot spots"-locations with unusually high levels of chronic crime and disorder. By using a survey to zero in on physical features of such locations, police can make suggestions to businesses, residents, and government about physical changes that can reduce or prevent problems. Such a survey might begin by defining the area to be surveyed. The survey instrument developed would provide structured terms for recording and validating the amount of lighting, the density of population, and the behavior of residents at various times of the day. In addition to data developed by the survey, other sources of information can help round out the picture of environmental assets and problems. Used both before and after implementing a problem-solving effort, opinion surveys and environmental surveys can enable police not only to evaluate the effort just past but also to improve on it when the effort moves on to new challenges elsewhere. A glossary and bibliography are provided as well as examples of survey instruments used successfully in several cities. The Uses of Surveys Part I: Surveying Citizens: A Guide for Police Of What Value Are Surveys? Surveys can be used for a variety of purposes. For example, they are useful in gathering data on how police are perceived by the public and in helping to determine police priorities. Regular surveys of the general public are quite useful for the latter purpose. The Reno (Nevada) Police Department conducts such surveys every 6 months. The Madison (Wisconsin) Police Department regularly surveys individuals who have had contact with the police to determine the quality of police-public encounters. Surveys can also be used to identify specific problems in target neighborhoods or among special populations. In Newport News, Virginia, a survey of residents in an apartment complex revealed concerns about the maintenance and physical structure of the complex. In Maryland, Baltimore County police officials routinely use surveys to diagnose community problems confronted by special problem-solving units. In other cities, community surveys have been used to help define community concerns about drug problems. Special populations-- the elderly, school children, women, minority groups, and others--can also be surveyed to learn their special concerns. Finally, multiple surveys can be used to evaluate problem-solving efforts. For example, before-and-after surveys can be used to determine changes in citizens' fear of crime as a result of a police intervention. Problem-solving units of the Baltimore County Police Department routinely use this technique to gauge their effectiveness. There is some evidence that door-to-door surveys by uniformed police officers may reduce crime and fear of crime, as well as enhance attitudes toward the police, independent of any information collected by the officers for the purpose of collecting that information (Pate et al., 1986; Pate, 1989; Uchida, Forst, and Annan, 1990). If true, then this may be one of those "good news/bad news" situations. The good news would be that surveys themselves could be used as a tactic in the control of crime and fear; although this is a resource-intensive tactic, it requires neither coercion nor the use of force. The bad news is that uniformed officers cannot conduct before-and-after surveys and expect to obtain genuinely unbiased answers to their problem-solving questions. Most citizens are unlikely to criticize police officers to their faces. Civilian interviewers are needed to make sure that effects of the data collection are not interfering with the effects of the problem-solving program. In summary, surveys can be used to achieve four goals: o Gather information on the public's attitudes toward police and neighborhood priorities. o Detect and analyze problems in neighborhoods or among special population groups. o Evaluate problem-solving efforts and other programs. o Control crime and reduce fear of crime. Except for the first goal cited, several alternatives to surveys can achieve these objectives. Whether surveys should be employed to achieve these purposes depends on the types of questions asked. Types of Questions To Be Addressed Police agencies collect a great deal of data during their normal operations that can be used to detect problems and judge anti-crime effectiveness. Because these data are a byproduct of routine police activities, the only additional cost to using them is the time needed for their analysis. Surveys of the public, however, are seldom part of routine police activity. These data are far more costly because a special effort must be undertaken to collect them. And surveys are not an appropriate way to collect data if there is a cheaper alternative of similar quality. Due to these additional costs, police managers need to know when it is worthwhile to conduct a survey and when it is not. Surveys can be used to address three types of questions. They are the primary method of answering questions about mental states--attitudes, beliefs, fears, and perceptions--of large numbers of people. They are useful for learning about people's behaviors and experiences, and they can determine group characteristics such as age, race, sex, education, employment, income, and housing conditions. Attitudes and Opinions Data describing people's attitudes and opinions can often be derived through surveys; it is impossible to directly observe people's thought processes. Surveys seeking information on mental states frequently address issues such as: o Attitudes toward police performance. o Fear of crime. o Future plans and intentions. o Concerns about specific problems. o Suggestions for police actions. Although there are no alternative sources of this type of information, surveys that seek information on mental states are most useful when people already have thought about the issue and can express themselves honestly. Asking citizens about their feelings concerning police actions they have never experienced will yield less reliable information than querying people who have already interacted with officers. Additionally, individuals may not give their true opinions but provide what they believe to be socially acceptable answers. For example, if a person is asked by a police officer whether he or she thinks the police are doing a poor, adequate, or good job, the respondent is more likely to rate the police higher than if a civilian were asking the question. Behavior and Experience Surveys are also useful for gathering data on individuals' behaviors and experiences. Common topics addressed in surveys of this type include: o Crime-prevention actions taken. o Experiences as victims of crime. o Experiences with the police. o Experiences with problems. There are limitations to using surveys to obtain accounts of behaviors and experiences: people forget and they misrepresent. People often have poor memories of experiences that occurred long before a survey is administered. To overcome the difficulties people have in remembering details of crime experiences long past, the Bureau of Justice Statistics queries people only about experiences that occur during the 6 months preceding an interview. People also have a tendency to reinterpret the past based on more recent experiences. For example, a citizen who has had multiple police contacts over a period of time is likely to base his or her opinion of the earliest contacts partly on later experiences. Finally, people may distort or downplay their responses when asked to recall behaviors that embarrass them or, alternatively, inflate and embellish a victimization or heroic experience if they believe this will make them look better. It is possible, for example, that when drugs were considered "cool" by teenagers, high school seniors interviewed about drug use overreported their rates of drug use; once drugs became socially unacceptable, high school seniors may have underreported their drug use. Nevertheless, it may be worth some expense to conduct surveys to learn about certain experiences. If an event is unrecorded, a survey may be the only way to learn about it. For example, to learn about victimization experiences of people who do not report crimes, one would have to interview them; their experiences will not be found in a crime report. Surveys have been used extensively to learn about drug use by people who have not been arrested (they represent the vast majority of illicit drug users). Despite the limitations of poor memory about victimization experiences or the current tendency to downplay involvement in drugs, there are few alternatives to conducting such surveys to obtain these data. Sometimes there are less expensive, alternative sources of information than surveys; surveys should not be used when better alternatives are available. For example, officers might evaluate the need for a project to control rowdy teenagers at a shopping plaza by conducting surveys of store personnel and customers, asking their perceptions of the problem. Other approaches that might be cheaper and more reliable would be to compare gross sales receipts at stores before and after a control project, to use traffic counters at entrances to the parking lot, or to count people entering the stores. Such measures provide direct evidence of behavior and are not subject to memory lapses and other distortions. However, if officers in the example above did not care about shopping behavior, but instead were concerned about how the merchants and shoppers perceived the police, then surveys would be the preferred tool. Characteristics A third way in which surveys are useful is in revealing characteristics of groups of people. This can include: o The characteristics of people living in a neighborhood. o The background of victims of crimes. o The personal history of offenders. Often there are many alternative sources of this type of information. Census data provide a great deal of information about neighborhoods. Characteristics of victims can be obtained from offense reports. Offender background information can be obtained from arrest reports. Police are often interested in using this information in conjunction with data on attitudes and opinions or behaviors and experiences. As a result, questions about the characteristics of a person answering a survey are often included on survey forms. The limitations that apply to experience and behavioral questions also apply to these data: people may forget or distort the answers. Limitations on the three types of survey data--the attitudes and opinions of respondents, their behavior and experience, and personal characteristics, including personal history--can be minimized through various survey methods. For example, civilian interviewers may be substituted for uniformed officers to obtain more accurate data on people's attitudes toward the police. A variety of questions may be asked on the same topic to check consistency. Much of what follows in this monograph are descriptions of techniques developed to minimize problems of poor memory and distortion. 2. The research reports describing these findings are not specific about what the officers actually did with the information they gained from the surveys. If they did little or nothing with the information, then the reduction of crime and fear of crime was a result only of the visibility of police or the interaction of the officers with members of the public during the survey. If, however, the officers addressed the problems revealed in the surveys, then the reduction of crime and fear may have been caused by the problem-solving efforts alone, the police-citizen contacts alone, or a combination of the two. 3. Police in one city asked residents of a housing complex to rate their satisfaction with the police and the public housing authority. The police received good marks but the residents were very critical of housing officials. Separately, the housing officials had completed a similar survey in which the police fared poorly (no questions were asked about housing officials). Although the questions in the two surveys and the methods of administering the surveys were not the same, some of the differences in the ratings of the police may have been due to who asked the question. Who Will Be Surveyed? Police officials often assume that they must interview everyone in the community to obtain useful information from a survey. If this were true, surveys would be so expensive that few would be conducted. In fact, survey research methods are designed to get the maximum reliable information from interviewing the fewest people. The procedure for selecting people to be interviewed is called sampling. This section discusses sampling procedures. A sample is a subgroup of the larger population that interests the researcher. During presidential campaigns the population of interest to pollsters are those citizens who have registered to vote. The pollster's goal is to interview only a portion of this population--the sample. Members of the sample who answer the survey questions are known as respondents. Based on the respondents' answers, the pollster will make inferences about the population. For example, if 53 percent of the people interviewed say they will vote for candidate X, the pollster might assume that about 53 percent of the registered voters (give or take a few percent) intend to vote for candidate X. Political polling and survey research is far more complex, but this is the basic approach used. A measurable characteristic of the population to be studied is called a parameter. The investigator defines a population sample and makes an estimate of the parameter. The estimate is called a sample statistic. For example, in estimating the proportion of the voting public who would vote for candidate X, this proportion of voters is a parameter. (Other parameters might be the average age of voters, how they have voted in the past, or how they feel about other issues.) The estimate of voters favoring candidate X, or 53 percent, is a sample statistic. To select a sample of respondents from a population, the investigator needs a list of the members of the population. The list from which the sample is drawn is called the sampling frame. If one is conducting a survey of sworn officers in a police department (the population), a list of department members could serve as the sampling frame. Unfortunately, in many surveys, a list of population members is seldom available to use as a sampling frame. In these situations a sampling frame must be created. The following example is drawn from the experiences of a number of police agencies that surveyed residents of public housing. It illustrates the process they used to develop and use a sampling frame. Example: A number of police agencies have used surveys to learn about the concerns and experiences of people living in public housing complexes. The residents of the housing complex comprise the population. Officers wanted to interview adults because children were less likely to give reliable answers. Further, residents who were not named on the lease were likely to avoid contacts with the police for fear that they might be kicked out of the complex. So the population was narrowed to those adult leaseholders living in the complex. A list of adult leaseholders is sometimes available from the public housing authority. But drawing a sample of respondents from this list could result in households of several adults being disproportionately represented in the sample. So instead of using the list of adult leaseholders, the police used a list of occupied apartments. The sample was taken from this sampling frame, and officers interviewed only the person who claimed to be the head of the household. The example cited above was a rather straightforward approach, and the sampling frame was a reasonable approximation of the population of interest. However, this is not always the case. To conduct a victimization survey, for example, one might start with a list of victims of crimes and then draw a sample. Because crime reports may list the names of victims, one might be tempted to start there. There is a major problem with this approach: some victims of crime never call the police, so no incident report is created. Nonreporting victims would be excluded from the sample, and these individuals could have had very different experiences from those persons who reported an offense. To overcome this difficulty, researchers draw their sample from the entire population of the area in question. During the interview they ask several questions about victimization experiences. When a respondent indicates that he or she has been a victim of a crime then a series of more detailed questions is asked. Because most people have not been crime victims, only a small percent of the total sample are asked the special set of questions. Although this is an expensive process, it is the only reliable method of finding victims who do not report crimes. The San Diego Police Department has been conducting a study of the characteristics of drug-dealing locations. This does not involve interviewing individuals, but it does provide an illustration of how a representative sample can be obtained without having a list of respondents. The study objective was to determine if the physical characteristics of drug-dealing locations were significantly different from locations in the same neighborhood without drug dealing. Answering this question could help determine what might be done to prevent drug-dealing locations from developing. A representative sample of drug-dealing locations was needed, but no listing of such locations was available. One option was to use arrest, warrant, and raid information in police records to identify all of the possible drug-dealing locations in the area. This option was eliminated as prohibitively expensive. Instead, a list of all 340 census blocks in the area was used as a sampling frame, and a random sample of 200 blocks was selected. For each address in the sample of blocks, information from police records (citizen calls about drugs, drug arrests, warrants, and field interrogations) and from patrol officers and narcotics investigators was used to list the dealing and nondealing locations. A random selection of dealing locations was made from this sampling frame. Although there is some unknown chance that a drug-dealing location may be missed (or that a nondealing location may be mistaken for a dealing location), the use of multiple indicators minimizes this risk. The use of multiple indicators of drug dealing is expensive, so a strategy of limiting the number of addresses analyzed prior to sampling was used. In this way, police achieved a representative sample of dealing locations. Simple Random Sampling There is always a chance that the sample is not representative of the population regardless of how it is selected. But some sampling methods minimize this risk and give the researcher the ability to calculate the probability that the results from the sample are very close to the results that would be obtained if the entire population were surveyed. In the preceding example, random selection was used to choose blocks and then addresses within blocks. Random selection is the most common way of obtaining a representative sample. It is popular for four reasons. First, random sampling removes the possibility that the person selecting the respondents inadvertently biased the sample; that is, selected a sample that was not representative of the population. Note that sampling bias has not necessarily been eliminated; only the sampling bias due to conscious or unconscious actions of the person doing the selection has been removed. If the sampling frame is biased, then the sample will not be representative of the population, even if random selection is used. Second, random sampling is simple to use. All one needs to do is assign an identification number to each member (person, address, etc.) on the sampling frame. The members can be numbered in some sort of sequence (alphabetically, for example) or in an arbitrary manner. All that matters is that each member has a unique number (not shared by another member). A random numbers table (often found in introductory statistics books) or a computer program (most spreadsheet programs designed for microcomputers have simple commands for selecting random numbers) can be used to select the respondents for the sample. Third, statisticians can estimate the probability that the results learned from the sample (for example, the proportion of respondents who believe drug dealing is the most important problem in their neighborhood) are really characteristic of the larger population and not due to chance. Fourth, there are many variations on random sampling--generally referred to as probability sampling--that fit the needs of most survey efforts. For example, if a police chief is interested in learning what problems are of particular concern to certain ethnic and racial groups in the city, a stratified random sample might be useful, especially if the ethnic and racial groups to be surveyed represent a small part of the total population. If Hispanics make up 5 percent and blacks 15 percent of the population, and the remaining 80 percent are whites, then a simple random sample of the population would reflect these proportions. This would mean that relatively few Hispanics and blacks would be in the sample. Either a large sample could be selected to make sure enough Hispanics and blacks were in the sample to draw valid conclusions, or the city could be divided into three geographic areas corresponding to the concentrations of the three groups. Three random samples of equal sizes could then be drawn, one from each area. Each sample would be representative of the area from which it was drawn, but in combination the three samples would not be representative of the city population: blacks and Hispanics would be overrepresented and whites underrepresented. This is not a serious problem because there are mathematical methods for compensating for such distortion. If one can calculate the probability of members of each area being selected for the sample, then the responses can be weighted by the number one divided by this probability. The weighting process is easily explained by an example. Suppose the city in a hypothetical example has 100,000 adults. The census tracts comprising contiguous black communities have an adult population of 20,000. (Not all blacks live in these census tracts; some whites and Hispanics live there.) The adult population of the census tracts that make up the Hispanic neighborhood numbers 6,000. The remaining 74,000 adults (primarily whites but some blacks and Hispanics) live in the other census tracts. A statistician has calculated that an appropriate sample size is 300 adults from each area. This means that the probability of selecting a respondent of any race in the black neighborhoods is 300/20,000 = 0.015; the probability of selecting a respondent of any race from the Hispanic neighborhood is 300/6,000 = 0.05; and the probability of selecting a respondent of any race from the other neighborhoods is 300/74,000 = 0.004. Another way of looking at these figures is that every respondent selected from the black neighborhood represents 66.67 ( = 1/.015 = 20,000/300) adults. Each respondent from the Hispanic neighborhood represents 20 adults (1/.05 = 300/6,000) and each respondent from the other neighborhoods represents 250 (=1/.004 = 300/74,000) adults. Multiplying the answers from respondents times the number of people they represent weights the data so that the results of the survey analysis reflect the proportions found in the population. Data from respondents sampled from black neighborhoods would be multiplied by 66.67, data from Hispanic neighborhood respondents would be multiplied by 20, and data from other neighborhoods would be multiplied by 250. This example of a stratified random sample is just one of many types of probability sampling that can be applied. Although these forms of sampling can be very complicated to design, they often result in substantial savings in data-collection resources, and the findings that result from the data analysis are often more definitive than the findings from unstratified sample designs. In short, the benefits of bringing in an expert in sampling may be far less than the savings of a specially designed survey. Nonrandom Sampling Various nonrandom sampling procedures may be simple, but they often leave open the possibility of drawing a biased sample. Furthermore, one cannot know how far the estimates obtained from sample statistics differ from the true population parameters. Nonrandom sampling procedures include: o Convenience sampling--selecting respondents based on how easy it is to get in touch with them. For example, to get an estimate of residents' views of community problems, an officer interviews everyone who shows up at a meeting held to discuss community problems. If the people who come to the meeting are not representative of the general population of the community--for instance, have higher incomes, show greater willingness to get involved in community affairs, have higher education, are less fearful of retribution, or have different perspectives on community problems--then the results of this convenience sample will not reflect the views of the community at large. This type of survey may provide useful information on those attending the meeting, but it is risky to generalize from such a sample to a larger population. o Accidental sampling--selecting respondents based on some arbitrary characteristic. For example, to learn how neighborhood residents feel about drug problems, an officer talks to every pedestrian he or she meets on a street during a tour of duty. Here, again, the respondents are unlikely to be representative of the population of the community. The type of person an officer meets on the street will depend on the time of day and where the officer is patrolling. In the evening, relatively few people may be on the street, and they are likely to be disproportionately male and young. During the day, more women may be present, but men and women who work during the day will be absent. People who are fearful of street behavior will also be harder to find on the street, so the elderly may be underrepresented. The location of the patrolling will also have an effect. If the officer is patrolling in front of bars on a Friday night, then he or she will talk to a very different population than when patrolling near a church on Sunday. There may be good reasons to interview people in the street, but these respondents are likely only to be representative of others who have similar lifestyles and are unlikely to represent the general population of the community. o Systematic sampling--selecting every nth person or household from the sampling frame. For example, to learn about community fears of crime, an officer decides to talk to people in every fifth house. This may yield a representative sample if there is not a systematic layout of the neighborhood that corresponds to the selection rate (every fifth house). For example, if the 1st and 10th house on every block is on a corner, then each house sampled will be at the same relative position on its block. Because corner houses may have a greater likelihood of being burglarized, the closer a house is to a corner the greater the chances that a household member knows a burglary victim. This could bias the level of fear estimated from the sample. However, if there is no correspondence between the sampling rate and the layout of the neighborhood, then this strategy can be quite useful. For example, if some blocks have 4 houses, some 10, and others 7 (in no particular order), then systematic sampling may provide a representative sample. Randomly selecting the starting point--in this example, the first house in the sample--removes the biases of the person doing the sampling. Although systematic sampling is clearly superior to accidental and convenience sampling, a great deal of effort must precede its application to ensure its suitability. The Madison, Wisconsin, police department surveys its "customers" weekly by sending a survey to everyone mentioned in every 50th case report (Couper and Lobitz, 1991). o Purposive sampling--selecting members of the sample to achieve a specific objective. For example, to learn how drug users find out where drugs are available, a group of arrestees is interviewed after an undercover reverse buy (that is, plainclothes investigators take over a dealing location and sell real or fake drugs to customers, who are then arrested). This group of suspected users may not be representative of a larger population of interest--and it is impossible to determine whether or not they are--but the information they provide may be potentially useful for learning about drug markets. Often, purposive samples are used to pretest survey questionnaires. In these situations a draft questionnaire is given to individuals especially selected because of some concern as to how they would complete the form. For example, a questionnaire to be read and filled out later by respondents may be given to a few individuals with little education to determine if it is understandable. Or, a draft questionnaire asking for information on criminal behavior may be given to a number of known offenders whose responses can be checked against official records to determine if respondents are likely to fill it out completely and honestly. In summary, the objective in using a purposive sample is less to make an estimate about some larger population than to learn a specific set of facts about a few hard-to-find individuals. One hopes the group selected is representative of some larger group of similar people, but because one cannot be certain, no claim of representativeness is made. This differs from accidental and convenience sampling in that those strategies are associated with a claim-- usually insupportable--that the sample is representative and no attempt is made to select people with a special characteristic of interest. This section discussed the rationale behind sampling, introduced some standard terms common to survey research, and reviewed some basic sampling strategies. Random sampling, or one of its many variants, is clearly the preferred method, although systematic sampling is often useful if certain conditions can be met. Purposive sampling is also useful in answering some types of questions, but inferences to larger populations are suspect or unreliable when this strategy is followed. Finally, accidental and convenience sampling should be avoided because results based on these strategies can be highly misleading. The next section discusses how many respondents to select for a sample. In that section and those that follow, assume that some form of random sampling is used unless specifically stated otherwise. How Many Will Be Sampled? One of the questions most commonly asked by those conducting a survey is, "How many respondents should be in my sample?" This question is so important and goes so close to the heart of the rationale for sampling that it seems strange to report that the question has no definitive answer. Except for the simplest surveys, only ballpark estimates can be made of optimal sample size--that is, the smallest sample that will provide the most valid and reliable results. Books have been written on how to determine optimum sample sizes and the many formulas that can be used, depending on the sampling procedure and the known characteristics of the population. In this brief primer on survey methods, only the primary factors that must be taken into account in determining sample sizes will be described. There are two factors that are important in determining the sample size: cost and confidence in the findings. For any random (or systematic) sampling procedure selected, the greater the confidence desired in the results, the more the survey will cost. This is because confidence depends in large part on the size of the sample once the sampling strategy has been selected. As noted in the previous section, however, for any given situation there are sampling strategies that provide high confidence at relatively low cost, other strategies that offer high confidence at a high cost, and still other strategies that are costly yet produce little confidence. For this reason, it is worthwhile to discuss how various factors influence sample sizes and how the selection of a particular type of sampling strategy can reduce the sample size needed for a desired level of confidence. Until now the term "confidence" has been used without defining its meaning in statistics. As stated in the previous section, the researcher's objective is to estimate a population parameter with a sample statistic. If 86 percent of the adults in a sample of city residents believe that the police are doing a good or very good job at controlling drug abuse, then one might like to conclude that 86 percent of all adults in the city feel this way. In reality, it is highly unlikely that exactly 86 percent of the adults in the population feel this way, but the percentage should be close--within a few percentage points. One might claim to be very confident that the police-approving population percent lies within a range or interval, for example, between 83 percent and 89 percent. Statisticians commonly express such levels of confidence in percentage terms, so that an investigator might report the results as follows: "We are 95-percent certain that the percentage of the adult population who believes the police are doing a good or very good job is between 83 percent and 89 percent." Or alternatively, "There is only a 5-percent chance that the true population parameter is higher than 89 percent or lower than 83 percent." Commonly, researchers attempt to achieve 90-percent, 95-percent, or 99-percent confidence (with 95 percent and 99 percent more common than 90 percent). Confidence, then, has two components: the size of the interval in which the population parameter is sought and the level of confidence that the population parameter actually falls within the interval. In the example above, the interval is 3 percent on either side of the sample statistic of 86 percent, and the level of confidence is 95 percent. As one narrows the confidence interval, the larger the sample size needed for a given level of confidence. In other words, the more precisely the population parameter is defined, the more respondents will be needed. Equally true, the greater the level of confidence desired at a given range or interval, the larger the sample size needed. Therefore, a survey requiring a high level of confidence in a very precise estimate will be more costly than a survey designed to get a ballpark estimate at a moderate level of confidence. Levels of precision and confidence are usually set early in the planning of a survey and are adjusted as more details become known about the sampling strategies available and the likely costs. In the example above, the objective was to determine the opinions held by the adult population about the police. Included were all the adults and all their views--pro and con. If concerns were limited to the way a small portion of the population felt about the police compared to the larger group, a larger sample would have to be selected for a given level of precision or confidence. For example, to contrast the opinions about police held by victims of crime with those held by nonvictims, a very large sample is needed because the experience of victimization is relatively rare. Or to use another example, a researcher plans identical surveys in two large neighborhoods at opposite ends of town to determine neighborhood problems. Of special interest are the problems of the elderly. In neighborhood A, census data show that half the adults are over age 65; in neighborhood B, only about 10 percent of the residents are in this age group. To obtain similar levels of confidence and precision about problems of the elderly in the two neighborhoods, a much larger sample size will be needed in neighborhood B (where the elderly are relatively scarce) than in neighborhood A. In short, to calculate the frequency of a rare attribute of respondents, a larger sample size is needed than if the attribute is common. Stratified random sampling is well suited to this type of situation. For example, if census data or information from agencies that serve the elderly indicate that in neighborhood B the elderly were concentrated in a cluster of apartment buildings, then neighborhood B could be stratified or subdivided so that this cluster is in one study area. Respondents could then be randomly selected by areas within neighborhood B in such a way as to assure that a large enough sample of elderly were included. The sample could then be weighted to make valid population parameter estimates. This could result in a lower cost survey without sacrificing precision and an acceptable level of confidence in the results. As a rule of thumb, if an attribute of interest is held by fewer than 40 percent or more than 60 percent of the population to be studied, then stratifying the sample may provide a cost savings. Note, however, that the feasibility of stratifying depends on the existence of a suitable sampling frame. Without a sampling frame, stratification is very difficult. In the examples thus far, two population groups have been compared, but the same logic applies to multiple categories. Suppose that the population of neighborhood X consists of equal members of adults aged 18-30, 31-45, 46-60, and 61 and over (25 percent of the population in each group). By contrast, neighborhood Y is overwhelmingly younger: 60 percent of persons are 18 to 30 years old; 20 percent are 31 to 45 years old; 15 percent are 45 to 60 years old; and 5 percent are aged 61 and over. If the views of elderly persons are particularly desired, then a larger sample would be needed for neighborhood Y--unless a stratified random sample can be used--to achieve the same level of precision and confidence. But if the views of the elderly are not a major focus of this survey, it is possible that neighborhood X would require the larger sample. Why? This would be required if age appeared related to interviewee responses (for example, their opinions about the police). Neighborhood X is more heterogeneous than neighborhood Y, or, in the terms of statisticians, neighborhood X has a higher variance in age than neighborhood Y. This suggests that the variance in opinion about the police in neighborhood X is higher than in Y. And the higher the variance in the responses, the greater the sample size needed to achieve a predetermined level of precision and confidence. In this situation, stratifying on age in neighborhood X might be useful to keep costs down. Finally, the greater the number of comparisons to be made in subgroups within the sample, the greater the sample size needed. In the previous examples, comparisons were made only among different age groups. In the first example, there were two groups in question, in the second example there were four. A sufficient number of respondents is needed in each category so that the comparisons can be done with a predetermined level of precision and confidence. This suggests that larger sample sizes will be required to answer the questions posed in the second example (four groups) than to answer the questions posed in the example with two groups. Note that the number of groups compared is determined by investigators conducting the survey, based on their substantive concerns. Note also that the number of groups can increase rapidly. In addition to comparing the four age groups in the second example, a researcher may want to determine if female and male respondents at each age view the police differently. That would require eight groups. In addition, race may be a factor. If three races are represented--white, black, and other--then a comparison of age, gender, and race categories would involve 24 groups (4 age x 2 genders x 3 races). Clearly, one needs to define the most important comparisons prior to collecting the data. Furthermore, one should determine if certain comparisons are really needed. For example, although there is interest in comparing the views of whites, blacks, and others, is there a genuine need to compare the views of young white males with those of middle-aged black females? If such a need actually exists, then a larger sample size is required. It is worth noting that unless the population is small--fewer than 100, for example--the population size has very little influence on the size of the sample needed. The sample size needed to obtain a desired level of precision and confidence in a neighborhood of 800 residents will be about the same as is needed to make the same estimates at the same level of precision and confidence for a neighborhood of 2,000 people, everything else being equal. If the population is very small--perhaps 50 or fewer--there may be no point in sampling because everyone could be interviewed. How Will the Respondents Be Contacted? The previous sections covered the tradeoff between costs and quality and ways to keep costs down while maximizing precision and level of confidence. It was demonstrated how selection of a sampling strategy and sample size can be chosen to get the most "bang for the buck." This section will continue the discussion by describing how various methods of reaching respondents affect costs and survey results and how a data collection strategy that keeps costs down and achieves acceptable results can be selected. There are three basic types of data collection strategies: a questionnaire can be mailed to everyone in the sample for them to complete and mail back; the sampled respondents can be interviewed by telephone; or the researcher can go to the respondents (at home, in the office, on the bus, or wherever they are) and interview them in person. Mail Surveys Mail surveys are an inexpensive method of obtaining large sample sizes. Instead of having to pay someone to talk to respondents, the surveyor can simply buy postage stamps and envelopes. As usual, a person gets what he or she pays for. Mail surveys are notorious for having low response rates. The response rate is the proportion of sampled respondents who provide information. Many sampled respondents do not send back completed questionnaires; this effectively lowers the sample size. Low response rates have two problems. First, by lowering the number of respondents whose answers can be analyzed, the survey results will be less precise and there will be less confidence in the results. Second, if the nonresponses are not systematic, then the sample of returned responses can be treated as a random sample, but will be representative of the population. Often, people who do not respond have common characteristics related to certain facts or opinions the survey is trying to discern. For example, if nonresponders are uneducated, poor, and suspicious of the police, then a mail survey soliciting public attitudes toward the police will misleadingly overestimate the confidence the public feels in the police; many of those harboring negative feelings will not send back the questionnaire. If the response rate is high--about 90 percent, for example--the bias due to selective responding is usually so minor it can be ignored. But if 40 percent or more of the sample do not respond, then there are serious questions about how well the responses represent the larger population. Unfortunately, mail surveys are more likely to have response rates of 60 percent or less; an 80 percent or greater response is rare. For example, a questionnaire mailed to 5,462 residents of Lansing, Michigan, by the police department yielded 2,328 returned questionnaires, a response rate of 42.6 percent (Trojanowicz et al., 1987). There are several approaches to raising the response rate of mail surveys. First, publicizing the mailing can sensitize the public, reminding them to mail the questionnaire back. However, because individuals are selective about the sources of information they pay attention to, this also could unbalance the sample. For example, if publicity is made through press releases to the local news media, only those persons who read the paper, watch television news, or listen to radio news will get the message; others will not. If publicity is provided through local churches, then churchgoers may be overrepresented in contrast to adults who do not attend church. If publicity is to be used, it must be conducted in a broad enough manner--via multiple information outlets--so that important groups of respondents are not missed. However, one category of people will always be missed by a mail survey: people without addresses such as the homeless and those who move frequently (members of the military and college students, for example). Another method of improving the response rate to mail surveys is to send a followup letter to all respondents within a short period of time--1 week to 10 days after the mailing of the questionnaire. The followup mailing reminds the recipients that the survey is important and asks again for their assistance. Because anonymity is important, the letter should mention that the sender does not know who already has and has not returned the questionnaire; it should express thanks to those who have already returned it. A well-designed questionnaire--easy to read, appealing to the eye, and asking questions of interest to respondents--is likely to get a higher response rate than a poorly designed one. The following section will discuss how to construct the survey questions, but first the reader should recognize the importance of nonsubstantive considerations. Seemingly trivial considerations-- such as using colored paper and colored ink (instead of black ink on white paper); employing a larger, more impressive type face than is normally produced by a typewriter or word processor; generously providing white space instead of packing questions tightly together; and choosing a good quality reproduction and printing process-- influence the rate of returns. In other words, a typewritten, photocopied questionnaire will, in all likelihood, produce fewer responses than a version with a more professional look. Finally, remember that it costs money to return questionnaires by mail. To improve the response rate, enclose an addressed, stamped envelope for the purpose. Instead of pasting stamps on all the return envelopes, check with the local post office to see if a preprinted return stamp might be cheaper and easier to use. Every week the Madison, Wisconsin, police department sends out surveys to a systematic sample of people who have contacted the police that week. Enclosed with the survey instrument is an addressed, stamped return envelope. The mailing has a response rate of 35 percent to 40 percent (Couper and Lobitz, 1991). Apart from the response rate, mail surveys have another deficiency that may negate the cost savings: there is no opportunity to assist respondents who have trouble interpreting the questions. If a question is ambiguous, respondents will either make their own interpretations (which may be inaccurate) or give up and leave the question blank. Therefore, special attention needs to be paid to ensuring that the wording is clear and unambiguous. If many of the individuals in the sample have limited education, the questionnaire should be tested on a group of similar persons to determine if they can read the text and understand how to answer the questions. If a sizeable proportion of the population sample is believed to be illiterate, do not use a mail survey. It is also important to determine whether a significant proportion of the respondents are more conversant in another language than English. If so, consider translating the questionnaire into that language and conducting a separate pretest of the translated questionnaire to ensure that nothing was lost or added in the translation. Telephone Surveys Instead of mailing survey questionnaires to determine the public's views about the police, the Reno, Nevada, police department telephones a random sample of citizens every 6 months. Calling respondents by phone and asking them questions is more expensive than sending them a questionnaire, but response rates are usually higher, often 75 percent to 90 percent, and the quality of the answers is usually higher. Remember, however, that if many of the sampled respondents do not own telephones, this strategy will not work. This is an important consideration for surveys conducted in public housing complexes and in low-income neighborhoods. Response rates for surveys by telephone are usually higher than for mailed surveys because the investigator can call back repeatedly until a respondent answers, and people are less likely to hang up on an interviewer than to throw a questionnaire in the trash. In practice, a researcher cannot continue to telephone people indefinitely; a limit must be set on the number of callbacks--two, three, or four--to be attempted. The more callbacks the higher the response rate-- but the higher the cost. An advantage of using telephone surveys is that if respondents are unclear about the meaning of a question they can seek clarification from the interviewer. This means that all the interviewers must provide the same answers to the same questions, consistently. Obviously, it is better to ask questions that do not require clarification. Thus, as is the case with mail surveys, a great deal of attention should be paid to how questions are worded and how people may interpret certain phrases. This is especially important when dealing with different ethnic groups and people whose first language is not English. If a significant proportion of the population sample does not speak English, the questionnaire should be translated, and interviewers fluent in the other language should be recruited. Like the mail survey questionnaire, the phone survey instrument should be pretested on a small group of individuals believed to be similar to persons in the sample. Additionally, individuals conducting the interviews need guidance and training. This should involve standard scripts that interviewers follow when introducing themselves to respondents and answering questions they may have. Although some discretion should be left to interviewers so that they do not appear to be reading a script, a fair amount of standardization is needed to ensure that the interviewers are not influencing--intentionally or unintentionally--the responses. Training should cover how to conduct telephone interviews in general, as well as how to administer the particular instrument to be used in the survey. Such training usually involves some classroom time for reviewing the questionnaire and answering questions about it. Several rounds of rehearsals should then be scheduled. The first round might involve trainees interviewing each other to get used to the questionnaire. A second round might consist of the trainees interviewing various department members by telephone. In both the first and second rounds it is sometimes useful for the person simulating the respondent to ask difficult questions so that the trainee-interviewer learns how to cope within the guidelines offered. Finally, trainees should conduct the pretest. This will suggest revisions in the questionnaire, or survey instrument, and will reveal difficulties interviewers may encounter. Telephone interviewers can be either department members or people especially recruited for this purpose. With the collaboration of a local university, the Reno police use both students and volunteers. The only cost of their service to the Reno Police Department is for the coffee and doughnuts the volunteers consume. Inperson Interviews The highest response rates can usually be achieved with personal interviews, which usually are more costly to administer. Personal interviews allow repeated followup and clarification of questions. Particularly if a questionnaire is complex, personal interviews may be required to ensure that respondents can supply the answers. Inperson interviews are particularly useful if many of those in the sample have no telephone. In addition, an interviewer can directly observe the respondent and sometimes collect information without asking a question. As with telephone surveys, training and guidance need to be given to the interviewer. Unlike phone and mail surveys, however, the interviewers' safety in the field must be taken into consideration if the questionnaire is to be administered in high-crime areas. When it comes to choosing who will actually administer the survey, there are a number of alternatives. Perhaps the most expensive alternative is to have police officers conduct the survey. This alternative has another drawback: respondents may not give candid answers to some questions. For example, questions about attitudes toward the police will be inflated in the positive direction, and questions about illegal activities of respondents will produce underestimates of the true level of these activities. However, officers may be very useful in getting information about neighborhood problems. Not only are respondents likely to welcome an officer's asking about such problems, but officers will also be able to ask followup questions and probe for important details that an inexperienced, unsworn interviewer might leave unexplored. In addition, as a community relations tool, officer-interviewers can be effective apart from the information they gather. Surveys give officers a chance to meet community members they seldom have a chance to talk to, and the meeting is neither confrontational nor an emergency. Finally, there is some evidence that door-to-door interviews by officers may have a crime suppression effect (Pate, 1986; Uchida, Forst, and Annan, 1990). To the extent that community relations and crime-suppression effects are more important than information gathering, and to the extent that these effects do not bias the estimates of population parameters, then survey interviews conducted by officers may be worthwhile. The Baltimore County Police Department routinely has officers assigned to special problem-solving units conduct neighborhood surveys. The Newport News, Virginia; Tampa, Florida; and Philadelphia, Pennsylvania, police departments, among others, have used uniformed officers to collect survey information about neighborhood problems. In Tulsa, Oklahoma, and Atlanta, Georgia, unsworn police employees have been used to conduct surveys. It is usually cheaper to use civilians to collect survey data. They are often easier to schedule, and the quality of their interviewing (assuming competent training) is high. Furthermore, there is less chance that their physical presence will influence the responses of the respondents. However, the community relations value of civilians is lower, they probably have no crime-suppression effect, and special measures may be needed to provide security in high-crime areas. Civilians can be recruited from a number of sources. One might contract with a professional survey research firm. The firm would assist with all phases of the survey, from designing the sampling strategy to analyzing and reporting the data. Interviewers are usually well trained and very professional. The out-of-pocket cost to the police department is relatively high compared to other alternatives. If a number of civilians are employed by the police department's crime prevention or community relations units, these individuals may be good interviewers and may be less expensive to use than sworn officers. One might encounter many of the same problems of biased data without some of the benefits of using officers; nevertheless, this approach can be useful. The Atlanta Bureau of Police, for example, used civilian employees to conduct a survey of public housing residents and determine neighborhood problems. Volunteers also can be used effectively for such surveys. For one project, the San Diego, California, police department gained the assistance of a local university professor in recruiting students to conduct a survey on drug problems in a high-crime area. Several officers were assigned to each city block while the interviewers were at work. There were no incidents, and the information was very useful. It is also possible to recruit retirees or members of citizens' groups to conduct surveys. For example, a police department might interest a neighborhood group in interviewing residents of its own area. Care must be exercised to ensure that these volunteers adhere to the interview protocol and do not try to influence their neighbors' responses. Summary There are a number of alternative data collection strategies that can be used, depending on budget and quality concerns. For ballpark estimates that will not be used in critical decisionmaking, a well-constructed mail survey may be useful. But if relatively exact estimates of population parameters are needed, or if the results will be used to make decisions that will have major impacts on people's lives, telephone and inperson surveys are more appropriate. Telephone and inperson surveys require that the interviewers be trained and provided with guidance on how to administer the survey questionnaire. Regardless of the method of administering the survey, a pretest should be conducted before a final instrument is developed. Whichever strategy is adopted, care must be taken in the construction of the survey instrument to ensure that respondents understand the questions and can provide honest answers. This is the subject of the next section. What Questions Will Be Asked? Until now, the construction and content of questionnaires and data collection instruments have not been considered. It should be clear from the previous discussions, however, that the questionnaire must be tailored to fit the needs of the data collection strategy. For example, a survey instrument suitable for a telephone interviewer will not be suitable for a mail survey. General Considerations There are a few key issues to resolve before developing a questionnaire: o What are the specific purposes of the survey and what kinds of questions are most likely to yield responses that are consistent with those purposes? It is important to clarify the goals of the survey project to minimize the number of questions asked. For example, a police department is planning a survey of 200 city residents to examine their fears of crime and their perceptions of the magnitude of the drug problem. A police official asks whether the survey should also include a question about recent victimization experience because victims may be more fearful. Victimization is relatively rare; few respondents in a sample of 200 will report having been victimized. Should such a question be included? Probably not, unless a high-crime neighborhood is being surveyed. Without clear goals, the number of questions tends to mushroom. This increases the time it takes to administer each survey, which is a burden on the respondent and the interviewer, and may reduce the response rate. With too many questions asked, many of the responses are not analyzed and the time expended to collect the information is wasted. In short, fight the temptation to include everyone's favorite question, which is often prefaced by, "As long as we are doing a survey, we might as well ask ..." o How will the survey be administered--by mail, phone, or in person--and how much opportunity will the respondent have to seek clarification? Self-administered questionnaires--either sent through the mail or handed to the respondent by a personal interviewer--need to be professional in appearance, easy to read, and easy to complete. Make sure that the lines and spaces for answers are clearly marked and far enough apart that stray marks do not confuse someone reading the completed questionnaire. Instructions should be short, clear, and polite. They should also be clearly marked so that someone who rapidly reviews the questionnaire does not accidently skip over them. If it is impossible to ask a simple question, easily understood by respondents who fill out the form by themselves, consider eliminating the question or using a different data collection strategy. Once the questionnaire is completed and returned, assume that the respondent answered the question that was posed, not his or her own interpretation of the question. If this assumption seems implausible, then asking the question was a waste of time. If an interviewer is administering the instrument, the questionnaire does not have to have a professional look, but it still should be easy to follow and be designed to minimize errors when an interviewer marks down the responses. o Who will fill out the survey instrument--the respondent or an interviewer--and what is his or her educational level, training, and proficiency in the language of the survey instrument? A questionnaire that examines problems in a community of college students will be very different from a questionnaire developed for a similar study in a low-income public housing project with a high proportion of high school dropouts; this is true even if the surveys are designed to obtain the same basic information with the data collection strategy. If the survey will cover an educationally heterogeneous population, then a single questionnaire should be developed that will be understandable to all respondents. If an important segment of the population does not understand English, then separate, translated instruments need to be developed. o How much time will it take to complete the survey and is this a reasonable amount of time to impose on respondents? When persons are asked to complete a questionnaire or answer some questions for a survey, it is an intrusion into their lives; it takes up their time and makes personal inquiries. Most people are willing to allow such an intrusion if it is for a good cause and the time burden is not too onerous. Ten to fifteen minutes to complete a questionnaire is reasonable, but if the subject is important (concerns about crime, drugs, and personal safety usually are considered important), the survey instrument is designed well, and the interviewer well trained, then survey times of 30 to 45 minutes may be possible. The time it takes to complete a mail survey is only slightly linked to the number of pages in the questionnaire. It is better to have a multi-page, well-formatted instrument than a short, cramped one. For telephone and personal surveys, the interviewers' time as well as the respondent's is consumed. It is a good idea to make time estimates when the instrument is pretested. Even at this late stage, the questionnaire may be edited to make it more cost effective. How To Ask Questions There are various ways of asking questions on a survey instrument, depending on the issues explored and the data collection strategy employed. Here are a few of the most common methods. Remember, however, as with other topics discussed in this monograph, there is much more to the subject matter than can be covered here. Consulting an expert and reviewing survey instruments that are acknowledged to be effective and well constructed is a highly recommended course of action. Asking about behavior or experiences Questions about behavior or experience generally are framed in one of two ways: "In the last n time period, have you done a specific action?" or (2) "In the last n time period, have you been involved in this specific type of event?" Note first that there is a time period for reporting the behavior or experience. Respondents will give very different answers to the following questions: A. In the last 6 months, have you witnessed what you believe to be drug dealing on the block in front of your residence? Yes 1 No 0 Do Not Recall 8 B. Have you ever witnessed what you believe to be drug dealing on the block in front of your residence? Yes 1 No 0 Do Not Recall q 8 With example A, there will be many fewer positive responses than with B, but they will reflect more recent experiences. Because example B has no time boundaries, drug dealing witnessed years ago but no longer occurring will be reported along with recent drug dealing. In short, without time limits for recalling behavior and experience, one can obtain very unreliable estimates. If you are interested in determining behavior or experience over a lifetime, that should be explicitly stated in the question. (Note: the coding numbers (1, 0, and 8) are used here to key answers for data entry.) C. Have you ever witnessed what you believe to be drug dealing on the block in front of your place of residence? Yes 1 No 0 Do Not Recall 8 Note that because people change residences often, example C may be answered Yes even if the drug dealing was witnessed in another neighborhood or city. The above examples carefully place the behavior in question in a defined location with reference to the respondent's home. This type of phrasing would be useful for determining the prevalence and concern about drug dealing for a community, possibly as part of a problem-solving effort. However, to determine how much drug dealing people witness in their daily lives, a more general question would probably be asked. D. In the last 6 months, have you witnessed what you believe to be drug dealing in the city of Pheasant Hills? Yes 1 No 0 Do Not Recall q 8 Example D provides a larger area for the respondent to report drug dealing, yet drug dealing witnessed outside the city is purposely excluded. However, there is an implied assumption that respondents know when they are inside the city of Pheasant Hills' boundaries. This may be plausible if there is very little or no drug dealing in adjacent communities or if the border of the city is clearly delineated. However, in a dense urban cluster of cities and towns, municipal boundaries may be hard to recognize. In such a setting, one would want to specify some other geographic delimiter that is obvious and well recognized. If it cannot be identified, the researcher has the choice of dropping the question or retaining it while recognizing that the respondents' answers will be difficult to interpret. Finally, notice that the definition of drug dealing is unclear in these examples. If the investigator is interested in learning about citizens' perceptions of drug dealing, these questions are reasonable. But if a more precise count of drug dealing in neighborhoods is needed, answers to these questions can overcount or undercount the number of times citizens see such behavior. Behavior that appears suspicious to respondents but does not constitute drug dealing is likely to be reported in the survey. On the other hand, street dealing that is extremely subtle or drug exchanges that take place in private settings will not be reported. To obtain more precise estimates of actual drug dealing on the street, a more precise question or set of questions needs to be asked. In example E, a series of questions have been asked about specific behaviors that have been witnessed and how often they have been witnessed. If a strict definition of witnessing drug dealing were employed, only affirmative answers to question E2.1 would be used as an indicator. Alternatively, an investigator might decide that a reasonable indicator of witnessing drug dealing is witnessing an exchange of money on more than one occasion (E1) and seeing the same people involved (an affirmative answer to E1.1). E. In the last 6 months, have you witnessed people exchanging money on the block in front of this residence? Yes 1 (If "yes", please go to E1) No 0 (Go to F) Do Not Recall 8 (Go to F) E1. In the last 6 months, how many times have you witnessed the exchanging of money in the block in front of your residence? One Time q 1 (If "one time", please go to E2) 2 or 3 Times 2 4 or 5 Times 3 6 to 10 Times 4 If two or more times 11 to 20 Times 5 Go to E1.1 More Than 20 Times 6 E1.1. Have you seen the same people making exchanges on different occasions during this 6-month period? Yes 1 No 0 Don't Know 8 E2. During any of the exchange(s) you witnessed, did you see an exchange of things other than money? Yes 1 (If "yes", please go to E2.1) No 0 (Go to F) Don't Know 8 (Go to F) E2.1.Did you see what you thought were drugs being exchanged? Yes 1 No 0 Don't Know 8 There are other criteria for witnessing drug dealing that could be selected from these questions, and there are many other questions that could be asked. What is important about this example is that it adds specificity to the definition of drug dealing. In fact, in this example, the definition of drug dealing is not left to the respondent. The survey's developer defines drug dealing based on the pattern of multiple-choice answers provided. For example, drug dealing might be defined broadly by accepting any answer of Yes to question E as evidence of dealing. More restrictive definitions can be developed from E1 by establishing a number of exchanges witnessed (2+, 4+, 6+, 10+, or 20+) as the definition, or the investigator might decide that a Yes to question E1.1 would define drug dealing. More strictly, a Yes to question E2.1 might be required to define drug dealing. Other definitions could be used as well, and one need not be bound by a single definition. When the data are analyzed, the researcher might compare results from using different definitions to determine whether the results change dramatically with the definition selected. A high level of specificity comes with a cost, however. What was once a single question is now a series of questions. More time will be needed to administer the survey, and more time will be needed to analyze the results. In some circumstances, one may be able to find a more specific single question, but often more questions will need to be asked. This example uses what is called a skip pattern. The respondent is directed to specific questions, depending on how he or she answered previous questions. If question E was answered No, then the respondent would have skipped over all of the questions in the example and answered some other questions that followed later. By answering Yes the respondent is directed to a set of questions that probes for more details. Skip patterns are quite useful when there are a series of questions that only some of the respondents will be asked. Researchers must ensure that respondents understand which questions they are and are not supposed to answer. In this example, written instructions follow an answer, along with directional arrows and a graphic design that directs the respondent to a box containing the next set of questions to be answered. Care must be taken when editing and revising questionnaires with skip patterns: moving just one question in a survey may create a need for adjustments in the instructions to a number of skip patterns. It is important to test all the possible alternative patterns prior to the actual survey to ensure that the questions make sense and are easy to follow. Asking about opinions and attitudes There are a number of ways to elicit opinions and attitudes in surveys. One common way is to ask respondents how much they agree or disagree with a series of statements. F. I feel safe walking around my neighborhood after dark. 1 Strongly agree 2 Agree 3 Neutral 4 Disagree 5 Strongly disagree G. Open-air drug dealing is a serious problem in my neighborhood. Strongly agree Agree Neutral Disagree Strongly disagree H. Police in Pheasant Hills are discourteous and rude. Strongly agree Agree Neutral Disagree Strongly disagree Notice that one has a choice of how to present the statement. Compare examples F, G, and H to I, J, and K below. I. I do not feel safe walking around my neighborhood after dark. Strongly agree Agree Neutral Disagree Strongly disagree J. Open-air drug dealing is not a serious problem in my neighborhood. Strongly agree Agree Neutral Disagree Strongly disagree K. Police in Pheasant Hills are courteous and well mannered. Strongly agree Agree Neutral Disagree Strongly disagree Because the statements are designed to provoke agreement or disagreement, they cannot be phrased in a neutral manner. However, the statements should not all be written with a positive (or a negative) slant; the slant of question should be alternated between negative and positive. Accordingly, in answering four questions that explore fear, a very fearful respondent should agree with two and disagree with two. Another approach is to ask a question and provide a series of answers from which a respondent selects. L. How safe is it to walk around your neighborhood after dark? Very safe Safe Somewhat safe Unsafe Very unsafe M. How serious a problem is open-air drug dealing in your neighborhood? Not a problem Minor problem Moderate problem Major problem N. How courteous are police in Pheasant Hills? Very courteous Courteous Not very courteous Discourteous Very discourteous Instead of placing multiple-choice answers below a question, as in examples L, M, and N, a scale could be provided and respondents asked to circle the most appropriate value in the scale, as in the examples below. INSTRUCTIONS: For the following questions, circle the score that matches your views most closely. O. How safe is it to walk around your neighborhood after dark? 1 2 3 4 5 very very safe unsafe P. How serious a problem is open-air drug dealing in your neighborhood? 1 2 3 4 5 no very problem serious Q. How courteous are police in Pheasant Hills? 1 2 3 4 5 very very courteous discourteous A third way to elicit opinions and attitudes is to pose a hypothetical situation and ask respondents to give the outcome they think is most likely. R. If you were driving through your neighborhood and a police officer stopped your car for a minor traffic infraction, the officer would be: 1 2 3 4 5 very very courteous discourteous S. Suppose your apartment were broken into while you were not at home. If your neighbors saw the burglar break in, what do you think they would do? 1. They would call the police. 2. They would call the apartment manager. 3. They would call someone else. 4. They would try to stop the burglar themselves. 5. They would watch the burglary and investigate. 6. They would wait until you got home and tell you who did it. 7. They would not do anything about the burglary. 8. Other. Please describe 9. Do not know. Note that with this last question respondents can give multiple answers. A neighbor could call the police and watch the burglary and investigate, for example. During the analysis of this question, items S.1 through S.9 should be treated as nine separate Yes/No questions. Asking about characteristics of respondents An investigator often wants to ask some basic questions about attributes and characteristics of the respondents: age, sex, race, income, marital status, welfare benefits, and so on. For questions of this type, the researcher should list alternatives and allow the respondent to check the alternative that fits best. T. Sex of the person answering this questionnaire. Male Female U. Age of the person answering this questionnaire. 17-19 years old 20-25 years old 26-30 years old 31-40 years old 41-50 years old 51-60 years old 61+ years old V. Please mark the race or ethnic group that most closely describes you. White African-American Hispanic Asian Other The categories presented should reflect the characteristics of the respondents to be interviewed and the issues the survey will try to address. For example, if the survey inquires about a person's area of origin and many of the respondents are from Southeast Asia, the researcher may want to subdivide the term Asian into several categories, such as Korean, Japanese, Vietnamese. If a fine gradation of ages is not needed, the questionnaire might employ only four age groupings, such as 17-25, 26-40, 41-60, and 61+ years old. Two important considerations must be taken into account when creating categories. First, a respondent must fit into only one group (note, for example, how the age groups in example U do not overlap). Second, all respondents must be classifiable into one group or another; no one should be left out. In example V, the category "other" was added for this purpose. The categories in example U are appropriate if the questionnaire is directed only to adults, that is, those aged 17 or older. If this were a mail survey, and a researcher could not be certain that only adults would complete the questionnaire, a category for ages 16 and younger should be added. For some queries, regarding age or income, for example, the question might just ask for a number, as below. W. What is your age in years? X. What is your annual gross household income? Open-Ended Questions To this point, with the exception of examples W and X, only questions for which a range of potential responses were provided have been discussed. Questions for which the responses are predetermined are called close ended. In contrast to close-ended questions, one may ask open-ended questions. Open-ended questions allow respondents to provide an answer in their own way. Consider example Y. Y. Describe the most serious problems in your neighborhood. Here the respondent can write (or dictate, if an interviewer is administering the questionnaire) whatever response he or she feels appropriate. Obviously, many different responses are possible. The advantage of open-ended questions is that respondents can tell their own story in their own words. This also allows for responses that a survey's developer may not have considered when preparing the questionnaire. The disadvantage of open-ended responses is that it is difficult to enter the data into a computer and analyze the responses. Because people use their own words, it is difficult to compare responses. For example, consider the following two possible replies to example Y. 1. "There are not many problems on this block. On Saturdays a few of the men get together and drink some in the park over there, but they don't bother anyone. I suppose I worry most about having my house robbed. The biggest problem is that a lot of these kids get out of school and cannot get a job." 2. "There is a lot of drinking and carrying on in the park. People are always having their houses broken into, and it is not safe to walk on the streets at night." Is drinking in the park a problem for both of these respondents? Is having one's house robbed the same as having one's house broken into, or are these two different problems? How would you classify the problems of job scarcity and street safety at night? While open-ended questions give much richer responses, it is not clear what should be done with this information. If the responses to the questionnaire are to be tabulated, the researcher will have to classify them and provide numerical codes for each type of response. This means that for open-ended questions, the investigator will have to read all the responses and develop categories. This is a time-consuming process. For this reason, it is sometimes useful to ask an open-ended question and follow it by a close-ended question that lists the responses of greatest interest. The open-ended question allows respondents to tell their own story and nominate responses the researcher may not have thought of. At the same time, the close-ended questionnaire focuses attention on issues the survey is designed to address. An advantage of this strategy is that when the results are reported, they can be supplemented with tables, charts, and figures and direct quotes. Designing the Questionnaire There are many other types of questions and formats for eliciting valid answers. Readers should examine and compare professionally constructed survey questionnaires to appreciate how questions can be asked. Here are a few tips on how to elicit valid responses from respondents, encourage respondents to complete a questionnaire, and reduce errors when collecting the data. Ask value-neutral questions Honest responses are needed from respondents to reach sound conclusions, so it is important that the questions and the array of multiple-choice answers allow respondents to express their true opinions, as happens in examples F, G, and H. Although the questions are phrased as statements, respondents may agree or disagree in a format offering equal potential for agreement and disagreement. Contrast the range of responses allowed in example H with those listed in example Z. Z. Police in Pheasant Hills are courteous and well mannered. Strongly agree Agree Neutral no opinion In example Z, respondents who disagree with the statement will either mark "neutral," "no opinion," or what is more likely, skip the question. They may feel that the questionnaire is not designed to allow them to express an honest opinion, so they will refuse to complete the survey. The results of the survey will show views strongly favorable to the police, but these results will not be valid. Make sure that the allowed responses measure the same thing The examples given above rate the responses along some continuum (for example, from observing something frequently to observing it infrequently, from strongly agreeing to strongly disagreeing, from feeling very safe to feeling very unsafe) or they provide categories for responses (for example, male/female or race). Example AA illustrates a poorly constructed question. It tries to evaluate fear of crime in a neighborhood but mixes up several measures of fear in the response categories. AA. How often do you walk around your neighborhood after dark? Every night Only with someone else Occasionally Never The example has several problems. For example, a respondent could legitimately mark two boxes ("only with someone else," and "occasionally"). This immediately suggests that the second possible response is unlike the other three and should be dropped. But there is another problem. "Every night" is a relatively specific response whereas "occasionally" is more vague. One should either pose the question with defined timeframes such as "every night," "once a week," "once a month," "less often than once a month," or offer general time terms--"often," "occasionally," "seldom," "never." Depending on the choice of scale, the question may need to be revised to fit the possible answers more closely. Ask only one question at a time If only one question is asked at a time, there will be little problem interpreting the answers. If a survey item contains two questions then the investigator will not know which question the respondents answered. Consider a questionnaire in which the following question about drug dealing is asked: BB. When does drug dealing take place in your neighborhood? During the day only At night only Night and day Have not seen drug dealing Because this query was not preceded by a question asking whether the respondent notices drug dealing in the neighborhood, it contains the implicit question, "Is there drug dealing in your neighborhood?" Therefore, an investigator will not be able to tell if persons choosing the last response in the series are saying that they know of drug dealing but have not seen it, or that they believe no drug dealing is taking place. Both are legitimate but very different responses. Survey questions must be unambiguous, and the responses must allow only for one interpretation. Ask what you mean Example BB illustrates another problem. Presumably the person asking the question wants to know what time of day respondents have seen drug dealing and believes that the most meaningful categories are day, night, or both. If this is the question, then once it has been established in a previous question that drug dealing has been noticed by the respondent, the survey questions should read: "When have you seen drug dealing in your neighborhood?" Pay attention to packaging and details Make the questionnaire easy to follow; decide whether it will be self-administered as part of a mail survey or whether a trained interviewer will complete it for the respondent. The following guidelines should be considered when designing a survey instrument: o Give clear and concise instructions. If the instructions must be complex, give visual clues (as were used to show the skip pattern in example E) to aid the interviewer or respondent. o Provide enough space between questions to make it obvious that they are distinct and separate. o When a respondent is expected to check or mark a category, provide respondents with brackets q or, better yet, a box to hold the check mark. o Group the questions by topic or by question style. For example, questions dealing with perceptions of neighborhood safety could be placed in one area of the survey instrument, and questions on attitudes about the police department in another. Alternatively, all questions using a five-point scale (see examples O, P, and Q) should be grouped, and all statements for agreement or disagreement (see examples F, G, and H) should be grouped. o Place sensitive questions toward the end of the survey, if possible. This minimizes the chances that a respondent will skip the entire questionnaire because of a sensitive question early in the document. Precode the questionnaire To analyze information obtained in the survey, a computer may be helpful. If the questionnaire contains many questions, if the sample size is large, or if there are many comparisons to be made among different types of respondents, the researcher will almost certainly need to use a computer. To enter the data on a computer, codes must be established for each response to a question. Codes translate the answer into a number. For some questions, this is not a problem. For example, the codes for the answers to examples W and X are the answers themselves because the answers are specific numbers. Similarly, the codes for questions O, P, Q, and R are obvious from the question. For other questions codes will have to be created. In examples A through E, codes have been provided--0 for No, 1 for Yes, and 8 for Don't Know. In question F, codes of 1 through 5 have been assigned to the answers. In question S, codes 1 through 9 are shown. Assigning codes is very simple. If there is some natural order or rank to the responses (more to less, high to low, stronger to weaker), then the codes should reflect this. If there is no natural order, the assignment of a number to a response is arbitrary--0 is assigned to No and 1 to Yes because they are easy to remember. Alternative codes are always possible--1 to No and 2 to Yes, for example. It is important to use the same coding scheme for all questions with the same format to reduce the chance of error. Special codes for missing data are useful. In examples A through E, the response Don't Know was assigned an 8. If a respondent does not answer a question, the investigator may want to code this as well. All codes should be developed prior to administering the survey; the investigator has a choice of whether or not to show the codes on the questionnaire itself. Their appearance on the survey instrument is helpful when the data are entered into a computer file. The person doing data entry does not have to look up the codes, and this can reduce data entry errors. On the other hand, placing the codes on the questionnaire can clutter it up and may be confusing to the respondents. If the survey is self-administered, relatively simple, and contains only a small number of codes, then coding should not be shown on the survey instrument. Pretest the survey instrument The precise wording and format of a questionnaire may have been the subject of much thought, but until it is tested on people like those in the population sample, an investigator cannot predict whether respondents will interpret the questions correctly. Some revisions in the content and wording of the questions and in the instructions, format, and organization of the questionnaire will be inevitable. It is useful to ask probing questions of those who pretest the questionnaire ("Were any questions hard to understand?" "Was the time needed to complete the questionnaire excessive?" "Were the instructions clear?"). A pretest may be time consuming, but it can eliminate many problems before they become serious. How To Make Sense of the Data Once the data are collected, they require analysis. For large sets of data--many questions or many respondents--a computer will probably be needed. Someone will have to read each questionnaire, note how each question was answered, determine the code for each answer, and enter the codes into a data file. This must be done with care to minimize data entry errors (for example, typing a 0 when a 1 is intended or skipping a question). Having an easy-to-follow questionnaire, established codes, and trained data entry people will help minimize these errors, but someone should check the data that were entered to determine if mistakes have been made. Randomly comparing a sample of questionnaires to data entered from them will provide an estimate of the frequency of errors. Although analysis techniques are discussed here only in the most basic manner, a more detailed introduction to analyzing data in policing can be found in Using Research: A Primer for Law Enforcement Managers (Eck, 1984). As in all stages of a survey project, more advanced texts and experts should be consulted while conducting the analysis of survey data. Once the data have been entered, there are four types of analyses to be performed. First, the characteristics of the sample must be determined. Second, a determination is needed of how representative the sample really is of the population being studied. Third, an investigator may want to make inferences from the sample to the population it represents. And fourth, the investigator may want to determine if there are relationships among the attitudes, behaviors, and characteristics identified in the sample population. Characteristics of the Sample During this most basic stage of data analysis, the frequency, central tendency, and dispersion of responses to each question will be calculated. The frequency of responses to a question is the number of people in the sample who answered the question in each of the possible ways. With a Yes/No question, for example, the number of persons answering Yes, the number answering No, and the number who did not answer show the frequency of responses for this question. If the numbers of responses are used, this is called the absolute frequency. If the percentage of respondents answering each way is used, this is called the relative frequency. Both the absolute and the relative frequency should be calculated. The central tendency is the average, or typical, response to a question. There are three types of measures for central tendency. The most common measure of central tendency is the mean, the sum of all the responses divided by the number of individuals responding to the question. For example, to determine the mean income of persons in a sample, reported incomes of all respondents would be added, and that total then divided by the number of respondents reporting incomes. (Some respondents may have ignored this question, so they would be excluded from the calculation.) Although the mean is the most commonly used measure of the average response, it is inappropriate for many types of questions. (A detailed explanation of situations in which the mean is the appropriate measure is found in Eck, 1984.) When the codes for a response have an order (for example, smallest to greatest, or least agreement to most agreement), but the responses cannot be added numerically (for example, adding "strongly agree" to "disagree" has no meaning), then the median is the most appropriate measure of central tendency. The median is the number that divides the number of responses in half. For example, if half of 100 respondents answered question R by circling 4 or 5, another 35 circled 3, and the remaining 15 circled 2, the median would be 3.5 because 50 percent of the respondents are above this score, and 50 percent are below. Or, if the ages of persons in this sample of 100 respondents had been recorded by categories (as in question U), then summing the categories and dividing by 100 would be meaningless. Instead, one would examine the frequency data for the question and determine the age category of the 50th and 51st respondents. The median age of respondents in a category is the age category between two answers; half of the respondents will have ages below the median, and half will have ages above the median. For questions with answers that have no order (for example, sex, race, or religion of the respondents), the mode is the measure of central tendency to be used. The mode is the answer given most often. If 73 of the 100 respondents answered No to question E, another 20 answered Yes, and 7 marked, "Don't Know," then No is the modal response. Measures of dispersion show how spread out the responses are. If all responses gave the same answer to a question, there would be no dispersion. If each response was selected by a similar number of people, there would be a great deal of dispersion. As with the central tendency, there are a number of ways of obtaining a measurement. If the mean is the most appropriate central tendency measure, the standard deviation or variance of the responses is the statistical method to be used. The method for calculating standard deviations and variances is beyond the scope of this monograph, but every statistical program will calculate these for a user, as will many hand calculators. They can be calculated from common microcomputer spreadsheet programs. Another measure of dispersion, useful when using medians to measure central tendency, is the range of the data. The range is the lowest and highest score for a question. Standard statistical programs will calculate all of these measures easily. More important than the mechanics of the calculations, however, is how the information will be used. The first step in using this information is to do an error check, looking for questions that have higher-than-expected missing cases or whose frequency of responses is very different from that expected. Before proceeding with the analysis, these matters should be checked to ensure that the data were entered correctly. Second, one begins to find answers to some of the substantive questions: Are respondents concerned about drugs? Do they have positive attitudes about the police? Are they fearful of crime? Representativeness of the Sample Regardless of the sampling strategy followed, the data must be checked to determine whether they are representative of the population. Random samples do not guarantee a representative sample; they merely make it likely. Furthermore, nonresponses can make the sample unrepresentative. The principal method for checking representativeness is by comparing answers to a few of the questions with information known about the population from other sources. For example, when members of the Newport News Police Department conducted a personal survey of residents in an apartment complex, they collected information on the income, age, race, and education of respondents along with other information. Because this apartment complex made up an entire census tract, these data could be compared with census data for the same area. Police analysts found no substantial differences between the proportions found in the sample and those found in the census information (for example, the percentage of blacks was similar in both data sets, as was the average income). This result suggested that the sample was likely to be representative of the entire adult population of the complex. It may not always be possible to make such a comparison, either because there are no data from a source other than the survey or because comparative data exist but are organized in a way that makes it difficult to achieve comparisons. For example, a survey may be for a single neighborhood, and the most comparable data are aggregated for several neighborhoods. This text has assumed that the sample was drawn using simple random sampling that did not involve stratification. If a stratified or other more complex random sampling procedure was used, then direct comparisons could not be made until the sample data were weighted to reflect the probabilities of selecting each respondent. Making Inferences About the Population There are two types of inferences that can be made about the population, based on sample data. First, characteristics of the population can be determined from what is learned from the sample. Second, one can determine whether or not there are relationships among the characteristics of members of the population, for example, whether the age and sex of a person have an influence on fear of crime. Such relationships will be discussed in the following section. The simplest method of determining characteristics of the population is to use the characteristics of the sample as the best estimate of the characteristics of the population; that is, one assumes that the relative frequencies, central tendencies, and dispersions found in the sample are very similar to the relative frequencies, central tendencies, and dispersions that would be found if everyone in the population had been surveyed. To make estimates of absolute frequencies in the population, the data must be weighted to reflect the probability of selecting each respondent. Each case is weighted by the inverse of chance of selection. For example, if a simple random sample of 200 households were selected out of a population of 560, the chances of selection are 200/560 or .357, so each case would be weighted by 560/200 or 2.8. In other words, each sampled case represents 2.8 households in the population. If 150 of the 200 respondents had seen drug dealing outside their residences in the last 6 months, then the best estimate of the total number of residents in the population who had seen drug dealing would be 150 x 2.8 = 420. In this example, all respondents are weighted the same because each had the same chance of being selected. If a stratified random sample had been used, respondents in each stratum, or subpopulation, would have different weights. For example, instead of a simple random sample, residents on the north side of the neighborhood had a 50 percent chance of selection, and those on the south side had a 30 percent chance of selection, and 100 respondents were selected from each part of the neighborhood. If 75 respondents from the north side and 10 respondents from the south side saw drug dealing in the last 6 months, then the best estimate of the number of residents in the entire neighborhood who witnessed drug dealing during this period is 183 (2 x 75 + 3.33 x 10 = 183.33). These are estimates of the population parameter, not exact figures. The true parameter will vary somewhat from the sample statistic. The question is, how much will it vary? A statistician can calculate the probability that the population parameter lies within set limits around the sample statistic. The broader the limits, the greater the probability that the parameter is within them, but the less precise the measurement. As discussed above, the greater the sample size, the narrower these limits can be drawn for some preset level of confidence. Commonly, a survey analyst will want to be 95-percent or 99-percent confident that the parameter is within certain limits (that is, that there is only a 5-percent or 1-percent chance that it lies outside these limits). Knowing the statistic of interest from the survey, the confidence limits desired, and other information available from statistical analysis of the data, the analyst will be able to calculate the bounds that provide this level of confidence. These calculations are beyond the scope of this basic overview of survey methods; however, most introductory survey research books discuss how these estimates can be made. What is important to remember here is that the estimates of population parameters are not guaranteed to be the precise figure found in the population; they will vary from advance estimates, and there are methods for learning how much they are likely to vary. Estimating Relationships Often the purpose behind a survey is to determine if people with certain characteristics or attitudes are likely to have another, specified attitude or to behave in a particular manner. When analyzing relationships, social scientists usually talk about variables. A variable is something that has different values in different circumstances. Because questions in a survey can be answered in different ways--that is, take on different values depending on the respondent's attitudes, behavior, or characteristics--questions are often treated as variables. Two variables are positively related if when one changes (increases or decreases) the other changes in the same direction. Two variables are negatively or inversely related when one changes and the other changes in the opposite direction. If, for example, respondents who are highly concerned about drugs also rate the police highly, then these two variables are positively related. If nonwhites rate the police lower than whites, then the variable "race" is negatively related to "rating the police highly." Two variables may be causally related or noncausally related. A noncausal relationship means that neither variable causes the other, they merely happen to be associated, perhaps because a third variable is causing both of them. For example, one might discover that in a sample of respondents, race is related to gender--that if the respondent is African-American, the respondent is more likely to be female. Race does not cause gender or vice versa. However, some other process may have created this relationship, perhaps the sampling process itself. In a causal relationship one variable is causing the other, almost always as a partial cause. In statistical analysis, the causes are called independent variables, and the effects are the dependent variables. In a noncausal relationship, there is no distinction between dependent and independent variables. Whether the investigation is of a causal relationship or a noncausal relationship depends on the theory and hypotheses of the analyst; the data do not specify whether the relationship is causal or not. Previous research, theories of human behavior, logic, and common sense dictate whether the relationship is causal. The data can suggest whether the relationship is real or not, how big it is, and whether it is positive or negative, but not the type of relationship. There are many methods for establishing relationships, but only the most basic method, the use of tables, is described here. First a general case is examined to show how calculations are made. Next a specific example is used to illustrate the calculations. To analyze a relationship using a table, one places the values of a variable to be evaluated at the head of the columns and places the values of the other variable at the head of the rows, as in exhibit 1. Each cell--lettered a through y--would contain the number of respondents who have the value of X corresponding to the column and the value of variable Y corresponding to the row. So q will equal the number of respondents who gave a value of 2 to question X and the value of 4 to question Y. The sum of columns or rows are called marginals, and the sum of the column marginals equals the sum of the row marginals and the sum of all the cells, the grand total. If the cells along the diagonal a, g, m, s, and y contain most of the respondents, then there is evidence of a strong positive relationship. If the diagonal cells u, q, m, i, and e contain most of the respondents, then there is evidence for a strong negative relationship. The more spread out or varied the respondents are from the diagonals, the weaker the positive or negative relationships. Although exhibit 1 shows a five-by-five table, these concepts apply to tables of any dimension. When analyzing a causal relationship, one calculates only the marginals for the independent variable (X) and divides each cell by its corresponding marginal to obtain a percent of the marginal. Then one compares these percents in the direction of the dependent variable (Y). Using exhibit 1, one would divide cells a, f, k, p, and u each by its sum to have the column percent for each of these cells. The same would also be done for the other four columns. Then one would compare along the rows to see if the percents varied from column to column. Suppose an investigator wanted to determine if respondents who were concerned about drugs were more likely than those unconcerned to participate in a neighborhood anti-drug effort. The investigator might include the following two questions in a survey: Drug selling and use are problems in my neighborhood. Strongly agree Agree Neutral Disagree Strongly disagree I would participate in a neighborhood anti-drug group if one were formed. Strongly agree Agree Neutral Disagree Strongly disagree After looking at the frequency of responses for each question separately, the investigator decides to collapse, or consolidate, the two "agree" categories into one and to collapse the two "disagree" categories. The table analyzed will have three columns for each variable and nine cells. Exhibit 2 displays the row percents (with numbers of respondents in parentheses) for this hypothetical example. By reviewing and comparing each column in turn, it becomes clear that concern about drugs increases willingness to participate. A total of 61 percent of the respondents who disagree with the statement that drugs are a neighborhood problem also disagree with the statement that they would be willing to participate in a neighborhood campaign against drugs. Only 24 percent of the neutral respondents and 3 percent of those respondents agreeing that drugs are a problem disagree with the idea of participating in a neighborhood organization. Twenty-four percent of the respondents who disagree with the statement that drugs are a problem were neutral about their willingness to participate. A total of 36 percent of those neutral on drug problems were neutral on participation, while less than 16 percent of those who felt drugs were a major problem were neutral on participation. Finally, 15 percent of the respondents who did not feel that drugs were a neighborhood problem agreed that they would participate. Of those neutral on drug problems, 39 percent would participate. And more than half the respondents who felt drugs were a major problem were willing to participate in a neighborhood anti-drug group. Clearly, the more that respondents in this example felt that drugs were a problem in their neighborhood, the more willingness they expressed to get involved to do something about it. Suppose an investigator had reason to believe that this relationship was not universal. In particular, he or she believed that apartment dwellers were less willing to participate in anti-drug campaigns than owner-occupants of single-family homes (in this neighborhood, there are no renters of single-family homes and no condominium owners). Because some neighborhoods have many apartment buildings, determining the validity of this assumption was important for developing a neighborhood-based anti-drug program. To determine if home ownership makes a difference in willingness to participate, the same analysis as above was conducted, but it controlled for whether the respondents were renters or owners (information presumably collected in the survey), meaning that separate analyses were conducted for renters and owners. In other words, the variable owner/renter was held constant; it did not vary. These results were then compared with the initial, uncontrolled results. When the owners' responses are evaluated, the same basic relationship emerges, increasing concern about drugs leads to increased willingness to participate in a neighborhood anti-drug program. Furthermore, the relationship seems also to apply to renters. The major difference between renters and owners seems to be the greater willingness of owners to participate in a neighborhood drug program, given their perception of the drug problem. Note that ownership is treated as the independent variable; the perception of drug problems is treated as the dependent variable. This exhibit is calculated from the marginals of the independent variable shown in exhibit 3, panels A and B. It shows that fewer renters seem to agree with the statement that there is a drug problem in their neighborhood. From this analysis, we have evidence that among respondents in this hypothetical example, ownership influences perceptions of a drug problem; this in turn influences willingness to participate in a neighborhood anti-drug program. In a causal analysis, control variables are always independent variables. They are introduced into the analysis to test whether they would change the relationship between the initial independent and dependent variables. In the above example, the relationship did not change because it turned out that the control variable did not directly affect the dependent variable, but affected only the independent variable. But this is not the only possible result of introducing a control variable. It is also possible to discover the following: o The control variable directly affects the dependent variable, but the independent variable has no direct effect on the dependent variable and instead directly affects the control variable. o The control variable causes both the dependent and the independent variable, but the independent variable has no impact on the dependent variable. o The independent variable has a different effect on the dependent variable depending on the value of the control variable. (This is what was expected when the control variable was introduced in the example above.) o The independent variable and the control variable are noncausally related, and both have a direct effect on the dependent variable. Two conditions must be met in order to introduce a control variable into an analysis. First, there must be a sound reason to believe that it could change the relationship. The reason could be based on prior theory or the investigator's experience. Second, the control variable must be separately related to the independent variable and the dependent variable. While the first condition cannot be tested for with the data, the relationship of a prospective control variable to the independent and dependent variable can be analyzed separately, prior to introducing it as a control. Significance Testing This text has discussed relationships among variables without discussing criteria for accepting the presence of a relationship. It is possible that a researcher may observe a relationship in the data that is due to random chance and not to a true relationship in the population. There are statistical tests that determine the probability that an observed relationship is due to chance. The use of these tests is called significance testing. To construct a significance test, an investigator first establishes two competing hypotheses: the null hypothesis and the alternative hypothesis. The null hypothesis states that there is no relationship between two variables in the population and that the observed relationship is due to chance. The alternative hypothesis states that the relationship in the population is as shown in the data. The investigator then picks a maximum probability that the observed relationship is due to chance. This is called the significance level. Typical significance levels are .01, .05, and .10. That is, there is a 1-percent, 5-percent, or 10-percent chance, respectively, that if there were no relationship in the population, the observed relationship could arise by chance. The lower the significance level, the greater the confidence one has that the results are not due to chance. Note that significance levels are interpreted in the same way as confidence levels. The mathematical formula to be used in the test depends on a host of factors, including the sample size, the type of data being used, and how the relationship is being analyzed. A great variety of significance tests can be applied, too many to permit examination here of the formulas that can be used. Suffice it to say that the test indicates whether the probability of the relationship's being due to chance--when there is no real relationship-- is greater or less than the significance level selected. If this probability is equal to or greater than the significance level, then the null hypothesis cannot be rejected. This means that the possibility that there is no relationship in the population cannot be ruled out. Notice that there is no claim that a relationship does not exist, only that this possibility cannot be ruled out. If the probability is less than the significance level, then the null hypothesis can be rejected in favor of the alternative hypothesis. This means that the chances of there being no relationship in the population are so remote that the investigator will act as if the alternative hypothesis is true. While the test provides evidence that the alternative hypothesis is true, it is not proof that it is true. There is still a small, but real, probability that the observed relationship did arise by chance. Conclusions In this introduction to survey research for policing, this text has focused on the costs and benefits of various decisions about neighborhood surveys. For example, the text has discussed how increasing sample sizes provides greater precision and confidence, yet results in a more time-consuming and costly study. Techniques have been described that improve the information gained from a survey while keeping costs down. For example, often a researcher may choose among different sampling strategies that can provide equally precise results at the same level of confidence, yet one strategy permits interviewing of fewer respondents. The basic choice is between using survey research methods or using other methods that have no basis in science such as a convenience sample or an accidental sample. It might be argued that because survey research methods are so complex, the expense of gaining the expertise and carrying out the needed steps makes them cost-ineffective. This may be the case in certain circumstances, but the careful researcher knows that the validity of results may suffer greatly from a convenience sample, although there is no way of knowing by how much. Selecting a convenience sample of a few citizens and asking them questions that have clear, socially desirable responses may provide some information about community needs and problems, but there is no way of determining whether the opinions expressed are representative of the population under study. Policing is now acknowledging the value of social science methods for informed decisionmaking. An increasing interest in the application of survey research methods to community and police problems is evidence of this trend. Consistent and diligent application of survey methods to police work will lead to more informed decisions and improved collaboration with community members. This book is only a starting point; police leaders who want to make surveys part of their operations should tap the expertise of the many college-educated officers and other employees with advanced degrees who are already in policing. For additional help they should seek out local experts in survey methods at nearby universities, marketing research firms, and other corporations, and consult more advanced texts on the subject. Part II: Environmental Surveys for Problem Solving: A Practical Guide Environmental Surveys and Problem Solving These surveys seek to assess, as systematically and objectively as possible, the overall physical environment of an area. That physical environment comprises the buildings, parks, streets, transportation facilities, and overall landscaping of an area as well as the functions and conditions of these entities. Although this guide focuses on the physical environment, police are concerned about how the physical environment affects the social environment. In other words, they want to find out how features of the physical environment contribute to crime and disorder by facilitating offenders and inhibiting nonoffenders. Therefore, environmental surveys are a component of a larger problem-solving process. The problem-solving process can be divided into four stages: scanning, analysis, response, and assessment. In the scanning stage, officers identify problems by grouping similar calls and incidents together. During the analysis stage, officers systematically analyze problems by collecting a wide variety of information, from inside and outside the police department, about factors or conditions that may be contributing to the problem. This information is used in the response stage to develop specific strategies to resolve the problem. Finally, in the assessment stage, officers evaluate the effectiveness of their response. The first letters of these four stages are often combined to refer to "the SARA model" for the problem-solving approach. Results from the evaluation may be used to revise the response, collect additional information, or redefine the problem. Environmental surveys can prove useful in each of the four stages of problem solving. Periodic environmental surveys can help identify problems that are just begi