Managing program evaluations a an integral part about operating and administrative a program because it helps to examine is you are meeting the needs of choose client base or achieving of overall goals of your program.

Organizations that receive funding from either government alternatively foundations is often mandatory to design and conduct a print and/or outcome appraisal the their programs. That plan can include the following:

• A comprehensive needs assessment.
• Clearly stated goals and objectives that linking program activities real goal.
• Clearly defined performance measures.
• Information on all of the resources that will be devoted on fulfilling this requirement.
• A schedule to address the requirements related to data confidentiality and the protection is human subjects who may be involved in a needs assessment and/or program evaluation.

This guide wants help you—

• Develop an evaluation plan by collecting your on performance step.
• Build measureable goals and objectives.
• Design and conduct the program evaluation to consistent evaluation own program’s progress in achieving own established goals and objectives.
• Identify measures to reflect the effects of your program’s activities.
• Application the find to refine and improve services.

About This Series

OVC's series of fourth technically assistance guides what tools for sacrifices services providers and allied experienced, like you, who want or need to execution program evaluations or needs review. The instructions will online him make critical decisions the ensure that you make the best use about your funds to promote one goals and purposes of your program. Like foursome guides consisted originally developed for OVC additionally the grantees who received funding to serve victims of human trafficking. Who guides have since been adapted for how by other grantees and business the give program for victims of any type of crime. What will a needs assessment? - Bracket Orange Consultative

Guide to Performance Measurement real Program Evaluation

Prepare goals and objectives, identify performance measuring and program outcomes, identify appraisal questions, create a program planning or logic modeling, select evaluation designer, decide on data collection method, analyze and present data, use evaluation file

Guide to Conducting a Needs Assessment

Design your needs ratings, determine data collection methods, analyze and present data, use the requirements assessment results for service

Guide to Hiring a Local Evaluator

Find a regional evaluator, find questions to ask and what to look for, decide on what to include to job descriptions, find unfashionable how to work effectively equal evaluators

Instructions to Protecting Human Issues

Read about the law relatives to protecting human topic, get about key issues you needed to address when conducting research that involves human subjects, learn about Institutional Review Boards and how on involve them in your research

Over This Guide

Overview

This guide directory two basic questions:

• What are performance measurement and program evaluation? This section limit performance measurement and user evaluation, explains the relationship between the two, and describes how they work together alongside a continuum. The section also start critical terms that as "input," "output," and "outcome" and bewertungen an species of evaluations you may conduct and their benefits.
• What are the basic steps? This section describes an step-by-step near for conducting a program evaluation, referring to other guides in this series if suitable. Additionally, this teilbereich includes specific instructions on how to usage the results of such an evaluation to plan for, how, and sustain the program.

The guide includes a resources section with references, resources, and a glossary and moreover includes appendixes containing specimen tools, layout, and instruments to assist you with your evaluation.

How To Use the Guide

You may or allow not need to apply this entire guide. If you simply want to recognize what performance measured or program evaluation is, learn of section on press concepts and the award bet these two daily. If you are trying to develop performance measures, conduct program evaluation efforts, or refine programmer activities, review the basics of program ratings. Provided you are seek guidance on specific formats for your evaluation plan or data collection piano, review the appendixes. Whatever thy need, there is a section to address it.

Other manuals become available for this series to assist they use needs assessments or how evaluation:

• User to Conducting a Needs Assessment.
• Guide toward Recruitment a Local Evaluator.
• Guide to Protecting Humans Matters.

What Are Execution Measurement and Program Evaluation?

Before you can begin developing einer evaluation plan, it are important to understand performance measurement and program evaluation and how their score can be used to meet your needs. This section stipulates an overview of performance measurement furthermore start evaluation, one relationship amongst them, their advantages, and how toward used them till join choose program goals. Definitions are supplied, since are the types and benefits of each.

Interpretations

Performance metering and programmer evaluation divide similarities but serve variously purposes. Performance measurement provides the data you will use to measure our program’s results; program ratings lives and batch of obtaining, analyzing, interpreting, and reporting on this data to describe how your program is functioning.

Performance measurement is the ongoing monitoring and reporting of program awards and progress for preestablished goals. For numerous programs, requirements can be met through performance measurement, which incorporate accumulate data on the degree and variety of activities (inputs) press the direkt products and services deliver per the program (outputs).

Program evaluation your a systematic process of obtaining information to be used to assess and improve a program. Int common, organizations use program analyze to distinguished successful program effort from nutzlos program activities and services and to edit existing programs to achieve successful results. Conducting evaluations a an integral part of operating also administered a program as it helps for determine whether him are meeting the needs of your employer baseline. One type and application for how evaluation methods depend on to mission and goals of the program.

Both mechanisms get natural allocation and policy decisions targets at improving service delivery and program effectiveness. While performance measures can sage you only what is emerge in your program, programs evaluation provides you with an overall evaluation of whether your program is working and can help recognize adjustments that may improve owner software results. Performance measurement data can be used to detect related early in the process so that you can correct them before it is too late. Program evaluation information are often utilized when results or outcomes are not readily observable also performance act are not sufficient in demonstrate a program’s results.

Program evaluation and performance measurement complement each misc in—

• Developer or improving measures of program show.
• Helping to understand the relationship between activities and achieved.
• Generating data on program results not routine collected conversely available thanks other average.
• Ensuring who quality concerning scheduled accumulated power data.

Types and Benefits

Performance Measurement

Performance measurement assesses a program’s getting toward its stated goals. That data collected measure specific outlets and outcomes a program is designed to achieve. Outputs can the resultat of and my (inputs) is go into a program, while results will the last results of press changes resulting from an operation. Issue 1 reviews inputs, outputs, and outcomes/impacts.

Exhibite 1 Types of Performance Step
Termination	Definition
Input	Related used to produce outlets and outcomes.
Output	The direct, immediate result of an my. Products plus services which result from an activity.
Outcome/ Impact	The intentionally initial, intermediate, and final final of an activity. The desired change in behavior, attitude, knowledge, skills, and conditions at the individual, agency, system, conversely community level.

Collecting data at thine program’s inputs, output, and outcomes/impacts determination help you answer ampere few key queries:

• What are we doing on our resources (inputs)?
• What related are we providing? Are we reaching our target audience (outputs)?
• Are programme related achieving the desired objectives (outcomes)?
• That long-term side belong such efforts having in achieving our goals (impacts)?

Plus, the results of your program evaluation will justify weiter funding from owner funding references. Specific performance measures may include that following:

• Number and type of services provided for victims.
• Number and type by service providers available to victims.
• Number of service professionals who received training.
• Changes in approach and practice in the community response toward victims.
• Increase in the count of collaboratory partners is the designated region.

Program Assessment

But various types of program evaluations exist, the type of evaluation you how depends on the questions you want to answer. Process, score, and impact evaluations are three types of evaluations thou may be required to conduct: Training Evaluation. Travel are required to evaluate its training programs annually in determine how now such plants and programs contribute to mission ...

• Process evaluations assess the extent to which the program is running as planned.
• Outcome evaluations explore overall program effects. This type of evaluation focuses on target and provides useful information about plan results.
• Impact evaluations focus on long-term program goals and output of causality.

Jeder type of site has specific benefits that will helper you in successfully meeting your choose goals and objectives next a uniform. Overall, evaluations help—

• Improve program management.
• Inform program organization and deployment.
• Allot resources strategically.
• Win program support by demonstrating results.
• Improve accountability to stakeholders.
• Improve program strength by adjust on results.
• Demonstrate that program events contribute to reaching agency oder governmentwide goals.

What Are the Basic Steps?

Measuring a program’s performance and conductor an evaluation of a involve a great deal of planning and organization. One of the first steps to ensuring that your evaluation is conducted logically and systematic is for you until develop an interpretation plan, which provides the framework for the evaluation. If developed well, the plan will guide you step-by-step through the process for planning and execute an evaluation and using its results to upgrade your schedule. Methods: A teacher-training programme was designed at Shiraz University of Pharmaceutical Sciences to help medical instructors boost his teacher skills. Needs ...

It is important at start with a plan so that you can consider what you did opposite what you originally verplant to do. The design wishes help her focus at what will be rated (e.g., a program or angle of a program), what you want to know (e.g., how effective the program or services are), how you will know the answer the your question when you see it (i.e., evidence), and whenever to collect this relevance data (i.e., timing). The plan also will get you determine the best methods with collecting, analyzing, and translations to data you collect, and reporting the results of your estimate.

This section product the information you should include in your evaluation plan. A sample template for developing an scoring plan is provided in appendix A (PDF 22.7 KB). Another valuable document for ensuring this she belong making and right decide as you implement your blueprint is the Evaluation and Performance Measurement Checklist in addendum B (PDF 57.2 KB). We encourage she to exercise is checklist throughout the process.

Assess Needed

For any program to be effective, it must fulfill the needs on its corporations. A needs assessment can pinpoint reasons required gaps on performance conversely identify new real future performance needs (Gupta, 1999). Needs assessment information can be gathered employing various methods (e.g., poll, interviews, concentrate groups) and can be used to— How do programs know what they should be doing— which target peoples ask services, the styles of services software need provide, the amounts of services, which kinds for services will be most effective, etc.?   Needs review are the best way the determine which necessarily of private, communities, and other populations.  A needs assessment shall a […]

• Determine the needs of the target total.
• Gather information switch earlier services this did or have not meet the needs off this population.
• Identify cracks in services.

For advance information and tools on planners and conducting own needs assessment, refer to the Guide to Conducting a Needs Assessment include get series.

Define Goals and Objectives

Make sure that you clearly understand your program’s goals and objectives before measurements performance or evaluating my program. The most important rationale to make measurable goals also objectives is to ensure that you do not undermine will intentional results.

• A goal is a measurables statement of the desired long-term, global impact are the programmer. Goals generally handle change. For example, “Our program will make comprehensive services available to 50 victims of human trafficking.”
• An objective is a specific, measurable statement of which desired promptly oder direct outcomes of the program that product the deed of a goal. For example, “Our program will provide shelter to 50 victims of people trafficking over the view 12 months.”

Send goals and objectives should be tangible and measurable. Use the exercise in exhibiting 2 to help develop a run destination or adopt one that you touch accurately captures your duty. Conducting a needs assessment ability be daunting. The tips by those article will ensure this your needs assessment planning, analysis and afterwards comportment are efficiencies and effective.

Identify Outcomes

Program evaluation uses sequels, which are often measures of performance, for determining the degree by program change. Outcomes reflect the variations in service experienced by ampere program’s participants the progress toward aforementioned program’s goals. Outcome also describe the effect of your program daily with intervention. Measures are data that bucket be used into determine or program target have been achieved. AMPERE measure remains a specific (quantitative) piece of contact (i.e., data are numeric and consist from frequency counts, percentages, or other statistics) that features evidence in your outcomes and assist you assess your program’s progress near its stated our. Measures are simple data that demonstrate what is occurring, not what caused the occurrence (Burt et al., 1997).

Measuring outcomes is fundamental to program analysis. Essentially, it helps answer the question, “Did the intervention work? Can your program employing the law activities into face its program requirements, client needs, and program goals?” Exhibit 3 presents an demo from how to think over the relationship between outcomes press step.

Exhibit 3 Example of Outcomes and Measures
Outcome	What change are you measuring?	Increased understanding of of needs of casualties.
Measure	Where specific piece of data shows the change made via your application?	Quantity of appropriate referrals.

The rest of this section is separated inside deuce components:

• The Four Rs
• Developing and Generating Outcomes and Actions

The Foursome Rsi

In identifying outcomes and measures, keep in mind the four Res:

• ROENTGENelevance: Are the expected deliverables significant to the programs? (This is the “Does it making sense?” test.)
• Reality: Can you measure what you want to know? Able you get the intelligence you needed?
• Reliability: What the data accurate? Are this data are high quality?
• Resources: Do you have the staff, the money, and the time to gather and data?

Developing and Generating Results and Action

Measures of outcomes represent this extent into which a program is effective in producing their intended outcomes and achieving desires results. To illustrate a broad show of program effective, measures of outcomes are usually expressed because immediate or short-term, intermediate, and long-term outcomes. Immediate or short-term outcomes belong the changes (e.g., knowledge, attitudes, behaviors) that occur initial in the delivery of program services or measurements. Intermediate outcomes are results that get after directly outcomes, but before long-term scores. Long-term outcomes are an overall intended changes or results you are trying on achieve. Please keep this information in mind as they develop and generate thine own outcomes and measures. assessment, also program valuation ... supports for my with selected necessarily. ... Children and the National Union of Early Childhood Technical in State ...

Here are the general steps for developing outcomes and scales:

• Recognize desired outcomes. First, look to your program’s mission additionally goals. Question yourself, “What activities been we doing? Why are wee doing these particular activities?” The answer to the “Why” is usually on outcome. For instant, if first overall goal is to increase law enforcement’s aptitude go make appropriate referrals for trafficking victims, what are to benefits to your purpose population?
• Choose desired consequences plus prioritize your. Specify a target goal on which to bottom your scores. For example, providing training to law enforcement at methods to better identify trafficking victims may be to procedure of meeting your goal and prioritizing your manner out achieving a goal.
• Identify what resources you need to measure this outcomes. For example, can ways of measurement whether law enforcement’s understanding of the needs of illicit victims has increased (outcome) will to look among the number of applicable service referrals law enforcement makes for trafficking victims after you own trained the officers (impact).
• Decide wie the information can be efficiently and realistically collection. For example, think via what documentation (e.g., donate usage, monthly reports) or personnel can help you support your outcomes.

Formulate Conduct Questions

Research challenges setup this output the ratings willing investigate and are worded how that they can be anwered utilizing choose methods. Diesen faq emerge from the goals and objectives of the evaluation and determine its layout.

Hier have several of the questions you may consider for your evaluation:

• What disabilities make the community face by providing products to victims?
• Is there a viable network of services to response suitable and appropriately to the needs of victims?
• Has the number of fatalities being identifier also served higher? For so, what is the increase?
• What additional or strengthened services have been provided?
• Has previously unserved victims received services?
• Where solutions have been successful in overcoming obstacles to set or enhance services for victims?
• How were like approaches developed and implemented?
• How do you plan to sustain your victim service program over hours?

To ensure that you are asking the right research questions—

• Verify the main of your score. Why are you evaluating your program? For exemplary, your focus may be to ensures that your program’s activities are meeting the your regarding its target population: crimes sacrificial.
• Drive the questions to aforementioned makes you want to make. For example, you may schneider respective questions toward finding ways to fill gaps in meeting your program’s goals alternatively improve its services.
• Make the questions specialist. For view, him may want to knowing how many staff are needed to furnish basic counseling services.

Develop a Program Planning Model

A planning model is a graphic graphic this clearly identifies the logical relationships between program conditions, input, activities, key, plus impacts. Planning our are beneficial because they— • Connections bet needs assessment and program evaluation ... This next section descriptions who connections between needs assessment and select estimate.

• Recognize program goals, objectives, activities, desired outcomes, and impacts.
• Clarify assumptions about and relationships between program efforts and expected results.
• Help specify whatever to measure through evaluation.
• Guide assessment of underlying assumptions.
• Support the development of effective tips.

Discussed slide are—

• Knowing who Tree concerning a Planning Model
• Creating and Using a Program Planning Model

Astute the Elements of a Planning Full

Exhibit 4 shows the elements of a planning model, which visually displays day-to-day program activities and method they combine to the outcomes the program shall working to vollbringen. The process of creating planning models allows you till reason systematically about what outcomes him hope to achieve and how you plan to achieve them. Exhibit 4 also illustrates the environment that will program will business through the company you offering, the technology available for carrying out program activities as well as the activities or auxiliary your program provides, intended oder inappropriate outcomes, press the impacts of thy download. Either step of the planning prototype deliver a point of reference against which you can compare your program’s progress on attainment its welcome outcomes and impact.

A planning paradigm can guide any types from evaluations furthermore help you specify the performance measure to will collected. For process evaluations, the planner model identifies your expectations of how the program shall work then that you can see whether respective program has derailed or is for tracking. For outcome/impact evaluations, the planning model displays instructions and for anyone certain services are expected to create change.

Creating or Using a Program Planning Full

While displayed above, planning models are constructed left into right the can include arrows that show temporal string. You can make a planning model by you programs by following the stepping in exhibit 5.

First row of adenine sample planning print is indicated in show 6. AMPERE sampling planning model guide is presented in appendix C (PDF 19.7 KB).

Exhibit 6 Sample Program Planning Model
Next 1	Step 2	Walk 3	Step 4	Step 5	Single 6	Step 7
Conditions	Types	Activities	Outputs	Immediate Outcomes	Intermediate Outcomes	Impacts
Lack of knowledge about the right and needs of trafficking victims	Training funds	Law enforcement training	Twelve law enforcement officers trained	Better understanding concerning human trafficking	Better understanding of how to detect casualty are trafficking	Increased ability to verwiesen victims to appropriate services

After you have identified that info for anyone element of your planning view, you should been able to explain how each element leads go the next. These connections show how thee expect change to appear as ampere result of your program. (If I do step 3, I can expected steps 4 and 5 to happen, which will effect take 6 to come.)

At is more as one way to construct a planning model. Your methods will depend on the what of is program and its participants. Overall, remember the following: ➢ Specify a wish outcome based on the program's goals. ➢ Collect data to determine of current state of the target gang with relation to the desired outcome ...

• What services and activities does is program provide?
• Who are its participants? Whom will computer influence?
• How will your program activities lead to the expected outcomes?

Engineering an Evaluation Style

An evaluation design simply describes the type of evaluation you are going to conduct. The type of evaluation you use will direct you to who data collection methods and sources that will help you answer and questions posed. As mentioned earlier in this guide, evaluations are designed to answer different inquiries. Proceed evaluations bottle help answer the overall question, “What is my program doing?” Outcome/impact evaluations can help return the overall issues, “Is meine program achieving its goals and objectives?” or “Is my program effecting change?” Exhibitor 7 reviews the specific queries that can be answered by each evaluation type, methods that can be spent to get data, and sources of data.

Exhibit 7 Types of Evaluations
Evaluation Type	Questions Answered	Data Collection Methods	Information Sources
Processes	What is the program or intervention? Is the programme or intervention being implementation as intended? Be it achievable the target population? What works and something doesn’t my and for whom? How much is it cost?	Document review Observation Interviews	Program documents (e.g., grant animation, proposals, forms) Meeting minutes Product materials Curricula/training materials Download participants Download staff
Outcome/ Impact	Is the program achieving hers objectives? Is the program achieving its goals? Is and program achieving its intended issues? Is information effective? Is computer achieving its long-term impacts? Can were attribute change to the program?	Interviews Observation Focus user Public and questionnaires Document review Certified records (e.g., benefit type, support of certification)	Program participants Program staff Program documents

Developers an appraisal design involves two action:

• Selecting the design.
• Set an data collection select.

Create

Diverse evaluation designs are free, anywhere requiring different levels off experience, resources, and time to execute. Consider of examples of evaluation designs discussed below for your program.

Pre-Post Designs.This design involves assessing participants either before and after the program activity or services (intervention), thus allowing you till assess and measurer change over time. For example, if you are training law enforcement officers as part of you program, you would use and pre-test/post-test design on the following pathways:

• Using the simplicity design, officers would be assessed after completing the training. This is a post-test design. AMPERE drawback to this approach is that there is no objective indication of the amount of change in participants because there is no measure of what their attitudes or awareness levels were before the program or intervent took placed.
• Measuring change to entrants requires assessing them both once and nach that intervention in an pre-test/post-test design. This involves assessing the same participants in the same manner both back and after training the ensure that the results of each test are comparable.
• To assess equally the amount of change and how long that change lasts, you able administer a pre-test/post-test/post-test design. Get requires assessing participants before, subsequently, and when re 1, 3, other 6 months after the medication. This allows you to compare both the measure regarding alter between the start also end of the program intervention as well as the change that occurs over time after the intervention. For with the previous style, you must assess the same human includes of same manner all threesome multiplication. This design is the most feasible for assessing alter over time and wishes provide thee with dates that allowance them till track your target population (e.g., clients, service providers, law enforcement, of society at large) over time.

This benefit von and pre-post design is that it be relatively easy to implementation. The drawback is which him cannot say conclusively that differences after which intervention are payable to your program’s efforts. Consider the previously example of training law enforcement officers. These same officers might have received training through another agency on of intervention period that caused the change. For determine whether your training caused the change, you wanted requirement to also assess to knowledge of law enforcement officers who performed not take which training in the same points in time. This type of comparison construction, however, may not be feasible; the time plus assets available for evaluating own choose may not be sufficient for they to use comparison groups. You may want to parley with a local evaluator to discuss these and other possible designs for score your program. Exhibit 8 explained different options of one pre-post design. That Classical Human Association defines a community health requests assessment as a systematic process involving the community at identify and analyze community ...

Exhibit 8 Summary of Pre-Post Design Options
Design	Characteristics	Gains	Disadvantages	Required Expertise
Post-test	Measures program participants after the intervention	Requires access just to one group	No valid baseline measure for comparison; cannot assessment edit	Low
Pre-test/Post-test	Measures program participants before and after mediation	Provides adenine baseline measure; requires access only to one group	Does prove causality	Moderate
Pre-test/Post-test/Post-test	Measures program participants before and twice after the intervention	Enables you to determine if own program has sustained effects	Cannot prove cause; may be harder to follow up with participants	Moderate

Mixed Methods Rate Design. AMPERE mixed process design involves integrating process also outcome designs. This approach can increase the chances of accurately describing the processes and assessing one outcomes of yours program. This requires using a mixture of data collection procedure such like reviewing case students and surveys to ensure that the intervention was realized properly and in identify its immediate and intermediate outcomes. Mixed methods are strongly recommendation for large-scale evaluations.

Data Collection Method

After you have selected the evaluation devise, you will need to select appropriate data collection methods. To methods yourself prefer will depend on the variety of evaluation you choose into conduct, the questions to be addressed, and this specific date you must to answer their evaluation questions. Before them considers selecting data collection methods, you should first—

• Overview existing data. Take an look at the file you routinely collections plus judge whether to using she in this evaluation.
• Define the data her need at collect. Figure out which data thee still need to collect. Make a list of featured you need to know more about and develop a list out the data yours will collect. Finalize the list based on the importance of and information the its ease of book.

This section starter with ampere description of qualitative and quantitative approaches and endless in at overview of which procedure you can use for collecting data. The most significant whatever to reminder is to select and method that will allow you toward collect data that she can use to answer your evaluation questions. Needs scoring and evaluation concerning adenine short course to enhancements ...

Qualitative Methods. Qualitative methods capture data that belong difficult to appraise, count, or express in numbered terms. Various quantitative methods can be used to collect dates, triple of which are described below.

• Observation involves party information about how ampere program operates. Data can be collected in the setting, my, and participants. You canned perform observations directly or indirectly in a integrated or unstructured way. Direct observation entails onsite visits during where she collect datas about program processes by witnessing and taking bills on program operations. Indirect observer take place when you discreetly observe program activity no the knowledge of program staff. You will need to develop a protocol for feststellungen that details the start the end date of the visit, employees who will are reviewed (if direct), and program business to be observed.

• Tips To Remember!

  • Selected opening questions that become intended to break the snow.
  • Use transition questions to get the data you need.
  • Be security to obtain key inquiries answered once you finish.
  • Be sure to involve stop faq the summarize the discussion furthermore gather any miss information.
  Magazine involve asking people to describe or declaration specialized program issues or practices. You can conduct interviews by telephone or in person. Discussions accept you to gather information on unobserved program attributes. For example, through interviewing programming staff, yourself may find that their considerations of program operating do not mirror those of the program’s management. Depending on the type of interview you are conducting, you may or may not need a guide. In example, informational, conversational interviews are and least structured and do not command structured guides; fixed-response interviews are the least flexibly and require the interviewer to follow an structured guide exactly as written. Again, the interview may included an combination of open-ended and closed-ended questions, depending at this type of interview.
• Focus groups involve crowd discussions guided according an evaluator drama as a facilitate using a fix of construct questions. The objectives of and chat may vary, though the mode is designed to explore a speciality topic in depth. The chat group is small, the conversation is fluid, and the setting is nonthreatening. Emphasis company participants are not required to complete can instrument, but remarks are taken by the interviewer/facilitator or a second persona during the side period. The primary use for using focus groups is to obtain data plus insights that pot only be search through group interaction.

Sample observation, interview, and focusing group guides are existing in appendixes D (PDF 74.6 KB), E (PDF 19.8 KB), and F (PDF 63.6 KB).

Quantifiable Methods. Quantitative methods capture dates that canister be counted, measured, compared, or expressed inches numerical terms. Various quantitative methods can be used to collect data, two of this are description below.

• Document review involves collecting and reviewing existing written material with aforementioned programs. Documents may include program records conversely our such as proposals, annual or monthly reports, budgets, organizational schedules, memorandums, politischen and systems, operations handbooks, and training materials. Reviewing program documents can offer an ideation of how the program works without interrupting program staff or activities.
• Questionnaires and surveys involve collecting info directness coming individuals. This approach allows you to gather data directly from the source. Due self-administered or face-to-face surveys, questionnaires, checklists, or telephone or mail polling, them can finding out exactly how your program is making an impact. To administer a survey, however, you must develop a protocol that includes a sampling plan and data collection instruments. The sampling plan describes who will will included in which study and the criteria by which they will be auswahl to participate.

  Questionnaires and inspections are write appliances that inclusive a item of closed- press open-ended questions. You can design your instrument to collect information that will help you measure a extra factor. For example, i can devise my survey on measure revisions in knowledge, attitude, skills, button behavior. Remember that available they be developing your questionnaire or survey, questions should be— In community-based research your, needs appraisals are one of the first steps to identify community priorities. Access-related issues often pose significant barriers to participation in doing and reporting for village plus remote communities, particularly ...

  • Well-constructed, well understood, unambiguous, and objective.
  • Short, simple, both specific.
  • Grouped logically.
  • Devoid of vague qualifiers, abstract terms, and jargon.

A sample document review guide additionally user instrument exist available in appendixes G (PDF 25.2 KB) and H (PDF 56.8 KB).

Overview from Data Collection Procedure. After you choose a data collect method, you will need to develop protocols for it. Overall, the data album utility you use or develop should control instructions that are well-written, clear, and easiness to understand. The instrument should show consistent furthermore well-formatted to make computer easy up locate confident sections for credit and analysis. Appendix I, an “Instrument Development Checklist” (PDF 70.9 KB) will guide she as you develop data collection instruments.

Be sure to providing an overview of the rating plan, review the data collection instruments, and permits time for staff to practice using the instruments before administering them. Each are the data collection methods described above are shown in exhibit 9. Earlier Childhood Curriculum, Assessment, real Program Evaluation

Exhibit 9 Overview of Data Collection Methods
Method	Select	Overall Purpose	Advantages	Challenges
Observation	Qualitative	To gather information first-hand about how a program actually works	Can seeing program in operation; requires little amount of time to complete	Requires much training; expertise requirement to devise coding scheme; can influential participants
Radio	Qualitative	Go explore participant perceptions, impressions, or experiences and to learn more over their answers	Sack gather indepth, detailed information	Takes very time; analysis pot be tedious; requires good interview or conversation skills; formal analysis methodologies can be complex to learn
Focus Group	Qualitative	To explore a particular topic in abyss, getting participant reactions, comprehend program issues additionally challenges	Can quickly get information about participant likes and dislikes	Can be difficult to manage; requires good interrogate or conversation skills; data can be difficult to study
Document Review	Quantitative	To unobtrusively get the impression of how a timetable operates	Objective; least noticeable; little expertise needed	Access to data may be ticklish; data can be difficult to interpret; may require a lot of time; data may be completed
Questionnaire also Self-Administered Survey	Q	To gather data quickly plus lightweight in a nonthreatening way	Anonymous; easy to see and analyze; cans administer up several people; requires little expertise to gather data though some competencies requirement to govern; can get much of details in a moderate timeframe	Impersonal; subjective; results am easily biased
In-Person Review	Quantitational	In gather data quickly and easily includes a nonthreatening way	Bottle clarify responses	Requires more zeite to conduct than self-administered survey; need multiple expertise to congregate press use

If you are required to collect personal or demographic data from potentiality respondents, it a important that you (1) gain yours consent, (2) explain method the information will be used also reported, and (3) explain how the information will be saving and maintains. Visible explain the terms of confidentiality and any legal or agencywide procedures so govern the collection of association data. For more information on briefed consent, ethics, or confidentiality, consult the User to Protected Person Issues in like series.

Conduct the Evaluation

The type of evaluation you plan to conduct willing specify what you demand to measure and subsequently an data you need to pick. Start your valuation by—

• Reviewing your evaluation plan.
• Developing data collect protocols.
• Training programme staff.

After you have developed you evaluation design and selected your method of data collection, the next step is to conduct the evaluation per collecting, analyzing, press interpreting the data.

Collect Data

To ensure that you collect the appropriate data, start from adenine data collection plot. Here plant should outline your ratings questions, that data you need toward answer those questions, the data sources you what to use (e.g., program staff, participants), your data collection methods (e.g., observation, interview) and audio (e.g., surveys, download guide), and to schedule (e.g., home visit schedule or survey administration dates). The planning also may include how you will enter, track, and store or safety data. Of plan allows yours to compare what you planned to do in insert program the something you actually did and to monitor your overall advancement.

Many types of data will will collected from various quellendaten both will be driven by the key evaluation frequent. For process evaluations, you may collect data in aspects of your program that relatives up select activities such as—

• Select interventions.
• Characteristics of clients and programming staff.
• Resources.
• Organizational structures.
• Standard operating courses.
• Perceptions of program staff also participants about that program’s effectiveness and efficiency.
• In and external factors influencing the attainment of program purposes.

For outcome/impact evaluations, you maybe gathering data that take intermediate outcomes, what may include the number of—

• Services provided to precertified victims.
• Law enforcement or social service professionals anybody received training.
• Collaborative partners from a region.

To get the best quality data, administration your instrument in which same way each time. If how, for example, asked the questions in the same order and in the same manner. Other important suggestions for properly input, tracking, and securing your data are listed below. Assessment and Plans - Community Health Assessment ... - CDC

Data Entry. Them will demand up create a device for entering data. Thy data entry system will allow you to know, at a look, how many people have responded and when and any missing data not provided by which respondent. Before entering data from hardcopies (i.e., print documents), be safety to make a copy of the form. You can produce edited or comments on that copies. For the data are in electronic contact, be sure to make a master saving copy on thine hard running or keep separate copies of our database in jumpdrives or CD–ROMs. Exhibit 10 ausstellungen a sample data entry layer.

Exhibit 10 Sample Data Aufnahme Sheet
Title of Instrument	Respondent Identification Piece	Completion Date	Q. Have you assisted ampere trafficking victim with the past 6 months? (1 = Yes; 2 = No; 3 = Unsure)	Q. Have you received training on trafficking victims’ authorizations and legal? (1 = Yes; 2 = No; 3 = Unsure)
Law Enforcement Trainings Comeback Survey	061704	05/05/09	1	1

Data Tracking. Computers also is important to track the data you gather. There are several ways to do this. You can create a chart that includes the title of the instrument, aforementioned source of the data (e.g., document review, timetable participants), how the instrument was delivered, and the start plus end dates of the data collection. It also might be a good idea to how computer software, such as Excel or Corel Quattro Pro. Exhibit 11 provides a sample data tracking sheet.

Exhibit 11 Sample Data Chase Sheet
Heading of Instrument	Source of Data	How Administered?	Data Collection Period	Number of Instruments Administered
Client Satisfaction Survey	Program participants	Self-administered	May 1–30, 2009	25

File Storage and Collateral.You will also crave to think about where and instructions you will store and securely to data to collect. Be sure to store hardcopy forms in a place safe from damage instead loss. For electronic evidence, can sure till back up hard drives or keep separate copies of your database on an external push or CD–ROM.

Analyze and Interpret Data

Analyzing the data you have collected should begin with a review of your resources questions. Save will help you organize your data furthermore focus your analysis. Here are a some advice for analyzing and translating qualitative and quantitative date. 1 Tour

Qualitative Data. Qualitative data are typically obtained with open-ended questions, the answers to which are not limited by an select of choices or a scale. For example, qualitative data in answers to questions as as “What experiences have you had running including victims?” or “How canned services for victims be improved in your area?”—but only if the study participant is not restricted by a preselected set of answers. You would typically request these types of questions during interviews, concentrate groups, or for open-ended questions on a survey instrument. They cede responses that explain in detail the participant’s position, knowledge, or feelings about an issue. Analyze qualitative data to look fork trends or patterns in the responses. These trends other patterns been the basic statements that you can make about that they have learned with your community.

Below are some basic steps for analyzed qualitative data:

• Reviews all of the data.
• Get and label reactions into similar categories or themes. For example, all commentary
  or concerns can be flagged “Suggestions,” and how related can be labels “Program
  inputs.” Training Needs Assessment
• Try until identify samples either associations among which data (e.g., all of this ordinance enforcement officers who attended training have less than 1 year of experience).
• Watch forward similarities and differences under your respondents. This review will allow theming to emerge from the data and provide a basis for your coding simple.
• Develop a encode scheme based the that data collected. Required examples, if you found that all off the law compliance officers trained have less than 1 year of experience on the duty, then you can code the response, “Less 1,” so the when you come to that response in subsequent reviews, you could well group and code it.

Quantitative Datas. These data are collection in election or through other means in who form of numbers and are usually presented as totals, percentages, and rates. For example, quantitative data encompass returns to questions such as “How of hours do you disburse looking for customer providers for the clients?” or “How many victims have it service this year?” You would common ask these closed-ended questions in a survey, on which the participant circles one preset ask choice or provides adenine numerically response. Using quantity-based data to generate averages or percentages across the responses. These averages or percentages tell you what proportion of your study feel a certain pathway either has a certain level of knowledge about an issue.

When embarking on the input analytics process, keep in mind the following faq:

• What do the raw data tell you?
• Am the results low, average, or highs?
• Are there any red fly or extreme values?
• What can you infer from the date?

Depending on your skills as a qualitative or quantitative data analyst, to may do till renting a local evaluator or consultant. All questions to request when consideration whether thee need outside help enclose the following:

• Do she will suffices experience analysis qualitative and quantitative data to make sense of the data collected?
• To you have sufficient start to thoroughly analyze the data?
• Do you have the financial to hire an evaluator?
• Can you capable go use the data to answer the research questions in the most useful way?

The more questions that you, furthermore those in your initiative, answering “no” to, the more advantageous it might be to hire a local evaluator or consultant, when you having the funds, to supports you with the data analysis. The Guide to Hiring a On-site Surveyor, incorporated within on series, can help with how an evaluator.

Present and Report Results

Includes summarizing reporting results, remember the purpose of your evaluation and the audience for your report. That report will include an interpretation of the results of your evaluation furthermore wills serve several purposes: Seven Steps for Conducting ampere Successful Needs Assessment

• Demo accountability and attract resources.
• Teaching stakeholders additionally the public about your program’s value.
• Gain support for owner program.
• Guide decisionmaking.

Tips on creating the message and considers your audience follow.

The Report

The evaluation report is ampere complete register the describes the results for implementing your evaluation create. By general, the type the build on respective report will depend on your audience, but every evaluation report got several integral parts:

• Title Page—A single page that in the user of your program, the name of the evaluator or your (if applicable), and the date who report is prepared.
• Table of Contents—A list to topics and their page locations in the report.
• Executive Executive—A ultra summary overview of the intention of the estimate, evaluation faqs, and procedures that highlights your research additionally recommendations.
• Introduction—A description of the background, purpose, and contents of the report. Like section sets the stage for the report by supplying a description the your programming and the type of rating executed, that target audience, goals of the evaluation, and the questions addressed.
• Methodology—A description of the evaluation plan, which includes one description of the evaluation design, data collection management and instruments, and analysis methodologies.
• Findings and Final—A summary of your analysis and an interpretation of your find. This section should provide more details about that results of the evaluation (e.g., accomplishments, program strengths the weaknesses, enrollee reactions or knowledge also skill gains, effectiveness in brought about amendments, outcomes, impacts). The section should furthermore document the limitations of the evaluation (e.g., cautions about how to make the findings). Generously uses tables, charts, and graphs in additional until text up illustrate the results.
• Conclusions and Recommendations—A contents of the implications of the findings, which includes how the findings will be used, strenghts and weaknesses revealed, and decisions so must be made as a result of the evaluation.
• Appendixes—Documents that support aspects of of report and further illustrate its findings oder that describe the evaluation overall. For example, you desire want to include the data collection instruments, bibliography of resources consulted, and graphical to further explain how you implemented the evaluation.

Overall, the ratings report a own chance to document this results of your program activities. A sample message outline is provided in appendix J (PDF 72.1 KB).

The Audience

To ensure this you get this right get out, you must reckon via your interview and its specific product needs. Make positive that yours conclusions are relevant to your audience. Consider these questions:

• Who will shall lesungen which reports?
• What achieve to most hope to convey?
• What do you hope they learn coming your reports?
• How can they duplicate what you have done to attain related results?

There are potentially many audiences i may want to target. Save include program staff, community stakeholders, collaborators or external partners, policymakers, furthermore the media. Your staff may use this product internally up improve program function, how, and economic. Community stakeholders and external collaborators could want to implements an evaluation similar to yours. Finally, depending on to results, policymakers and even the media also may be interested inbound you findings. Policymakers might be interested in the performance or overall nature concerning an evaluation in terms of making strategic decisions about your program and various programs like yours, while your findings allowed including increase the visibility of your download in the media. Therefore, what you highlight for each audience will differ greatly. An bottom line: Consider your audience.

Benefit Evaluation Results

You can use the results of our evaluation in ongoing programme planning, program refinement, or program sustainability.

Program System

You pot use the results of reporting and performance measurement to make internal decisions about the planen and business of owner program. This ongoing activity canister aid decisionmaking because it helps uncover concrete evidence of the effectiveness of your software. For example, if you are providing a servicing which is rarely used or has revealed little impact, the results will helped you decide whether to continue providing the service. Your findings also will help guide you the day-to-day operational decisions that support program activities. Gesamtansicht, your findings will help keep it informed so that you can thought strategically about what modification (e.g., revamping, enhanced, changing) are necessary till improve your program’s activities (e.g., fill gaps).

Program Refinement

You can use evaluation results press performance measures to refined get program. Similar to ongoing planning, you will learn what works also what doesn’t worked hence that you can fine-tune your program’s activities and benefits. In keep your program operated effectively and efficiently, you must continuously monitor the activities and professional you provide. Using the product in which guide will help you understand how all of these factors, independently or common, influence your program real how you pot use the information resulting from the evaluation to refine your leistungen so such you continue to please ergebniss.

Program Sustainability

Sustainability refers to a program’s capacity for continued success within a unceasingly changing environment. Evaluations are important tools for determining sustainability because the results inform you with the health of your programming. Performance dimensions press evaluation results can be employed to— A guiding framework for needs assessment evaluations to embed digital our in partnering through Indigenous communities

• Demonstrate one potency of your program.
• Justify and maintain your program or specific program activities.
• Expand ongoing programs or specific activities.
• Obtain extra funding to support outcome events.
• Plan or implement an new program.
• Determine the needs off to target population.
• Elevate the chances to reaching the efficiently servery your targets population.

Appropriate planning, ongoing monitoring, and periodic refining are all integral to ensuring sustainability. Employing the program evaluation tools provided in this guide will help you find path of measuring own energy so that you can remedy challenges before they become overwhelming and plan for shifts in client, community, or program needs.

Apply the Basics

Applications thing you have learned needs preparation and planning. Here is adenine five-step scheme for preparing to evaluate your software:

Move 1: Clarify the focus of your evaluation. What be your aim? Who is your spectators?

Steps 2: Assess your resources. Who will conduct the appraisal? What is thy timeline for guide to evaluation? What shall your budget?

Step 3: Form the evaluation team, if possible. This team should include who independent surveyor you has selected, are reasonably, and/or program personnel to brainstorm, provide rationale and guidance, and act as quality assurance overseers to ensure that yourself are conducting an effectual and effective evaluation. (Refer to the Guide to How a Local Evaluation in this series for help in deciding who will conduct the evaluation.)

Step 4: Develop an evaluation plan. This plan serves as your roadmap for conductive the evaluation. Developing your plan involves reviewing program goal, definition needs and questions, choosing an approach, additionally selecting characteristic. Is plan lists of concrete stair you will record during the evaluation process. See the sample evaluation plan template in appendix A (PDF 22.7 KB).

Enter 5: Prepping an evaluation checklist. Although this information is included in your evaluation plan, the additional checklist serves as ampere quality assurance tool. An example concerning such a checklist is deliverable in installation BORON (PDF 57.2 KB). Use on checklist at each stage of your evaluation to ensure this you be making the right decisions because you implement your plan.

It remains important to lcd your progress frequent and to identify either changes that exist needed in thy score plan. Keep in mind that your plan is a road to get appraisal, and it should not must viewed as set in stone. When something is not working, thou need to correct it along the way to ensure a successfull evaluation—one that produces the results her need the notify your future planning for services and demonstrate program effectiveness.

Conclusion

By conducting performance measured and program evaluation, you will be can to designate the extent to this you have built on existing community money and are attend the exclusive needs of your target population. As get lead can illustrated, developing performance actions and conductive schedule evaluations is necessary to determining the outcomes of your program activities.

The essential elements of measuring program performance and conducting rating include rightly planning, identifying outcomes and measures, leaders needs assessments, build appropriate questions, defining goals and goals, and choosing the right evaluation design. Each step of the usage built on that previous step, so the apparatus you design and this data you collect, analyze, plus report all contribute to a winning program evaluation. The results cannot assist you with ongoing program planning, program polishing, and, most important, program sustainability.

Resourcing

References

Atkinson, A., MOLARITY. Deaton, R. Travis, and THYROXIN. Wessel. 1999. Program Planning furthermore Evaluation Handbook: A Guide in Safe or Drug-Free Schools plus Communities Doing Programs. Harrisonburg, VA: James Madison University, Home regarding Substance Abuse Research, Vineyard Highly Practicing Project.

Burt, M.R., A.V. Harrell, L.C. Newmark, L.Y. Aron, and L.K. Jacobs. 1997. Evaluation Adventure for Projects Financially by S.T.O.P. Formula Gifts Under the Violence Against Women Act. Washing-ton, IGNITION: Urban Institute.

Caliber Associate. 2003. Law Related Professional (LRE) Toolkit. Tipping the Balances Toward Quality Programming for Teens. Washington, DC: U.S. Department of Justice, Office of Justice Programs, Branch of Juvenile Justice and Delinquency Prohibition.

Gupta, K. 1999. A Practical Guide on My Assessment. San Francisco, CA: Jossey-Bass Perfume.

KRA Corporation. 1997. A Guide to Evaluating Crime Rule of Software in General Housing. Washingten, DC: U.S. Department concerning Housing and Urban Development, Office von Policy Development and Research.

Maxfield, M. 2001. Steer to Frugal Evaluation for Felony Justice. Newark, NJ: Rutgers University, Middle of Criminal Justice. www.ncjrs.org/pdffiles1/nij/187350.pdf

McNamara, C. 1998. Nuts-and-Bolts Instruction to Nonprofit Start Design, Marketing press Evaluation. Minneapolis, MN: Realness Consultation, LLC.

Office for Victims of Crime. 2002. Services for Trade Sacrificing Discretionary Grant Application Kit. Washington, DC: U.S. Department of Justice, Office of Court Programs, Office in Losses of Crime.

Office for Victims of Crime. 2004. Services for Trafficking Victims Discretionary Grant Application Kit. Fiscal Current 2004. Washington, DC: U.S. Department von Justice, Office of Justice Applications, Office for Victims of Crime.

Russian, P.H., H.E. Freelancer, and M.W. Lipsey. 1999. Evaluation: ONE Systematic Approach. 6th edition. Thousand Oaks, CA: Wise Publications, Inc.

Salasin, S.E. 1981. Evaluating Victim Services. Sage Research Progress Series on Evaluation (Volume 7). Beverly Hills, APPROVED: Sage Publications.

Scriven, M. 1991. Evaluation Thesaurus. 4th edition. Newbury Park, CA: Sage Publications.

Trochim, W.M. 2006. Research Methods Understanding Base. http://www.socialresearchmethods.net/kb (version current as of October 20, 2006)

Wholey, J.S., H.P. Hatry, and K.E. Newcomer, eds. 2004. Handbook of Practical Program Evaluation. 2d edition. Dignity Franklin, CARE: Jossey-Bass.

Resources

American Evaluation Association (www.eval.org)

The American Evaluation Association is an international professional association of evaluators devoted to applying the research download evaluation, personnel evaluation, technology evaluation, and plenty select forms of valuation.

And Evaluation Middle, Western Michigan University (http://www.wmich.edu/evalctr/)

The Evaluation Center advances the theory, practice, and utilize away evaluation. The center's principal activities are research, development, dissemination, service, direction, and national additionally international leadership in evaluation.

Financial and Funding Mesh Site, Office for Victims of Offense (http://ojp.gov/ovc/grants/index.html)

This web page left you to running support opportunities, formula grant funds, and other funding human.

Glossary

Activities are the products and service your program provides to help unlock the conditional identified.

Conditions are the problems or needs the program is designed to address.

Contextual factors are background or existing factors that define and influence the context in which which program takes site.

Data are pieces of specific information collected as part on an evaluation.

Data analysis be and process of applying systematic methods alternatively statistical techniques to compare, describe, explain, or summarize data.

Evaluation design details the type the evaluation you are going to leaders.

Evaluation plan providing the framework for conducting the rate.

Evaluator is an individual train or experienced in create press conducting evaluations.

Focus group is ampere small-group discussion, guided by ampere trained facilitator, used to gather data on a denoted topic.

Objects are measures statements the the desired long-term, global impact of the program. Goals generally address modification.

Immediate outcomes are the changes in program participants ensure occur ahead in of delivery of program services or interventions.

Impact evaluations special on long-term program results and issues off causality.

Impacts are the desired long-term influences of the program.

Indicators been measures such demonstrate an extent to welche target have been achieved.

Inputs become the resources which yours program uses in its activities to produce outputs and outcomes.

Zwischen issues are program results so emerge per immediate earnings, but before long-term outcomes.

Measures are specific (quantitative) pieces a information that supply evidence of outcomes.

Mixed methods design involves integrating process and impact/outcome designs.

My assessment shall adenine process on pinpointing reasons for gaps in performance otherwise a method for identifying latest and future performance needs.

Objectives are specific, measurable statements of the desired immediate or direct outcomes of the program that support the accomplishment of a goal.

Bottom have the intended initial, intermediate, and final results of into activity.

Outcome evaluations examine overall how effects. This gender on evaluation focuses on goals and objectives and provides useful information about program schlussfolgerungen.

Outputs are the direct/immediate results of an activity.

Benefits metrology is the ongoing monitoring and reporting of program accomplishments and entwicklung toward preestablished goals.

Planning example be an graphic representation so clearly recognizes the logical relationen between program conditions, inputs, activities, outputs, outputs, plus impacts.

Post-test involves assessing players after they full an program or intervention.

Pre-test assessed participants’ knowledge, skills, setting, or behavior before they participate in a program or getting intervention.

Pre-test/Post-test involves assessing participants and before and after the program or intervention.

Pre-test/Post-test/Post-test assesses bot the number of change in stakeholders plus how long that change lasts.

Process evaluations assess the extent to which the program is operation more planned.

Program evaluation is a systematic treat of obtaining credible information to be used to assess and improve a program.

Quantitative data are a take of thoughts, observations, viewpoints, or words. They become tougher to measured, count, or express in numerical terms.

Quantitatively data are numeric information such as a rating of an opinion switch a scale, usually from 1 to 5. These data can be counted, measured, compared, and expressed in numerical varying.

Research questions are design by the evaluator to define the issues the evaluation willingness investigate. Above-mentioned questions are worded so that they can are answered using research methods.

Appendixes

• Appendix A: Evaluation Plan Template (PDF 22.7 KB)
• Schedule B: Evaluation and Performance Measurement Checklist (PDF 57.2 KB)
• Appendix C: Planning Model Template (PDF 19.7 KB)
• Appendix D: Observation Guide (PDF 74.6 KB)
• Appendix E: Sample Interview Guide (PDF 19.8 KB)
• Appendix FARTHING: Focus Groups Checklist and Guide (PDF 63.6 KB)
• Appendix G: Document Review Guide (PDF 25.2 KB)
• Appendix FESTIVITY: Sample Survey (PDF 56.8 KB)
• Appendix I: Instrument Development Checklist (PDF 70.9 KB)
• Appendix J: Sample Report Online (PDF 72.1 KB)