Frequently Asked Questions
Toolfind contains a wide variety of tools-encompassing different outcome areas, focusing on varying stages of change, and using differing methods and sources for gathering information. When considering the many tools available, take your time and be thoughtful. Just because a tool matches your outcome area of interest does not mean it is the right tool for you. For more help in finding the right tool, go to Finding a Tool That Fits! This guide will help you consider your needs and priorities and before determining whether available tools fit your program’s purpose and audience, measurement plan, capacity & resources, and the population of youth you serve.
Toolfind is a searchable database containing detailed descriptions of 36 tested tools that are available to programs in 14 youth outcome areas. The table shows the 14 outcome areas and examples of elements of the outcome area that tools might measure.
|Outcome Area||Measures related to this outcome might examine:|
Academic/College & Career Readiness Outcomes
Teacher ratings of youth’s academic progress
Youth self report of academic progress
Youth knowledge and skills in core academic areas
Programs designed to promote specific academic skills and content are encouraged to work collaboratively with the local school system to coordinate testing efforts. In addition, a few selected academic measures can be found on the FAQ page of Toolfind.
|Higher Education/Employment Readiness|
• Future expectations/aspirations
• College Planning
• Job/Career awareness
• Job seeking skills
• Work Place conduct/Work Habits
• Time management skills
• Level of participation/engagement in school or program activities
• Interest in what they are doing or learning
• Intrinsic motivation, initiative, persistence
• Sense of/beliefs regarding their academic competence
• Attitude towards school and motivation to achieve
• Study habits and homework completion
For academic performance see "Academic Skills".
• Youth possess particular skills and beliefs related to resiliency such as: a positive attitude and self-concept, values achievement, feeling supported by adults in their life.
• Program/community offers external supports and environments believed to promote resiliency. (Note that external assets are not youth outcomes per se, but rather the conditions believed to promote youth's resiliency.)
|Emotional Well Being|
• Ability to cope with frustration and manage anger
• Perception of one's own ability to make a difference (self-efficacy)
• Positive outlook towards life, future
For positive self concept, confidence see "Self Concept"
• Conduct in program or school (e.g. abides by rules, cooperative with staff requests)
• Absence of challenging/disruptive behaviors Absence of socially aggressive behavior
• Ability to maintain self control, regulate own behavior
• Youth has respect for authority/rules/limits
For delinquent or risky behaviors see "Healthy Lifestyles".
• Self Confidence
• Sense of competence socially, academically
• Positive identity-sense of self
• Healthy Habits
• Reports of, and attitudes towards, risky behaviors
• Ability to resist peer pressure
• Knowledge of nutrition and healthy lifestyles
• Safety (in home, school, and community)
For measures of general conduct in program/school see "Positive Behavior".
|Adult Youth Relationships|
• Respectful adult-youth interactions
• Emotional support for youth
• Feelings of trust, intimacy, understanding between youth and adults
• Youth feel that adults care about them, that they are important, that they matter
• Parent-Teacher communication
• Supervision and monitoring of their child’s after school activities/time
• High expectations for academic success/monitoring homework completion
• Involvement in school activities
• Parent-child communication
• Emotional bond/trust between parents and youth
|Peer Relationships/Social Competence|
• Ability to relate positively to peers-uses social skills
• Ability to form friendships
• Ability to resolve conflicts constructively
• Peers/friends viewed as social supports
For socially aggressive behavior see "Positive Behavior".
Youth Leadership and Civic Outcomes
• Youth participate in running of program
• Youth set personal goals-challenge themselves
• Youth lead a project or mentor other youth
• Youth stand up for their beliefs, advocate
|Problem Solving/Decision Making|
• Youth draw upon information and experiences when making decisions/solving problems
• Youth apply problem solving skills
• Youth consider future implications and consequences of their decisions
• Knowledge of citizenship, community systems, government and the democratic process
• Participation in community service/volunteer projects
• Connections with community groups
• Awareness of community resources
The Toolfind Profile Page includes the following information about each tool:
Features of the tools:
- Tool Name
- Cost of tool
- Age group(s) that the tool applies to
- Number of items in the tool
- Scales or sub-scales included in the tool
- The outcome areas the tool measures
- Whether the tool is available in Spanish
- Whether the tool is available online
- Whether the tool is ready to use
- If the tool comes with instructions for use
- If training is required or offered
- If software or scoring or reporting services are available with the tool
- If an online database is available for users
- Field testing
- Reliability and validity testing
- Availability of comparative data
- If the tool has been used in afterschool research or evaluation
- If the tool is linked with an approach
- If the tool is part of a larger assessment package
- If the tool has been adopted by cities/states, consortiums of programs, or large initiatives
- Whether the tool's researcher must be notified before the tool is used
- If there are pre-qualifications required and what (if any) qualifications are needed to use the tool
- Special notes or considerations
- Special instructions to follow when using the tool
- Research related to the tool
- Credits or acknowledgements to include if you use the tool
Identifying and Contact information:
- Author(s) or organizations that developed tool
- Year published
- Where to obtain the tool
- Where to get more information about the tool
- Where to get contact or permission information about the tool
- Where to get additional materials or research related to the tool
It offers newly available tools and up-to-date information.
Toolfind includes several tools that have been used in large scale after school evaluations but have not been easily available -- until now. It also offers up-to-date pricing, contacts and background on each tool, including information that may NOT be available on product web sites!
It provides a wide variety of tools to choose from.
The variety of tools in Toolfind gives programs many choices for finding measures that meet their needs, interests and resources. For instance:
- Some tools in Toolfind target one outcome, others examine numerous outcomes, and some also examine aspects of program quality.
- Some tools are "ready to use," while others require you to customize them to fit your outcomes of interest.
- Some tools focus on early outcomes, such as youth's experiences in the program, while others focus on "intermediate" outcomes that might be gained after longer participation in the program, and still others examine long term outcome areas that require much more time to show change-such as self-concept.
- Tools encompass various age groups and respondents (e.g. youth, staff, parent, teacher).
- Many tools listed in Toolfind are available with services for data entry, analysis and reporting.
It gives you lots of ways to search and compare tools.
Toolfind has been structured so that you can browse all tools or search by outcome area or age groups. It includes a brief summary as well as a full-length profile of each tool that meets your search criteria.
It's not a recommended list of tools.
All tools listed in this directory were found to meet certain selection criteria (see Selection Criteria). However, it is important to note that Toolfind is not a recommended list of tools. Only you can decide if a tool is right for you.
It's not a guide to program quality assessment.
While several of the tools listed in this directory also examine aspects of program quality, Toolfind was not intended to be a resource for quality assessment tools. For a guide to Quality Assessment tools, see "Measuring Youth Outcome Program Quality: A Guide to Assessment Tools, 2nd Edition" published by the Forum for Youth Investment, Impact Strategies. (http://www.forumfyi.org/content/measuring-youth-program-quality-guide-assessment-tools-2nd-edition)
It's not a guide to family, community-level outcomes.
The purpose of Toolfind is to identify tools that measure YOUTH outcomes-not family outcomes, or community-level outcomes.
It's not a "How to Measure Outcomes" guide.
Toolfind gives information about measurement tools, not how to develop or use outcome measurement systems. Information related to designing and conducting an outcome evaluation can be found in FAQ What resources can I use to get ready for outcomes evaluation?
It's not a collection of ACTUAL tools.
Toolfind includes detailed DESCRIPTIONS of tools-not copies of the tools themselves. To preview or obtain copies of tools, you can click on the website hyperlink provided or you can contact the author or organization using the information provided. ain copies of tools, you can click on the website hyperlink provided or you can call the source of the tool using the information provided.
Changes to tool wording, ratings and scales
Every tool listed in Toolfind has undergone field, reliability or validity testing. This means that tremendous effort, care, and testing has gone into the exact wording of questions, the options for responding or rating those questions, and decisions about which sets of questions (scales) measure which outcomes. NEVER change the wording of any question, the rating options or add/ eliminate any questions from a scale unless there are instructions that specifically tell you that this is allowed.
If a list of items/questions are said to be part of a "scale", "sub-scale" or "item-set", then ALWAYS use ALL items/questions in that scale must be used together. Do not use single items from any measure unless instructions indicate that it is OK to do so.
Instruments from publishing companies
If you purchase a tool from a publishing house, then most likely copyright laws protect the survey. This means that you may NOT alter the survey in any way or even reproduce the survey in whole or part WITHOUT explicit written permission from the publisher. Feel free to use the contact information in the profile to ask questions.
Using individual scales from a survey
Many tools are comprised of multiple scales (i.e. sets of questions). In many cases, it is fine to use a single scale. In fact, when a survey has been designed to measure multiple outcomes, usually you can simply pick and choose which outcome--and therefore, which scales-- to use. In other words, you don't have to use the whole survey. However, a tool designed to focus on a single outcome, (e.g. leadership), may contain several sub-scales that are designed to work together to provide you with a comprehensive picture of that one outcome area (e.g. sub scales might be: sets personal goals, challenges themselves leads others, and stands up for their beliefs). For single outcome tools, it is usually advisable to use all sub-scales.
Acknowledging your source
In cases where you are constructing your own survey using available scales, you will ALWAYS need to acknowledge your source/s in your printed surveys and in any reporting of results.
Here is how to reference your sources as a footnote:
Please see the "Instructions for Credits and Acknowledgements" section of your selected tool to be sure you are using the correct wording.
Instructions that accompany the tool
If you have important questions about alternate uses or aspects of use that are not covered in the instructions, you should contact the publisher, researcher or organization that developed the tool. Contact information is included in the profiles.
To be considered for Toolfind, tools had to meet the following selection criteria:
- Applies to one or more of the listed outcome areas
- Has undergone reliability and/or validity testing, or has been extensively field-tested and is being widely used.
- Is available for use by UNITED WAY programs.
In addition, tools meet at least one of the following preferred criteria:
- Free or affordable.
- Available online.
- Minimal qualifications needed to administer.
- Been used in afterschool settings or evaluations.
- Comes with support or software to help with scoring, interpretation and reporting.
- Has available data that allows programs to compare their findings with scores from large, and/or demographically diverse samples.
The UNITED WAY has had a long-term interest in increasing the focus on measuring benefits to program participants. We believe what organizations learn from measurement can help improve programs, communicate their value to supporters, and enhance benefits for youth. With Toolfind, we hope to encourage agencies to use research-based tools when measuring outcomes, both to enhance confidence in the results and to focus attention on those outcomes that are known to be important for youth development.
By their nature, the tools in Toolfind focus on quantitative results. Qualitative approaches that derive data from observations, interviews, or verbal interactions and focus on the meanings and interpretations of the participants are valuable in their own right and can also be important complements to quantitative approaches.
Our work on Toolfind began with an analysis of information about the outcomes of UNITED WAY affiliates' youth programs. We also scanned the standardized tools affiliates currently use to identify the outcomes each tool addresses. We concluded that a short list of key outcomes could meet the needs of many programs, and that there are available research-based tools that address these outcomes.
We hope Toolfind will also be a vehicle through which funders of youth programs can focus their attention on those outcomes that sound research identifies as being most clearly associated with positive results. By working collaboratively with a variety of funders, we hope to simplify the messages programs receive about measurement and expand the resources available to them.
A comprehensive scan of available measures in 14 youth outcome areas was conducted in 2005-2006. A 2nd scan was conducted during the winter/spring 2011. Key sources for the scan were in education, psychology, youth development and after-school such as:
- Available compendiums of youth outcome measures.
- Publications of large-scale after-school research studies.
- Instrument publishing houses.
- Web sites for research organizations and intermediary after school organizations.
- General on-line searches.
The following resources yielded several appropriate tools that are included in Toolfind. Readers may want to consult these resources themselves when seeking other measurement instruments that fit their needs. This list is not a recommendation of the following sites, nor is it meant to be comprehensive.
- Child Trends. (2003, March). Indicators of Positive Development Conference. Washington, D.C.: Child Trends (www.childtrends.org/Files/ConferenceSummary.pdf).
- Harvard Family Research Project. (2008, August). Measurement Tools for Evaluating Out-of-School Time Programs: An Evaluation Resource. Cambridge: Harvard Family Research Project. (http://www.hfrp.org/publications-resources/browse-our-publications/measurement-tools-for-evaluating-out-of-school-time-programs-an-evaluation-resource).
- Harvard Family Research Project. (2009, May). Data Collection Instruments for Evaluating Family Involvement. Cambridge: Harvard Family Research Project. (http://www.hfrp.org/family-involvement/publications-resources/data-collection-instruments-for-evaluating-family-involvement).
- Partnerships for After School Education (PASE). (2010, September). Afterschool Youth Outcomes Inventory. New York: Partnerships for After School Education (PASE). (http://www.pasesetter.org/reframe/tools.htm).
- Pratt, C. C. Katzev, A., Ozretich, R., Henderson, T., and McGuigan, W. (1998). Measuring Outcomes for Oregon’s Children, Youth, and Families: Building Results III. Oregon State University, Family Policy Program. Prepared for the Oregon Commission on Children and Families: 530 Center St. NE, Suite 405, Salem, Oregon 97301. (503) 373-1283 (http://www.hhs.oregonstate.edu/hdfs/results-accountability).
- Program in Education, Afterschool & Resiliency. (2009). Assessment Tools in Informal Science (ATIS). Belmont: Program in Education, Afterschool & Resiliency (PEAR). (http://www.pearweb.org/atis) .
- RMC Research Corporation Denver. Compendium of Assessment and Research Tools (C.A.R.T.) for Measuring Education and Youth Development Outcomes. Denver: RMC Research Corporation Denver. (http://cart.rmcdenver.com/)
- Wilson-Ahlstrom, A., Yohalem, N., Dubois, D., & Ji, P. (2011) From Soft Skills to Hard Data: Measuring Youth Development Outcomes. Washington, D.C.: Forum for Youth Investment. (http://www.forumfyi.org/).
The following list includes assorted guides, other websites, and publications that may be useful to readers who want to learn more about evaluating youth outcomes.
- Allen, T., & Bronte-Tinkew, J. (2008). Outcome Evaluation: A Guide for Out-of-School Time Practitioners Part 4 in a Series on Practical Evaluation Methods. Washington, DC: Child Trends. http://www.childtrends.org/Files//Child_Trends-2008_01_07_OutcomeEvaluation.pdf
- Chinman, M., Imm, P., & Wandersman, A. (2004). Getting to Outcomes 2004: Promoting Accountability through Methods and Tools for Planning, Implementation, and Evaluation. Santa Monica, CA: RAND Corporation. www.rand.org/publications/TR/TR101
- Curnan, S., & LaCava, L. (2000). Getting ready for outcome evaluation: Developing a logic model. Community Youth Development Journal, 16 (1), 8-9. www.cydjournal.org/2000Winter/hughes_S1.html
- "Evaluation Forum." A list of evaluation resources/publications. http://www.evaluationforum.com/publications
- Geiger, E. & Britsch, B. (2003). Out-of-School Time Program Evaluation: Tools for Action. Portland, OR: Northwest Regional Educational Laboratory. http://educationnorthwest.org/webfm_send/148
- Harvard Family Research Project. Site includes multiple resources available related to after school evaluation. Visit address below or call (617) 495-8594. http://www.gse.harvard.edu/hfrp
- Hatry, H., Cowan, J., & Hendricks, M. (2004). Analyzing Outcome Information. Washington, DC: The Urban Institute. www.urban.org/publications/310973.html
- Lampkin, L. M., & Hatry, H. P. (2003). Key Steps in Outcome Management. Washington, DC: The Urban Institute. http://www.urban.org/publications/310776.html
- Little, P, S Dupree, and S Deich. (2002). Documenting Progress and Demonstrating Results: Evaluating Local Out-of-School Time Programs. Cambridge, MA: Harvard Family Research Project and the Finance Project. http://www.hfrp.org or www.financeproject.org
- Morley, E., & Lampkin, L. M. (2004). Using Outcome Information: Making Data Pay Off. Washington, DC: The Urban Institute. http://www.urban.org/publications/311040.html
- National 4-H Council and the University of Arizona. (2009). Building Partnerships for Youth. Tucson, AZ: The University of Arizona. http://cals-cf.calsnet.arizona.edu/fcs/bpy/index.cfm
- United Way of America. (1996). Measuring program outcomes: A practical approach. Alexandria, VA: United Way of America. http://www.unitedwaystore.com/product/measuring_program_outcomes_a_practical_approach/program_film
- Weiss, C. H. (1998). Evaluation 2nd ed. Upper Saddle River, NJ: Prentice Hall.
There are many tools and resources available to assess changes in youth academic skills. The list of resources below includes a few sources to get you started. After school and other youth serving programs interested in assessing academic skills are strongly encouraged to contact their local school system in order to coordinate testing efforts.
Houghton Mifflin Harcourt
McGraw Hill- CTB
Pearson Clinical Assessments
Pro Ed Publishers
Western Psychological Services (WPS)
Other Resources for Academic Assessment in Afterschool:
Assessment Tools in Informal Science (ATIS). Program in Education, Afterschool & Resiliency (PEAR). http://www.pearweb.org/atis
Communities Organizing Resources to Advance Learning (CORAL). http://www.coralsanjose.org/index.html
DIBELS Data System. University of Oregon Center on Teaching and Learning.
National Partnership for Quality Afterschool Learning. Southwest Educational Development Laboratory. www.sedl.org/afterschool
Route 21. Partnership for 21st Century Skills.