UCCF Evaluation Approach
Process, Quality, Learning, & Impact
The University Consortium for Children and Families (UCCF) is dedicated to high-quality training, dynamic learning, and positive impact on the practices of human services workers and the lives of those they serve. To accomplish these goals, the UCCF employs a utilization-focused and developmental evaluation framework to continuously assess and improve training for human services workers, their organizations, and the communities they serve. Four major aspects of training are routinely evaluated: process (e.g., training completion, implementation, and procedures), quality (e.g., training approach, perceived learning, content, and relevance), learning (e.g., transfer of knowledge and skills), and impact (e.g, immediate and longer-term changes in attitudes, work behaviors, practices, and outcomes). The UCCF Evaluation Group (including evaluators from California State University Long Beach, California State University Los Angeles, the University of Southern California, and University of California Los Angeles), in partnership with training staff and the UCCF Assessment and Accountability Committee (comprised of a diverse group of stakeholders from across human service agencies, Universities, and other community members), design and conduct evaluations that include tracking, measurement, analysis, and reporting of these training aspects, as appropriate for each training provided. Common approaches to evaluating each of these components of the training process are outlined below.
UCCF Evaluation Framework
- Dimensions of the training process evaluated include tracking of training completion, fidelity of implementation, and training procedures
- Both institutional data and primary data collection (e.g., surveys and observations) used combined to evaluate training process
- Dimensions of training quality evaluated include training approach, perceived learning, content, and relevance
- Measurement instruments include a mix of qualitative and quantitative assessments, including self-report surveys, interviews, or focus groups with trainers and trainees
- Dimensions of trainee learning evaluated include acquisition of knowledge and skills relevant to the goals and outcomes identified for trainings
- Measurement of knowledge and skill acquisition includes pre/post assessments that often incorporate vignette-based measures that provide learners the opportunity to demonstrate the ability to apply new knowledge or skills to realistic case scenarios
- Outcomes evaluated subsequent to training include immediate and longer-term changes in trainee attitudes and practices or behaviors. For more broad-based training approaches (e.g., academy training), comprehensive and longer-term indicators of training impact for trainees (e.g., retention, fidelity of practices), and outcomes (e.g., health and welfare) for the communities’ human service workers support are assessed.
- Multi-method approaches are employed to measure longer-term post-training knowledge and skill application among trainees. Broader and longer-term indicators of training impact, among trainees and those they serve, are often assessed though analysis of existing data (e.g., administrative), or primary data collection (e.g., surveys, focus groups, interviews).
Dissemination of Evaluation Information
Evaluation reporting is routinely disseminated for all trainings to diverse stakeholder groups, including UCCF Management and Staff, the UCCF Assessment & Accountability Committee, human service agency partners (e.g., the Los Angeles Department of Children and Family Services), and other stakeholders (e.g., community members). Dissemination occurs through innovative approaches to data visualization (often hosted on this site), narrative reports, presentations, and small group meetings. Stakeholders with access to the UCCF Evaluation Site (See https://eval.uccf.la/) may review and interact with evaluation information such that they can tailor reporting to meet their unique information needs. For access to the UCCF evaluation site, please contact the Robert Blagg, UCCF Director of Evaluation (email@example.com)
UCCF Evaluation Site
The UCCF Evaluation Group is dedicated to empowering stakeholders to use data to make more timely and sound training, practice, program, and policy decisions. This goal is accomplished in part by putting vital information in the hands of users or stakeholders in the form of data visualizations. This site is a primary conduit through which data is shared with stakeholders.
This site will continue to expand as stakeholders define their information needs. If you have any questions, please contact the UCCF Director of Evaluation (Robert Blagg – firstname.lastname@example.org; 310-825-0852).