Research and Evaluation
CRP research scientists apply requisite skills and capabilities in defining policy problems and conditions.
CRP research scientists use a wide variety of web-based search engines, directories,databases, and other information sources to conduct reviews of literature. CRP also builds and maintains readily accessible and fully documented research databases, while preparing comprehensive research reports of the study background, methods, findings, interpretations, and conclusions in publishable forms. CRP's capabilities utilizing quantitative approaches include survey research methods, meta-analysis, experimental research/design, quasi-experimental research/design, and web surveys. CRP offers a wide range of qualitative capabilities, including case studies, content analysis, ethnographic studies/research,cognitive interviews, focus group research, key informant interviews, and participant observation. While neither quantitative or qualitative methods should be viewed as exclusive, CRP also offers services in methodological triangulation, which provides a stronger research design, resulting in more valid and reliable findings.
Evaluation ResearchCRP executes culturally relevant, context-sensitive, and methodologically sound evaluation methods in collaboration with stakeholders to support decision-making, accountability, and program or policy planning priorities. CRP has expertise in designing all types of evaluation studies, including formative, process, and outcome evaluations, using an array of evaluation design methods. The evaluation designs CRP uses most frequently include one group posttest only design, one group pretest/post-test design, nonequivalent comparison group (quasi-experimental) design, and experimental design.
Samples of Work
Focused Reporting With Data From the National Assessment of Educational Progress (NAEP)National Assessment Governing Board (NAGB) has contracted with CRP to produce reports, primarily using data from NAEP, focusing on education topics of current interest. CRP will produce reports on rural education, private and charter schools, large urban school districts that participate in the NAEP Trial Urban District Assessment, and inclusion of English language learners and students with disabilities in NAEP. Using less well-known NAEP data, these reports will take the form of infographics or other digital media. The reports will provide insights on these topics, as well as show the tremendous range of information about students, teachers, and schools that is collected with each NAEP assessment.
NAEP - Early Childhood Longitudinal Study (ECLS) Cognitive Interviews and ReportCRP conducted cognitive interviews (often referred to as a cognitive laboratory study or cog lab) for one of the National Assessment of Educational Progress' (NAEP) special studies: NAEP-ECLS (Early Childhood Longitudinal Study). NAEP is administered by NCES, part of the Institute for Education Sciences within the U.S. Department of Education. The NAEP-ECLS survey questionnaire was designed to validate student responses to socioeconomic-status-related questions by comparing them to parent responses from NAEP-ECLS. The cognitive interviews were conducted with fourth and fifth grade students recruited from the metropolitan Washington, DC area. CRP made every effort to engage students from a range of demographic groups with regard to gender, race/ethnicity, socioeconomic background, and urban and suburban settings.
NAEP Pretesting of Survey Questions for 2017 Core, Reading, Writing, and Mathematics Questionnaires - Cognitive InterviewsCRP recruited and conducted 86 cognitive interviews of principals, teachers, and fourth, eighth, and twelfth grade students in the Washington, DC metropolitan area to include the cities of Washington, DC, Alexandria, and Arlington, and the counties of Fairfax, Prince George's, and Montgomery. The cognitive interviews will support the development and validity of newly developed survey questions for the NAEP assessment, particularly informing which questionnaire items should be used during the upcoming 2016 — 2017 NAEP administrations. To meet the timeline for this aggressive schedule, CRP trained approximately 15 doctoral students and retired educators in cognitive interviewing techniques and procedures.
NAEP Focus GroupsCRP conducted 14 focus groups (combination of virtual and face-to-face) composed of 4th and 8th grade administrators, reading and math teachers, and students. The National Assessment of Educational Progress commissioned the focus group research to better understand the following constructs in the context of reading and mathematics: (a) familiarity with technology, especially as it relates to mathematics and reading (e.g., e-readers, tablets, software, apps, and Internet resources); (b) access to technology relevant for learning at home and at school, including patterns of access across student groups; (c) ways in which technology (especially tablets, smart phones, and Internet resources) changes students' experiences with text, numbers, and calculations; (d) how the use of technology facilitates instruction and learning, both in and outside of the classroom; (e) shifts in the nature and type of skills required of students as they read and/or do math in technology-rich environments. Eight (8) states were randomly selected for recruitment of administrators and teachers for the virtual focus groups originating from CRP. CRP chose the largest population center of each state (urban/suburban) and a random rural population area.
Peer Health Exchange (PHE) Program Student SurveyPeer Health Exchange (PHE) is a national nonprofit organization that gives teenagers the knowledge and skills they need to make healthy decisions. CRP worked closely with PHE to assess the impact of PHE health education using the PHE Outcome Evaluation Survey tool in the following areas: (a) Alcohol, Tobacco, and Other Drugs; (b) Nutrition and Physical Activity; (c) Sexual Health; (d) Mental Health; and (e) Decision Making and Communication. CRP provided statistical expertise to analyze pretest and post-test data for appropriate statistical significance. Students in four sites completed pre- and post-surveys: Chicago, IL; New York City, NY; San Francisco, CA; and Washington, DC. Pair sampled t-test and McNemar Chi Square statistical tests were used for data analysis. CRP cleaned the data collected using the PHE survey and prepared a report to reflect the impact of PHE education on each area nationally and site-wise. CRP also provided recommendations to administrators regarding survey design and survey scales for the PHE survey.
District of Columbia Public Schools (DCPS) HIV/AIDS Education Extensive StudyThis project, which used a goal-attainment model, featured formative and summative evaluation components to assess five major areas spanning grades 4—12: (1) school HIV / AIDS policies; (2) teacher training; (3) curriculum implementation; (4) student knowledge, attitudes, and behavior; and (5) family involvement and community linkages.
Centers for Disease Control and Prevention (CDC) Research Monograph: Cultural Competence for Providing Technical Assistance, Evaluation, and Training for HIV Prevention Programs - A Research SynthesisThis CRP-prepared CDC research monograph was hailed as a significant contribution to the literature on behavioral and social science research on HIV/AIDS prevention planning and on culturally specific HIV/AIDS programmatic applications targeting racial and ethnic populations. It included an exhaustive review of over 400 articles on HIV/AIDS prevention programs nationwide.
Policy Assessment and Analysis
Briefing Alerts - National Assessment of Educational ProgressOn a monthly basis, CRP prepares policy briefs for the National Assessment of Educational Progress (NAEP) program, commonly referred to as "The Nation's Report Card." These Briefing Alerts provide timely information on education issues, research, events, and trends, including educational measurement and assessment. Recent editions have examined and illuminated a broad range of current and emerging issues related to NAEP, such as the potential impact of the NAEP Technology and Engineering Literacy (TEL) Assessment on K-12 engineering education, the Every Student Succeeds Act (ESSA), noncognitive factors, and within-and between-school factors influencing the black-white achievement gap. Briefing Alerts also have explored implementation and the equitable delivery of the Common Core State Standards, 21st-century skills as constructs or educational outcomes and fundamental transformations shaping their demand, deeper learning initiatives, and games-based learning and assessment.
The following publications are representative of CRP's contributions to policy emphases spanning the field of behavioral health.
CRP Supports SAMHSA in the Development of Congressional Report on Underage DrinkingA report on underage drinking addressing the serious public health issue that significantly affects young people under the age of 21
Identifying and Selecting Evidence Based InterventionsGuidance document for the Strategic Prevention Framework State Incentive Grant Program
AIDS - An Extensive Study (With Executive Summary)CRP's HIV/AIDS study prepared for DC Public Schools, had broad policy implications
SAMHSA Conference on Trauma and Behavioral Health: Lessons Learned From the First Cohort of the Iraqi-SAMHSA InitiativeThe Iraq-SAMHSA monograph on the first cohort of Iraqi behavioral-health practitioners
SAMSHA/CSAP A Comprehensive Plan for Preventing and Reducing Underage DrinkingCRP's seminal reports to Congress on underage drinking
Cultural Competence for Providing Technical Assistance, Evaluation, and Training for HIV Prevention ProgramsCDC monograph on the integration of cultural competence into HIV/AIDS prevention programming
SAMSHA/CSAP Prevention and Wellness Pilot Listening SessionsA report on a major behavioral health initiative at SAMSHA
SOCIAL SECURITY ADMINISTRATIONCRP facilitated an evidence base that informed the development of substance use disorder and a co-occurring mental disorder policy
The Surgeon General's Call to Action to Prevent and Reduce Underage DrinkingCRP prepared this report, which seeks to raise awareness and engage government, individuals, and private sector institutions in promoting action on underage drinking and its adverse effects
CRP's program management capabilities and its staff's knowledge of techniques and practices place CRP in a key position to support clients' program needs ranging in complexity and scope. Whether supporting mission-critical priorities, managing technical requirements, or providing specialized training and technical assistance, CRP's goal is the same: to ensure that its clients' return on investment for program management support meets or exceeds expectations. CRP accomplishes this goal by applying a structured and comprehensive portfolio of methodologies to clients' specific policy and/or programmatic needs. CRP's demonstrated success in managing contracts is attributable in large part to its staff's dynamic understanding of programmatic concepts, processes, competencies, and tools. Well-defined program plans and work schedules—the nucleus of CRP's program management model-serve as critical success factors and blueprints for the fulfillment of client program objectives. CRP's program management staff can capably facilitate the tracking of various functions and staff activities, estimate and maximize resource utilization, and adjust project interdependencies while ensuring responsiveness to client expectations.The National Assessment for Educational Progress (NAEP) is a critical national instrument in assessment and educational reform. In support of NCES/NAEP priorities, CRP manages a comprehensive program of work in collaboration with senior NCES staff members. CRP also collaborates with other NCES/NAEP contractors to include ETS, AIR, HumRRO, Fulcrum, and Westat in implementing specific phases of the national assessment process. CRP further manages logistical requirements for NCES/NAEP meetings and expert panels, including the National Assessment Governing Board. CRP, providing full-time on-site staff, coordinates production support for NCES/NAEP public release reports and manages content review for all NCES/NAEP publications and materials. CRP also provides an array of expert consultants who are nationally regarded in advancing the policy and programmatic aims of the NCES reports. These experts provide editorial reviews of and comments on NCES reports, and serve as technical reviewers, writing briefings on educational trends and topics that are relevant to NCES. Some of the experts are currently working with NCES and the Commissioner, coordinating responses to requests for information from the Secretary of Education, Congress and other federal agencies, the media, and constituency organizations. These experts exhaustively examine each NCES report to assess varied factors, including significance of the topic to the field of education; quality of the scholarship; appropriateness of the conclusions; clarity of the presentation; and adequacy of methods, analyses, and language, if causal claims are advanced.For the Center for Substance Abuse Prevention (CSAP), within the Substance Abuse and Mental Health Services Administration (SAMHSA), CRP provided program support to SAMHSA/CSAP's Centers for the Application of Prevention Technologies (CAPTs) in their efforts to transfer substance abuse prevention knowledge, apply prevention skills, and translate prevention science into practice for the prevention field. CRP managed an array of technical support services aimed at advancing policy and programmatic priorities encompassing SAMHSA/CSAP's Service to Science Initiative, data-driven substance abuse prevention planning, the Strategic Prevention Framework (SPF), and health reform. The scope of activities included Service to Science national, regional, and Native American-focused academies; awarding of Service to Science mini-subcontracts (in excess of $4 million) for building evaluation capacity; travel coordination for CAPT expert consultants; Web-based training; logistics support to conferences, meetings, and workshops; prevention and wellness listening sessions; and the development of publications and other resource materials.Among SAMHSA's goals and priorities is the implementation of its Strategic Initiative on Returning Veterans and Military Families. Since 2006, CRP was one of SAMHSA's principal partners in working to ensure that needed behavioral health services are accessible to our nation's service members, particularly its returning veterans. CRP coordinated and managed all aspects of the nation's first behavioral health conference on returning veterans and their families, jointly sponsored by SAMHSA and the U.S. Departments of Defense and Veterans Affairs. CRP also managed SAMHSA's follow-up national conference and inaugural policy academy in 2008, both of which emphasized an interactive and iterative strategic-planning process aimed at enhancing behavioral health systems and services for returning veterans and their families. Each event was conceptualized and planned by military and civilian interagency planning committees whose logistical arrangements were coordinated and managed by CRP.
Training and Technical Assistance
To advance the strategic imperatives and/or policy objectives of its clients, including Federal, state, and local government agencies, CRP provides customized training and technical assistance (T/TA) informed by a research-based approach to the conceptualization, development, and implementation of solutions. CRP also provides T/TA services to community-based organizations and program grantees. The cornerstone of CRP's operating approach to T/TA is "Transferring Knowledge to Practice and Practice to Knowledge"—that is, CRP trainers and other T/TA staff emphasize the importance of converting knowledge and data on the key issues affecting each client or client discipline into relevant and timely services, programs, policies, and products. CRP IT staff and consultants also provide training and technical assistance on developing and enhancing operational procedures and new innovative solutions to address current and emerging IT challenges.
CENTER FOR SUBSTANCE ABUSE PREVENTION (CSAP) - CENTERS FOR THE APPLICATION OF PREVENTION TECHNOLOGIES (CAPT)When SAMHSA's CSAP needed a partner to manage its national portfolio of T/TA activities in collaboration with CAPT, SAMHSA/CSAP selected CRP. We coordinated an array of T/TA deliveries through several subcontractors; planned and managed a broad spectrum of T/TA workshops and conferences; managed a network of expert T/TA consultants; and directed the Agency's Service to Science Initiative, which provided evaluation-capacity building T/TA to prevention interventions and programs. SAMHSA/CSAP relied on CRP to manage its mini-subcontract program for "building the field" of evidence-based prevention, resulting in an investment of over $3,750,000 in mini-subcontract awards over a five-year period. We also managed SAMHSA/CSAP's pilot Prevention and Wellness Initiative and worked with the CAPT in convening and reporting a series of T/TA listening sessions in geographically dispersed sites across the country.
POLICY ACADEMIES ON CO-OCCURRING DISORDERSCRP planned and managed four T/TA policy academies on co-occurring substance abuse and mental disorders for SAMHSA, which enabled the Agency to effect systems change for transforming how co-occurring services are delivered. We coordinated pre-academy planning meetings, designed and developed strategic action templates used to guide strategic action planning sessions, facilitated the T/TA sessions, coordinated the delivery of on-site T/TA at each academy, and led state and tribal teams in constructing an estimated 46 strategic action plans for enhancing co-occurring systems of care.
SAMHSA'S RETURNING VETERANS INITIATIVESAMHSA selected CRP to provide an array of T/TA services in support of its Returning Veterans Initiative, the predecessor to the Agency's Strategic Initiative on Returning Veterans and Military Families. First, we managed the nation's inaugural conference targeted at addressing the imminent and special behavioral health needs of returning veterans and their families. CRP facilitated the initial inter-agency planning committee meeting and prepared a report, which informed the subsequent planning and implementation of the national conference. Second, we convened and managed SAMHSA's first policy academy on returning veterans and their families, which was preceded by planning visits to each state.
At DOI's request, CRP provided training and technical assistance to Atlanta Cluster Initiative (ACI) institutions on marketing their research capabilities to DOI offices and programs. Using the medium of interactive videoconferencing technology, CRP implemented a research and training symposium that captured and streamed live sessions on this topic to personnel from ACI member institutions, including Morehouse and Morris Brown colleges and Clark Atlanta and Spelman universities.
CDC National Minority AIDS Initiative (NMAI)/Young Men Who Have Sex with Men INTERVENTION PLAN AND PROCESS MONITORING TRAININGCRP provided technical assistance and logistical support by developing curricula and implementing a series of regional training workshops for participating NMAI community-based organizations. Specifically, CRP was contracted to perform the following tasks: develop and implement a training curriculum for NMAI grantees; develop and implement a training user manual for grantees and trainers; conduct train-the-trainer sessions (during which a CRP master trainer trained others to serve on the training team); develop an evaluation instrument to measure the success of the trainings; write a summary report of all training activities; develop a TA tracking system; and provide follow-up TA to NMAI grantees.
In conjunction with the HRSA Maternal and Child Health Bureau, and in collaboration with the Centers for Disease Control and Prevention, CRP conducted six national training courses in maternal and child health (MCH) epidemiology. For each course, CRP invited professionals in state and local health departments who had significant responsibility for collecting, processing, analyzing, and reporting MCH data to participate in a one-week intensive training program. The training curriculum was designed to build participants' conceptual, technical, and analytic skills in the effective use of epidemiological data and other applications relevant to their day-to-day work.
A team of CRP logistical and technical staff conceptualized, planned, and implemented a five-day conference for the W. K. Kellogg Foundation's National Fellowship Program. The programmatic and technical design of the conference model allowed more than 100 Kellogg fellows, Foundation personnel, and guest experts to select and pursue a track from among four program areas for experiential learning. Each learning experience included site visits to facilities and agencies that embodied the desired learning objective. Additionally, we spearheaded the research effort emanating from the conference; developed a set of recommendations for related sub-topics; and conducted a series of case studies focusing on cultural diversity as it relates to education, employment, health, and housing from both local and national perspectives.
Consultation, Facilitation, and Events Management
CRP provides assistance in conceptualizing, planning, and managing all aspects of program initiatives. With our expert consultation and evaluation capacity-building components, we are able to strengthen organizational capacity and to document evidence and outcomes of program success. We are able to apply evidence-based practices and policies in unique settings. CRP IT staff and consultants also provide consultation support on integrating innovative solutions to address changing operating requirements.
SERVICE TO SCIENCE INITIATIVE - SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION (SAMHSA)CRP advanced the policy and programmatic priority structure of SAMHSA's Service to Science Initiative on two mission-critical fronts. First, it built a strong case for and upgraded the evidence of Service to Science programs' effectiveness in enhancing locally developed prevention efforts. Second, CRP expanded the institutionalization of evidence-based prevention practices throughout Service to Science programs, enhancing their prevention science infrastructures and improving prevention outcomes for their targeted populations.
PLANNING GROUP ON IRAQ MENTAL HEALTHSenior CRP staff and consultants provided expertise, advice, and guidance in support of efforts to help rebuild Iraq's mental health infrastructure resulting from and after the U.S.-Iraq war. CRP worked closely with SAMHSA and other agencies in Iraq, England, Egypt, and Jordan to coordinate action-planning sessions that led to the development of comprehensive plans for this initiative.
SAMHSA ACTION PLANSCRP's facilitation services staff have assisted in the development of nearly 50 SAMHSA-funded state and Tribal action plans for improving systems of care for persons with co-occurring disorders and for returning veterans and their families. The work focused on ensuring that state and Tribal teams reach consensus, resolve divergent views, eliminate barriers, and prevent fragmentation, thereby facilitating improved collaborations and avoiding duplication of services and systems.
CRP facilitated a three-day expert panel meeting as part of a CDC initiative focusing on cultural competency in HIV/AIDS prevention programs. The meeting was devoted to illuminating the issues that shape and inform the cultural competency components of HIV/AIDS prevention programs targeting racial and ethnic populations. The meeting addressed four key intervention strategies: risk reduction/health education, street outreach, prevention case management, and counseling and testing. CRP facilitators engaged the expert panel members in crafting an innovative model for culturally competent HIV/AIDS prevention programming.
Office of Refugee and Resettlement (ORR) and Health Resources and Services Administration (HRSA)CRP's facilitation services staff addressed ORR's mission-critical programmatic needs by conducting sessions that provided ORR personnel with a wealth of information and insights into the needs of and challenges faced by newly arrived refugees to American society. For HRSA, CRP staff facilitated workshop sessions aimed at strengthening the data-analytic capacity of the agency's maternal and child health workforce located in state and local health agencies.
CRP's facilitation services advanced DOJ policy and programmatic priorities aimed at strengthening services for ex-offenders, incarcerated fathers, and their families. Through several structured forums, CRP facilitators identified and explored opportunities for forging public/private and interagency collaboration in response to the special needs of this vulnerable population.
In support of this foundation's National Task Force on African-American Men and Boys, CRP conducted workshops in a number of geographically dispersed sites across the U.S. The workshops culminated in a seminal report titled Repairing the Breach, which CRP staff prepared in conjunction with Kellogg's staff and consultants. That report, presented to the Task Force, contained a number of far-reaching policy recommendations and constituted a significant contribution to the literature on African-American males.
CRP facilitated a series of workshop sessions for Tuskegee University's National Center for Bioethics in Research and Health Care, the nation's first bioethics initiative to employ a multi-disciplinary approach to the exploration of core issues that underlie research about medical treatment for people of color. The facilitated sessions impacted all areas of the Bioethics Center's organizational development, including strengthening operational efficiency, formulating and implementing evaluation methodologies, enhancing staff development, developing grant funding proposals, identifying and recruiting expert consultants, and creating and maintaining databases of Center constituencies.
Innovative Technology: Technology With You in MindCRP offers demonstrated capabilities in producing innovative and integrated technology solutions for Federal, commercial, and nonprofit organizations to meet their strategic imperatives, policy goals, and operating priorities. Integrating technologies also eliminate duplication, enhance organizational efficiency, and build on industry best practices. Our information technology (IT) professionals typically begin such engagements by conceptualizing and constructing an integration framework from which the emerging technologies will be executed. Specific services we are prepared to provide include the design, development, implementation, and deployment of integrated enterprise solutions; enterprise architecture design and capacity planning; and the management of migrations to new applications and software. Utilizing a customer-focused approach, and well-honed management processes and procedures, CRP takes into consideration technological arrangements of cutting-edge electronic multi-media tools, such as: • Electronic evaluation software • Online registration • Videoconferencing • Website development • Web-based file sharing • WebEx/webinars From our expert team of software program writers, technology technicians, website designers, and social network specialists to our staff of graphic and technical artists and 508-certified compliance personnel, CRP can provide dynamic technology solutions in real-time for ongoing projects.