Connecticut’s Evidence-Based Workgroup

THE MEANING OF EVIDENCE BASED

Evidence-based prevention strategies refers to programs, policies, or other strategies that have been evaluated and demonstrated to be effective in preventing health problems based upon the best-available research evidence.

PURPOSE OF CONNECTICUT’S EVIDENCE-BASED WORKGROUP

Connecticut’s Evidence-Based Workgroup (EBW) exists to increase the use of and fidelity of evidence- based prevention practices and programs in Connecticut. The work complements national efforts by the substance Abuse and Mental Health Services Administration and the Centers for Disease Prevention and Control to promote the use of evidence-based practices and evidence-based strategies.

PROCESS

The Connecticut Department of Mental Health and Addiction Services (DMHAS) Prevention and Health Promotion Unit assembles a diverse group of prevention and health promotion subject matter experts (SMEs). DMHAS takes a leadership role in identifying candidates to serve as SMEs. The composition of the EBW varies depending on the topic. The SMEs volunteer their time to review and/or develop resources and tools to increase the use of and fidelity of evidence-based prevention practices and programs. The group meets formally at least four times per year. The EBW receives staffing support from the Connecticut Prevention Training and Technical Assistance Service Center (TTASC).

EBW AREAS OF FOCUS

DMHAS supports a robust, statewide prevention infrastructure. This includes a commitment to support local prevention councils as well as prevention coalitions that may receive additional funding. A deeper analysis of coalition-based prevention efforts revealed: a) wide spread use of evidence-based programs (EBPs); and b) uneven level of implementation fidelity, particularly in the use of environmental strategies. Environmental strategies aim to change the context in which substance use occurs. This includes community conditions, standards, institutions, structures, systems, and policies.

The EBW members identified three environmental strategies as areas of focus and placed an emphasis on developing fidelity standards for these strategies. The following pages provide the work products associated with each of these environmental strategies.

Page 4 Enforcement strategies to reduce access to and use of substances, especially alcohol Page 11 Social marketing campaigns

Page 13 Community prevention coalitions

CONTACT

For additional information, contact Dawn Grodzki, Behavioral Health Program Manager, Department of Mental Health and Addiction Services – Prevention and Health Promotion Unit, 410 Capitol Avenue, Hartford, Connecticut, 06134. Tel: (860) 418-6772 e-mail: dawn.grodzki@ct.gov

Enforcement Strategies

CONTEXT

Environmental strategies focus on populations and affect large numbers of people through the adoption of systems and policy change and ongoing effective enforcement. Effective enforcement tends to result in widespread adoption and sustainable changes to the environment.

The EBW identified five enforcement operations most relevant to Connecticut’s prevention coalitions:

  1. Alcohol and tobacco compliance checks

  2. Party prevention and controlled party dispersal

  3. Reducing third party transactions through operations such as “Shoulder Taps”

  4. Enforcing impaired driving laws

  5. Preventing the use of fake and illegal ID’s

 

EBW PROCESS

The EBW members used as a reference point the fidelity standards and guides developed by the Pacific Institute for Research and Evaluation for the US Department of Justice Office of Juvenile Justice and Delinquency Prevention.

The EBW members condensed these guides into a fidelity matrix and included references to other scientific evidence that demonstrated the effectiveness of these enforcement strategies.

ENFORCEMENT OPERATIONS MATRIX

Pages 4 to 10 contain the evidence-based enforcement operations matrix to facilitate a higher level of implementation fidelity consistent with the research establishing them as evidence-based prevention strategies.

Enforcement Operations: Elements for Evidence-Based Practice

 

Compliance Check Operations

  • Planning Compliance Investigations

  • Working with Community Political Leaders

  • Working with Prosecutors, Courts and Regulatory Agencies

  • Work with Industry/Retailers

  • Select, Train & Document Volunteers

  • Procedures/Protocols for Conducting Investigations

  • Data Collection

  • Working with Media

  • Follow-up/next steps to Prevent Underage Sales

Party Patrol and Controlled Party Dispersal

  • Limiting Access to local party “hot spots”

  • Nuisance Abatement

  • Party Prevention

  • Controlled Party Dispersal Plan

  • Controlled Party Dispersal

  • Developing Community Support

  • Working with the Media

Reducing 3rd Party transactions

  • Surveillance

  • Shoulder Tap Enforcement Programs

  • Working within the Community

  • Strategies to reduce familiar 3rd party transactions

  • Overcoming barriers

Enforcing Impaired Driving Laws

  • Challenges to enforcing zero tolerance laws for youth drinking and driving

  • Overcoming barriers

  • Working with the media

Preventing the Use of False Identification

  • What retailers, servers and sellers can do to prevent sales to persons under 21 using False IDs

  • What communities and law enforcement can do to reduce the use of Illegal IDs

  • What the State can do to assist in reducing illegal ID use

Compliance Check Operations

Element

Description

Relevance

Sources of Data

Planning Compliance Investigations

  • Communities must decide on their purpose for compliance checks. Arrests, administrative actions, retailer education, community education?

  • Achieve compliance with current laws regarding underage alcohol sales

  • Reinforce no-use and no underage sales messages

  • Reducing Alcohol Sales to Underage Purchasers. Revised 2010

  • Journal of Studies on Alcohol and Drugs, 75(1),158–169 (2014).Ralph Hingson , Sc.D., M.P.H.a*, and Aaron White

,Ph.D.a

  • Journal of Studies on Alcohol and Drugs, 74(6),852–858 (2013).Darin J. Erickson , Ph.D.a*,Derek J. Smolenski , Ph.D.,M.P.H.b, Traci L. Toomey ,Ph.D.a, Bradley P. Carlin ,Ph.D.c, and Alexander C.Wagenaar , Ph.D.d

  • Joel W. Grube, William DeJong, Maureen DeJong,Sharon Lipperman- Kreda,Brad S. KrevorFirst published: 09 July 2018 Drug and AlcoholReview

  • Rebecca J. Schweitzer, Yan Yan Wu & Claudio R. Nigg (2017) Compliance checks for underage cigarette and alcohol purchase attempts and associated factors in retail outlets in Hawaii,Drugs: Education,Prevention and Policy, 24:5,384-391, DOI: 10.1080

/09687637.2016.1197182

Working with Community Political Leaders

  • Build relationships and retailer support to prevent underage sales. Establish goals for the program to support retailers to pass compliance check operations

  • Community and political support is critical for long term success for operations that help to prevent youth access to alcohol

Working with Prosecutors, Courts and Regulatory Agencies

  • Dept. of Consumer Protection, Liquor Control can provide a complete list of all licenses in your community, and may assist in conducting operations

  • Liquor Control Board can take administrative action against retailers who fail compliance checks

  • Police and community leaders must work with prosecutors to ensure they take criminal cases seriously and see these as a priority for the community

  • All retailers or a random sample are checked

  • Both administrative (DCP alcohol licensing) and criminal sanctions (local police) are available options

Work with Industry / Retailers

  • Build community partnerships with local prevention efforts, retailers and police to promote compliance

  • Provide merchant education programs

  • Industry/retailers see local prevention efforts and police as partners in eliminating youth retail access to alcohol

Select, Train & Document Volunteers

  • Recruit youth involved in local prevention efforts, including police explorers, scouts, etc.

  • Youth receive a minimum or two hours of training for these operations

  • Trained minors complete all documentation required by Liquor Control and DMHAS

  • Youth volunteers are representative and divers

  • Youth volunteers meet requirements for age, history and appearance

  • Proper documentation protects both the youth volunteers and the program

Procedures/Prot ocols for Conducting Investigations

  • Local Law Enforcement, community prevention efforts and youth read and understand CT’s “ Alcohol Compliance Check Operations: Protocols and Procedures”

  • CT Procedures and protocols are followed to ensure operation legality and success

Data Collection

  • Data collection protocols, as outlined in CT’s “Alcohol Compliance Check Operations: Protocols and Procedures” are followed.

  • Trend data comparisons

  • Proper documentation ensures operation follow up, including administrative or criminal sanctions are followed

Working with Media

  • Media exposure of operation outcomes informs the community of the level of compliance with underage sales laws among community retailers

  • Media exposure of operation outcomes can be very useful in fostering voluntary compliance

Party Patrol and Controlled Party Dispersal

Element

Description

Relevance

Sources of Data

Social Host Laws

  • Laws that prohibit adults from providing alcohol to minors or a place to consume alcohol for minors

  • Legal tool to educate adult on the possible consequences, including “public announcements” of allowing youth to use alcohol in their home or on their properties

 

 

 

 

 

 

 

 

 

 

  • Preventing and Dispersing Underage Drinking Parties. 2000, Revised 2005. Reviewed 2011

  • Debra S. Srebnik Ph.D., Dana Kovalchick, B.A., Linda Elliott, Journal of Drug Education Volume: 32 issue: 1, page (s): 13- 23 Issue published: March 1,2002

  • Flewelling, R.L., Grube,J.W., Paschall,

M.J. et al.Am J Community Psychol (2013) 51: 264

 

Limiting Access to local party “hot spots”

 

  • Local parks, beaches, woods, etc. can be party “hotspots” for underage drinkers

  • Limiting access to these “hotspots” can prevent parties and related consequences such as DWI, car crashes, alcohol poisoning, etc.

Nuisance Abatement

  • These laws include criminal activities such as health and fire code violations, excessive noise, property damage, drug dealing and other activities associated with youth parties

 

  • Can prevent further parties at chronic party sites where problems have been known to occur

 

 

Party Prevention

  • Offer alternatives to underage drinking parties, including alcohol free parties planned by youth

  • Build community relationships to access party information and break up parties before they start

 

  • Provide youth with safe, alcohol free activities

  • Build community norms against underage drinking parties

Controlled Party Dispersal Plan

  • Legally sound enforcement plan to intervene and safely disperse an underage drinking party

  • Sufficient human resources are required

  • Prevents DUI, car crashes, etc.

 

 

Controlled Party Dispersal

 

  • Protocols for these operations include briefing, deployment, scene security and processing

  • Officers must be properly trained in conducting safe party dispersal

  • Allows for safe party dispersal including any medical needs, safe transportation of intoxicated youth, parent notification and assessing the party site for damage or other criminal activity

  • Properly trained officers minimize risk and operational problems or mistakes

 

Developing Community Support

 

  • Build relationships and establish goals for the program to provide safety for youth and appropriate consequences

 

  • Political support is critical for long term success for operations that help to reduce youth drinking

Working with the Media

  • Media exposure of operation outcomes informs the community of the level of compliance with social host and underage drinking laws

  • Media exposure of operation outcomes can be very useful in fostering voluntary compliance with these laws

Reducing 3rd Party Transactions

Element

Description

Relevance

Sources of Data

 

 

Surveillance

  • Strategy for assessing where & when 3rd party transactions are occurring

  • Surveillance could be conducted during routine police patrols, special operations, or even by civilians in cooperation with local police

 

  • Identify “hotspots” for 3rd party transactions and future Shoulder Tap operations

 

 

 

 

 

 

 

 

 

 

 

 

 

 

  • Strategies for Reducing Third Party transactions of Alcohol to Underage Youth. 2004, Updated 2012

  • Rhonda Jones-Webb, Traci L. Toomey, Kathleen M.Lenk, Toben

F. Nelson, Darin J. Erickson

 

Shoulder Tap Enforcement Programs

  • Recruit and train youth involved in local prevention efforts, including police explorers, and scouts

  • Youth attempt to solicit adults outside a licensed off-premise retailer to buy alcohol for the minor volunteer

  • Trained minors complete all documentation required by local police

  • Reduce underage consumption and access to alcohol

  • Collaborate with police to enforce underage drinking laws

  • Raise public awareness of the problem of furnishing alcohol to minors

 

Working with the Community

  • Build relationships and establish goals for the program that provide safety for youth, appropriate consequences for “buyers” that the community and its leaders can support

  • Community and political support is critical for long term success for operations that help to prevent youth access to alcohol

Strategies to reduce familiar 3rd party transactions

  • Increasing community awareness of the dangers of underage drinking

  • Increasing community awareness of the legal consequences of providing alcohol to youth of hosting underage drinking events

  • Consistent enforcement of social host and youth provision laws

  • Many adults are unaware of the laws governing provision of alcohol to youth or social hosting

  • Some adults are not aware of the consequences of these laws or feel they are not enforced

Barriers to reducing 3rd party transactions

  • Lack of resources to conduct Shoulder Tap operations

  • State statutes that allow minors to drink in on-premise establishments with a parent/guardian

  • Negative perceptions of Shoulder Tap programs (safety, entrapment)

 

  • Barriers may prevent implementation of 3rd party transaction operations

 

Overcoming barriers

  • Establishing the resource priority for 3rd party transaction programs

  • Changing negative perceptions of 3rd party transaction programs and building support among the community including community leaders, law enforcement leaders, prosecutors and courts leaders, law enforcement leaders, prosecutors and courts

  • Obtain the necessary resources for 3rd party transaction operations

  • Community support is critical for long term success of 3rd party transaction operations

 

 

 

Working with the Media

 

 

  • Use media to create and demonstrate community support

  • Publicize enforcement efforts to deter 3rd party transactions

  • Media can assist in building community awareness of the problem with 3rd party transactions and generate support for enforcement operations

  • Increasing awareness of enforcement operations of unlawful 3rd party transactions increases the sense of risk among potential providers of alcohol to minors, increasing

deterrence of these activities

 

Enforcing Impaired Driving Laws

Element

Description

Relevance

Sources of Data

 

Challenges to enforcing zero tolerance laws for youth drinking and driving

  • Laws are difficult to enforce because offenders with low BAC’s are not likely to display erratic driving

  • Officers must show probable cause to administer an alcohol (BAC) test

  • Youth often believe zero tolerance laws are not strictly enforced

  • Barriers may prevent enforcement of impaired driving laws and decrease safety on our roads and highways

 

 

 

  • Enforcing Impaired Driving Laws. 200, Revised 2010

  • Bigelow GE, et al. Identifying Types of Drug Intoxication: Laboratory Evaluation of the Subject Examination Procedure. Washington, DC: NHTSA 1985, DOT HS 806

  • Compton RP. Field Evaluation of the Los Angeles Police Department Drug Detection Program. Washington, DC: NHTSA1986. DOT HS 807 012;

  • Adler EV, Burns M: Drug Recognition Expert (DRE) Validation Study. Phoenix: Arizona Governor’s Office of Highway Safety; 1994.

Overcoming barriers

  • Be alert for any signs of alcohol, and that any amount is a violation for drivers under age 21

  • Use sobriety checkpoints as an opportunity to identify young drivers who have been drinking

  • Build community support for zero tolerance laws and identify problems with laws and policies that may limit enforcement and

decrease safety

 

  • Training for law enforcement to detect impaired driving in minor drivers will increase safety on our roads and highways

  • Community support is critical to improving safety on our roads and highways

 

 

Working with the media

 

  • Using media to amplify deterrence

  • Using media to create and demonstrate community support

  • Increasing awareness of enforcement of impaired driving laws increases the sense of risk among potential providers of alcohol to minors, increasing deterrence of these activities

  • Media can assist in building community awareness of the problem impaired driving law violations and generate support for enforcement operations

Preventing the Use of False Identification

Element

Description

Relevance

Sources of Data

Types of Fake ID’s

  • Borrowed

  • Fake/Fraudulent/Fictitious ID’s

  • Knowledge of strategies to detect borrowed or fake ID’s

 

 

 

 

 

 

 

 

 

 

  • Law Enforcement Guide to False Identification and Illegal ID use. 2001, Updated 2011

  • Journal of Health Economics Volume 57, January 2018, Pages 102-112

 

Elements of state issued ID’s to prevent fraud

  • Training retailers in the security elements of state issued ID’s such s Ultra-Violet images, ghost images, Optical variable Device, holograms, Opacity Mark, Bar Codes, etc.

 

  • Increasing the ability for retailers to detect fake ID’s

What retailers, servers and sellers can do to prevent sales to persons under 21 using False IDs

  • Establish strict polices for checking ID’s

  • Training of retail staff on laws and retailer policies

  • Provide retail staff with necessary tools to properly check ID’s

 

  • Trained and prepared retailers will prevent underage sales using fake ID’s

 

 

What communities and law enforcement can do to reduce the use of Illegal IDs

  • Assessment of current laws/policies and enforcement of these laws/policies

  • Mandatory ID Checking policies

  • Prioritizing police response to calls for service involving fake ID’s

  • Provide retailer training on detecting and dealing with cases involving fake ID’s

  • Establishing community commitment to address the use of fake ID’s

  • Increase enforcement of fake ID violations

  • Increase the ability of retailers to detect fake ID’s and take appropriate action

What the State can do to assist in reducing illegal ID use

  • Vertical ID’s for persons under age 21

  • ID checking laws

  • Increase sanctions for those using fake ID’s

  • Suspend driver’s license for use of fake ID’s

  • Establishing the state’s commitment to address the use of fake ID’s

  • Increase enforcement of and appropriate sanctions for fake ID violations

Suggested Process and Timing for Administration and Completion

The figure below shows the recommended sequence of action to complete the CVAT process.

Coalition Vitality Assessment Tool v2.0 Users Guide and Resources

 

Purpose

The Prevention Coalition Vitality Assessment Tool (CVAT) creates an opportunity for coalition leaders and coordinators to assess the vitality and strength of their coalitions. The CVAT is an organizational assessment tool that can be used annually to identify the strengths and limitation of a community prevention coalition across four important domains: 1) Coalition composition; 2) Leadership and staffing;

  1. Plan and implementation; and 4) Sustainability.

 

The results can inform decisions about how to prioritize improvement efforts of the coalition and what training and technical resources can support these improvement efforts. For example, benefits of completing the CVAT process include:

The Connecticut Prevention Training and Technical Assistance Service Center (TTASC) will provide assistance in completing the CVAT process in 60 days or less followed by ongoing implementation supports (e.g., assistance with developing a project dashboard, coalition coaching plan).

SOCIAL MARKETING CAMPAIGNS

CONTEXT

Marketing techniques and tools commonly used in the private sector play an increasing role in implementing and disseminating social change strategies. Social marketing now represents a powerful set of tools for segmenting and identifying focus populations; designing, positioning, testing, and refining products and services; and promoting community action – including changing social norms and policies.

Environmental strategies implemented by community prevention coalitions typically contain a social marketing component. Implementing an evidence-based social marketing campaign with high fidelity means a community prevention coalition must have capacity and resources to measure the impact and effectiveness of the campaign. Many prevention coalitions hire media and marketing firms to develop and support implementation of a social marketing campaign. However, these engagements rarely included the use of campaign implementation fidelity guidelines.

EBW PROCESS

The EBW: a) conducted a literature review; b) reviewed existing technical assistance tools developed by TTASC for use with community prevention coalitions; and c) developed a “Fidelity Checklist for Implementing a Social Marketing Campaign.” Use of the Fidelity Checklist – including during the planning and consultant selection process, will increase the likelihood of implementation fidelity for environmental strategies that include a social marketing component.

The EBW members used following research as the foundation to develop the Fidelity Checklist:

  1. Guide to Marketing Social Norms for Health Promotion in Schools and Communities Michael

P. Haines, M.S. H. Wesley Perkins, Ph.D. Richard M. Rice, M.A. Gregory Barker, Ph.D. A Guide to Marketing Social Norms for Health Promotion in Schools and Communities is a publication of the National Social Norms Resource Center. www.socialnorm.org Copyright © 2005 National Social Norms Resource Center.

  1. An Overview of the Social Norms Approach Alan D. Berkowitz Independent Consultant www.alanberkowitz.comTo be published as Chapter 13 in L Lederman, L Stewart, F Goodhart and L Laitman: Changing the Culture of College Drinking: A Socially Situated Prevention Campaign, Hampton Press.

  2. The 7 Step Montana Model on Social Norms Marketing Toolbox at https://chsculture.org/wp- content/uploads/2014/09/The-Toolbox.pdf

  3. The following document consists of excerpts taken from a file posted on the Social Norms cite. For the full document go to: http://www.edc.org/hec/socialnorms/theory.html Social Norms Approach by Alan D. Berkowitz, Ph.D., Independent Consultant, Trumansburg, NY

  4. Ten Steps for Developing a Social Marketing Campaign. www.samhsa.gov/capt/tools- learning-resources/developing-social-marketing-campaign

Domain 1. Coalition Element Coalition: Scoring Rubric
1 Point 2 Points 3 Points 4 Points
Limited to no capacity Concerns or Issues limit coalition effectiveness Sufficient capacity or capabilities to produce results Strong capacity or capabilities
Guiding Principles Vision & Mission No vision or mission statement exists. Members ad lib the vision and mission An outdated vision or mission statement exists. A current vision or mission exists, and 51% to 75% of members can articulate the vision and mission. A current vision or mission exists and at least 76% of members can articulate the vision and mission.
Coalition composition Required sectors Active participation by 3 or less required sectors. Active participation of 4 to 6 required sectors. Active participation of 7 to 9 required sectors. Active participation of 10 to 12 required sectors.
Demographic diversity & inclusiveness Demographic diversity does not reflect the community or priority populations as well as youth and parents relevant to the Action Plan. Demographic diversity reflects the community in general; does NOT reflect priority populations as well as youth and parents relevant to the Action Plan. Demographic diversity reflects community; gaps in representation of priority populations as well as youth and parents relevant to the Action Plan. Demographic diversity reflects community and priority populations as well as youth and parents relevant to Action Plan.
Youth participation No vehicle or mechanism exists to engage youth or develop leaders. Vehicle or mechanism exists to engage youth and/or develop leaders and is NOT used or supported at this time. A vehicle or mechanism exists to engage youth and develop youth leaders that provide input to the coalition. Youth serve as coalition members and equal contributors at the coalition meetings.
Coalition meetings (full group) Member roles No formal description of member roles and responsibilities and/or expectations. Use of some written descriptions for roles, responsibilities and expectations; information conveyed orally and as needed Written documentation of member role, responsibilities, and expectations; formal orientation / onboarding process followed Written documentation of member role, responsibilities, and expectations; formal orientation / onboarding process followed + mentoring and/or formal check-in process
Meeting frequency & duration Coalition meetings occur less than 3 times per year. Coalition meetings may not follow a regular schedule. Coalition meetings occur at least 4 times per year. Coalition meetings may not follow a regular schedule. Coalition meetings occur 5 to 7 times per year and follow a regular meeting schedule. Insufficient time may exist to finish business. Coalition meetings occur 8 or more times per year and follow a regular meeting schedule. Sufficient time exists to finish business.
Meeting organization No process or standard protocols in place to share in advance meeting agenda, prior meeting notes, or other documents. Process or standard protocols exist to share in advance meeting agenda, prior meeting notes, or other documents. Process not used or followed. Uniform process exists to share meeting announcements, agendas, and materials sent at least 10 days in advance of the meeting. Process followed most of the time. Uniform process exists to share meeting announcements, agendas, and materials sent at least 10 days in advance of the meeting.
Meeting attendance Regular coalition meeting attendance less than 25%. No formal quorum used. Regular coalition meeting attendance represents 26% to 50% of total membership. Possible use quorum and attendance tracking. Regular coalition meeting and formal attendance tracking with at least 51% to 75% of total membership. Use of quorum may or may not exist. Regular meeting attendance exceeds 76% of the total membership. Mechanism for quorum exists.
No vision or mission statement exists. Members ad lib the vision and mission. An outdated vision or mission statement exists. A current vision or mission exists, and 51% to 75% of members can articulate the vision and mission. A current vision or mission exists and at least 76% of members can articulate the vision and mission.
Possible use quorum and attendance tracking. membership. Use of quorum may or may not exist.
Meeting feedback / climate No process exists to collect feedback about participant meeting experience. Informal (oral) and/or inconsistent process exists to collect feedback about participant meeting experience. Regular, written process exists to collect feedback about participant meeting experience. No formal process or protocols to review data by coalition leadership. Regular, written process exists to collect feedback about participant meeting experience; formal process exists and used by coalition leaders to review feedback / improve climate.
Committees, work groups, or action teams Formation Coalition does not use committees or work groups. Coalition uses groups. No formal protocols exist to organize participants or leadership. Coalition uses groups. Written guidelines or practices exist to form committees or work groups and leadership. Guidelines not always followed. Coalition uses groups. Written guidelines or practices exist to organize committees or work group and leadership in context of coalition structure.
Charge Coalition does not use committees or work groups. Committee charge defined in general terms, may or may not be written down, and may not clearly connect to Action Plan. Committee charge clearly defined with a scope of work (timeline, deliverables); gaps may exist in alignment with Action Plan Clear charge, scope of work, deliverables, timeline, and realistic to staffing. Alignment exists with Action Plan.
Level of support Coalition does not use committees or work groups. Any committee or work group does not receive any formal staff support; volunteer support uneven at best. Committee or work group receives an explicit level of commitment from staff or volunteers sufficient to deliver results.
Public participation (including youth) Coalition does not use committees or work groups. Public (non-coalition) members may participate. No formal guidelines or process. Committee or work group receives an explicit level of commitment from staff or volunteers; support may not be sufficient to deliver results.
Culture Partner engagement by coalition members Less than 25% of coalition members engage partners outside of coalition meetings or demonstrate interest to do so. 26% to 50% of the coalition members express interest in engaging partners or assist in this work; training & support needed and may or may not be available. 51% to 75% of coalition members willing and able to talk about coalition work with partners and stakeholders; additional training and support available. Public (non-coalition) members may participate. Written guidelines exist to explain roles, responsibilities, expectation, and decision-making.
Guiding Principles Vision & Mission No vision or mission statement exists. Members ad lib the vision and mission. An outdated vision or mission statement exists. A current vision or mission exists, and 51% to 75% of members can articulate the vision and mission. A current vision or mission exists and at least 76% of members can articulate the vision and mission.
Data-driven culture & approach Coalition work and decision-making based on anecdotal, unsubstantiated, or outdated information. Less than 25% of members use community and prevention data as reference. A few (3 to 5) champions exist. 26% to 50% of coalition members reference community and prevention data sets; group norm emerging. More than 50% of coalition members reference community and prevention data sets; group norm exists.
TOTAL SCORE COALITION DOMAIN
Domain 2.Leadership Elementent Leadership & Staffing: Scoring Rubric
1 Point 2 Points 3 Points 4 Points
Limited to no capacity Concerns or Issues limit coalition effectiveness Sufficient capacity or capabilities to produce results Strong capacity or capabilities
Coalition Chair(s) Clarity of role No formal, written description of leadership roles and responsibilities. No tools or processes to assess leadership efficacy. Limited formal, written description of leadership responsibilities. Leaders complete tasks without tools or processes. Clear documentation of roles, tools and processes exists. Coalition leader complies partially with responsibilities of the leadership role. Clear documentation of roles, tools and processes exists. Coalition leader complies fully with responsibilities of the leadership role.
Process and protocols No procedures or guidelines exist on leadership selection and term length. Procedures or guidelines exist on leadership selection and term length; coalition does not follow or use them. Procedures or guidelines exist on leadership selection and term length; coalition adheres to the greatest extent possible. Clear procedures or guidelines exist and followed for leadership selection, term length, and leadership succession planning.
Coalition Leaders (includes committee / work group leaders; NOT coalition staff) Prevention / Health Promotion Experience Less than 25% of leadership designees maintain up-to-date knowledge of prevention / health promotion field. 26% to 50% of leadership designees maintain up-to-date knowledge and working knowledge of prevention / health promotion field. 51% to 75% of leadership designees maintain up-to-date knowledge and working knowledge of prevention / health promotion field. At least 76% of leadership designees maintain up-to-date knowledge and working knowledge of prevention / health promotion field.
Collaborative Leadership Experience Less than 25% of leaders hold experience leading a community / volunteer groups. 26% to 50% of leaders hold experience leading a community / volunteer groups and can navigate group dynamics effectively. 51% to 75% of leaders hold experience leading a community / volunteer groups and can navigate group dynamics effectively. At least 76% of leaders hold experience leading a community / volunteer groups and can navigate group dynamics effectively.
Leadership development & succession Opportunities No leadership rotation in past 24 months or conversely high rates of turnover. Coalition addresses leadership replacement as needed; no formal plan or approach. Coalition leadership positions use term limits and produce regular openings; informal process to identify and develop potential leadership candidates. Coalition leadership positions open regularly (e.g., term limits) and process exists to deliberately develop leaders.
Selection No formal process exists. Group relies on volunteers or default options. Semi-formal process used on an as-needed basis. May or may not involve objective criteria. Formal process exists to select/elect leaders; may not be based on objective criteria or followed consistently. Formal process exists to select/elect leaders based on objective criteria and followed consistently.
Orientation / Onboarding No formal methods exist; each leader uses own, self-directed process. Informal leadership orientation process, varies based on other leaders and staff. Formal leadership orientation and onboarding process exists and followed. Formal leadership orientation, onboarding, transition plan, and mentoring available.
Youth leaders Coalition does not include youth members or a youth group. Youth not on coalition. Action Plan strategies affect youth; informal promotion to youth about future opportunities as members. Coalition members include youth or a youth group; informal process and pathway exists to develop youth leaders. Coalition members include youth; a clear process and pathway exists to develop youth leaders
Coalition coordinator Clarity of Role No formal job description exists that articulates roles, responsibilities, deliverables, and standards for performance. No explicit annual work plan performance. No explicit annual work plan. Formal job description exists that articulates roles, responsibilities, deliverables, and standards for performance. However, document does not reflect current work. Annual work plan may or may not exist. Formal job description exists that articulates current roles, responsibilities, deliverables, and standards for performance with 50% to 75% accuracy. Annual work plan exists in at least generalities. Formal job description exists that articulates current roles, responsibilities, deliverables, and standards for performance with regular process to update and amend. Annual work plan exists and updated.
Qualifications / Experience Coordinator does not maintain minimum credentials and/or experience to hold job. Coordinator maintains minimum credentials and experience for job. Gaps exist in technical competencies or knowledge important to make meaningful progress. Coordinator exceeds minimum credentials and experience for the job, maintains core competencies to perform at satisfactory level; may participate in continuing education. Coordinator exceeds minimum credentials, holds all core competencies and knowledge to perform duties, and pursues ongoing prof development and continuing education.
Level of Effort Staffing levels not sufficient to support coalition functioning; no or minimal effort to address disconnect (e.g., more volunteer help; revise plan). Staffing levels sufficient to maintain critical coalition functions at a reasonable level. Additional effort to address disconnects may or may not occur. Adequate staffing levels exist to support coalition and Action Plan. Efforts by coalition members to provide volunteer support or leverage partner resources. More than sufficient staffing levels exist to support coalition and Action Plan. Volunteer and partner resources aligned for optimal efficiencies.
Supervision No formal process exists to provide structured and regular feedback about job performance. Clear lines of supervision for coordinator may not exist. Annual review and/or immediate supervisor reviews occur without structure or regularity. May or may not produce a productive outcome. Annual review process and guidelines exist; immediate organizational supervisor available for reviews 2 to 4 times per year; peer mentors may or may not exist. Annual review process and guidelines in place for Coalition chair to review progress + monthly reviews by immediate organizational supervisor + connection with peer mentors.
Administrative support Level of effort Less than 5 hours weekly of regular access to administrative/fiscal support provided to coalition coordinator by coalition, partners / vendors, and/or sponsoring organization. 6 to 10 hours weekly of regular access to administrative/fiscal support provided to coalition leader by coalition, partners / vendors, and/or sponsoring organization; may or may not be sufficient level to support coordinator. 11 to 20 hours weekly of regular access to administrative/fiscal support provided to coalition leader by coalition, partners / vendors, and/or sponsoring organization; adequate level to support coordinator. 20+ hours weekly of regular access to administrative / fiscal support provided to coalition leader by coalition, partners / vendors, and/or sponsoring organization; sufficient level to support coordinator.
Alignment of skills to work Mismatch in competence or knowledge to perform work; high errors and/or level of supervision required Partial (50%) alignment between work tasks and required knowledge and competence. Must invest high level of quality control. Uneven productivity. General alignment (51% to 75%) between work tasks and required knowledge and competencies. Reasonable quality control required. Moderate productivity. Strong alignment (76% or above) of work tasks and required knowledge and competence. Minimal supervision required; high productivity and low error rate.
Domain 3. Action Plan Element Plan Development & Implementation Scoring Rubric
1 Point 2 Points 3 Points 4 Points
Limited to no capacity Concerns or Issues limit coalition effectiveness Sufficient capacity or capabilities to produce results Strong capacity or capabilities
Fundamental knowledge of process SPF Less than 25% of leaders hold knowledge about or experience with SPF framework. 26% to 50% of leaders hold knowledge about or experience with SPF framework. 51% to 75% of leaders hold knowledge about or experience with SPF framework. At least 76% of leaders hold knowledge about or experience with SPF framework.
Community planning Less than 25% of leaders hold knowledge about or experience with community planning processes. 26% to 50% of leaders hold knowledge about or experience with community planning processes. 51% to 75% of leaders hold knowledge about or experience with community planning processes. At least 76% of leaders hold knowledge about or experience with community planning processes.
Data access and use Data sources Coalition relies on the professional knowledge of the coordinator. Coalition relies on the professional knowledge of the coordinator and a subset of coalition members. Coalition coordinator and/or members developing written inventory of data sources. Written inventory exists, updated annually.
Data access Coalition relies on coordinator to facilitate access. Coalition relies on coordinator and the connections of a small set of coalition members to facilitate access. Coalition members and partners facilitate access to most relevant data sets; some gaps exist. Coalition members and partners facilitate access to the most relevant data sets. Process in place to close any “gaps” in access.
Access to data experts Coalition membership does NOT contain an expert on quantitative and/or qualitative data and methodology. Data experts may not be included or consulted. Coalition membership does NOT contain an expert on quantitative and/or qualitative data and methodology. Limited access to resources or experts. Coalition membership does NOT contain an expert on quantitative and/or qualitative data and methodology; resources or relationships exist to involve as needed. Coalition membership contains an expert on quantitative and/or qualitative data and methodology; or coalition maintains formal access (partnerships) to data experts.
Community needs assessment Process occurs irregularly and with limited or partial data sets; may rely heavily on opinions, anecdotal information, or qualitative information. Partial needs assessment process conducted (updated) every five years; does not include comprehensive data sets such as demographics, community indicators, population-based surveys, focus groups, and key informant interviews. Comprehensive assessment process conducted (updated) every five years, OR partial process occurs every three years using demographics, community indicators, population-based surveys, focus groups, and key informant interviews Comprehensive process conducted (updated) every three years using diverse data sets such as demographics, community indicators, population-based surveys, focus groups, and key informant interviews.
Coalition process to develop Action Plan Risk and protective factors Coalition applies general assumptions about most relevant risk factors based on literature from the field. Coalition applies general assumptions about most relevant risk factors based on literature from the field. Incomplete effort to identify community-specific risk and protective factors. Coalition defines clearly the relevant risk and protective factors by domain (e.g., individual, family, peer, school, community) for priority populations; factors not linked closely to data sets. Coalition defines clearly the relevant risk and protective factors by domain (e.g., individual, family, peer, school, community) for priority populations, and links to data sets.
Priorities Coalition does not clearly identify priority substances or populations – “universal” language. Coalition identifies priority substances OR priority populations – not (yet) a population-specific approach. Coalition identifies priority substances and priority populations; approach does not include specificity (e.g., age, race/ethnicity, geography). Coalition identifies priority substances and priority populations with specificity (age, race/ethnicity, geography) around which the Action Plan will focus.
Priorities align with community agendas Limited effort to understand how priorities correspond with broader strategic objectives in community. Priorities correspond only indirectly with strategic agendas of elected officials and/or partners (e.g., schools, hospitals). 1 coalition priority correspond directly with strategic agendas of elected officials and/or community partners (e.g., schools, hospitals). 2 or more priorities correspond directly with strategic agendas of elected officials and/or community partners (e.g., schools, hospitals).
Strategies The coalition does not include relevant strategies in its plan. The coalition includes vague and/or universal strategies; disconnects exist across Action Plan. Clear articulation of specific strategies. However, selected strategies not well integrated to produce the biggest impact. Clear articulation of specific strategies; integration to support multiple goals and objectives across the Action Plan.
SMART objectives The coalition does not include objectives in its Action Plan. Action Plan objectives incomplete across all criteria of “specific, measurable, actionable, relevant, and time-bound.” Action Plan objectives align with criteria of “specific, measurable, actionable, relevant, and time-bound.” Action Plan objectives align with SMART criteria AND align to support outcomes valued by other institutional / community partners.
Action Plan implementation process Resources No or insufficient resource plan exists to implement Action Plan. May assume coordinator will figure it out. Resource plan exists to implement Action Plan; must close up to 25% gap in funding/resources. Plan may be driven only by coordinator. A resource plan exist to cover 76% to 100% of implementation. Coalition uses a committee and/or group process to develop and update the resource plan. A resource plan exist to cover 76% to 100% of implementation using diversified funding sources. Use of a group process to develop and update the resource plan.
Quality assurance No process exists to assess the extent to which Action Plan implemented with fidelity. Coalition coordinator uses informal approach and methods to assess implementation fidelity. Coalition coordinator uses formal and systematic approach and methods – albeit limited in scope, to assess implementation fidelity. Action Plan and includes clear implementation standards, benchmarks, and QA methods to assess implementation fidelity.
Evaluation Coalition does not use any evaluation methods. Evaluation methods and processes emerging in coalition work. No formal evaluation underway. Coalition organizes and manages evaluation process for critical Action Plan strategies. Process behind schedule. Coalition organizes and manages evaluation process for critical Action Plan strategies. Evaluation process on-schedule.
Updates to coalition No standard, regular process to update coalition on planned v. actual progress for Action Plan. Coalition coordinator uses informal (e.g., oral) process to update coalition on planned v. actual progress for Action Plan. Coordinator uses systematic process and/or dashboard to update coalition on planned v. actual progress for Action Plan. A coalition group and coordinator use systematic process and/or dashboard to update coalition on planned v. actual progress for Action Plan. Share results with community.
TOTAL SCORE PLAN DEVELOPMENT & IMPLEMENTATION DOMAIN
Domain 4. Sustainability Element Sustainability Scoring Rubric
1 Point 2 Points 3 Points 4 Points
Limited to no capacity Concerns or Issues limit coalition effectiveness Sufficient capacity or capabilities to produce results Strong capacity or capabilities
Fundamental knowledge of sustainability Coalition members Less than 25% of leaders hold knowledge about or experience with sustainability planning and resource development. 26% to 50% of leaders hold knowledge about or experience with sustainability planning and resource development. 51% to 75% of leaders hold knowledge about or experience with sustainability planning and resource development. At least 76% of leaders hold knowledge about or experience with sustainability planning and resource development.
Coordinator No experience in fundraising and/or resource development. Limited experience and/or technical competencies in resource development and sustainability. Coordinator holds prior experience (successful or unsuccessful) and/or subset of core competencies necessary to support sustainability planning and resource development. Coordinator holds necessary knowledge, competencies, and experience to support sustainability planning and resource development.
Current Plan Existence No written resource development plan exists beyond fundamental information such as a current and/or multi-year program budget. A current and/or multi-year program budget exists; discussions occur irregularly and informally about future funding. A current and/or multi-year program budget exists with preliminary outlines of how the coalition will support priority work in the event of changes in resources (expansion, contraction). A resource development and/or sustainability plan or framework exists and identifies goals for growth, diversification, funding / resource opportunities, prospects and timelines.
Priorities / goals No sustainability priorities exist; primary focus may be to support a full- or part-time staff person. No formal coalition consensus process used; sustainability priorities may relate more to funding opportunities than to need and/or results. Coalition members hold a general consensus of sustainability priority actions without conducting a formal or extensive process. Data will support selection of priorities and help make the case. Coalition members use a data-driven process to select sustainability priorities (growth, contraction) and describe goals objectively (resource amounts, timing, and impact).
Diversity The coalition relies on one funding source that may be time-limited or vulnerable. The coalition relies primarily on one funding source that may be more stable and static (i.e., not growing). The coalition receives direct funding from multiple sources that account for up to 20% of the total budget + in-kind contributions from resource partners. The coalition receives direct funding from multiple funding sources that account for more than 21% of the total budget + in-kind contributions from resource partners.
Use of data to make case The coalition does not use data and/or results to make a case for resourcing Action Plan priorities. The coalition uses general and somewhat current data and/or results to make a case for resourcing Action Plan priorities. The data relates may not relate directly to the priority in question. The coalition uses general, current, and relevant data – including evaluation data, and/or results to make a case for resourcing Action Plan priorities. The data relates directly to the priority. The coalition uses specific, current, and compelling data – including evaluation data, and/or results to make a case for resourcing Action Plan priorities. Case statement may include cost-benefit, cost saving, and/or economic impact.
Process for financial reporting, resource development & sustainability Structure No formal group process or guidelines exist. Working assumption that coordinator will complete any work or “time-limited, grant funded” project. No formal group process or guidelines exist. Coordinator uses self-directed process in response to circumstances. Coalition relies on a small subset of coalition members that may convene as a group or offer individual support to the coordinator. Coalition convenes regularly a committee to advance sustainability and resource development work. Coordinator allocates time to support this work.
Frequency Coalition discusses funding when issues exist (e.g., end of grant). No systematic process exists to share financial reports / resource development plans. Coordinator may share information orally. Coalition uses systematic process to share financial reports / resource development plans at least 2 times per year. Coalition uses systematic process to share financial reports / resource development plans at each coalition meeting.
Partnership / innovation No interest in and/or capacity to pursue partnerships or other resource development activities. Willingness exists to explore new partnership and funding opportunities; follow-through driven by individual efforts (e.g., coalition member, coalition coordinator). Coalition open to explore opportunities with partners and/or engage in more entrepreneurial approaches (e.g., events, sponsorships, incorporation into operating budgets); informal guidelines exist driven mostly by coordinator and/or small number of coalition members. Coalition maintains guidelines and/or decision parameters to explore/screen funding opportunities with partners and/or engage in more entrepreneurial approaches (e.g., events, sponsorships, incorporation into operating budgets).
Annual appeals / online giving No process in place to conduct annual appeal or collect online giving. Building blocks exist to add online giving options and/or establish or strengthen annual appeals or flagship fundraising events (lead or as a partner). 1 to 4 years of experience in building coalition visibility and presence using online giving options and/or flagship events (lead or as a partner). 5+ years of experience in online giving, annual appeals, or flagship events (lead or as a sponsor); may be exploring or earning income through alternative revenue streams.
Level of effort Coalition members and/or coordinator commit less than 2 hours per month to research funding opportunities and/or cultivate prospects / partners. Coalition members and/or coordinator commit 2 to 4 hours per month to research funding opportunities and/or cultivate prospects / partners. Coalition members and/or coordinator commit 5 to 20 hours month to research funding opportunities and/or cultivate prospects / partners. Coalition members and/or coordinators commit at least 21 per month to research funding opportunities and/or cultivate prospects / partners.
Quality Assurance No process in place. Assume coalition coordinator or lead organization addressing any issues. Coalition coordinator discusses topics as needed. No formal guidelines in place to maintain quality assurance. Coalition conducts at least an annual reflection / review process to adjust/update sustainability and resource development plans. Coalition conducts quarterly progress checks and convenes an annual reflection / review process to adjust/update sustainability and resource development plans.

Coalition Vitality Assessment Tool v2.0 – Discussion Notes Template

On pages 12-15, you will find a tool to help you document notes from your self-assessment discussion.

Evidence supporting score Suggestions for improvement?
Domain 1: Coalition
Guiding Principles Vision & Mission
Coalition Composition

• Required sectors

• Demographic diversity & inclusiveness

• Youth participation

Coalition Meetings (full group)

• Member roles

• Meeting frequency & duration

• Meeting organization

• Meeting attendance

• Meeting fdbk / climate

Committees, work groups, or action teams

• Formation

• Charge

• Level of support

• Public participation (including youth)

Culture

• Partner engagement by coalition members

• Data-driven culture & approach

Evidence supporting score Suggestions for improvement?
Domain 2: Leadership
Coalition Chair(s)

• Clarity of role

• Process and protocols

Coalition Leaders (includes cmte / work group leaders; not coalition staff)

• Prevention / Health Promotion Experience

• Collaborative Leadership Experience

Leadership development & succession

• Opportunities

• Selection

• Orientation / onboarding

• Youth leaders

Coalition Coordinator

• Clarity of Role

• Qualifications / Experience

• Level of Effort

• Supervision

Administrative Support

• Level of effort

• Alignment of skills to work

Evidence supporting score Suggestions for improvement?
Domain 3: Action Plan
Fundamental knowledge of process

• SPF

• Community planning

Data access and use

• Data sources

• Data access

• Access to data experts

• Community needs assessment process

Coalition process to develop Action Plan

• Risk and protective factors

• Priorities

• Priorities align with community agendas

• Strategies

• SMART objectives

Action Plan implementation process

• Resources

• Quality Assurance

• Evaluation

• Updates to coalition

Evidence supporting score Suggestions for improvement?
Domain 4: Sustainability
Fundamental knowledge of sustainability

• Coalition members

• Coordinator

Current Plan

• Existence

• Priorities / goals

• Diversity

• Use of data to make case

Process for financial reporting, resource development & sustainability

• Structure

• Frequency

• Partnership / innovation

• Annual appeals / online giving

• Level of effort

• Quality Assurance