From an Innovative Concept to a National Model 2006&endash;2016: Our Story

Introduction

Over the last decade the Denver Preschool Program has evolved from an innovative concept to a national model for what a locally funded, high quality early childhood education initiative can be. This report details the story of that evolution and provides key insights and lessons learned from that experience.

This is the story of the Denver Preschool Program.

It is a story of unwavering focus on the city’s commitment to ensuring every Denver child gets the best possible start to his or her education. It is a story of community collaboration and commitment. It is a story of continuous learning, evolution and improvement. And it is a story of keeping our promises to our key partners: Denver’s families, early education providers and taxpayers.

We are humbled by our success and the fact that other communities in Colorado and across the country are interested in learning from our experience. We don’t purport to have the formula for success. We recognize that each community has unique circumstances that will shape their approach to this issue. However, we do believe there are important lessons to be learned from our experience and we are eager to contribute to the field of early childhood education and serve as a partner to others who seek to nurture young children in their communities. We hope that by sharing our story we can inform and advance the work of others in a constructive and meaningful way.

In partnership,

Jennifer Landrum

Former DPP President and CEO

Mike Yankovich

Former DPP Board Chair

MAKING IT HAPPEN: VISION, ESTABLISHMENT & EXTENSION

EARLY FAILURES INFORMED LATER SUCCESS

The roots of the Denver Preschool Program can be traced back to the early 1990s when Denver Mayor Wellington Webb brought new focus and community attention to children’s issues, including care and education for young kids. Mayor Webb undertook many important and successful initiatives to support vulnerable kids.

However, Webb’s highest profile initiatives to address the needs of children both failed. In 2000 and again in 2001, Webb championed sales tax increase proposals that would have provided funding and support for a wide range of programs to benefit vulnerable children, including funding for child care subsidies. Both times Denver voters rejected the “kid tax” measures by fairly wide margins.

Denver’s Mayor in the early 1990s, Wellington Webb

Analysis of the measures and campaigns found that they were unsuccessful for three primary reasons. First, they were viewed to be too broad and lacking focus, as each measure would have provided funding for a number of support services for low income children, including child care, school-based health care, food support and after-school programming. Second, they lacked a broad base of support among voters. While social service providers and the early childhood community backed the measures, there wasn’t broad public understanding of the need for or support of the proposals. Third, supporters lacked the financial resources necessary to run competitive campaigns. While unsuccessful, the efforts were valuable learning experiences that informed and shaped the successful 2006 measure.

PLANNING & PREPARING FOR A PUBLIC CAMPAIGN

John Hickenlooper was elected Mayor of Denver in May 2003. His Mayoral campaign platform included a focus on early childhood education, specifically high quality preschool. During his first month in office Hickenlooper launched Invest In Success, an early childhood initiative.

To implement the initiative, Hickenlooper established two key committees: The Mayor’s Early Childhood Education Commission (Commission) and the Mayor’s Leadership Team for Early Childhood Education (Leadership Team). The Commission was comprised of 15 members with expertise in early childhood education and was charged with advising the Mayor’s Office on early childhood education and child care issues. The Leadership Team was a larger group of approximately 30 business and civic leaders charged with “ensuring that all of Denver’s kids have access to high quality early childhood education.” The Commission and Leadership Team began formally collaborating in 2004. They formed three shared workgroups and a joint Steering Committee.

The work of the Commission and the Leadership Team was substantially facilitated and supported through a partnership with the Gary Williams Energy Company and its philanthropic arm, the Piton Foundation (Piton). Gary Williams and Piton brought invaluable business and nonprofit expertise, as well as considerable financial resources and technical assistance, to the effort.

Through months of work together the joint Steering Committee established five priorities. They recommended that a new early childhood education program for Denver should:

1. Be independent of City government, but accountable to elected leaders and tax payers;
2. Focus on quality rating and improvement of preschool programs;
3. Empower parents to choose the early education program best for their child;
4. Be universal and open to all children in Denver; and
5. Prioritize children in the year just before kindergarten (most commonly, 4-year-old children).

These principles became the pillars of the Denver Preschool Program and are still evident in the program today.

In Colorado, raising public revenue at any level of government requires a vote of the people. As such, it became clear that any effort to establish a new early education program in Denver would require a ballot initiative. In 2005, the Piton Foundation and Gary Williams Energy hired two local political consulting firms, The Kenney Group and SE2, to advise the joint Steering Committee on the campaign process.

BUILDING PUBLIC WILL FOR PRESCHOOL

The political consultants started their work by focusing on public opinion research, message development and timing. To establish a baseline of support for the concept and develop messaging, they conducted a series of focus groups in November 2005, followed by a telephone survey poll of 500 likely Denver voters in December. The research found that among likely voters there was general support for preschool, but a low level of understanding of the importance of preschool and a belief that early education was primarily a parent’s responsibility. Compared to other community needs, funding preschool was simply not a priority for Denver voters.

Armed with this information and an understanding of historic trends in voter turnout, the consulting team recommended targeting the November 2006 general election. It was a gubernatorial election and so would result in higher turnout among sympathetic voters who tend not to vote in off-cycle elections. Equally important, this approach provided a year to educate voters about the issue and hopefully elevate it on voters’ priority list.

The research found that among likely voters there was general support for preschool, but a low level of understanding of the importance of preschool and a belief that early education was primarily a parent’s responsibility.

An education campaign about the value and importance of preschool was conducted in the early part of 2006. This effort, which was largely funded by foundations and 501(c)3 organizations, focused on messaging about brain development in early childhood, creating equal opportunity for all children and preparing children to be successful in kindergarten and beyond. The education campaign was built around television advertising. In total this effort lasted about three months and cost approximately $350,000. The education campaign was separate from the political campaign, but informed by the research and message development work done in late 2005 and guided by the same players. It concluded in March 2006.

2006 CAMPAIGN

While the education campaign was underway publicly, the joint Steering Committee worked behind the scenes to develop the specific ballot language, gauge support, identify vulnerabilities and build relationships with key stakeholders and partners, particularly Denver City Council members. As it was developed and refined, the program was named the Denver Preschool Program (DPP). Leaders polished the vision to ensure the program delivered on two explicit objectives: (1) maximize access to and participation in preschool programs for all 4-year-old children in Denver in order to improve school readiness and academic success; and (2) promote quality improvement of preschool programs available to those children.

The political campaign, known as Preschool Matters, officially launched in April 2006 and ran through Election Day, November 7, 2006. The campaign’s key tasks were to:

• Build campaign infrastructure and raise funds; in total, the campaign effort cost $1.5 million - $1 million raised cash, $500,000 in-kind support;
• Secure City Council support to refer the measure to voters on the November ballot; This happened in August 2006 and the effort was officially labeled Question 1A;
• Build an effective media campaign, consisting of television advertising and direct mail to likely voters, as well as earned media; and
• Execute a paid field campaign (door knockers) to turn out the vote among sympathetic voters.

Mayor Hickenlooper was the most visible champion for Preschool Matters, but a number of high profile business and civic leaders supported the measure publicly. The campaign collected endorsements from dozens of well-respected community organizations and relied on those supporters and their networks to help raise awareness and support for the proposal. The campaign also actively educated and engaged early care and education providers to help them understand the nature of the proposal and how it would benefit the families they served, as well as their programs. This work during the campaign phase was an important early step in building trust and partnership in the provider community.

Like all political campaigns, Preschool Matters had its critics. The most vocal opponents were the Anti-Defamation League, which opposed the proposal because faith-based preschool programs were included among providers eligible to participate, and the Colorado Convention and Visitors Bureau, which opposed it because of concerns that the sales tax increase would negatively impact convention business in Denver. While these groups made their opposition known through news coverage of the campaign, there was no funded opposition effort. At the time, Denver had two daily newspapers, The Denver Post, which endorsed the measure, and The Rocky Mountain News, which editorialized against the measure.

Election Day came and went and the outcome of the Question 1A was still unclear. The vote had been so close that a recount was required. Luckily, the recount was conducted relatively quickly and within about a week of the election supporters got the good news that Denver voters approved Question 1A by 1,815 votes.

Sales tax revenue began accruing to the program in January 2007, so there was no time to waste in establishing the new nonprofit that would administer the program. A transition team was put in place and Mayor Hickenlooper appointed the initial Board of Directors, which included one member appointed by Denver City Council. That group worked for approximately six months and then appointed an Interim Executive Director. The first tuition credits were distributed to providers on behalf of the families in September 2007 and the first permanent Executive Director of the organization was installed in November 2007.

2014 CAMPAIGN

Because of the 10-year sunset provision in the 2006 policy, Denver Preschool Program leaders would need voter support again to continue the program. Since the first campaign and election, public awareness of and support for early education had grown significantly, as evidenced by the fact that President Barack Obama called for universal high quality preschool for all American children in his 2013 State of the Union address. Later that same year, organization leaders made the decision not to wait for the full 10-year authorization period to pass, but to rather seek public support for program extension in 2014, another gubernatorial election year. This ensured that if the reauthorization campaign was not successful the first time, there would be another opportunity to win voter support before the program’s authorization expired. With the strong support of Mayor Michael B. Hancock, Denver Preschool Program leaders prepared for another campaign.

Reauthorization provided a key opportunity to revisit the policy established in the City ordinance that governs the program. By and large, the policy established in 2006 had proven quite effective. There was no pressing need to significantly revise the program’s framework. Rather, the program had matured and the benefit of experience taught program leaders and city officials where policy should be tweaked to create an even-more effective program (details of those policy changes are noted in the following sections regarding program structure, governance and operations.) The revised ordinance extended the program through 2026, 10 years from the expiration date of the original program authorization.

The reauthorization effort benefitted tremendously from the robust program evaluation and copious data available about the impact of the program (described in detail later in this paper). The concept had been proven effective, so the task at hand was to convince Denver voters to extend and expand an existing tax in support of a successful program. As such, a lengthy education effort was not needed and the campaign itself was much more modest in both time and budget.

The 2014 campaign lasted five months and cost about $530,000 - $427,000 raised cash and $100,000 in-kind media. The Denver Post editorialized in support of the measure twice. The Rocky Mountain News, which had opposed the proposal in 2006, was no longer in business. The primary opponent of the original proposal in 2006, the Anti-Defamation League, remained neutral on the question of reauthorization having been convinced that the program had been implemented in such a way as to ensure that public dollars were not subsidizing religious education. Voters approved reauthorization with just over 55 percent support.

KEY INSIGHTS

Crafting a focused proposal with clearly stated outcome objectives was key. Earlier efforts to fund a broad package of children’s programs failed – twice.

Establishing foundational tenants about program structure early in the policy development phase was helpful in guiding the policy development and ultimately the program.

Securing significant financial and technical assistance resources from an angel investor early on provided essential capacity to do the necessary planning and preparation work that supported a successful campaign.

Building a broad base of active support across business and civic leaders, in addition to children’s advocates and educators, was key to executing a successful campaign and securing public support.

Assessing the public’s understanding of and support for early childhood education programs well before launching the first campaign was essential. Doing so clearly indicated the need to undertake an educational effort about the value and benefits of preschool before asking taxpayers to invest in it.

While the policy established in 2006 was largely successful, some revision was needed in the program structure and governance to address changing needs as the program transitioned from a start-up to a mature organization. The 10-year sunset provision and 2014 reauthorization provided a timely opportunity to revise some components of the policy structure.

PROGRAM STRUCTURE, GOVERNANCE & OPERATIONS

INDEPENDENT NONPROFIT

An independent nonprofit corporation, Denver Preschool Program, was established for the sole purpose of administering this program. The organization enters into an annual contract with the City and County of Denver to administer and expend Denver preschool tax revenue. While a private entity, DPP is contractually obligated and committed to public transparency and accountability and as such the organization operates in accordance with state transparency laws that apply to public entities.

When originally established, the City ordinance required the Denver Preschool Program to have a governing Board of Directors, consisting of seven members, including one member of the Denver City Council. All private citizen members of the Board are appointed by the Mayor and confirmed by City Council to serve three-year terms and are not permitted to serve more than three terms. The City Council representative on the Board is selected by City Council. The original ordinance also required the establishment of a 25-member Board of Advisors to make non-binding recommendations to Board of Directors on policy issues regarding early childhood education and administration of the program. They were, in essence, the content area experts.

When the ordinance was revisited ahead of the 2014 reauthorization campaign, two important changes were made to the governance. First, the Board of Directors was expanded from seven members to a minimum of 11 and a cap of 15 members. Second, the Board of Advisors was eliminated. While the Board of Advisors played a critical role during the start-up phase of the organization, it was not as relevant or necessary to the operations of the mature organization.

FUNDING

The 2006 measure established a .12 percent (12 pennies on a $100 purchase) sales tax increase. The 2014 measure increased the tax to .15 percent (15 pennies on a $100 purchase) sales tax. The increase was needed to meet the growing demand of students in full- and extended-day programming, to reinstate and sustain year-round programming and to keep up with the rising cost of preschool. All funds are collected by the City and County of Denver and are subject to annual appropriation by City Council. Sales tax revenue fluctuates and its performance is tied to economic conditions, which complicates projections, but historically the sales tax has generated approximately $9 - $12 million in revenue. With the increased tax rate approved in 2014, the tax is projected to generate approximately $18 - $20 million going forward.

There are pros and cons to any public funding mechanism. Sales taxes can be volatile and are subject to economic changes. The Denver Preschool Program certainly experienced the impact of sales tax declines during the Great Recession of 2008 and 2009, and was forced to take painful steps to contract the program accordingly. That said, sales tax bases tend to increase over time and create the opportunity for capacity growth in the program, whereas other public funding sources, including mill levies, grow relatively slowly and cannot similarly accommodate program growth.

The Denver preschool tax revenue can be used for seven explicit purposes, as laid out in the enabling ordinance:

• Administrative expenses – this includes salaries and office expenses, routine business expenses and reimbursement to Board members for expenses incurred in connection with their service on the Board. Administrative expenses were capped at 5 percent in the original ordinance and increased to 7 percent in the revised ordinance. This is the only expenditure that is explicitly limited by ordinance.
• Tuition credits.
• Outreach to parents and the Denver community.
• Assistance with preschool enrollment.
• Technical assistance and direct grants to preschool providers for the purpose of improving program and service quality.
• Contracting with qualified experts to design and help implement a quality improvement system for preschools.
• Establishment of independent evaluation in order to provide measurement of program performance and child outcomes and the preparation of associated performance reports.

While there are variances year-to-year, the organization’s budget typically follows this approximate revenue allocation model:

• Tuition credits – 70 percent of revenue
• Quality improvement – 10 percent of revenue
• Community outreach – 5 percent of revenue
• Enrollment support and customer service – 3 percent of revenue
• Evaluation – 3 percent of revenue
• Other contracted services – 2 percent of services
• Administration – 4 percent of revenue (limited in City Ordinance to 7 percent)

Additionally the organization maintains an operating reserve to help weather economic downturns and ensure there is revenue to close out the program should voters choose not to renew it at sunset.

Tax revenue is the primary funding source for the program; however, nothing in the enabling ordinance prohibits the Denver Preschool Program from securing private revenue to supplement public resources. While DPP is permitted to secure private contributions through grants or other sources, the organization has carefully considered its approach to private funders and limits requests to special projects that enhance its core programming.

TUITION CREDITS

Expanding access to and participation in preschool programs is one of two primary objectives of the Denver Preschool Program and tuition credits are the vehicle for reaching that objective. Any Denver resident who is a parent or legal guardian of a preschool-aged child who is also a Denver resident is eligible for a tuition credit to be used to enroll that child in preschool in the year prior to the year in which the child will be eligible for kindergarten. Because Colorado law states that the age eligibility for kindergarten is 5 years old, this practically means that the subsidy is available to families of 4-year-old children whose birthday is on or before October 1.

The City ordinance that was approved in the reauthorization campaign empowers the Denver Preschool Program Board of Directors to consider providing subsidies to younger children. While the Board has not yet exercised that option, it is newly available to them.

Tuition credits are administered on a sliding scale, with the amount of the credit being inversely related to the family income of the recipient and related to the quality of the preschool provider with whom the credit is used. As such, low-income families choosing high quality preschool programs are eligible for the highest subsidies. The average tuition credit distributed in 2014-2015 was $303 per month for a student attending a full-day program.

Tuition credits are provided to any participating preschool on behalf of the parent or guardian. The credits follow the child should the child move during the preschool year. Preschools are eligible to accept tuition credits if they are duly licensed under the Colorado Child Care Licensing Act, or are an early childhood education program administered by Denver Public Schools, and if they agree to participate in a three-part quality rating and improvement program.

In the early years, the Denver Preschool Program provided subsidies for year-round education programs. However, when the Denver economy suffered as part of the Great Recession, sales tax revenue took a significant hit. As a result, program leaders were forced to consider how to contract the program to meet lower-than-projected revenue. While the Board of Directors considered a number of policy options, they ultimately decided to reduce the number of months that coverage was provided. Instead of providing subsidies for year-round enrollment, the organization would subsidize only nine months of a preschool year. This was a painful decision for program leaders and it had a very real impact on Denver families, as well as Denver early childhood education programs.

The 2014 reauthorization increased the amount of the sales tax dedicated to the program, which allowed DPP to reinstate year-round subsidies, beginning in the summer of 2015. Further, as a result of this experience, and thanks to the increased investment of public resources, DPP established and funded a firm reserve target to buffer against future economic volatility. Fulfilling promises to the program’s key stakeholders – families and providers – is a key principle of how the Denver Preschool Program operates and the organization is committed to doing all it can to avoid having revenue fluctuations impact parent and provider experiences with the program.

COMMUNITY OUTREACH & ENROLLMENT

The Denver Preschool Program is only effective if Denver families know about the program and apply to receive funding. As such, DPP has taken a thoughtful and comprehensive approach to developing and implementing community outreach and enrollment strategies since its inception. Both functions – outreach and enrollment – are managed by DPP staff and executed by contractors.

Community outreach entails paid media (advertising), earned media (news coverage), social media, community events and provider partner support. All media – paid, earned and social – is executed through contracts with various consulting firms, including those with expertise in communications strategy, branding, advertising, direct marketing and digital content. Specific elements of the strategy have evolved over time, allowing the program to adopt changing practices in the field and reflecting community and program realities at the time.

However, consistent throughout has been the understanding that our preschool providers are the best asset to reaching families and encouraging them to participate in DPP. Throughout a cyclical communications process DPP communications efforts have always been child-focused, and maintained a clean, professional, look and brand targeted at reaching adults. DPP strives to differentiate itself from its providers so as not to be mistaken as a preschool provider itself.

In addition to media, DPP has maintained a strong presence in Denver neighborhoods by hosting events and sponsoring innovative initiatives. DPP was instrumental in creating the One Book, One Denver program, which was a city-wide initiative promoting early literacy and providing free books to young children. This initiative grew into the One Book 4 Colorado program that now distributes a book to every 4-year-old in Colorado each year. In 2013, DPP began hosting Preschool Showcase events designed to educate parents about DPP and connect them to preschools in their neighborhood. While the Showcase events have evolved over time, they remain an important marketing tool for DPP and for the preschools themselves.

Like community outreach activities, enrollment services are executed by a third-party contractor and managed by Denver Preschool Program staff. The contract for enrollment services includes: a call center to interface with families and providers, application, attendance, and tuition credit processing. This vendor also holds a tremendous amount of data, which is critical to the program’s evaluation efforts.

Finally, it is worth noting the important role that child care providers play in both educating families about DPP and helping with enrollment. In many cases, families connect with their provider before they connect with DPP. As such, the provider is the trusted referral to DPP and an important ambassador for the program. DPP works to ensure providers have up-to-date resources, information and materials necessary to help connect families with DPP.

QUALITY RATING AND IMPROVEMENT

Improving the quality of available child care and preschool options is the second primary objective of the program and so quality rating and improvement activities are embedded throughout the Denver Preschool Program. Most significantly, DPP makes participation in quality rating and improvement one of only two requirements for provider participation (the other being state licensure).

To implement the quality rating and improvement activities, the Denver Preschool Program used only the Qualistar Rating™ or the National Association for the Education of Young Children (NAEYC) accreditation framework from the 2007-2008 school year through 2012-2013 school year to rate providers and develop quality improvement plans for them. These systems focus on measuring the structural components of early learning, including characteristics of the learning environment, teacher / staff qualifications, ratio and group size and family partnerships. Qualistar used a star rating scale of zero (provisional) to four stars, with four stars being highest quality. DPP providers that attained NAEYC accreditation were given a rating equivalent to four-stars on the Qualistar Rating™ scale.

In the 2012-2013 school year, in response to a growing body of research about the importance of teacher-child interactions, including the findings of a DPP evaluation study, the Denver Preschool Program added significant investments in CLASS®, the Pre-K Classroom Assessment Scoring System, to its quality rating and improvement activities. CLASS® utilizes classroom observation to evaluate the nature of teacher-child interactions. CLASS® compliments the Qualistar Rating™ and NAEYC accreditation framework to provide a more complete picture of program quality. To implement this change, DPP created its own five-level quality rating system in the 2013-2014 school year, which combined the CLASS® with either the Qualistar Rating™ score or NAEYC accreditation, and began requiring that each DPP classroom receive the CLASS® observation once every two years.

In addition to requiring participation in quality rating, the Denver Preschool Program provides technical assistance and grant funding to child care providers to undertake quality improvement activities. These activities can include acquisition of learning materials, early childhood education coursework for providers and participation in conferences, training and coaching aligned with the provider’s quality improvement plan. DPP has worked to continuously review and improve its quality improvement resource offerings, leading to the creation of a new quality improvement framework in 2015 based on the core values of intentional teaching, highly effective teachers and positive teacher-child interactions. These core values are prioritized when supporting providers in creating their QI plans and in selecting how they will use their quality improvement funding allocation.

Finally, as noted above, the amount of a family’s tuition credit is determined, in part, on the quality rating of the preschool they choose. This policy creates an incentive for parents to choose highly rated programs, thus driving consumer demand for quality in the market.

Quality rating in early learning programs is a quickly evolving area in the field of early childhood education. The State of Colorado launched a new quality rating and improvement system in 2015 tied to the state’s licensure of child care facilities. Known as Colorado Shines, the new state system assesses quality in five domains:

1. Workforce Qualifications and Professional Development
2. Family Partnerships
3. Leadership, Management and Administration
4. Learning Environment and
5. Child Health Promotion.

While there are many similarities between Colorado Shines and the DPP Classroom Rating, there are also some key variations. As such, the Denver Preschool Program adopted a two-year plan to align its quality rating and improvement plan with the state’s system and move away from having its own DPP Classroom Rating. This extended transition period, now complete, was designed to ensure participating child care providers could benefit from the investments they have previously made to secure a quality rating and provide ample time for them to transition to the new system. While CLASS® was not recognized as part of the old Qualistar Rating™, providers can earn points within Colorado Shines if they achieve certain CLASS® observation benchmarks, which is one important reason why DPP chose to align with the state’s new rating system.

Remaining nimble and being prepared to respond to changes in the field of early education is another key principle of how the Denver Preschool Program operates, as is evident in the evolution of the quality rating and improvement activities of DPP.

EVALUATION

Recognizing that Denver was undertaking an innovative approach to funding early education, measurement and evaluation has been a key tenant of the program since its inception. The Denver Preschool Program contracts with a small group of academic institutions and private companies to conduct a comprehensive annual evaluation to show, in measurable terms, how the program benefits Denver children. A key partner in this effort is Denver Public Schools (DPS), which provides achievement data for DPP and non-DPP students throughout their time in DPS used in the DPP longitudinal student outcomes study.

The evaluation looks at impact on student achievement, impact of preschool quality and program operations. Evaluation of student achievement centers around two primary questions: Are students ready to learn when they enter kindergarten? How long after preschool do its benefits last? Program evaluation looks at impact on availability of quality preschool programs and efficiency and efficacy of Denver Preschool Program operations.

More information about the impact of the program is available later in this paper, but the bottom line is that data shows that DPP students are ready to learn when they enter kindergarten and they are outperforming non-DPP student on reading, writing and math assessments when they reach third and fourth grades.

Recognizing that the field of early childhood education is developing and evolving rapidly, the annual data collected through this rigorous evaluation process is invaluable to informing program operations and supporting efforts to continuously learn and improve the Denver Preschool Program. In addition to informing internal operations, the student achievement and program operations evaluations are made available through annual reports to the Mayor, City Council, City Auditor and the public. As stated earlier, the data and research measuring impact of the program was particularly valuable in the 2014 campaign to secure continued public support for the program.

KEY INSIGHTS

There are pros and cons to any public funding mechanism. Whatever public funding source is used, program leaders must be cognizant of potential pitfalls or limitations of that structure and be empowered to address them.

Employing a mixed-delivery system that empowers parents to choose the preschool program right for their child – public, private, community-based, school-based or faith-based programs – and leverages other public funding sources, including Head Start and the Colorado Preschool Program, have been key in DPP becoming a truly universal program that works for the vast majority of Denver’s families.

While the mixed-delivery system is a net strength of the program, it has created some administrative and policy challenges in working with and supporting both Denver Public School programs and community based providers. Leaders implementing a mixed-delivery system program must be sensitive to the differing opportunities and challenges facing the various provider types and must be nimble in establishing program parameters that work for all provider types.

The Denver Preschool Program operates in the context of the evolving field of early childhood education and is reliant on a public funding source that is subject to changing economic conditions. Organization leaders must have the authority, flexibility and courage to adapt to new research and best practices and manage through changing economic conditions. Further, they must support program partners – namely families and early childhood education providers – in understanding and adapting to the changing landscape.

Program evaluation has been critically important, not only in measuring effectiveness and revising the program as needed, but also in maintaining public support and making the case for continued investment during the reauthorization campaign.

COMMUNITY IMPACT

STUDENT ACHIEVEMENT: SCHOOL READINESS AND LONGITUDINAL RESULTS

Annual evaluations are conducted to determine how students participating in the Denver Preschool Program progress in their preschool year, how they present for kindergarten and their performance through their early elementary years. Evaluators examine five key questions related to student achievement:

1. Do children make progress in their development while in Denver Preschool Program early childhood environments (i.e. language, literacy, mathematics and social-emotional development)?
2. To what extent and in what areas are children who participate in the Denver Preschool Program ready for kindergarten?
3. Do children from different income levels and with different primary languages make similar progress in their development while in Denver Preschool Program early childhood environments?
4. Do children who receive Denver Preschool Program tuition credits compare favorably with the district as a whole on assessments administered by Denver Public Schools in kindergarten and beyond?
5. Is attendance at higher-rated preschool programs associated with greater kindergarten readiness and long-term academic success (as measured by the Transitional Colorado Assessment Program - TCAP)?

At the highest level, the annual student achievement evaluations have found that Denver Preschool Program students are ready to learn when they enter kindergarten and they are outperforming students who don't participate in DPP on reading and math assessments when they reach third and fourth grades.

Some of the data from the most recent annual evaluation of student achievement that inform and support this key finding include:

• Kindergarten readiness assessments consistently show that Denver Preschool Program children make growth in early literacy and math skills, receptive vocabulary and social-emotional skills.
• In 2013 the Denver Preschool Program’s first cohort of preschoolers took the third-grade Transitional Colorado Achievement Program or TCAP. Results show that 64 percent of third-graders that participated in Denver Preschool Program posted advanced or proficient reading scores compared to 58 percent of students that did not participate in the Denver Preschool Program.
• Results of the second class of preschoolers to take the third-grade TCAP in the spring of 2014 are consistent with the 2013 results: 64 percent of students that participated in the Denver Preschool Program posted advanced or proficient reading scores compared to 56 percent of students that did not participate in the Denver Preschool Program.

In addition to core academic performance metrics, Denver Preschool Program looks at developmental benchmarks for young children with the understanding that social-emotional development has a direct impact on school readiness and learning. When looking at behavioral skills, most DPP children rate high for initiative, self-control and attachment. Protective factors were a concern for fewer than 15 percent of children. Further, teachers identified behavioral concerns for fewer than 10 percent of children.

Denver Preschool Program data shows low-income and dual-language children demonstrate the greatest growth in kindergarten readiness. This finding is consistent with national research. However, unlike national experience, DPP data also shows that children across all gender, racial and socio-economic categories demonstrated growth and sustained student achievement through the early elementary years.

While there is much to celebrate, the data and evaluation experience also points out the limitations of the Denver Preschool Program approach. Vocabulary is very hard to impact in a single-year intervention with 4-year-olds. DPP experience shows that children who come to preschool with deficits in vocabulary do not make sufficient progress in this area in just one year. Language acquisition is one clear example of the critical learning that happens in the infant and toddler years. In this area, the data illustrates the fact that a single year of preschool is not a silver-bullet solution to addressing the needs of the diverse student population that DPP strives to serve.

To ensure the program’s standards are sufficiently rigorous, the evaluation framework is regularly revisited and revised. The Denver Preschool Program strives to establish challenging expectations for itself and the preschool providers it partners with, in order to give DPP students the greatest opportunity to succeed. To this end, we regularly ask ourselves whether we are measuring what matters most to impacting student success. Looking to the future, we hope to develop effective measures to evaluate more abstract, but critically important executive functioning skills such as the ability to work effectively in teams, to switch between tasks, sustain attention, take turns and control emotions.

As early participants in the program progress through their education and more students participate in the Denver Preschool Program and as we refine and strengthen our frameworks for evaluation, more data and more meaningful data will become available to enable even richer analysis of the impact that the program has had on student achievement.

PRESCHOOL QUALITY IMPROVEMENT

Evaluation has shown that, when provided resources and support to undertake the work, Denver’s preschool providers have embraced quality rating and improvement. The most recent evaluation shows that 80 percent of the Denver Preschool Program’s participating providers take advantage of available quality improvement process and resources. Providers report that they have modified their hiring standards and curriculum, and are increasingly taking advantage of the coaching support offered by the Denver Preschool Program in order to improve quality.

The Denver Preschool Program considers a provider to be “high quality” once they have achieved at least three stars on the Qualistar Rating™ scale. As of the most current analysis for the 2013-2014 school year the Denver Preschool Program had 252 participating sites. Of the 252 sites, 207 sites (82 percent) had received a Qualistar Rating™ of either three or four stars. The vast majority of Denver Preschool Program students are enrolled in high performing programs. Over 35 percent of children were enrolled in 4-star classrooms, while 57 percent attended 3-star classrooms.

DPP's evaluators have also looked at how DPP provider quality changes over time. With the first DPP providers having joined the program in 2007-2008 and receiving a quality rating once every two years, DPP has a growing data set to examine how quality levels change over time. As of April 2014, 100 sites (with 168 DPP classrooms) had received their second rerating (defined as two ratings after their initial rating). For these sites, the evaluator found that nearly all classrooms that received a provisional, 1-star or 2-star Qualistar Rating™ on their initial rating had received a rating of 3-stars or 4-stars on their second rerating. The below chart shows how Qualistar Ratings™ changed for these 100 providers by their 2nd rerating when compared to their initial rating.

The data is promising in that 100 percent of sites initially rated provisional or 1-star improved by their 2nd rerating, 81 percent of sites initially rated 2-stars improved to at least 3-stars during their 2nd rerating, 36 percent of sites initially rated 3-stars improved to 4-stars during their 2nd rerating, and 81 percent of sites initially rated 4-stars maintained that level during their 2nd rerating.

The below chart shows how quality continues to increase over time:

The chart shows how the number of 4-star sites increase and 2-star sites decrease the longer they participate with DPP. While only 20 percent of sites earned 4-stars on their first rating, 54 percent of sites have earned 4-stars by their 3rd rerating, and the percentage of 2-star sites has dropped from 15 percent to 4 percent by their 3rd rerating.

While quality rating is central to the Denver Preschool Program, no quality rating system can be perfect. The rating scale is reflective of a number of variables as observed and evaluated at a given point in time. Conditions change within a preschool that can impact quality – teacher and staff turn-over, changes in physical environment to name just two – but the Qualistar Rating™ was valid for two years and the new Colorado Shines Rating is valid for three years. This can be to the benefit or detriment of the provider depending on any changes that occur between ratings. However, experience has shown that, in general, once a provider achieves a high quality rating level they tend to maintain that rating.

PROGRAM OPERATIONS

In the face of historic economic volatility and three leadership changes over the decade of its existence, the Denver Preschool Program has consistently received good feedback from its key partners – families and child care providers – about program operations. Provider complaints have mostly centered around the quality rating system. Historically parents have noted challenges with the tuition credit application process, mostly around application complexity and time required to complete it, but in recent years that has improved as the organization contracted with a new enrollment vendor in 2012 and undertook efforts to streamline the process and provide more assistance.

KEY INSIGHTS

Measuring the Denver Preschool Program’s impact across the three domains of student achievement, availability of quality preschool programs and organization operations has been essential in fulfilling the commitment to continuously refining and improving the DPP and in building community trust in and support for the program.

Evaluating student achievement impacts requires looking at student performance at various points-in-time, focusing on growth and measuring social-emotional development, not just academic performance.

When provided with resources and supports to implement it, most preschool providers embrace quality improvement efforts.

We have yet to find the perfect evaluation framework for measuring either student achievement or provider quality, but have not allowed the perfect to be the enemy of the very good.

LOOKING AHEAD

From 2007 to 2017 the Denver Preschool Program has invested more than $92 million in tuition support and helped ensure nearly 46,000 Denver students have entered kindergarten ready to learn. During the 2015-2016 school year DPP distributed $12.8 million in tuition support and nearly $2 million in quality improvement dollars and served 5,079 kids. With reauthorization secured through 2026, DPP is poised to more than double that impact in the years ahead. As we look ahead to the significant opportunities on the horizon, three are worth highlighting here.

First, the evolution of the Denver Preschool Program's approach to quality rating and improvement provides a tremendous opportunity to continue to learn from, refine and improve the DPP model. Measuring provider quality and implementing quality improvement efforts in early childhood education is a rapidly evolving field. We are committed to regularly and rigorously evaluating our approach and learning from our experience, as well as drawing from research that is emerging elsewhere.

Second, we are excited about the opportunity to contribute to the growing national conversation about the efficacy and importance of early learning. We believe that our experience and our organization’s evaluation results can be informative and useful to others who share our goals of nurturing young learners. Further, we look forward to learning from the experience of others. Ultimately, we believe that over the next decade, early learning will continue to grow to be recognized by parents and policymakers alike as being every bit as important as the K-12 and higher education systems in America.

Third, is the opportunity to better align and integrate preschool into the life long learning continuum. Today, the Denver Preschool Program addresses a critically important, but very narrow slice of the educational spectrum. The infant and toddler years (birth through age 3) are a rich period of learning and development, as is particularly evident when you look at language acquisition and vocabulary development in that period. While the Denver Preschool Program’s primary focus will always be on ensuring every 4-year-old makes necessary progress in the pre-kindergarten year so as to start school ready to learn, there is tremendous opportunity for impact with even younger children. The revised City ordinance approved in 2014 gives DPP the option to explore if and how to support learning for younger children. Equally important, is ensuring quality kindergarten and early elementary experiences into which DPP students can progress, so that their investment in quality early learning experiences can be maximally beneficial. Ensuring graceful transitions from early learning settings to elementary school and supporting alignment across the learning continuum for children across the full early childhood spectrum, birth through age 8, is a tremendous opportunity for DPP in the years ahead.

As researchers, educators and policymakers come to better understand the promise that the early childhood years hold, there is new focus, energy and momentum in the field. The Denver Preschool Program is excited to be part of this change that will serve to create a stronger future for our children and our country.

NOTES

The Denver Preschool Program thanks Cody Belzley of Common Good Consulting, LLC for her invaluable partnership in researching and writing this paper.

We also wish to acknowledge and thank the following Denver Preschool Program partners for offering their thoughtful input and comments in the drafting of this paper:

Linda Campbell, Community Volunteer
Pamela Harris, President & CEO, Mile High Early Learning
Lynea Hansen, Senior Vice President, Strategies 360
Anna Jo Haynes, President Emeritus, Mile High Early Learning
Theresa Peña, Former Chair of the Board of Directors, Denver Preschool Program

Additional information and resources are available at our website: dpp.org