Archive for the ‘Top 5 reader favorites’ Category

School planning in the era of Race to the Top

Wednesday, August 3rd, 2011


Race to the Top (RTTT) has arrived in full force. School districts across the country are already feeling its impact. In New York State, RTTT is driving the Board of Regents reform agenda focusing on three areas:

  • Common core standards
  • Data driven instruction including school-based inquiry teams
  • New teacher/leader evaluation systems based, in part, on student performance

Figure 1 - Click to Enlarge

The challenge


Just a decade after the adoption of No Child Left Behind, the educational ship of state is once again charting a new course. The challenge to each school district? To climb aboard the ship and not be lost in its turbulent wake.

Districts will thrive if they can overcome inertia without a substantial expense of resources, including wasted time, energy and budget. District leadership will need to adjust their current trajectory and redirect resources to a new set of priorities quickly and efficiently.

A model plan


Chenango Forks Central School District provides a model of planning dexterity that other school districts may want to emulate. For the last seven years, the Forks has followed a disciplined comprehensive educational planning process that includes a set of strategic performance targets, rolling priorities and long-term strategies. Typical human behavior would have suggested that the Forks district team would hold tightly to the plan they were heavily invested in and resist change.

Instead, Associate Superintendent Kathleen Dixon posed the challenge very clearly: how do we incorporate the work required of us by RTTT while maintaining our distinct local character and priorities?

Figure 2: Click to Enlarge

The district team embraced Mrs. Dixon’s challenge from the start, looking with open, flexible minds at how to accomodate the RTTT requirements while making the plan uniquely Chenango Forks’. The first task was to swap some new performance targets for old ones. They added

  • NYS Grade 3-8: ELA & math % proficient, disaggregated by all students, students with disabilities and economically disadvantaged students
  • Regents ELA: % cohort college ready (> 75)
  • Regents Math: % cohort college ready (> 80)

At the same time, they eliminated five performance metrics from their plan while modifying another three (see Figure 1 – click to enlarge).

The Forks breakthrough


The real breakthrough came as the team prepared to take a fresh look at the district’s improvement strategies. The facilitation plan was to have the group explore a wide set of possible new strategies and then converge on a final set of priorities and strategies. However, as the team broke for lunch, Nicole Knapp, the Forks elementary school principal, approached the facilitator and said, “I think I can speed up this whole process.” She left an outline of a brand new way to articulate the district’s strategies.

In effect, Mrs. Knapp had completed a matrix analysis that identified

  • New strategies required by RTTT.
  • Current district strategies that complimented the RTTT strategies.
  • Current strategies that reinforced the district’s recent investments and unique priorities.

All other strategies were jettisoned, allowing the Forks to emerge from the planning session with a tight, focused plan that redirects scarce resources to a new set of priorities that align with RTTT, the Regents reform agenda and the district’s distinct local character (see Figure 2 – click to enlarge). For example, the plan retains these unique priorities:

  • Use the Chenango Forks’ Writing Continuum to guide curriculum planning and all instruction.
  • Implement the five-year technology plan to increase achievement and to promote 21st century skills for students, staff and parents.
  • Incorporate measures of 21st century skills in both the APPR and local assessments.

When the district team returned from lunch, they immediately resonated with Mrs. Knapp’s work and were quite excited by it. She had achieved what Edward DeBono calls “the insight re-arrangement of available information.” The team tweaked her work, approved the plan and adjourned about four hours early.

Yes, the Forks is aboard the ship of state and settled into a state room that they find very familiar and quite comfortable.

Tags: , , ,
Posted in All blog posts, K-12 education, Top 5 reader favorites | 1 Comment »

How to: Multi-criteria Analysis – Part 1

Monday, April 18th, 2011


You Are What You Decide asked: So, how do you decide? Based on your gut intuition, a formal analysis of trade-offs, your core beliefs, multiple criteria, the alignment to your vision for a preferred future? Or something else?

This three-part post will walk step-by-step through a multi-criteria analysis, a formal decision process you might use when making a weighty decision. Here are the steps:

Part 1:

  • Step 1: Complete your research
  • Step 2: Generate and weight criteria (i.e., define your desired outcome)

Part 2:

  • Step 3: Identify and/or create options, alternatives or solutions
  • Step 4: Complete a side-by-side comparison

Part 3:

  • Step #5: Objectively assess each option against each weighted criterion
  • Step #6: Make your decision

The steps are listed linearly for ease of understanding. But rarely does a decision process unfold without iteration. Thoughtful multi-criteria analysis leads you to loop back-and-forth between steps, clarifying and refining your thinking as you go. For example, although you may begin with an initial set of options, that set will likely be refined and enriched as you complete your research, generate and weight criteria, apply the criteria and move to decision-making.

Step 1: Complete your research


Whether buying a car or house, choosing a college or making some other consequential choice, you need first to become well informed.

  • What do I need to know?
  • What are my requirements?
  • Are there constraints?
  • What are the options?
  • How can I compare those options fairly?

In my experience, folks make two critical mistakes at this phase of decision-making. They do not invest sufficient time or resources in research, and they move to decision-making before they have investigated a wide range of options.

Consider the college search process. In terms of financial investment, it is one of the more significant decisions a typical family will make. In New York State, the total four-year investment at a public institution approaches $80,000 and at a private institution often exceeds $200,000. In some cases it tops out at $250,000.

Nonetheless, it is common for overwhelmed families to skimp on or rush through the information gathering phase. I have heard parents remark that they couldn’t afford to purchase the standard college guides, such as the Princeton Review ($22.99), because they are too expensive! I know students who neglected to use amortization calculators to determine the true cost of their student loans, both in terms of monthly payments and total payback with interest. Families often visit few college campuses before making their final selection. The consequences of these short-cuts are predictable. For example, after dropping her child off at college for the first time, a mom I know lamented: I was surprised how rural it was.

Multi-criteria case study

Consider this case study. Prism recently facilitated a diverse team of school district stakeholders through a successful multi-criteria decision. The Elmsford Union Free School District facilities planning task force was grappling with a complex decision. Elmsford’s Dixson Primary, a PK-1 building built in 1894, enjoys the love of many community stakeholders but is considered woefully inadequate to serve the needs of its students. The issue is emotional and many task force members arrived with preconceived notions: some, that the building needs to be closed; others, that building should be renovated. There was conflict and disagreement.

The planning process required that the team step back from their preconceived notions and thoroughly research the issue. For example, they spent considerable time walking through the facilities and researching state education law, financing and other issues. They needed to build a shared knowledge base so that their conversations — including their disagreements — could be informed by fact wherever possible. As shared knowledge grew and many untested assumptions proven invalid, the conversation became less polarized and more informed.

Step 2a: Generate criteria


Once they were well-informed, the task force began to list its criteria. A criterion is a standard, rule, or test on which a judgment or decision can be based. For example, if you are buying a house, a criterion might be “number of bedrooms” or “size of the yard.”

The Elmsford task force went through a creative process, diverging to consider many criteria and then converging on a final set of eleven. They defined their criteria clearly and simply, using a few words, like a book title, followed by a succinct definition. They agreed to eleven criteria, including “Multi-purpose: there is a large, multi-purpose room that has a stage and that can accommodate the school population” and “Flexibility: the facility has the flexibility to accommodate short- and long-term growth,” among others.

A family selecting the best college can use a similar process. Using 3 x 5 cards, articulate one criterion per card until all possible criteria have been listed. As a family, share, clarify, combine and refine your criteria. Then converge on a final set. Like the Elmsford task force, you will have defined your desired outcome and now know explicitly what you want to achieve. In the parlance of IT professionals, you will have completed your requirements analysis. Or, simply put, now that you know what you want, you just might get it.

Step 2b: Weight criteria


With the decision criteria agreed to, you can now weight them. First, order the final set of 3 X 5 cards in declining order of relative importance. Then assign a weight (between 0 – 100) to each criterion, making sure that the sum of all weights totals 100. Then begin to build a decision matrix, either using a table or, preferably, a spreadsheet. Place the weighted criteria in the top row. Later, you will add the options into the first column. See the sample decision matrix above. Or feel free to download the working spreadsheet.

The Elmsford task force used Prism’s Group Decision Support System to weight their criteria by completing a paired comparison analysis. View their results in the chart below.

Having identified and weighted your criteria, you are well on your way to creating a sound decision framework. In How to: Multi-criteria Analysis – Part 2, we discuss how to advance this framework by adding a set of options and completing a side-by-side comparison.




Tags: , , ,
Posted in All blog posts, How do you decide, Top 5 reader favorites | Comments Off

The Absurdity of New York’s School Accountability System

Monday, February 28th, 2011


The New York State Education Department (NYSED) released school report cards Thursday, February 17. A close look at the report cards reveals a very important, unreported story. We now have an absurd situation in New York State. For many schools districts to hit their accountability targets, they will have to improve at a rate that the NYSED does not consider credible.

“We need…a new set of assessments that are rigorous and dependable.”


In the spring of 2009, Board of Regents Chancellor Merryl H. Tisch was widely quoted saying that she was “skeptical” of the rate of improvement of grade 3 – 8 test scores. For example, between 2007 and 2009 New York City improved grade 3 – 8 English language arts (ELA) pass rates from 51% to 69%, a 35% increase. Chancellor Tisch also stated that the tests were too “predictable” and lacked “rigor.” She found the scores “suspicious.” The tests needed to be more “defensible,” and “dependable.”

In response, the NYSED made the 2010 ELA and math assessments more robust and raised the scoring cut points that determine the level of performance a student has achieved (i.e., not meeting, partially meeting, meeting or meeting learning standards with distinction). Consequently, across the state fewer students met learning standards, lowering schools’ test scores.

In fact, test scores dropped considerably and — in poor urban, suburban and rural areas — precipitously. Although the recently released school report cards use the old cut points for determining whether a school’s scores hit 2010 accountability targets, the new, higher cut points will be used to calculate 2011 accountability. At the same time the NYSED did not lower the 2011 accountability targets.

Can schools reasonably achieve their accountability targets with the new, higher cut points?


Consider this example. The first chart illustrates a typical rural school district’s five-year ELA test score trend based on the old cut points. The second chart substitutes 2010 scores calculated using the new cut points. Note the considerable decline in the district’s 2010 scores when the new, higher cut points are used. All student performance drops 17%, economically disadvantaged student (Low SES) performance declines 27% and student with disability (SWD) performance plummets 53%. (Such decline in district performance is common across the state: only the slope of the decline varies.)

The 2011 ELA accountability target is 155. Therefore, to hit its targets, this district’s economically disadvantaged students will have to improve by 38% and its students with disabilities by 154%—in a single year. The NYSED has a complex formula for achieving what it calls “safe harbor.” Based on that formula, the students with disabilities will have to score 75, a 23% increase. Each of those rates of improvement surpasses thresholds that provoked Chancellor Tisch’s skepticism in 2009.

Plot the NYSED’s targets out to the 2014-15 school year and the situation becomes almost surreal. Economically disadvantaged student performance must increase by 79%, more than 12% per year. Student with disability performance must increase by a whopping 228%, nearly 27% per year. Just to achieve safe harbor each of the five years, students with disabilities will have to improve by at least 93% or 14% per year.

Basically, what we have are two trains barreling into a wreck. The NYSED decided both to increase test scoring difficulty and to maintain aggressive improvement targets from 2011 through 2015. Considered separately on their own merits, both decisions are probably rational and defensible. However, taken together as part of a complex system, they produce this absurdity: To hit their accountability targets based on the new scoring cut points, many school districts will have to improve at a rate that the state does not consider credible.

The inherent contradiction within the current NYSED accountability system is exposed. Across the state, school children, their parents, guardians, teachers, principals, superintendents, boards of education and communities at large now face a no-win situation. If a district such as the one discussed above hits its targets, it will be treated with skepticism; if it doesn’t, it will fail.


Tags: , , , ,
Posted in All blog posts, K-12 education, Top 5 reader favorites | 2 Comments »

Is America in a Creativity Crisis?

Monday, July 19th, 2010


Last week, Newsweek published The Creativity Crisis with this subheading: For the first time, research shows that American creativity is declining. What was the evidence for this disturbing claim?

College of William & Mary researcher Kyung Hee Kim analyzed 300,000 scores of children and adults on the Torrance® Tests of Creative Thinking. She discovered that although creativity scores had been steadily rising until 1990, “since then, creativity scores have consistently inched downward. ‘It’s very clear, and the decrease is very significant,’ Kim says. It is the scores of younger children in America — from kindergarten through sixth grade — for whom the decline is ‘most serious.’”

Three days after the Newsweek piece appeared, my local newspaper published an article with the following lede:

“Nicholas Nieves is a veteran test taker. The Horace Mann Elementary student is 10 years old. ‘We do practice tests and practice tests and more practice tests,’ the soon-to-be Binghamton fifth-grader said. ‘And then we do the real thing. We take a lot of tests.’ But does he learn from the tests? ‘Not really,’ Nicholas said.”

Are Nicholas’s insights related to Ms. Kim’s? I suspect so — as are numerous other phenomena that may be sapping a child’s natural inclination to exercise the imagination and be creative. These days, children are less likely to experience active, free and unstructured play and more likely to experience passive, guided and structured play. Just as a for instance, the Kaiser Family Foundation reports that “8-18 year-olds devote an average of 7 hours and 38 minutes to using entertainment media across a typical day” and that doesn’t count the 1 hour and 35 minutes they spend each day sending or receiving texts.

If Ms. Kim’s findings are accurate, the news is disturbing for the nation’s children and our future prosperity. Consider. Scholars have been tracking those administered the Torrence tests to see how well performance on the test predicted creative accomplishments in adulthood. The findings? “Those who came up with more good ideas on Torrance’s tasks grew up to be entrepreneurs, inventors, college presidents, authors, doctors, diplomats, and software developers. Jonathan Plucker of Indiana University recently reanalyzed Torrance’s data. The correlation to lifetime creative accomplishment was more than three times stronger for childhood creativity than childhood IQ.”

Obviously, the converse will prove true: the less creative our children, the fewer innovative and productive adults our nation will produce. But the situation is not dire. See The Antidote for America’s Creativity Crisis for a discussion of the intentional teaching of creativity.

Tags: ,
Posted in All blog posts, Creativity & problem solving, Top 5 reader favorites | 1 Comment »

The Rashomon Effect

Monday, April 12th, 2010


Here’s a typical pattern in collaborative decision-making. A team meets, works through a series of issues and reaches agreement on a course of action. They leave the meeting confident that there is consensus and alignment. They begin to act on the details of their agreement.

In the days and weeks that follow, however, it becomes increasingly apparent that team members have left the meeting with different and sometimes conflicting interpretations of their agreement. “That’s not what I heard” is the common refrain. Consequences can include wasted time, poor work product, and — more seriously — distrust and suspicion as team members begin to question one another’s integrity and start to speculate on ulterior motives.

Yes, team members sometimes lack integrity and speculate on ulterior motives. However, very often the scenario described above results from the “Rashomon effect,” named for Akira Kurosawa’s cinematic masterpiece, Rashomon. In the film, four individuals witness an horrific crime. Each then recounts the story with absolute honesty but in mutually contradictory ways. The Rashomon effect is the “effect of the subjectivity of perception on recollection, by which observers of an event are able to produce substantially different but equally plausible accounts of it.”

There are two simple strategies to prevent the Rashomon effect from undermining a team’s agreement.

First, replace verbal with written agreement. As the group converges, display the exact language of the agreement’s details — on a flip chart or preferably on a projection screen. Simple bullets will suffice. Team members will engage with the language, clarifying its nuances and making it increasingly clear and unambiguous.

Second, replace casual with explicit agreement. Once the agreement’s details are displayed, survey the team to test publicly each member’s understanding of and support for the final agreement.

This public commitment to a written, explicit agreement significantly reduces the risk that team members will produce “substantially different but equally plausible accounts of” that agreement and, therefore, you will have avoided the hazards of the Rashomon effect.

Tags: , ,
Posted in All blog posts, Consensus-building, Decision-making, Top 5 reader favorites | Comments Off