The New York State Board of Regents has raised the bar for graduation rate performance beginning with the 2009-10 school year results. The graduation rate target has increased from 55% to 80% for all students and for all subgroups of students.

Currently, statewide graduation rates hover around 70%. For many schools, graduation rates are significantly lower, especially for subgroups such as economically disadvantaged students or students with disabilities. For example, according to the state education department, just 56% of Black and 55% of Hispanic students who started 9th grade in 2005 had graduated after 4 years, by June 2009.

So the challenge to school districts is daunting, especially for those in poor rural, suburban, small urban and large urban areas. The educational issues facing districts are well known: retention and overagedness, poverty, home conditions, discipline, attendance, problems at work, drug and alcohol abuse, teenage pregnancy, among others.

It’s three years later: do you know who is in your 2007 cohort?

In New York State, there is also a data issue facing districts: year-to-year, month-to-month and day-to-day, they do not know accurately who is in each “active” accountability cohort and what each member’s status is vis-a-vis graduation. Wait a minute, you say. What about their student management systems?

Each year, a district’s student management systems purge inactive students from the data set. Most students are inactive because they have transferred from the district and are no longer a part of its official accountability cohort. However, inactive students also include those who have dropped out or transferred to GED.

So with each new school year, last year’s dropouts and transfers to GED disappear from the student information system, even though they are still members of the official accountability cohort. These students remain invisible to the district—its administrators, guidance counselors, teachers and staff—until they re-appear in the cohort verification reports in the spring of the cohort’s fourth year of enrollment. How can district personnel expect to manage a cohort effectively if they do not know who is in it?

Is there any way a district can query to get an accurate accounting of cohort membership? Yes, it is possible through a Regional Information Center (RIC), but there are two caveats.

• Doing so requires very strong database skills, which most districts do not possess.
• Even with strong database skills, the process is extremely onerous and time consuming.

Consider the following example provided by a very able and talented district data manager.

To establish accurate membership in the official 2007 accountability cohort (i.e., the cohort of students who enrolled September 2007) after the third year of their enrollment, I would have to run six separate queries of the database. I would query for the 2007 cohort, for each of three years and for active and inactive students. Each query would produce an extract with separate tables based on school and location (e.g., BOCES, etc.). I would then need to join more than a dozen separate tables and filter the duplicates.

The critical point is that after the first year of a cohort’s enrollment, we do not know accurately who is in that cohort. Therefore, we cannot actively manage that cohort with any confidence, dexterity or effectiveness. We really need easy and timely access to an accurate listing of each official accountability cohort.

Here is how Sandra Sherwood describes the challenge from her role as Superintendent at the Dryden Central School District: “Keeping track of our student cohort sounds so simple, but it is very complex and we find ourselves surprised in the spring of each year to learn that members of a cohort are missing.  While we have gained much with the use of electronic databases, the roll-over process from year to year results in students being dropped from our active database and then during their graduation year, they suddenly reappear in the state’s list.  We need to be able to review and update the state database for our inactive students more easily before spring of their fourth year so that we can reach our graduation targets with confidence and be assured that students are not improperly being removed from their cohort. Right now, we receive the updated cohort list too late to make a difference for our students.”

An emerging cohort management tool set

Districts are under tremendous pressure to increase graduation rates and decrease dropout rates and they are working diligently to do so. Easy access to accurate and up-to-date lists of each accountability cohort would allow them confidently to track each cohort’s graduation status—by total cohort, cohort subgroup and each cohort member—in order to intervene with students who are lagging or seriously at-risk and to ensure their successful graduation.

For example, the Lockport City School District has boot-strapped in File Maker Pro a very useful cohort tracker tool that that allows guidance and administrative staff to track the graduation status of each cohort based on a set of critical factors: performance on Regents “gate” exams, total credits, foreign language, absences, tardies and discipline referrals. The system is color coded to identify students lagging or off track for graduation. It has alerts based on a student’s status related to disability, dropout, Regents exams, credits, absences and discipline. It can be sorted based on cohort year, graduation status and NCLB subgroup. (See screen shot above. Yellow indicates a student is lagging against a critical factor; red indicates the student is at-risk. Alerts are at right.)

Rob LiPuma, Lockport’s Director of Assessment & Technology and Chief Information Officer (CIO), has spearheaded the project. “Our guidance and administrative staff are working diligently to track the status of each cohort. And their efforts have borne fruit. We have improved our four-year graduation rates from 71% (2003 cohort) to 81% (2005 cohort). Cohort tracker was a critical capability facilitating that improvement.” What’s his biggest frustration? “My guidance and administrative colleagues are demanding greater accuracy in cohort tracker, and frankly we are having a challenging time getting up-to-date listings of official membership in each accountability cohort. There is simply no way to do so currently.”

The NYS Center for Rural Schools’ cohort tracker demonstration project

John Sipple, Director of The New York State Center for Rural Schools (NYSCRS), sees great promise in cohort tracker. “Whenever we meet with school district leadership and demonstrate Lockport’s cohort tracker, districts tell us they have a critical need for this capability. So we are sponsoring a demonstration project to expand access to cohort tracker capability to other districts. The plan is to introduce an easy-to-use desktop application that does not require its users to have database skills or knowledge of FileMaker Pro or any similar database program.

“We at the NYSCRS are working with Prism Decision Systems, LLC and Decision Mechanics Limited to build a cohort tracker desktop application that will be available to up to twelve school districts in phase one. In phase two, we are hoping to expand access to other districts across the state, either through broader distribution of the desktop application or through a web service.

“However, the effectiveness of the cohort tracker application will clearly depend on the availability of up-to-date listings of official membership in each accountability cohort. To exploit the cohort tracker’s capability, school districts will need easy access to that data.”

Download a PDF version of this white paper: Actively Managing a Cohort to Graduation

John Sipple, Cornell University
Andrew Tait, Decision Mechanics Limited
Sean Brady, Prism Decision Systems, LLC

Ever since I read Sidney Parnes’ The Magic of Your Mind more than twenty years ago, I have firmly believed that having more options is preferable to having fewer options. On the other hand, just about every time I go to the grocery store, I wish there were fewer choices. So I am conflicted: my training and belief system tell me the more options, the better; my experience, sometimes just the opposite.

Research now validates the conflict I experience.

Sheena Iyengar’s fascinating presentation, On the Art of Choosing, discusses the impact of culture on decision-making. One of her key research findings is that the American belief that more options are better than fewer is not universally held by all cultures: “Though all humans share the basic need and desire for choice, we don’t all see choice in the same places or to the same extent.”

I find this interesting but not surprising. What did surprise me is her assertion that even for Americans, having too many options can result in poorer decision-making: “When there are too many choices to compare and contrast, the process of choosing can be confusing and frustrating. Instead of making better choices, we become overwhelmed by choice, sometimes even afraid of it. Choice no longer offers opportunities but imposes constraints.” In fact, Iyengar’s research demonstrates that when you give people “ten or more options when they are making a choice, they make poor decisions.”

To view Iyengar’s full discussion of the impact of culture on decision-making, click below. If you wish to go directly to her dismantling of the assumption that the “more choices you have, the more likely you are to make the best choice,” fast forward to 8 minutes 10 seconds.

How did you do? Miss anything? If you did, you are in good company. I, for one, counted the number of passes correctly but failed to see the gorilla in the 1999 version. And in this new version I missed the player in black dropping out. On a positive note: I did notice the color change!

Cognitive scientist Daniel Simons showed the video to 76 University of Illinois students. As reported in i-PERCEPTION, for those who had not seen the 1999 version, only 56% noticed the gorilla (23 out of 41). Just “11% of subjects noticed the curtain change, and 16% noticed the change to the number of players on the black team. Only 1 participant noticed both the curtain and the player change.”

Simons defines the phenomenon as inattentional blindness, “the failure to notice unusual and salient events in their visual world when attention is otherwise engaged and the events are unexpected.” In a Seed Magazine interview, he discusses the implications to our everyday life:

In inattentional blindness you’re not seeing something that’s right there because your attention is engaged. The most obvious practical application of that is driving. We intuitively think that if something important happens right in front of us, we will see it…Dan Levin has done studies where he just asked people, would you notice if something like this happens? He shows people the video, gives them the instructions, points out the gorilla, and then asks them “how likely would you be to notice the gorilla if you were doing this task and counting the passes?” Ninety percent of people say they’d notice. Regardless of how you ask that question, you get high confidence, and a high percentage saying “yeah, of course I’d notice that.”

That’s the intuition that’s interesting, and that’s the one that’s dangerous. If we were completely aware of these limits on attention, we wouldn’t do things like talking on cell phone while driving: We would know that it would make us just that much less likely to notice something. But we don’t have that insight into our own awareness. It’s only in that rare case where you actually have an accident that you become aware that you’ve missed something.

As Simon’s studies have demonstrated, we are all prone to innattentional blindness. What impact does it have on our problem solving? What salient events in our visual field are we missing, even though we are confident we are not? How often do we see what we are looking for and not what is there?

Creativity is a discipline that can be taught. Although many consider this assertion to be self-evident, others vehemently disagree. A common retort: No matter how much training and practice I receive, I will never be a Mozart or a Shakespeare or a Beatle. But this misses the point. Few would make the claim that creative genius can but taught. But each of us can become better creative thinkers with quality training and sustained practice.

Consider this analogy. When I was a kid, I never suffered the illusion that I would be the next Jerry West or Oscar Robertson, but I did practice a lot of basketball and became fairly proficient at the game. Last fall, while killing time at the local YMCA, I went into an empty gym, picked up a basketball and made 14 free throws in a row. A fluke? Absolutely. But decades later, the muscle memory remained.

Just as we practice an athletic discipline over and over until its skills become automatic, so, too, we can practice creativity skills until they become habits of mind. For more than 50 years the Creative Education Foundation (CEF) has championed research and delivered training in deliberate processes to improve creative thinking. The CEF’s approach distinguishes between two major creativity skills, divergence and convergence:

• To diverge is to explore options, to consider all possibilities, to extend in different directions often while departing from the norm.
• To converge is to critically evaluate options, to move toward a common conclusion, to reach agreement, to make a choice or decision.

Diverging before converging is not the natural pattern for most people. By our natures, most of us are more apt to criticize each new idea as it is shared and less apt to defer judgment while carefully considering all possibilities. But the simple fact is that training and practice can instill the habit of diverging before converging in just about anyone.

Both these major creativity skills have sub-skills. For example, divergence requires the ability to think fluently and flexibly. Fluency is the ability to generate a large quantity of ideas quickly. Examples of fluency sub-skills include brainstorming, free-noting, tolerance for ambiguity, among many others. Flexibility is the ability to see diverse and unusual relationships. Here, too, there are many sub-skills, including forced analogies, lateral thinking and morphological analysis. And — like shooting free throws over and over until the motion is preserved in muscle memory — each of these sub-skills can be practiced repeatedly until they become automatic habits of mind.

Yes, we can all learn to be more creative thinkers. We have the research base, the educational resources, the knowledge and skill. America can reverse the trend toward declining creativity among its youth.

College of William & Mary researcher Kyung Hee Kim analyzed 300,000 scores of children and adults on the Torrance® Tests of Creative Thinking. She discovered that although creativity scores had been steadily rising until 1990, “since then, creativity scores have consistently inched downward. ‘It’s very clear, and the decrease is very significant,’ Kim says. It is the scores of younger children in America — from kindergarten through sixth grade — for whom the decline is ‘most serious.’”

Three days after the Newsweek piece appeared, my local newspaper published an article with the following lede:

“Nicholas Nieves is a veteran test taker. The Horace Mann Elementary student is 10 years old. ‘We do practice tests and practice tests and more practice tests,’ the soon-to-be Binghamton fifth-grader said. ‘And then we do the real thing. We take a lot of tests.’ But does he learn from the tests? ‘Not really,’ Nicholas said.”

Are Nicholas’s insights related to Ms. Kim’s? I suspect so — as are numerous other phenomena that may be sapping a child’s natural inclination to exercise the imagination and be creative. These days, children are less likely to experience active, free and unstructured play and more likely to experience passive, guided and structured play. Just as a for instance, the Kaiser Family Foundation reports that “8-18 year-olds devote an average of 7 hours and 38 minutes to using entertainment media across a typical day” and that doesn’t count the 1 hour and 35 minutes they spend each day sending or receiving texts.

If Ms. Kim’s findings are accurate, the news is disturbing for the nation’s children and our future prosperity. Consider. Scholars have been tracking those administered the Torrence tests to see how well performance on the test predicted creative accomplishments in adulthood. The findings? “Those who came up with more good ideas on Torrance’s tasks grew up to be entrepreneurs, inventors, college presidents, authors, doctors, diplomats, and software developers. Jonathan Plucker of Indiana University recently reanalyzed Torrance’s data. The correlation to lifetime creative accomplishment was more than three times stronger for childhood creativity than childhood IQ.”

Obviously, the converse will prove true: the less creative our children, the fewer innovative and productive adults our nation will produce. But the situation is not dire. There is a clear antidote to this decline in the nation’s creativity. The intentional teaching of creativity will be the subject of the next blog post.

Like Homer’s ancient heroes, Lebron James is driven by a deep desire to defeat mortality and to live perpetually as the greatest and most admired basketball player ever. And he was likely traveling that path, until his certainty bias washed out the road.

Outside of Miami, fan and media response to his narcissistic self-annointing has been nearly universally scathing. Just a few examples:

• Fox Sports: LeBron’s new reality: He’s the villain now
• Red94 (fan blog): Lebron James leaves the Cleveland Cavaliers, cements self as villain for the ages
• Vanity Fair: LeBron’s “Decision”? To Avoid the Path of Greatness

How did James miscalculate so egregiously? He was a victim of certainty bias.

Robert Burton, author of On Being Certain, has exposed the hazards of “believing you are right even when you’re not.” His central premise: “Despite how certainty feels, it is neither a conscious choice nor even a thought process. Certainty and similar states of ‘knowing what we know’ are sensations that feel like thoughts, but arise out of involuntary brain mechanisms that function independently of reason.”

In an October 2008 article in Scientific American entitled The Certainty Bias: A Potentially Dangerous Mental Flaw, Burton discusses how to avoid its pitfalls: “I don’t believe that we can avoid certainty bias, but we can mitigate its effect by becoming aware of how our mind assesses itself…I’ve taken strong exception to the popular notion that we can rely upon hunches and gut feelings as though they reflect the accuracy of a thought.

“My hope is the converse; we need to recognize that the feelings of certainty and conviction are involuntary mental sensations, not logical conclusions. Intuitions, gut feelings and hunches are neither right nor wrong but tentative ideas that must then be submitted to empirical testing. If such testing isn’t possible (such as in deciding whether or not to pull out of Iraq), then we must accept that any absolute stance is merely a personal vision, not a statement of fact.”

No one reading this blog has been deified. But like Lebron James, we are all susceptible to certainty bias. Good thinking requires that we be aware of and avoid its dangers. As Burton concludes: “Only in the absence of certainty can we have open-mindedness, mental flexibility and willingness to contemplate alternative ideas.”

A crayfish has two defense mechanisms against predators: to freeze and hope the predator does not recognize it as a food source or to escape quickly by flipping its tail and swimming backwards. The Maryland study offered the crayfish “stark decisions — a choice between finding their next meal and becoming a meal for an apparent predator.” The crayfish could freeze and “remain close to a food source while at risk of being eaten by the predator” or it could flip its tail and escape the predator but put “critical distance” between it and its next meal.

How did the crayfish do when confronted with this clear cost-benefit dilemma?

The study concluded that crayfish are decisive decision-makers and adept at making trade-offs. “They carefully weighed the risk of attack against the expected reward” and took action “in a matter of milliseconds.” Their risk-benefit calculus was clear: when a predator was “moving too rapidly for escape,” the crayfish would not tail flip but freeze in order to remain close to the food source. On the other hand, when a threatening predator moved more slowly, the crayfish would tail flip to escape while sacrificing proximity to the food source.

Hey, if a crustacean can do it, we can do it! How do you calculate costs and benefits when making a decision? What is your decision calculus? Here’s a formula for you to consider:

Value = (Benefit – Cost)

When making decisions, you should always seek the highest value solution — that is, the solution that delivers the greatest benefit for its cost. Put simply, a good solution is worth more than it costs, and a bad solution costs more than it is worth. For example, let’s say you have three solutions to a challenge or opportunity:

- Solution A provides the greatest benefit at a cost of $X.
- Solution B provides 93% of the benefit but at 60% of the cost of Solution A.
- Solution C provides 60% of the benefit at 55% of the cost of Solution A.

What would you do? In this simple example, Solution B is the highest value option:

- Solution B provides 93% of the benefit but at 60% of the cost of Solution A.
- Solution B provides significantly greater benefit than Solution C at about the same cost.

Easy in concept; harder in execution. I know.

A future blog post will discuss the value formula above in greater detail. In the meantime, please see two current posts, Ben Franklin’s Trade-off Tool and The Matrix, for additional information on managing trade-offs and making cost-benefit decisions.

Science News recently published a great article on the use and abuse of statistics.

Odds Are, It’s Wrong, by Tom Siegfried, highlights some of the problems associated with testing hypotheses using statistical methods. These problems are well known within the statistics community, with “hundreds” of papers having been written on the subject. As Siegfried pithily observes, “if you believe what you read in the scientific literature, you shouldn’t believe what you read in the scientific literature”.

Statistical analysis is often presented in support of decisions. It’s rarely challenged. Yes, it’s sometimes ignored if it contradicts the views of key stakeholders. But it’s rarely challenged. This, we would suggest, is a consequence of:

• statistical analyses being considered to be objective; and
• decision-makers not being confident enough to dig around in the details.

Combine the reluctance to challenge statistical analyses with their widespread abuse and you clearly have cause for concern.

The article points out that, even among scientists, statistical literacy leaves much to be desired. For example:

• Statisical significance (e.g. p < 0.05) is often presented as a “black or white” binary concept. However, the use of 0.05 (…or 0.01…or 0.001) is completely arbitrary. In fact, with a p value of 0.05, there’s a 1 in 20 chance that the observed result is a fluke. Is that acceptable given the decision you have to make?
• Statistical significance at the 0.05 level is commonly equated to 95% certainty that the result could not have occurred by chance. This isn’t the case. You can’t draw conclusions about the likelihood of the hypothesis being correct based on its statistical significance. The correct interpretation, given a p value of 0.05, is that there is only a 5% chance of getting the observed result if no real effect is present.
• Studies also tend to equate statistical significance with practical significance. An example given in the article is that an expensive new drug may be statistically significantly better than an old one, but, if it only provides one new cure for every 1000 patients, that’s not of much practical use.

A final interesting observation made by Siegfried concerns random trials. Selecting groups at random provides no guarantee that they exhibit random traits with respect to the phenomenon of interest. Let’s say we walk out onto the street right now and select two groups of five professional strangers. How likely is it that both groups would express similar political views, for example? When discussing (as Siegfried does) drug trials, there are countless dimensions on which patients can vary.

Humorist Evan Esar was clearly onto something when he defined statistics as “the science of producing unreliable facts from reliable figures”…

In a Sunday, June 13, 2010 New York Daily News editorial, Washington, DC school chancellor Michelle Rhee summarized the list of creative contract provisions:

• Pay for performance: “Paid for with private money, and voluntary for teachers,” the pay for performance system “recognizes and rewards our most highly effective teachers for their individual accomplishments in raising student achievement.”
• Layoff provisions: “When a school undergoes a budget reduction and a layoff is necessary, that decision is made based on performance, not seniority.”
• Teacher placement: “A teacher cannot be placed at a school unless the teacher and the school principal agree.” Teachers “who cannot find a ‘mutual consent placement’ … are moved out of the system.”
• Elimination of tenure: Teachers rated as ‘ineffective’ are “immediately terminated from the system.” Those rated ‘minimally effective’ have their pay frozen and after two years are terminated.

According to Rhee, “in exchange for these reforms, teachers are receiving unprecedented levels of support, resources, professional development, voice in decision-making and pay – an increase of 20% over previous salary levels (with additional bonuses making it possible to make twice as much).”

Contract highlights are available from the Washington Teacher’s Union.

The New York Times reports that “heavy multitaskers” have more “trouble focusing and shutting out irrelevant information” than non-multitaskers. And they “experience more stress.”

Stanford researcher Eyal Ophir was studying whether or not our brains have evolved to make us “efficient and effective multitaskers.” Ophir divided study subjects into two groups: those who are heavy multitaskers and those who are not.

In one test, subjects were briefly shown an image of red rectangles and blue rectangles and asked whether any of the red rectangles had moved. In another test, subjects were briefly shown a number and a letter and then asked to say whether the number was odd or even, or whether the letter was a vowel or a consonant.

The findings? Multitaskers scored significantly more poorly on each test than non-multitaskers, because

• They had more trouble ignoring the “irrelevant information” (i.e., filtering out the blue from the red triangles).
• They took longer “switching between tasks” (i.e., focusing on the number or letter as instructed).

You can take the two tests yourself:

• Test how well you focus
• Test how fast you juggle tasks

It turns out that mutltitasking has other deleterious effects. The Times article also reports that a University of California, Irvine, study found that “people interrupted by e-mail reported significantly increased stress compared with those left to focus.”

The implications to your personal productivity are clear. Not only is multitasking inefficient, it is unhealthy. When addressing an important task that requires your full intellectual focus and engagement, eliminate distractions. Monotask. Shut down your e-mail client. Close your web browser. Turn your smart phone to silent. Log out of Facebook, Skype and G-Chat. Switch off the Twitter feed. Mute Pandora. Direct the full focus of your mental energies solely to the task at hand.

While writing this blog post, I followed all those prescriptions…honest…well…mostly!