(Two Sigma/Digital learning transformation series
Last time, when we looked at Daphne Koller's TED talk, we highlighted a pivotal 1984 result from famous education researcher Benjamin Bloom, the Two-Sigma "problem":
 |
| The two-sigma challenge: the effect of mastery approaches (cf. here & here also) and individualised, interactive, high-feedback, highly responsive and flexible learning processes |
Here, we see how students taught in 1:1 or very small group tutorials using methods that emphasise mastering content step by step, outperform those taught in conventional classes to the point where a "C-student" would get an A, and 98% of students taught that way would be at or above the level of performance of the top 20% of students taught conventionally.
That has significant implications for how we usually think about education approaches, resulting student performance and the "bright" vs. the "dunce."
(Including, the vicious spiral of frustration, inappropriately pigeonholing some students as "hopeless" for a given subject, and failure. Thence, wasted potential capacity relevant to -- or even crucial for -- development of our region. And, while it is common to point fingers at today's teachers who are always seen as not as good as those who went before [they were saying much the same 20 years ago, too, and they probably will say the same in twenty years time], it also points to the way we set up and operate education as an organisational system.)
Wiki, in its article on the two-sigma issue, has an apt, pointed summary:
Wiki, in its article on the two-sigma issue, has an apt, pointed summary:
Considering the significant outcomes of these studies on student performance, educational researchers can make a number of implications and conjectures for follow-up studies. Among them:
- Labeling students as low achievers is less relevant, since altering one or two variables can have significant positive effects on the average learner.
- Technology may simulate tutoring affects without the high cost of providing a live tutor for each student.
- Social aspects present in one-to-one tutoring may imply a larger role for consideration of sociality in (or the social nature of) learning.
A blogger at Incremental Thought has captured some of my own feelings:
Bloom wrote:
“The tutoring process demonstrates that most of the students do have the potential to reach this high level of learning. I believe an important task of research and instruction is to seek ways of accomplishing this under more practical and realistic conditions than the one-to-one tutoring, which is too costly for most societies to bear on a large scale. This is the ‘2 sigma’ problem.”I had three immediate reactions to Bloom’s statement. First, I am more determined than ever to be heavily involved in tutoring my children regardless of whether we go with a home school, private school or public school approach. Second, what does it say about our society that even the home cannot be a “safe bet” for one-to-one tutoring? Regardless of the reasons, I am troubled that an entire value system has been built that deems it “too costly” for us to bear. I wonder what the general attitude and level of education in our nation would be if families didn’t assume that education was something to be outsourced to an already overburdened and complicated public school system? Third, it struck me in the context of workplace mentoring. How often do I make myself available to be a mentor for junior software developers? How often am I seeking out other senior developers or managers to mentor me?
Another blogger, at isegoria.net, remarks:
My first reaction is surprise at the degree of the effect, but it should be obvious that advancing 30 students in lock-step means that many will be bored, a few will be in the sweet spot, and many will fall further and further behind, as the material builds on previous material they never learned.
So, my conclusion would be that conventional classroom teaching is largely a waste of time — but that’s not where educational experts place their emphasis:
Although much recent attention has focused on gaps in the achievement of different groups of students, the problem has been with us for decades. This paper presents the problem as one of reducing variation in students’ achievement, and reviews the work of renowned educator Benjamin Bloom on this problem. Bloom argued that to reduce variation in students’ achievement and to have all students learn well, we must increase variation in instructional approaches and learning time.I suppose they see it as Bloom’s Paradox.
In his original paper, Bloom notes that a full-size classroom can get one-sigma results by switching to mastery learning, where students are tested not just for a final grade on a unit but to uncover where they need to do further corrective work, so they keep at it until they get it right . . . .
It is odd, when you think about it, that we give students As, Bs, and Cs, and then advance them all to the next course, when they really should study the material until they earn a solid A before moving on — unless the goal of education isn’t conveying information but ranking students.
Now, you may wonder why I am highlighting the thoughts of bloggers who have happened on the results, instead of going to the professional literature.
I do so to highlight that a common-sense look at the data (which seems to be widely accepted by educators, though how they package it is of course in light of their own systems of thought and agendas) leads us to wonder what has been going on, and to think that maybe we need to change.
Yup: we need to change.
Which is always going to come up against all sorts of resistance and inertia that blocks or slows down change, sometimes for very good reason. Change is not a good in itself. (We need the right sort of change, and we need to be able to recognise it for what it is.)
But too often we stoutly, even angrily resist changes that are credibly beneficial for reasons that -- once we walk away from the power systems that prop up the conventional wisdom and actually listen to external stakeholders -- cannot stand the light of even basic common sense reflection.
This has the potential to be one of those cases.
So, we need to think very carefully indeed about what the research is telling us, and about what the digital technology revolution may be opening up for us.
(My agenda is of course the AACCS, but I am very interested in what is going on in secondary and primary level education as well. Not least, because the bridging studies, second chance secondary education component of the proposal, has to address finding and plugging the holes in the education of a good slice of the 80% or so of our region's young people who complete secondary level schooling -- as opposed to learning or education -- without an adequate level of certification or (much worse) actual useful and permanently acquired learning. I am also frankly concerned that our region may be drifting into a state where we are by default schooling our youth to become hewers of wood and drawers of water in a high tech digital age. [Note my suggestions here and here. Also, here, as a way to get out of thralldom to the major digital empires.])
As a step, I suggest that the astonishing potential impact of 1:1 tutoring is not so much in that itself, but in what it naturally promotes, a highly interactive, personalised, rapidly responsive, high-feedback and adjustment learning environment. (And BTW, I want to suggest that for certain strategic interventions, we need to have 1:1 or small group tutoring on the plate as key interventions. This also points out one of the reasons why cell groups and small work teams can be very effective, i.e. the high interactivity promotes effectiveness.)
So, let us now look at a table of useful interventions investigated by Bloom et al, as presented by Vockell of Perdue, redoing the table highlights to better reveal a pattern:
Bloom proposes the impressive but possibly attainable goal of trying to make group instruction as effective as individual tutoring. Bloom identifies several alterable variables that have been shown to provide a partial "solution" to this two-sigma problem.
Note that this emphasis on the two-sigma problem does not contradict the importance of academic learning time [--> ALT, aka "Time on Task"] discussed earlier in this chapter. Indeed, the main reason these strategies have such impact is because they promote the effective use of academic learning time.Table 2.7 lists modifiable factors that Bloom found to be related to enhanced learning. Note that on Bloom's scale a score of 2.0 indicates the level of improvement that would occur under individualized tutoring. This is Bloom's "ideal" score, and other pedagogical strategies can be regarded as effective to the extent that they approach this ideal. For example, when teachers assign homework, there is an average effect size of .30 compared to similar classes in which there is no homework assigned. When teachers not only assign but also grade homework, there is an effect size of .80 compared to similar classes in which homework is not assigned.
Table 2.7. Selected Alterable Variables That Influence Student Achievement Effect Size Percentile* Strategy 2.00 98 Tutorial instruction 1.20 86 Reinforcement 1.00 84 Corrective feedback 1.00 84 Cues and explanations 1.00 84 Student classroom participation 1.00 84 Student time-on-task 1.00 84 Improved reading / study skills .80 79 Cooperative learning .80 79 Homework (graded) .60 73 Classroom morale .60 73 Initial cognitive prerequisites .50 69 Home environment intervention .40 66 Peer and cross-age remedial tutoring 30 62 Homework (assigned) .30 62 Higher order questions .30 62 New science and math curricula .30 62 Teacher expectancy .20 58 Peer group influence .20 58 Advance organizers .25 60 Socioeconomic status (Included for contrast - SES is not easily alterable by teachers.) *The percentile indicates the percentile at which the "average" student would typically score if he/she received this treatment instead of "traditional" instruction. (Ordinarily, the "average" student would score at the 50th percentile.)
(Adapted from Walberg, 1984)
Vockell (1994) has reversed Bloom's logic and has discussed the "minus two-sigma problem." He reasons that if teachers teach really incompetently, they will lower students' performance by an effect size of 2.0. (Having no teacher at all could be perceived as the opposite of having an individual tutor.) He suggests that various activities in which teachers sometimes engage hurt student performance to the extent that they approximate a truly do-nothing teacher.
Plainly, nothing can replace the impact of the highly interactive small group or especially one on one tutoring.
[--> hint, hint on the importance of "cell groups" for discipleship and ministry training . . . similarly on organisation by "squads" led by competent and committed "NCO's" in getting capacity built through training and in getting effective work done . . . ]
However, a combination of interventions that promote interactive, active learning, that diagnoses and addresses gaps in required base, as well as responds to the diverse ways people learn, can make a big cumulative difference. We should not overlook the impact of time on task, graded homework (which motivates effort and provides hopefully rapid feedback) and enhanced reading and study skills.
This last may help explain a puzzling experience I had in teaching High School physics: one of the most effective learning interventions I made was to spend time in the classroom simply reading the textbook with students in rounds, paragraph by paragraph -- randomising who reads next -- and then pausing to discuss what we were reading and to work through examples interactively.
I now think that one factor at work was that students probably often find reading the textbook on their own a dreary, boring and often puzzling exercise that therefore naturally gets postponed again and again [until the "cram before the exam" rush which drastically shifts degree of motivation . . . ], reducing the book to a repository of homework problems. Which then became much more difficult challenges.
Interactive group work with people one likes probably dramatically enhances enjoyability, and it also sets up interaction, feedback, and mutual support and accountability. Video, animated computer simulations and multimedia would multiply that effect. However, I have more than enough experience of student project and study groups where one or two of the students seemed to be carrying a circle of free-loaders with them to think that small groups are a cure-all.
Can I plug for the homework notebook that can form a point of step by step accountability over work assigned and done, at home and school alike?
Similarly, can I point out the advantages of the three-ring binder file folder with well-divided sections as a way to organise work, notes etc? (If one is worried about students losing pages, if spiral notebooks are inserted you can get the best of both worlds, or one can punch three holes in manilla file jackets and affix folder sheets using treasury tags. That forms a sort of flexible notebook in a file jacket. Cf. here. Another trick is to use shoelaces or the like and cut and punched cornflake boxes to stack and tie together clusters of folder sheets or three-hole punched computer printouts into simple, mechanically bound "books." That gives a handy permanently bound reference book, on the cheap.)
I also plug for one to two page diagram-heavy handouts that students can work through and read. Indeed, I have come to a point where I routinely prepare a 1 - 2 pp summary handout for classes I taught, written up like a magazine article. Cf here for an example. One possibility here is to use blogs to present course summaries for reading, with use of discussion fora for student interaction. I am now seriously considering Disqus as a managed blog forum technology, in a world where there are trolls. However, discussion fora can be hard to manage. [Commenting in this blog has been suspended due to troll and hate site activity.])
The next major cluster is the process of chunking and sequencing the scope of learning so that students can build, test, debug and correct then successfully master and apply knowledge and skills in manageable steps in a context where they also have the big picture thanks to a well-structured introduction. The significance of cues and explanations on this should not be overlooked.
All of this is pointing to the significance of a group-oriented, individually responsive and adaptable, interactive, media-rich, easy to use digital learning platform as a powerful base for moving education towards the two sigma result envisioned by Bloom.
Hence, the potential of the 7" Android tablet in a folio with its own keyboard for perhaps US$ 120, as already highlighted in this blog. Let me show a typical picture yet again -- my Mom long ago stressed how it takes about ten exposures to break through the filters that block out messages in our environment:
But, again, it is not just a matter of getting machines into students' hands and expecting magic to happen: the above tablet is also a great gaming machine (or worse, a Youtube and porn viewing or Facebook etc machine . . . ) and students who zone out may be using this to waste their time. That is where the use of an interactive, tutor facilitated structured step by step learning process with accountability through needing to do and present effective work becomes key.
That's not all: we are here implying a considerable developmental effort to get together digitally based training materials that work in support of the sort of learning we need. That will call for educators, it will call for techies, it will call for research to ascertain and incrementally improve the needed features for maximum effectiveness, and more.
It will call for a fair degree of financial investment too.
But the alternative needs to be considered in light of its destructive consequences.