Every cohort has one. Somebody emails in week four asking if it’s too late to join, or logs in for the first time after three weeks of silence because a building crisis finally passed. If you’ve facilitated anything with a start date, you know the feeling of reading that email and doing quick math about what they’ve missed and what it would take to catch them up.
There are two things we usually do about it, and both of them are reasonable. The first is to bring the person along individually, sending them the recordings, summarizing what the group decided, checking in more often than we check in with anyone else. That works, up to the point where it stops scaling. The second is to close the enrollment window, which protects the group from the constant churn of new arrivals and makes the facilitation load predictable. That also works, and it works by turning some number of people away.
What I keep noticing is that we rarely talk about either of these as design decisions rather than logistics. A closed enrollment window gets described as protecting the cohort experience, which is accurate as far as it goes, and it’s also a decision about who the course is for. The person whose schedule opens up in October doesn’t get in until the next offering, assuming there is a next offering. That’s an access outcome produced by a design choice, and it deserves to be discussed with the same seriousness we bring to whether the discussion prompts are any good.
The vocabulary we inherited
Part of what makes this hard to see is that the research literature on online participation gave us a vocabulary built almost entirely out of leaving. When Kizilcec, Piech, and Schneider analyzed engagement in three Coursera courses, they sorted learners into completing, auditing, disengaging, and sampling patterns based on video watching and assessment submission. Ferguson and Clow later replicated that work across four FutureLearn courses and found seven patterns instead, which they named samplers, strong starters, returners, mid-way dropouts, nearly there, late completers, and keen completers.
That second list is worth sitting with, because “returners” doesn’t mean what a person outside the field would assume. In Ferguson and Clow’s taxonomy, Returners are learners who completed assessments in the first two weeks and then left, Strong Starters left after the first week, and Mid-way Dropouts left around the middle. The entire vocabulary describes exit trajectories, which makes sense given what the researchers were studying, and it also means that the person who steps away for three weeks and comes back has no name in the standard framework. There’s no category for the arc where somebody leaves and returns, because the frameworks were built to explain attrition.
Vocabulary does work on us. When the categories available all describe departure, the design problem gets framed as preventing departure, and the interventions follow from there.
Does the standard intervention work?
The standard response to intermittent participation is some combination of reminder emails, progress nudges, and plan-making prompts. This is what most LMS platforms are built to automate, and it’s what most of us reach for first.
The evidence for that approach isn’t good. Kizilcec, Reich, and a large team tested a set of established behavioral science interventions over two and a half years with roughly a quarter million students across 247 courses at Harvard, MIT, and Stanford. Scaling those interventions across varied contexts reduced their average effectiveness by an order of magnitude compared to what earlier smaller studies had found. In their forecasting work, a personalized policy that assigned each student the intervention predicted to help them most produced an estimated completion rate of 13.38 percent against 12.81 percent for no intervention at all, which isn’t a statistically meaningful difference. Their conclusion was that supporting diverse students online requires more than a light touch.
What that says to me is not that reminders are evil but that they’re aimed at the wrong thing. A nudge assumes the barrier is attention or intention. Cross, writing about adult participation decades before anybody had an LMS, sorted barriers into situational, institutional, and dispositional categories, where situational barriers are the ordinary facts of a life, including health, money, and family. For educators in buildings, the situational category is doing most of the work. Nobody misses week three of a course because they forgot it existed. They miss it because a student was in crisis, or their own kid was, or the schedule got rewritten on them. Sending that person a notification about their incomplete module is answering a question they didn’t ask.
Two moments nobody designs
The more useful research on this isn’t in education at all. Mark, Gonzalez, and Harris studied how knowledge work actually unfolds and found it heavily fragmented, with 57 percent of working spheres interrupted. Roughly a quarter of interrupted tasks weren’t resumed the same day, and when they were resumed, it took over 25 minutes and usually involved passing through at least two other tasks along the way. (The frequently repeated “23 minutes and 15 seconds” figure that circulates in productivity writing traces back to a magazine interview rather than to a paper, which is a small reminder to check the sources on numbers that sound too precise.)
Altmann and Trafton’s memory-for-goals model gets at the mechanism. Their work predicts that resumption depends on the availability of cues, and importantly, on cues present at the moment a goal is suspended and not only at the moment it’s retrieved. Longer and more demanding interruptions produce longer resumption times, because the suspended goal decays. When somebody comes back to a task after three weeks away, they aren’t just missing content but the thread of what they were in the middle of and why it mattered.
That points at two moments in a course that almost nobody designs: 1) what a learner leaves behind when they stop, and 2) what greets them when they come back. In most Canvas courses I’ve seen, including ones I’ve built, the answer to both is a progress bar and a list of things marked incomplete. The system knows exactly where you stopped and tells you nothing about what you were doing.
Designing those moments isn’t exotic. It means modules that end with something a person can pick back up, orientation material that stays available and current instead of expiring after week one, and a landing view that reintroduces the through line of the course instead of only reporting a completion percentage. It means writing content that assumes the reader has been away, which is a different register than writing content that assumes they were here last Tuesday.
What is the cohort actually carrying?
There’s a finding that complicates all of this, and I think it’s the most interesting piece of research in this whole area. Russell, Kleiman, Carey, and Douglas built four versions of an online course for middle school algebra teachers. One had a mathematics instructor, an online facilitator, and asynchronous peer interaction with a cohort moving together. One was fully self-paced with none of those supports. Two sat in between. All four conditions produced significant gains in teachers’ mathematical understanding, pedagogical beliefs, and instructional practices, and the outcomes were comparable across conditions. The authors were careful to note that their participants were self-selected and that the finding may not generalize past that particular course.
I don’t read that as evidence that facilitation is worthless so much as a question we should be asking more often, which is what specifically the cohort structure is load-bearing for. If the answer is knowledge outcomes, this study suggests it may be carrying less than we assume. If the answer is relationship, belonging, and the experience of being in something with other people, then the cohort is doing something the self-paced version genuinely can’t do, and the shared timeline isn’t overhead.
This is where I want to be careful, because the cohort programs I’ve watched work best are the ones where community is the actual product. Family and community education programs run this way on purpose. People are supposed to be in a room with each other, building something together over time, and that only happens when they’re moving together. The outcomes are real and they’re visible in ways that a completion report never captures. I’ve watched that work and come away wanting to design more of it, not less.
But the same feature that produces those outcomes is what makes the enrollment window close. And a closed window means the population served is the population whose lives happened to be stable in September. That’s a real selection effect, and it tends to filter along exactly the lines you’d expect, which is to say that the people with the least schedule control are the people most likely to be filtered out. When we describe a cohort program as accessible because it’s free, or local, or offered in the evening, we’re describing one dimension of access while a different dimension is doing quiet work in the background.
Where this leaves me
I don’t think the answer is to abandon cohorts for self-paced modules. That trades away the thing that makes the good programs good, and it produces the isolated experience that most people already associate with compliance training.
There are partial moves worth trying: 1) multiple entry points within a term, so that arriving in week four means joining the second wave rather than joining nothing, 2) cohort periods that are shorter and repeat more often, which lowers the cost of missing one, 3) a layer of material that exists outside the cohort entirely, closer to what Gottfredson and Mosher describe as performance support for the apply, solve, and change moments, so that somebody who can’t commit to the schedule still gets something more than a waiting list, and 4) peer interaction designed around artifacts that persist instead of conversations that expire, so that arriving late means reading what the group built rather than missing the only chance to be present for it.
None of those fully resolve it. A person who joins in week four of a program built on collective trust has still missed the weeks where the trust got built, and no amount of asynchronous scaffolding manufactures that retroactively. What I am less willing to accept is the version where we treat that as settled, and where the enrollment deadline goes on the flyer without anybody in the room asking who it removes.
If you’ve designed for the learner who shows up late, or decided not to, I’d like to hear how it went. You can reach me at licht.education@gmail.com, and there are more tools, articles, and resources at bradylicht.com.
References
Altmann, E. M., & Trafton, J. G. (2002). Memory for goals: An activation-based model. Cognitive Science, 26(1), 39-83.
Cross, K. P. (1981). Adults as Learners: Increasing Participation and Facilitating Learning. Jossey-Bass.
Ferguson, R., & Clow, D. (2015). Examining engagement: Analysing learner subpopulations in massive open online courses (MOOCs). Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, 51-58.
Gottfredson, C., & Mosher, B. (2011). Innovative Performance Support: Strategies and Practices for Learning in the Workflow. McGraw-Hill.
Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. Proceedings of the Third International Conference on Learning Analytics and Knowledge, 170-179.
Kizilcec, R. F., Reich, J., Yeomans, M., Dann, C., Brunskill, E., Lopez, G., Turkay, S., Williams, J. J., & Tingley, D. (2020). Scaling up behavioral science interventions in online education. Proceedings of the National Academy of Sciences, 117(26), 14900-14905.
Mark, G., Gonzalez, V. M., & Harris, J. (2005). No task left behind? Examining the nature of fragmented work. Proceedings of CHI 2005, 321-330.
Russell, M., Kleiman, G., Carey, R., & Douglas, J. (2009). Comparing self-paced and cohort-based online courses for teachers. Journal of Research on Technology in Education, 41(4), 361-384.
