Stephen Downes, (born April 6, 1959) is a designer and commentator in the fields of online learning and new media. Downes has explored and promoted the educational use of computer and online technologies since 1995.
In 2008, Downes and George Siemens designed and taught an online, open course reported as a “landmark in the small but growing push toward ‘open teaching.'”. Born in Montreal (Quebec, Canada) Downes lived and worked across Canada before joining the National Research Council of Canada as a senior researcher in November 2001. Currently based in Moncton, New Brunswick, Downes is a researcher at the NRC’s Institute for Information Technology’s e-Learning Research Group.
The following post is an extract of a larger piece which Stephen wrote for this blog project. Please download here the full text: Downes: Quality of MOOCS
The Quality of Massive Open Online Courses
In this short contribution I would like to address the question of assessing the quality of massive open online courses. The assessment of the quality of anything is fraught with difficulties, depending as it does on some commonly understood account of what would count as a good example of the thing, what factors constitute success, and how that success against that standard is to be measured.
With massive open online courses, it is doubly more difficult, because of the lack of a common definition of the MOOC itself, and because of the implication of external factors in the actual perception and performance of the MOOC. Moreover, it is to my mind far from clear that there is agreement regarding the purpose of a MOOC to begin with, and without such agreement discussions of quality are moot.
The primary criticism of what I will address in this chapter is that success is process-defined rather than outcomes-defined. Without outcomes measurement we cannot measure success, we can’t focus our efforts toward that success, we can’t become more competitive and efficient, we can’t plan for change and improvement, and we can’t define what you want to accomplish as a result. All this is true, and yet there is no measure of outcome or success that can be derived from designer and user motivations, or even from the uses to which MOOCs are put. The only alternative is to identify what a successful MOOC ought to produce as output, without reference to existing (and frankly, very preliminary and very variable) usage.
These outcomes are a logical consequence of the design of the MOOC. The same is true of a hammer. This tool is defined as a hand-held third-class lever with a solid flat surface at the business end. Anything that satisfies these criteria will, as an outcome, have the capacity to drive a nail into a piece of wood (whether or not any hammer is ever used in this fashion). It has to be under a certain weight to be hand-held, above a certain mass, and of a certain length, to be a lever, and of certain material and design to have a hard flat surface.
When we are evaluating a tool, we evaluate it against its design specifications; mathematics and deduction tell us from there that it will produce its intended outcome. It is only when we evaluate the use of a tool that we evaluate against the actual outcome. So measuring drop-out rates, counting test scores, and adding up student satisfaction scores will not tell us whether a MOOC was successful, only whether this particular application of this particular MOOC was successful in this particular instance.
The design of a MOOC is, in the first instance, as described above: it is a massive open online course, and the design is successful to the extent it satisfies those four criteria, and unsuccessful to the extent that it doesn’t. That said, however, there are many ways to create a massive open online course, and within that domain, some may be more successful than others. So we need to look at why we designed and developed the MOOC the way we did – why we made it massive, open, online and a course, as described above. Why this model, say, and not a traditional online instructor-led class, or an open online community, or any of a dozen other combinations?
What I begin with is the observation that each person has a different objective or motivation for taking a course, and has different needs and objectives (it’s a lot like dating that way – we think that everyone wants the same thing, but we find in practice that everybody wants something slightly different). We looked at what we called ‘sifters’ and ‘filters’ to create learning recommendation systems, resulting in work I presented at MADLat based on collaborative filtering. “Collaborative filtering or recommender systems use a database about user preferences to predict additional topics or products a new user might like.” There are different ways to approach this problem; I adopted what we called ‘resource profiles’ to characterize resources and make them accessible within a learning resources network. Since the work of filtering and selecting could now be done by the metadata, I turned to the question of what would constitute a successful network, which I addressed in 2005.
Partially influenced by earlier work I had done in networks (and especially the work of Francisco Varela) it was clear to me that the objective wasn’t to connect everything to everything, but to achieve an organization in such a way as to support cognition. The work of Rumelhart and McClelland suggested ways this organization could be defined in terms of nodes and connections and learning mechanisms to achieve what Churchland and others called “plasticity”. The structural properties I described in 2005 were drawn in large part from documents describing the design principles behind the internet. Finally, remarks by Charles Vest about the American university system led me to formulate what I now call the Semantic Principle, also in 2005 which crystalized as the ‘Groups and Networks’ presentation in New Zealand.
At the risk of repeating myself, let me say here that the Semantic Principle consists of four major elements: autonomy, diversity, openness, and interactivity.
Before discussing each of these briefly, let me describe the outcome a network design embodying the semantic principle will achieve. Such a system is not static; it is dynamic. It is self-organizing, and creates these organizations in response to (and as a reflection of) environmental input. It can be thought of as a highly nuanced perceptual system. Over time, it acquires a state such that it can (if you will) recognize entities and events in the environment as relevantly similar to those it experienced in the past, and respond accordingly. This knowledge is characterized as emergent knowledge, and is constituted by the organization of the network, rather than the content of any individual node in the network. A person working within such a network, on perceiving, being immersed in, or, again, recognizing, knowledge in the network, thereby acquires similar (but personal) knowledge in the self.
Or, to put the same point another way, a MOOC is a way of gathering people and having them interact, each from their own individual perspective or point of view, in such a way that the structure of the interactions produces new knowledge, that is, knowledge that was not present in any of the individual communications, but is produced as a result of the totality of the communications, in such a way that participants can through participation and immersion in this environment develop in their selves new (and typically unexpected) knowledge relevant to the domain. A MOOC is a vehicle for learning, yes, but it acts this way primarily by being a vehicle for discovery and experience (and not, say, content transmission).
Not every MOOC will produce this outcome, nor will this form of learning be experienced by every participant (particularly those who sample and leave early) but to judge from the commentary the experience of new and unexpected emergent knowledge is common and widespread     among many others.
Let me now turn to the four success factors that I argue tend to produce this result. My purpose here is not to describe each in any detail – I have done that elsewhere – but rather to consider each as a success factor, that is, to consider how each design elements contributes to this result.
Autonomy – this is essentially the assertion that members of the network (in this case, participants employ their own goals and objectives, judgments and assessment of success in the process of interaction with others. This is reflected, for example, in Dave Cormier’s assertion that “you determine what counts as success in a MOOC.” A collection of people working in a MOOC should be, for example, thought of as cooperating, rather than collaborating, because though they will exchange value and support each other, each will be pursuing his or her own objectives and depending on their own means and resources.
In our MOOC it was important that we not tell people what they ought to learn or what lessons they should take home from the presentations we made and the conversations we led. People perceive what they are looking for, and often only what they are looking for, and our well-intentioned attempts to guide their cognition could just as easily lead to participants missing the information most important to them. Similarly, we did not attempt to define how participants should interact with each other, but instead focused on supporting an environment that would be responsive to whatever means they chose for themselves.
Without autonomy, a MOOC is not able to adapt to the environment. Rather that enable each person to allow his or her unique perspective or point of view of the world to influence the course design or organization, they would instead reflect the perspective or world view of some organizer telling them what their objectives should be, what they should learn, what counts as success. It is important that each person respond to the phenomena – the communications of others – in their own way, positively or negatively, in order to generate a unique structure or organization.
Diversity – this is a natural consequence of autonomy, and in addition a success factor in its own right. While we typically think of diversity in terms of language, ethnicity or culture, for us diversity applied to a broad range of criteria, including location and time zone, technology of choice, pedagogy, learning style, and more. Participants, for example, could experience the course as a series of lectures, and some did, but many skipped the experience. Others treated the course as project-based, creating artifacts and tangible products. Others viewed the course as conversation and community, focused on interaction with other participants.
The major concern with diversity so broadly construed is that some people might be seen as ‘doing it wrong’. We were, for example, criticized for offering lectures, because it did not follow good constructivist pedagogy; our response was that connectivism is not constructivism, and that it was up to those who preferred to learn through constructivist methods to do so, but not appropriate that they would require that all other participants learn in the same way. Additionally, it should be noted that it did not matter whether some particular pedagogical choice was in some respects a failure, since the perceptual recognition that it is a failure constitutes success in its own right.
Without diversity, it is not possible to contemplate the possibility of a network having different states, or different types of organization. A collection of entities that is not diverse is inert, or worse, overly reactive, in that a change in one becomes a change in all. In a computer, we expect each bit of memory to contain different values of one or zero over time than others, for otherwise, our computer could do nothing more than blink off and on and off again. Any sort of complexity requires diversity, and any sort of learning requires complexity.
Openness – this is the idea that the boundaries of the network are porous and that the contents of the network are fluid. In practical terms, it means that participants of the course are free to enroll or to leave as they wish, and to move in and out of course activities equally freely (I once remarked  to ALT that what made my talk a success was defined not by the fact that they were all here, but by the fact that they could all leave (but hadn’t)). Openness also applies to the content of the course, and here the idea is that we want to encourage participants not only to share content they received from the course with each other (and outside the course), but also to bring into the course content they obtained from elsewhere.
Openness is necessary because – as the saying goes – you cannot see with your eyes closed. An a priori condition for the possibility of perception is openness to perceptual input. Learning requires perception, not only of the thing, but also of its opposite. If we were not open to the perception of evil, we would not be able to define good. If we are not open to the possibility of failure, we are not able to achieve success. We obtain these experiences through openness, by being open to other ideas, other cultures, other technologies, other people. The free flow of people and information through a MOOC is as important as the organization of the people therein.
An interesting side-effect of openness is that there is no clear line dividing those who are in the course and those who are not. The course resembles not a solid sphere but rather a cluster of more of less loosely associated participants (and resources, and ideas). In a connectivist course, for example, lurkers are seen as playing as equally important and valuable role as active participants. Off-topic discussions are not distractions but are rather seen as valuable outcomes. As members of the Bar Camp and unconference movement would say, the people who are there are the right people, and the outcome of the event was the right outcome.
Interactivity – through the years I have used various terms for this fourth element, including ‘connectedness’ and ‘interactivity’ but none of them suits exactly what is meant by this concept. It is not simply that members of the network are connected with each other, and that interaction takes places through these connections. It is rather the idea that new learning occurs as a result of this connectedness and interactivity, it emerges from the network as a whole, rather than being transmitted or distributed by one or a few more powerful members.
Another way to understand this property is to see it as the stipulation that the graph of network interactions or connections is not a power law distribution. In a power law distribution, one or a few members receive most of the connections, creating what I’ve called the ‘big spike’, and the each of the majority has only a few connections, resulting in what many people have called ‘the long tail’. This formation commonly occurs in dynamic networks, the result of what Barabasi identified as selective attraction: newcomers to the network tend to link to those people who are already popular, resulting in their disproportional growth in popularity.
Networks characterized by a big spike and long tail are not response to their environment, and can over-react to small stimuli, resulting in cascade failure and eventual network death. A more balanced (and dare I say, egalitarian) distribution of connectivity gives the network resilience, and the influence from one perspective cannot become disproportional simply because it came from an influential node. Each signal (each idea, each resource) must face not one challenge but many challenges as it is propagated, person to person, through the network.