Week 2: The Quality of Massive Open Online Courses by Stephen Downes

Stephen Downes,  (born April 6, 1959) is a designer and commentator in the fields of online learning and new media. Downes has explored and promoted the educational use of computer and online technologies since 1995.[1]

In 2008, Downes and George Siemens designed and taught an online, open course reported as a “landmark in the small but growing push toward ‘open teaching.’”.[2] Born in Montreal (QuebecCanada) Downes lived and worked across Canada before joining the National Research Council of Canada as a senior researcher in November 2001. Currently based in MonctonNew Brunswick, Downes is a researcher at the NRC’s Institute for Information Technology’s e-Learning Research Group.[3]

The following post is an extract of a larger piece which Stephen wrote for this blog project. Please download here the full text: Downes: Quality of MOOCS

The Quality of Massive Open Online Courses

In this short contribution I would like to address the question of assessing the quality of massive open online courses. The assessment of the quality of anything is fraught with difficulties, depending as it does on some commonly understood account of what would count as a good example of the thing, what factors constitute success, and how that success against that standard is to be measured.

With massive open online courses, it is doubly more difficult, because of the lack of a common definition of the MOOC itself, and because of the implication of external factors in the actual perception and performance of the MOOC. Moreover, it is to my mind far from clear that there is agreement regarding the purpose of a MOOC to begin with, and without such agreement discussions of quality are moot.

The primary criticism of what I will address in this chapter is that success is process-defined rather than outcomes-defined.[1] Without outcomes measurement we cannot measure success, we can’t focus our efforts toward that success, we can’t become more competitive and efficient, we can’t plan for change and improvement, and we can’t define what you want to accomplish as a result. All this is true, and yet there is no measure of outcome or success that can be derived from designer and user motivations, or even from the uses to which MOOCs are put. The only alternative is to identify what a successful MOOC ought to produce as output, without reference to existing (and frankly, very preliminary and very variable) usage.

These outcomes are a logical consequence of the design of the MOOC. The same is true of a hammer. This tool is defined as a hand-held third-class lever with a solid flat surface at the business end. Anything that satisfies these criteria will, as an outcome, have the capacity to drive a nail into a piece of wood (whether or not any hammer is ever used in this fashion). It has to be under a certain weight to be hand-held, above a certain mass, and of a certain length, to be a lever, and of certain material and design to have a hard flat surface.

When we are evaluating a tool, we evaluate it against its design specifications; mathematics and deduction tell us from there that it will produce its intended outcome. It is only when we evaluate the use of a tool that we evaluate against the actual outcome. So measuring drop-out rates, counting test scores, and adding up student satisfaction scores will not tell us whether a MOOC was successful, only whether this particular application of this particular MOOC was successful in this particular instance.

The design of a MOOC is, in the first instance, as described above: it is a massive open online course, and the design is successful to the extent it satisfies those four criteria, and unsuccessful to the extent that it doesn’t. That said, however, there are many ways to create a massive open online course, and within that domain, some may be more successful than others. So we need to look at why we designed and developed the MOOC the way we did – why we made it massive, open, online and a course, as described above. Why this model, say, and not a traditional online instructor-led class, or an open online community, or any of a dozen other combinations?

What I begin with is the observation that each person has a different objective or motivation for taking a course, and has different needs and objectives (it’s a lot like dating that way – we think that everyone wants the same thing, but we find in practice that everybody wants something slightly different). We looked at what we called ‘sifters’ and ‘filters’ to create learning recommendation systems, resulting in work I presented at MADLat based on collaborative filtering. “Collaborative filtering or recommender systems use a database about user preferences to predict additional topics or products a new user might like.”[2] There are different ways to approach this problem; I adopted what we called ‘resource profiles’ to characterize resources and make them accessible within a learning resources network.[3] Since the work of filtering and selecting could now be done by the metadata, I turned to the question of what would constitute a successful network, which I addressed in 2005.[4]

Partially influenced by earlier work I had done in networks (and especially the work of Francisco Varela) it was clear to me that the objective wasn’t to connect everything to everything, but to achieve an organization[5] in such a way as to support cognition. The work of Rumelhart and McClelland suggested ways this organization could be defined in terms of nodes and connections[6] and learning mechanisms to achieve what Churchland and others called “plasticity”.[7]  The structural properties I described in 2005 were drawn in large part from documents describing the design principles behind the internet. Finally, remarks by Charles Vest about the American university system led me to formulate what I now call the Semantic Principle, also in 2005[8] which crystalized as the ‘Groups and Networks’ presentation in New Zealand.[9]

At the risk of repeating myself, let me say here that the Semantic Principle consists of four major elements: autonomy, diversity, openness, and interactivity.

Before discussing each of these briefly, let me describe the outcome a network design embodying the semantic principle will achieve. Such a system is not static; it is dynamic. It is self-organizing, and creates these organizations in response to (and as a reflection of) environmental input. It can be thought of as a highly nuanced perceptual system. Over time, it acquires a state such that it can (if you will) recognize entities and events in the environment as relevantly similar[10] to those it experienced in the past, and respond accordingly. This knowledge is characterized as emergent knowledge[11], and is constituted by the organization of the network, rather than the content of any individual node in the network. A person working within such a network, on perceiving, being immersed in, or, again, recognizing, knowledge in the network, thereby acquires similar (but personal) knowledge in the self.

Or, to put the same point another way, a MOOC is a way of gathering people and having them interact, each from their own individual perspective or point of view, in such a way that the structure of the interactions produces new knowledge, that is, knowledge that was not present in any of the individual communications, but is produced as a result of the totality of the communications, in such a way that participants can through participation and immersion in this environment develop in their selves new (and typically unexpected) knowledge relevant to the domain. A MOOC is a vehicle for learning, yes, but it acts this way primarily by being a vehicle for discovery and experience (and not, say, content transmission).

Not every MOOC will produce this outcome, nor will this form of learning be experienced by every participant (particularly those who sample and leave early) but to judge from the commentary the experience of new and unexpected emergent knowledge is common and widespread [12] [13] [14] [15] among many others.

Let me now turn to the four success factors that I argue tend to produce this result. My purpose here is not to describe each in any detail – I have done that elsewhere – but rather to consider each as a success factor, that is, to consider how each design elements contributes to this result.

Autonomy – this is essentially the assertion that members of the network (in this case, participants employ their own goals and objectives, judgments and assessment of success in the process of interaction with others. This is reflected, for example, in Dave Cormier’s assertion that “you determine what counts as success in a MOOC.”[16] A collection of people working in a MOOC should be, for example, thought of as cooperating, rather than collaborating, because though they will exchange value and support each other, each will be pursuing his or her own objectives and depending on their own means and resources.

In our MOOC it was important that we not tell people what they ought to learn or what lessons they should take home from the presentations we made and the conversations we led. People perceive what they are looking for, and often only what they are looking for, and our well-intentioned attempts to guide their cognition could just as easily lead to participants missing the information most important to them. Similarly, we did not attempt to define how participants should interact with each other, but instead focused on supporting an environment that would be responsive to whatever means they chose for themselves.

Without autonomy, a MOOC is not able to adapt to the environment. Rather that enable each person to allow his or her unique perspective or point of view of the world to influence the course design or organization, they would instead reflect the perspective or world view of some organizer telling them what their objectives should be, what they should learn, what counts as success. It is important that each person respond to the phenomena – the communications of others – in their own way, positively or negatively, in order to generate a unique structure or organization.

Diversity – this is a natural consequence of autonomy, and in addition a success factor in its own right. While we typically think of diversity in terms of language, ethnicity or culture, for us diversity applied to a broad range of criteria, including location and time zone, technology of choice, pedagogy, learning style, and more. Participants, for example, could experience the course as a series of lectures, and some did, but many skipped the experience. Others treated the course as project-based, creating artifacts and tangible products. Others viewed the course as conversation and community, focused on interaction with other participants.

The major concern with diversity so broadly construed is that some people might be seen as ‘doing it wrong’. We were, for example, criticized for offering lectures, because it did not follow good constructivist pedagogy; our response was that connectivism is not constructivism, and that it was up to those who preferred to learn through constructivist methods to do so, but not appropriate that they would require that all other participants learn in the same way. Additionally, it should be noted that it did not matter whether some particular pedagogical choice was in some respects a failure, since the perceptual recognition that it is a failure constitutes success in its own right.

Without diversity, it is not possible to contemplate the possibility of a network having different states, or different types of organization. A collection of entities that is not diverse is inert, or worse, overly reactive, in that a change in one becomes a change in all. In a computer, we expect each bit of memory to contain different values of one or zero over time than others, for otherwise, our computer could do nothing more than blink off and on and off again. Any sort of complexity requires diversity, and any sort of learning requires complexity.

Openness – this is the idea that the boundaries of the network are porous and that the contents of the network are fluid. In practical terms, it means that participants of the course are free to enroll or to leave as they wish, and to move in and out of course activities equally freely (I once remarked [17] to ALT that what made my talk a success was defined not by the fact that they were all here, but by the fact that they could all leave (but hadn’t)). Openness also applies to the content of the course, and here the idea is that we want to encourage participants not only to share content they received from the course with each other (and outside the course), but also to bring into the course content they obtained from elsewhere.

Openness is necessary because – as the saying goes – you cannot see with your eyes closed. An a priori condition for the possibility of perception is openness to perceptual input. Learning requires perception, not only of the thing, but also of its opposite. If we were not open to the perception of evil, we would not be able to define good. If we are not open to the possibility of failure, we are not able to achieve success. We obtain these experiences through openness, by being open to other ideas, other cultures, other technologies, other people. The free flow of people and information through a MOOC is as important as the organization of the people therein.

An interesting side-effect of openness is that there is no clear line dividing those who are in the course and those who are not. The course resembles not a solid sphere but rather a cluster of more of less loosely associated participants (and resources, and ideas). In a connectivist course, for example, lurkers are seen as playing as equally important and valuable role as active participants. Off-topic discussions are not distractions but are rather seen as valuable outcomes. As members of the Bar Camp and unconference movement would say, the people who are there are the right people, and the outcome of the event was the right outcome.[18]

Interactivity – through the years I have used various terms for this fourth element, including ‘connectedness’ and ‘interactivity’ but none of them suits exactly what is meant by this concept. It is not simply that members of the network are connected with each other, and that interaction takes places through these connections. It is rather the idea that new learning occurs as a result of this connectedness and interactivity, it emerges from the network as a whole, rather than being transmitted or distributed by one or a few more powerful members.

Another way to understand this property is to see it as the stipulation that the graph of network interactions or connections is not a power law distribution. In a power law distribution, one or a few members receive most of the connections, creating what I’ve called the ‘big spike’[19], and the each of the majority has only a few connections, resulting in what many people have called ‘the long tail’[20]. This formation commonly occurs in dynamic networks, the result of what Barabasi[21] identified as selective attraction: newcomers to the network tend to link to those people who are already popular, resulting in their disproportional growth in popularity.

Networks characterized by a big spike and long tail are not response to their environment, and can over-react to small stimuli, resulting in cascade failure and eventual network death.[22] A more balanced (and dare I say, egalitarian) distribution of connectivity gives the network resilience, and the influence from one perspective cannot become disproportional simply because it came from an influential node. Each signal (each idea, each resource) must face not one challenge but many challenges as it is propagated, person to person, through the network.


[1] http://www.tdcorp.org/pubs/Outcomes_Measurement_Article.pdf

[2] http://www.downes.ca/presentation/90

[3] http://www.downes.ca/presentation/85

[4] http://www.downes.ca/presentation/32

[5] http://www.enolagaia.com/Tutorial1.html#Org&Str

[6] http://www.amazon.com/Parallel-Distributed-Processing-Vol-Foundations/dp/026268053X

[7] http://www.amazon.com/Scientific-Realism-Plasticity-Cambridge-Philosophy/dp/0521338271

[8] http://www.downes.ca/presentation/109

[9] http://www.flickr.com/photos/stephen_downes/252157734/

[10] http://www.downes.ca/post/212

[11] http://www.kakihara.org/papers/Kakihara&Sorensen_JGITM.pdf

[13] http://ivarsadventures.wordpress.com/2012/08/25/my-educational-dream-is-coming-true-with-mooc/

[14] http://edwoodworth.wordpress.com/category/mooc-journeys/

[15] http://digitalopened.blogspot.ca/2013/04/open-education-open-university-h817open.html

[16] http://www.youtube.com/watch?v=r8avYQ5ZqM0

[17] http://www.downes.ca/presentation/113

[18] http://www.openspaceworld.com/brief_history.htm

[19] http://www.downes.ca/presentation/37

[20] http://www.wired.com/wired/archive/12.10/tail.html

[21] http://www.amazon.com/Linked-Everything-Connected-Else-Means/dp/0452284392/ref=la_B001IGQIYW_1_1?ie=UTF8&qid=1366838681&sr=1-1

[22] http://www.downes.ca/post/53882

11 thoughts on “Week 2: The Quality of Massive Open Online Courses by Stephen Downes

  1. Dear all,

    I really appreciate that Steven declares OER as essential for the “openness” of MOOCs. In the current German MOOC fellowship competition with more than 250 proposals, only 4 have mentioned that they will use liberal CC licenses for their material, additionally I know that 3 more are aware of OER (but did not mentioned it within the proposal).

    As the competition is still running, I would like to ask if you could support our own contribution by voting for it :o)

    Is there a need for teachers and lecturers with solid knowledge about technology enhanced learning (TEL)? Are open courses for many (MOOCs) a good opportunity, especially if regular offers are rare? Are open educational resources (OER, licenced with CC BY-SA) a valuable contribution to society?

    If your answer is YES!!!, then please support our submission for a German MOOC fellowship. Just click [here], click on “Deutsch” (on the right top) and use your e-mail adress, it only takes some seconds. Vote German-speaking lecturers and teachers to TEL heroes! Thanks for your support and sharing!

    Best wishes,
    Sandra

    PS. Background of this proposal is our OER textbook about technology-enhanced learning (“L3T“), with 50 chapters, developed with 200 experts within 10 months, available as pdfs, apps, also as printed version ;) etc. Our ambitious MOOC proposal is currently at range 5 of 250 – but there are still some days left by 2013-05-23.

  2. Thanks to all involved at EFQUEL who came up with the idea to run the MOOC Quality Project; there is so much rhetoric around MOOCs these days and I think that this project is moving the rhetoric to the right direction.

    Week’s two article by Stephen Downes, who founded connectivism with George Siemens and ran one of the first MOOCs back in 2008, offers an opportunity to consider once more the theoretical underpinning of MOOCs as they have been envisaged by their creators and before venture capitalists became interested in their exploitation.

    Quality and the ways we measure it, is always challenging and although of critical importance, quality assurance may sometimes become a bureaucratic exercise by applying a Procrustean logic, which could be counter-intuitive and stifle innovation. As Downes acknowledges, ‘With massive open online courses, it is doubly more difficult, because of the lack of a common definition of the MOOC itself, and because of the implication of external factors in the actual perception and performance of the MOOC’.

    However, an area of difficulty which makes a number of educators uneasy with MOOCs, is the fact that, so far, online courses – regardless of their underpinning learning theory, be it behaviorism, cognitivism, constructivism, social constructionism or any combination of these – have been designed around explicit learning outcomes. Learning materials are selected based on those learning outcomes and well defined activities facilitate participants’ learning, while assessment is set against well defined criteria. In other words, the aim of most online courses so far has been to achieve Biggs’ ‘constructive alignment’. Connectivism and its resultant MOOCs, is challenging these principles by proposing that none of this is necessary, by allowing participants to choose what they want to learn, what activities they want to participate in and how they might be assessed. While this may have huge potential for informal learning from vast networks, when it comes to formal learning, this becomes inherently problematic in terms of assessing its quality. Standards are mostly about consistency; I would dare say that, it would be easier to standardize something which is consistently average, than something else that might be occasionally excellent but is highly inconsistent and unmeasurable.

    Downes claims that ‘The design of a MOOC is successful to the extent it satisfies those four criteria – massive, open, online and course – and unsuccessful to the extent that it doesn’t’.
    From a quality perspective, the most questionable of those criteria is whether it is a course or not. Some have argued that it is an experience, a happening or a networking event rather than a course. As an experience, MOOCs can allow ‘diversity, autonomy, openness and interactivity’; they may even occasionally achieve the creation of an ‘egalitarian network’ in which ‘a more balanced distribution of connectivity gives it resilience’, although this sounds rather aspirational to me. But it is those same inherent qualities of MOOCs that go against most course design principles, as they have been laid out over the last two centuries since the conception of the modern University.

    Looking forward to the conversation,
    Timos

  3. Dear Timos, thank you – I would very much like to emphasize what you say about the individual intention of learning in a MOOC context. It is a significant characteristic, that in a MOOC there can not be one and the same learning outcome assessment for all. MOOCs are rather characterized by the fact that, while there might be one intention of a learning outcome by the organizers, it is absolutely clear that the participants of MOOCs are selecting their own picks out of the MOOC. This becomes even more true since we know that in most MOOCs the intention is to provide a rich learning environment, which deviates from the normal course metaphor because the paced content presentation is not the sole meaning maker any more but all the rich social interaction and learner provided content, hints and links are creating an additional learning environment, especially in cMOOCs, but also in xMOOCs.
    So – my take out of that is that the ideal participant sof a MOOC is what we call the self-organised learner.
    MOOCs are learning environments for a special zype of learner: Those who have the ability to learn self-organised, kow what interests them and what theiy need to know, arecompetence learners, as they have available learning methodologies how to structure learning processes from chunks of materials, and are competent in assessing their own learning progress. These so-called self-directed, autonomous or self-forganised (Literature) learners are the ideal candidates for learning in MOOCs . Another aspect of virtue and skill which supports learning success in MOOC environment is social competence in so far as describes the ability of learners to interact in a meaningful way using technology to exchange experiences and oppinions, as well as questions and enter into conversations.

    So far my take on some of the points you made – let’s see how the debate evolves.

  4. Individualization AND formalization of learning goals

    Dear all,

    first of all thanks to the organizers for focussing on this higly interesting topic, and thanks to Stephen for his contribution. It insprired me to share some thoughts about the cMOOG/xMOOG-differenciation and a possible combination, based on some experiences won in our entrepreneurship programme (www.studiengang-unternehmertum.de – German only, sorry!).

    In my opinion an underestimated aspect of the MOOC discussion is the question of definition of the learning goals, or more precisely: the question of who has the power and the right to define them. An xMOOG structure represents the classical teacher/pupil or master/apprentice-relation: The one more educated and/or experienced and therefore fulfilling the role of a teacher sets the learning objectives and the means to controll them – questions, assessments, grades, finally academic titles. etc. As we all know this works fairly well as long as competent and trustworthy people are in charge of the teacher role, because it requires the trust that the teacher knows better what is good for the learner than he himself.

    In a cMOOG the goalsetting is mainly let to the participant. The freedom and motivation gained comes with a tradeoff in terms of structure, clarity and measureability. This is perfect for some, but hinders others, and it makes it much more difficult to organize coherently. The individualization of the learning process and thus the learning goals (or maybe better: the learning outcomes, since the goals may shift during the process) makes it virtually impossible to work with standardized formats and assessments.

    What to do? Just skip any form of given learning goals and formalized assessments and leave it to the learner? Or keep rigid structures and pay the price in terms of flexibility and motivation?

    A possible solution could be a formalized process in defining individual goals. For example the first task for every participant of a MOOC may be the identification of his or her own aims. This could be done by the selection or weighting of prestructured items or by free text, much as the introductary chapter of a paper. This approach will lead first to a reflection of the whole enterprise (and mabye for some to the conclusion that its better not to start at all than to drop out soon without much achieved) and second to an at least somehwat clearer understanding of the own personal goals, thus supporting committment, motivation and effort.

    This is an advantage by itself, but it could also be used to map the assessments against it. For example if multiple choice tests are applied, the selection of the questions could be algorithmed according to the goals set by the participant within a Likert-type system. In the case of a peer-reviewed paper the introduction including the learning objective would stem from the definition made right at the start of the MOOC – the learner has to proove his own claim.

    As we participate in the German MOOC fellowship competition (https://moocfellowship.org/submissions/zyklenbasierte-grundung-systematische-entwicklung-von-geschaftskonzepten) we plan to implement such types of individual goal setting, linked to assessments. Hopefully the Iversity system is going to support it…

    Best wishes to all and fruitful discussion
    Armin

  5. Often in thinking about the quality of a course we begin by asking what are the expected learning outcomes, and it would seem that one of the strengths of the cMOOCs approach is to question whether expected learning outcomes should be central to the design of a course, and hence to question whether they should be central to an assessment of quality, not only in MOOCs but in courses more generally. I do not think that this questioning means that we throw away learning outcomes, or our objectives, in running a course, just that we look at this aspect in a wider context when thinking about quality. Experience both as a learner and a teacher suggests that the learner’s objectives often do not coincide with the teacher’s, or with those of the host institution. I do not think that this is something that can be completely solved by simply asking learners what their objectives are (though that is a very sensible and helpful thing to do). Stephen Downes reports in his account of the first MOOC, that some participant’s objectives were actually disruptive ones (i.e. intention to discredit connectivism) where it might be counter-productive for them to declare their objectives! More generally participants may not be able to clearly articulate their objectives initially, and the end result of some courses may be to enable participants to articulate their objectives, rather than to achieve them.

    On a different issue, I think Ulf raises an interesting point about ‘self-organised learners’. A sensible quality criterion for any course is that the requirements for doing the course are clearly signaled, but perhaps it should do more than this, perhaps a course should provide means to enable potential participants to prepare themselves to participate. The word ‘open’ in education has had slightly different meanings over the years and I think it is being used in the phrase Massive Open Online Courses in rather the same sense it is used in the phrase Open Educational Resources. Back in the 1960s when the Open University in the UK was founded, the word open seemed to be being used in a slightly different sense, that is to mean reduction of barriers, so courses were open to those who did not have the necessary qualifications to start a degree in another university, it was open to students with disabilities, and it was open to potential students who might not have the necessary skills to be ‘self-organised learners’, but the university would support them in developing these skills so that they could take advantage of the courses. Now whilst xMOOCs perhaps clearly require ‘self-organised learners’, it might be that cMOOCs might contain opportunities for personal development as a learner that would enable them to demonstrate quality with respect to this aspect of openness.

    Best wishes

    Harvey

  6. Excellent piece. The references alone are golden. (And accessible without expensive journal subscriptions!) Here are just a few reactions to the 15 page version. I would be grateful for any comments — especially different points of view.

    1. On the definition of a MOOC early in the piece, I have the impression Mr. Downes is shouting against the wind. He’s describing what (his) early MOOCs were, and what he thinks they should be, and he suggests we be strict about this definition. But Coursera et al are mostly blowing in a different direction. Top-down, lecture-heavy, stand-and-deliver education driven by the interests (whatever they are) of elite and survival-concerned institutions alike. Sometimes time-bound, sometimes not. Call them xMOOCs, call them non-MOOCs, I’m not sure it matters. They’ve forked his idea and it doesn’t look like they’re turning back. There are plenty of exceptions of course. The digital learning MOOC from Edinburgh for example: https://www.coursera.org/course/edc
    But ignoring the proliferation of the less-connected versions of a MOOC won’t make them disappear or change. And maybe some (most?) learners don’t want them to change.

    2. There’s an assumption baked into the piece that learners care about learning as an end in itself, and that they’re willing to put in the energy to create instead of only consume. I don’t know whether that’s correct. It may be, on average, right now. Two thirds of participants in the Edinburgh MOOC mentioned above already have postgraduate degrees. They’re probably not looking for another degree. We’re trying to capture data on learner motivations in a qualitative way as part of a small research project on this topic, which we’d love people to engage with: http://www.futureofmoocs.org
    But my instinct is that, whatever the interests of current MOOC learners, if MOOCs – emphasis on the libre – are going to help fulfill the promise of democratizing learning – instead of exacerbating knowledge, education, and wealth gaps – then at least in the current environment they will have to help learners market themselves to employers. Ironically, this understanding of learner demand for employer-approved certifications has already led the xMOOC providers to offer non-open (non-gratis) components for testing and credentialing. EdX and Pearson, for example. And if there’s testing and credentialing, then presumably someone other than the learner is dictating what success means. Are there ways around this tradeoff? It seems under the current paradigm we can have true Downes-style MOOCs – open everything – that don’t meet the most critical needs of the learners with the most to gain, who view education as a way out of poverty and need a piece of paper – or a badge, or certificate, etc. – to demonstrate their competencies to potential employers, or we can have faux MOOCs that close off part of their offering to those who pay a reasonable fee for a credential. What will be the way out of this tradeoff? Can truly open MOOCs integrate gratis industry-recognized credentials into their offerings? Will employers move away from degree requirements? Will more learners go around employers altogether and start their own businesses equipped with knowledge and skills gained from MOOCs? Whatever the answers, MOOCs will only be relevant if they meet learners where they are and help them get where they want to go. Philosophical purity for its own sake is pointless in the real world.

    3. The success matrix (Massive, Open, Online, Course X 6 literacy dimensions) is interesting. But it’s complicated. It’s probably more useful for academics, less so for regular people. How about something a little simpler? One I’d propose is “to what extent are learners achieving their objectives?” This would be consistent with the spirit of the early MOOCs that Downes writes about (and with the commenter Armin above who emphasizes individual learning objectives). Whether a learner is participating in a MOOC to meet new people, practice and get better at blogging, prepare for passing a professional certification, impress the admissions staff at MIT and get an offer for admission, etc., they can express what their objective is and can later reflect on to what extent it was accomplished. And they can share these with the rest of us who can use that information to make sense of MOOCs. Maybe the (non-) MOOC platforms are already capturing this data, I’m not sure. This approach would make it meaningless to ask about the success of a particular MOOC in any general way. Success of each MOOC would be relative to the success of each learner, or group of similar learners. The specific features of the MOOC – learning materials, various technologies, etc. – would be adjusted not to fit some generic and rigid definition of what a MOOC is, but to maximize the likelihood of success of various groups of learners, as they define for themselves. For example, a perfectly open, connectivist-style MOOC that passed Downes’ 24-point success matrix would be a failure if all learners sought an industry-recognized credential and none of them received one as a result of participating in the MOOC.

    Thanks for the chance to comment. Would love any feedback here or on Twitter:
    https://twitter.com/philip_m_martin

    Phil

  7. I’m just beginning to look into open online education – but what I need is an ability to access the text work to teach me how to do it, possibly a lecture demonstrating how to apply the knowledge I learned in the text, a set of problems to solve to test my knowledge, and a discussion forum to ask questions in where I can get feedback. Have any of you been to the Khan Academy web site? Amazing! I also want a logical progression from the most basic of concepts up to the advanced. Along the way, some will move more slowly or quickly than others – individuals just move into or out of groups at the same level. I want to be able to do this at little to no cost.

    There needs to be a service that organizes all the information educators are creating to allow others to access it. Math has been taught for centuries – with the current technology it should be easy to organize it to where a person can say “I want to obtain all the knowledge someone with at least a bachelors in mathematics has,” and be able to get it on the internet. It’s been taught many times before! Organize it and make it available! It could be done with anything! What I see is an issue of fragmentation – many want to provide a MOOC, but it is not organized very well, and parts of what would be a full course are missing.

  8. Week 2 summary!

    Thank you!
    The first week of the MOOC Quality Project saw an impressive post by Stephen Downes. Thank you very much, Stephen, for your comprehensive take on the theme. Your suggestion to view quality as a performance of learning literacy with the formular to take the 4 elements of MOOCs „Massive, Open, Online and Course“ and mirror them in the light of the 6 literacy dimensions you outline is very profound. We believe it touches on much of the grounds which we will cover also in the weeks to come.

    Thank you also to the comments which we received, all of them very deeply reflecting the theme and very elaborated. We feel that underlying many oft he expressed thoughts is a discussion of pre-defined structured vs. emerging structure of learning, and the question of the role of the learner as the responsible part for co-producing learning quality in any learning scenario in interaction with his/her environment.

    We would like to welcome and thank Dave Cormier, one of the inventors of the actual term MOOC, who is leading us into the third week with a blog post on his thoughts on the issue of MOOC Quality.

  9. Pingback: Week 2: The Quality of Massive Open Online Cour...

  10. Pingback: bookmarks for June 26th, 2013 through June 29th, 2013 | Morgan's Log

Join the Discussion...