by 
The Scheller Teacher Education Program, The Education Arcade, and the MIT Game Lab recently launched the first two courses in EdTechX, a series of massively open online courses (MOOCs) focused on educational technology. Both MITx courses, 11.132x Design and Development of Educational Technology and 11.126x Introduction to Game Design, were developed on the EdX platform. Of the over 40,000 combined registrants, approximately seven percent or 3,000 students earned certificates for their participation.
For those of us involved in designing these courses, the story goes far beyond earning certificates. We wanted to push the envelope of what was possible. Since these were new courses, we had a blank slate from which to work rather than transferring a residential version of an MIT course to the digital environment. We maintained a deep commitment to replicate the interactive, collaborative, and intimate environment of our typical small, project-based courses delivered on campus. But how exactly do you build and support that sort of learning and community in massive online courses?
Our solution included several specific instructional design decisions to support elements we considered critical to the learning process — student and community engagement, student-constructed knowledge, and collaboration. Of course, we also made many mistakes along the way, which allows us to apply further refinements to our next two courses in 2015.
Student Engagement
In face-to-face classes, it can be challenging to know whether you are engaging students. We often have to rely on subtleties like facial expressions and sitting postures to provide valuable feedback to an instructor. How do you determine whether you are succeeding in reaching students that you cannot see or interact with in real-time? We needed to create different opportunities for actively engaging students throughout the course, starting with the primary medium of the MOOC: streaming video. To lay the foundation for student interaction, we settled on three primary forms of video-based content — interview style, action or demonstration-based videos, and activity breaks.
Interviews borrowed from the visual language of celebrity talk shows. We took advantage of our access to many of the pioneering voices in our fields and worked through foundational concepts right on camera. Instead of telling students everything we knew about a topic, they could hear reasoning and rationale from the horse’s mouth. Edited into five- to ten-minute segments, these informal and informational clips subtly suggested to students that knowledge doesn’t need to come from one authoritative voice, and that there is room for different perspectives, approaches, and philosophies to work together. Demonstration videos took on more of a how-to vibe, not unlike children’s science television and YouTube cooking channels. The instructors would not only introduce assignments and goals, but also demonstrate how they would tackle a sample problem. In one clip, Sara Verrilli and Philip Tan designed a board game in under ten minutes in front of the camera, and concluded the segment by challenging the students to do the same. These think-aloud videos aimed to reduce the anxiety of getting started by showing how even instructors can make mistakes. The presenters continuously talk through their process and decision-making, setting student expectations by demonstrating what reasonable effort and success might look like.
Every video series was punctuated by an activity break. In Design and Development of Educational Technology, one activity break encouraged students to stop the video, try their hand at juggling, and reflect on what it felt like to take on the role of the learner. This gave students the room to explicitly contemplate the learning theory distinct from the technologies. Other activity breaks afforded students the opportunity to gain hands-on experience with and consider possible applications of the tools and technologies — exploring and manipulating motion graphs with SimCalc, experimenting and exploring using simulations with tools like PhET Science Sims, Molecular Workbench, and StarLogo Nova.
Community Building
Learning is an inherently social process — we learn by explaining to our peers, challenging other ideas, and being challenged by others. The physical classroom presents opportunities for students and teachers to interact in these ways, forging peer and student-teacher relationships. Yet, these types of experiences are not often baked into MOOCs; many students experience online instruction in isolation. Seeing the social aspects of learning as critical to the process, we sought to create better student interactions in forums and live events to create a more intimate experience for students, creating an environment to increase the quantity and quality of peer interactions.
In the early weeks of both courses, the lead faculty — Eric Klopfer and Philip Tan — conducted Ask Me Anything (AMA) sessions on Reddit. AMA participants were able to get a feel for the faculty and how the course might go while also meeting some of their classmates. Both sessions helped to spark peer interactions and model respectful communication among prospective participants.
To support a diverse student body, in terms of geography, technical expertise, bandwidth, etc., we needed to build a broad event and social media infrastructure. We wanted to support students within their technological comfort zone while introducing a broader range of possible technology tools that could enhance ongoing collaboration and contribute to a sense of community.
The Design and Development of Educational Technology team held a number of live events using various technology tools spaced evenly throughout the course. These events were delivered on different technologies including Talkabout, Google Hangouts on Air, and Unhangout. This allowed participants to evaluate the benefits and challenges associated with the tools and provided them with opportunities to interact with each other and with the course team.
The Introduction to Game Design team presented live video in the form of broadcast on Twitch.tv, mirroring the practices of the gaming community. While attendance at these events was promising, scheduling for the real-time events were defined by staff working hours in the US Eastern Time Zone, limiting live participation from the global student population and forcing them to rely on online archives instead.
We considered the potential value in supporting more organic community development. Student-created social media pages and hashtags are a mainstay of the MOOC experience, so we created official pages on Twitter, Pinterest, Facebook, and Google+, allowing participants to communicate with fellow students through media that they found more familiar. Introduction to Game Design participants created an internet relay chat channel to support each other on the use of GameBlox as a development tool. Staff did not monitor or moderate these vehicles heavily but occasionally participated in them to guide students back to the course forums, where the team could focused our efforts to build community and support student interaction and collaboration.
Student-Constructed Knowledge
Unlike many MOOCs, both Design and Development of Educational Technology and Introduction to Game Design were designed around the principles of project-based learning. We sought to create opportunities for students to provide evidence of their learning and explore alternative methods of grading over typical online quizzes. Each of the activities and assignments throughout the two courses were designed to provide students with incremental knowledge and skills required to complete the summative course project.
For the Design and Development of Educational Technology, that meant preparing students to create and deliver a Kickstarter-like pitch about a new educational technology. Introduction to Game Design students created a digital or tabletop game using iterative game design processes. The projects not only provided an effective example of the theories discussed in the classes. They also provided students with a final goal. Weekly assignments all led to the summative final project, allowing students to think through their projects each week to integrate new course information, peer feedback and questions.
The projects for these two courses may have been too large a scope for students to handle in a short time frame. This was especially true given the amount of theoretical and technical content students were trying to process at the same time. Students were trying to incorporate new theories and best practices while having to become fluent in new technologies. This tension created a level of frustration on the part of students who were invested in completing the course with an admirable portfolio. Despite these frustrations, there was an overwhelming desire on the part of students to continue to make progress on their final projects. Many did complete them and gained valuable experience from doing so.
Collaboration and Peer Review
The final component we rethought was collaboration and peer review. Both are critical instructional strategies of residential courses delivered by the MIT Game Lab and Scheller Teacher Education Program. It was important to design both the technical infrastructure and the curriculum to ensure a positive collaborative experience for students.
From a technical perspective, we spent significant energy identifying the most effective forum and discussion tools for supporting community interaction and conversation. We also had to find solutions that would integrate easily with the EdX platform. We created a customized student forum system based on open-source technologies for both courses. The forum features seemed to work well overall, providing opportunities for students to interact in groups, share course artifacts, develop a personal profile, and highlight interesting work. Many of the students who had taken other MOOCs commented on the enhanced interactions provided by our forums.
Still, there is much more work to be done. Some students struggled to make connections with others with similar interests, in nearby locales, or with similar background knowledge and experience. There were also typical technical hiccups with peer review — students inadvertently submitting blank documents or inaccessible web links — but we are most committed to improving the value of the feedback provided to students.
In a MOOC, there are so many students that it is clearly not sustainable for the faculty, teaching assistants and other members of the course team to personally review and provide feedback on all submitted work. It was a priority for the course teams to get peer review to work well. We tried many different approaches from the start, such as establishing guidelines and norms about providing effective feedback on the course pages and modeling what feedback might look like in the forums.
Despite this, many participants received either no feedback on their work or feedback of low value due the lack of comfort, experience, knowledge, or context on the part of the reviewer. In an ideal world, peer review allows students to help support each other as they acquire and refine new knowledge and skills. As the courses progressed, it became evident that there was a disparate range of familiarity and experience with providing peer review among the course participants.
To help students become more adept at peer review, we increased our modeling of providing constructive and supportive feedback beyond the forums. The Design and Development of Educational Technology course team hosted a Google Hangout on Air in which Eric Klopfer, Scot Osterweil and Jen Groff reflected on a few student projects, asking probing questions to help students think about their project, their goals, and possible refinements. The Introduction to Game Design did the same with weekly videos. We shared standard peer-review guidelines and discussed comments from the forums and live chat to demonstrate examples of effective feedback.
Students were encouraged to create working and affinity groups that might serve as their personal learning community. Several groups were successful with members stepping up to serve as dedicated facilitators, ensuring that the group came together and stay on task. We are currently studying the most successful groups to determine how to better foster and support the behaviors that made them work.
We anticipate that building-in greater support for creating and maintaining self-directed collaborative groups, combined with enhanced modeling and supports for effective peer review, will result in fewer bland “Good job!” and “Fun game!” comments and more insightful questions that help students consider their projects under a new lens. This sort of reflection allows a student to make explicit and deeper connections between the course content and their own work.
Design and Development of Educational Technology and Introduction to Game Design validated that project-based collaborative instruction can succeed in the virtual world. A subset of students were deeply engaged and became integral members of a strong learning community. We will continue to iterate and identify ways to scale that positive experience for a larger percentage of students and make these successes feel like core components of our courses. Coming up in the EdTechX Series will be 11.127x Design and Development of Games for Learning, which starts April 1, 2015, and 11.133x Implementation and Evaluation of Educational Technology, starting in July, 2015.
