Back in July I wrote about the unfortunate experience San Jose State University had with online course provider Udacity, which builds massive open online courses (MOOCs). After experimenting with the online courses for six months (in a program called San Jose State Plus) the school decided to “pause” the program, because students taking the courses were failing too often.

But good news, the pass rates are up! But is that because the program improved, or just because different students took the courses? According to a press release about the program:

Udacity and SJSU released initial data from the summer semester of their SJSU Plus program. In short, pass rates were up and more in-line with those of SJSU on-campus classes as they were able to update the course content and pacing methodology, as well as provide additional student support, based on learnings from the spring data we collected.

Udacity reports that during the Spring semester experiment, pass rates were 24 to 51 percent. Pretty bad. But during the just-completed summer project, pass rates were much better, between 30 and 83 percent.

What changed? Well, Udacity explains that part of the improvement might have to do with learning from the past and changing course delivery:

We used data from the spring to understand what parts of our content worked, and what needed to be improved. We added hints for challenging exercises, and we added more course support staff to assist with online discussions and communications. We also changed the pacing methodology, informing students earlier and as part of their course experience when they were falling behind.

Well that certainly might be part of it, but the program also had different students. As Geoffrey A. Fowler writes at the Wall Street Journal:

Some of Udacity’s summer improvement in student performance they offered may be attributable to adjustments to the program offerings. But direct comparisons are difficult, because the student body demographics changed considerably between the semesters. The spring courses were focused on high-risk students who had previously struggled; the summer courses were open to students from around the world, many of whom may have been better prepared for college learning.

It shouldn’t really be much surprise that pass rates go up if students are simply better prepared when they start the courses.

Fowler reports that San Jose has now appeared to un-pause San Jose State Plus. San Jose State Provost Ellen Junn indicated that the college will “resume the online courses as soon as January, and may even consider adding more flexibility to the courses to allow students to complete classes on their own schedule.”

Daniel Luzer

Daniel Luzer is the news editor at Governing Magazine and former web editor of the Washington Monthly. Find him on Twitter: @Daniel_Luzer