More (Bad News) on the Udacity Partnership at San Jose State

This Udacity-San Jose State online college partnership is really just the gift that keeps on giving, isn’t it? At the beginning of the year, with great publicity, San Jose State announced an agreement with online course provider Udacity, which builds massive open online courses (MOOCs), to build a special MOOC series just for the college.

There were, let’s just say, a lot of problems with the project. In particular, the completion rate in the online courses was pretty low and the college seemed to be able to correct that problem only by enrolling different, better prepared students.

It gets worse. According to an article at Inside Higher Ed:

San Jose State University on Wednesday quietly released the full research report on the for-credit online courses it offered this spring through the online education company Udacity. The report, marked by delays and procedural setbacks, suggests it may be difficult for the university to deliver online education in this format to the students who need it most.

The report, funded by the National Science Foundation, details the setbacks the research team encountered as it began to evaluate results from the spring pilot project. In particular, it took months to obtain usable data from Udacity that tracked how students used instructional resources and accessed support services. The research team then had to spend several weeks awaiting clarifications and corrections to resolve accuracy questions.

Particular problems involve the quality of the data itself.

The spring pilot produced just 213 students whose results could be used for statistical purposes — the remaining 61 received an incomplete grade, dropped a course or were removed after data were pruned for inconsistencies. Survey response rates ranged from 32 to 34 percent, and the research team found “significant differences” between those who responded and the general student population.

All of this does not necessarily undermine the idea of online education in general. A course can still be very good even if researchers lack good information to demonstrate that the course is good, but the fact that the college and the company have such problems collecting, interpreting, and using data to make decisions suggest that there are some pretty serious barriers to using research and evidence to make responsible decisions about technology in education.

Washington Monthly - Donate today and your gift will be doubled!

Support Nonprofit Journalism

If you enjoyed this article, consider making a donation to help us produce more like it. The Washington Monthly was founded in 1969 to tell the stories of how government really works—and how to make it work better. Fifty years later, the need for incisive analysis and new, progressive policy ideas is clearer than ever. As a nonprofit, we rely on support from readers like you.

Yes, I’ll make a donation

Daniel Luzer

Daniel Luzer is the news editor at Governing Magazine and former web editor of the Washington Monthly. Find him on Twitter: @Daniel_Luzer