This Udacity-San Jose State online college partnership is really just the gift that keeps on giving, isn’t it? At the beginning of the year, with great publicity, San Jose State announced an agreement with online course provider Udacity, which builds massive open online courses (MOOCs), to build a special MOOC series just for the college.
There were, let’s just say, a lot of problems with the project. In particular, the completion rate in the online courses was pretty low and the college seemed to be able to correct that problem only by enrolling different, better prepared students.
It gets worse. According to an article at Inside Higher Ed:
San Jose State University on Wednesday quietly released the full research report on the for-credit online courses it offered this spring through the online education company Udacity. The report, marked by delays and procedural setbacks, suggests it may be difficult for the university to deliver online education in this format to the students who need it most.
The report, funded by the National Science Foundation, details the setbacks the research team encountered as it began to evaluate results from the spring pilot project. In particular, it took months to obtain usable data from Udacity that tracked how students used instructional resources and accessed support services. The research team then had to spend several weeks awaiting clarifications and corrections to resolve accuracy questions.
Particular problems involve the quality of the data itself.
The spring pilot produced just 213 students whose results could be used for statistical purposes — the remaining 61 received an incomplete grade, dropped a course or were removed after data were pruned for inconsistencies. Survey response rates ranged from 32 to 34 percent, and the research team found “significant differences” between those who responded and the general student population.
All of this does not necessarily undermine the idea of online education in general. A course can still be very good even if researchers lack good information to demonstrate that the course is good, but the fact that the college and the company have such problems collecting, interpreting, and using data to make decisions suggest that there are some pretty serious barriers to using research and evidence to make responsible decisions about technology in education.
