The authors compared traditional classroom instruction with web-based equivalent courses in a graduate program in library and information science. The study investigated differences in overall student performance, student attitudes towards their courses, and the learning approaches used. Findings indicate that there are no significant differences in student performances and that the students felt the web-based experience worked well in certain environments.
This study systematically compared the student learning outcomes and perceived effects of several for-credit courses offered via the World Wide Web with traditional on-campus sections of the same courses in the Master of Library and Information Science (MLIS) program at the University of Wisconsin-Milwaukee (UWM).
The purpose of the study is to gain a greater understanding of:
the appropriateness and effectiveness of web-based courses for graduate education
which technologies and approaches work best from students' and instructors' perspectives
how students can best be prepared to participate in web-based classes
how faculty can best be encouraged and aided in developing and implementing such courses
cost and time issues related to developing and delivering web-based courses
There is a
small but growing body of research at present comparing traditional and
web-based education. Schrum (1998, p. 53), in describing the state of
research surrounding web-based education, speaks to the need for comprehensive,
in-depth evaluations such as the present study: "The impact of
on-line courses has only begun to be investigated. To date, the
traditional distance education literature has focused on the design and
implementation of correspondence, compressed video, or satellite broadcast
delivery courses. That literature provides some parallels, but does not
directly inform the design and development of on-line courses."
Findings of the present study have implications for the development of web-based
courses in the graduate environment, as well as curricular redesign.
Six courses in the MLIS program at UWM were selected for inclusion in the study. Each course was offered with a traditional onsite section and a web-based section. Web CT was used to mount the web-based courses and permitted synchronous and asynchronous communication between the instructor and students. In all but one case, the courses were taught by the same instructor to control for differences between instructors in their presentation style, pedagogical approaches, and course content.
Students in each section of the course were surveyed at the beginning of the semester on their attitudes towards and perceptions of educational technology, specifically those technologies used in a web-based class. Students were surveyed again at the end of the semester on their educational experiences in the course.
A pilot course was selected in the summer of 1999 to help refine the survey instruments. Attitudes and grade outcomes for each course were compared between the two sections for each course, and more generally across the traditional and web-based courses. Including the pilot course, a total of 129 pairs of completed pre and post surveys by students in traditional sections and 53 in the web-based sections were received. A much lower response rate and generally smaller environments accounted for the lower number of responses in the web-based sections. Twelve students participated in in-depth follow-up telephone interviews regarding their experiences with web-based courses.
Similarly, instructors for each of these courses were interviewed on their attitudes on teaching in a web-based environment. Those instructors new to web-based courses were given in-depth, qualitative pre and post interviews.
In order to make sense of the qualitative data, coding procedures as described by Miles and Huberman (1994) and Newman (1991) were utilized, allowing the researchers to assign units of meaning to the descriptive or inferential information compiled during the study.
Key findings for quantitative and qualitative measures are briefly reported here. T-tests were conducted comparing average final course grades between traditional and web-based sections of the same courses. Overall, there were no significant differences in four of six courses (p>0.05). There were significant differences in two of the courses, in one case favoring the web-based course (p< 0.005), in the other, the traditional classroom environment (p<0.028). Students were asked in the post-questionnaire to rate the importance of different personal characteristics that students should bring with them to their respective instructional environments. T-tests reveal there were no significant differences in the importance of student flexibility, communication skills, and organization skills between the two environments. Students in the web-based sections, however, felt the attributes of self-discipline, self-motivation, technical expertise, and patience were more important in their environment than the students in the traditional classroom environment. Students were also asked to rate the importance of characteristics between the environments regarding the importance of instructor subject knowledge, flexibility, patience, communication skill, and ability to facilitate discussions. Students in the traditional environment felt a dynamic presence was more important in their environment, while students in the web-based environment felt instructor knowledge of information technology and organization skills were more important.
The in-depth qualitative interviews with students and instructors were conducted in order to complement the qualitative findings. As Buchanan (1999) has discussed, qualitative research in web-based education is novel, and yet holds great potential for understanding, and thus increasing, the efficacy of web-based education. The qualitative findings are reported on this web site and focus on student relationships, level of preparedness and effort devoted to web-based education, attitudes and perceptions of types of delivery, and differences among distance education and traditional students in terms of maturity, insight, abilities to communicate effectively, and real-world experiences.
The investigators are currently performing a finer-grained, in-depth analysis of the data, examining the influence of factors such as student demographics and the nature of the courses themselves, in addition to the specificity of the technology used and organizational approaches used by the instructors. Instead of individuals learning to teach online as they are engaged in their courses, this research will offer pedagogically sound suggestions for efficacious web-based instruction. The mixed-method evaluative approach holds great potential for elucidating the positive and negative attributes of traditional and online delivery of graduate level coursework.
For more detailed results of the study, consult the results section.
Acknowledgements
The authors acknowledge The University of Wisconsin System for funds through its Curricular Redesign Grant Program 1999-2000. We would also like to thank Jean Casanova, Jeannine Strunk and Colleen McFarland for research assistance.
References
Buchanan, E. (1999). Articulating a web-based pedagogy: A qualitative examination. Unpublished Doctoral Dissertation. University of Wisconsin - Milwaukee.
Miles, M.B. and Huberman, A.M. (1994). Qualitative data analysis: An expanded source book (Second Edition). Thousand Oaks: Sage.
Newman, W.L. (1991). Social Research Methods. Boston: Allyn & Bacon.
Schrum, L. (1998). On-line education: A study of emerging pedagogy. In Cahoon, B. Adult Learning and the Internet. (pp. 53-63). San Francisco: Jossey-Bass.
For a pdf file version of this overview, click on "Print" below