Professor and Acquisitions Librarian
Winona State University
AbstractThis paper describes the revision of an introductory information retrieval class by integrating more technology. All materials were provided on a single diskette in HTML format to try to improve learning while permitting an increase in class size. Creating the course materials was labor intensive. Student reaction was mixed. Many small technology problems arose. When everything worked, it worked very well though. We recommend trying this technology in a course with a greater credit size or in a course series.
I began development of Winona State University's (WSU) Library Science 100 in Fall 1995. The objectives of this one quarter hour class were to teach basic searching skills and strategies and also information evaluation skills. The purpose was to help students become effective users of cyberspace. We set the enrollment cap at 25, inasmuch as the course involved teaching critical thinking skills using a computer laboratory of 12 student computers. Initial enrollment was 21 students. We have offered the course at least once each quarter since Fall 1995, and the University's General Education Committee proposed that it become a basic skills course required of all students.
This would have entailed teaching at least 48 sections of the course each year, unless the course could be successfully restructured. That many sections is beyond the current ability of the library department, because of lack of staff. Due to switching into a semester-mode, WSU found it necessary to postpone the implementation of general education revisions for an additional 2 years, to give the library department time to revise the course. The semester mode will allow a 50% increase of course time, most of which will concentrate on depth of content, not breadth. The creator of the course attempted an innovation this fall that could expand the enrollment cap and hence reduce the number of sections required. Of course it would also be important not to erode the content or effectiveness of the course. Restructuring resulted in a paperless course. All course materials appeared on diskette, written in HTML. This also required revising the order of materials taught in the course.
The Former Course
Instructional methods used in the earlier version of this course (and in the version taught concurrently by another instructor) were traditional. The instructor delivered a lecture, using PowerPoint outlines and illustrations during the lecture to bring out key facts. The instructor encouraged questions and answers. The instructor also modeled searching behavior on his computer and displayed his screen on two wall monitors for the class to observe. There was a laboratory period during which students did lab projects for learning and practice. If time was available after the projects, students could do homework. The instructor was available in the laboratory during this time.
The course had formerly started with the history of cyberspace. There was discussion of the history of On-line Public Access Libraries (OPACs), and heavy use of them to teach searching skills and strategies. This is due to their relatively "clean" records, highly organized structure, relatively small size, and use of controlled vocabularies. The next presentation included early network tools: FTP, Telnet, Archie, Gopher, and Veronica. We also examined Web browsers, paying special attention to the evaluation of results and critical review of individual Web sites. All students wrote a short paper comparing two search engines, two Web directories, or Veronica and one search engine.
The course emphasized developing search strategies based on information needs--immediate and continuing. Most students were highly aware of immediate information needs, the type that demands specific information used within a limited time period. Writing a term paper is an example of need for specific, immediate information. Continuing needs demand more unspecified information over a long period of time--for example, in staying current within a professional field. Therefore the course also covered listservs and Usenet Newsgroups.
The class ended with the presentation of student group projects. These projects were critically annotated cyber-bibliographies on student-chosen topics. During the course, the students had several pieces of homework, two quizzes, a short paper, a group project and a final examination.
The Restructured Course
Student evaluations agreed that the course workload was too heavy and that there was need for more laboratory time. The instructor felt that these opinions were accurate. The purpose of changing the course structure was to address these concerns and to permit more students to enroll. The instructor would reduce the amount of lecturing and emphasize modeling the behavior. Homework assignments would require more critical thinking and less practice work. Practice work would be conducted through an interactive workbook. The interactive workbook, a key element for the course, would provide students a chance to practice skills and understand concepts. Because it would be interactive, if a student had a high failure rate, the instructor would receive contact via e-mail and could establish an individual teaching session, gaining time to provide this individual instruction from that saved in lecturing and grading homework. Students able to grasp the concepts from the lectures, readings, and workbook would not need to have individual sessions with the instructor.
The hope is that this refocusing of the instructor's time will raise the class' mean of learning. Teaching to mastery is the desired goal, recognizing that course restructuring is only a step to that goal.
The Restructuring Method: Software
Restructuring could have included several authoring tools: Coursebuilder, Asymetric Toolbox, MultiMedia Authorware or HTML. Price, availability, ease of interfacing with cyberspace, and the fact that Netscape Navigator was already being taught in the course determined the choice of HTML. Use of any authoring tool would have required that the students be taught to use that tool first. This would have meant teaching at least part of the Web browsing section before OPACs.
Changes in course outline. Shifting the course's outline enabled us, after the introduction to cyberspace and cyberspace history, to describe Web space in detail. We introduced the Netscape Navigator and covered basic functions of the program, although postponing some other functions. For example, we taught use of the "GO" button, but postponed teaching Usenet usage. We emphasized interactivity between various parts of cyberspace including the hard drives and disk drives of connected computers.
We introduced searching by directory or browser, but delayed full discussion. After the instructor felt that the students could successfully maneuver through the diskette, the instructor returned the course to its former outline, back into OPACs, believing that OPACs provide the most advantageous area for developing initial search skills and strategy. In essence, we plucked general concepts of Web space and the basic use of Netscape Navigator from their place in the old course outline and inserted them between the history of cyberspace and OPACs.
New elements added to course. Although the major addition to the course is the interactive workbook, it was obvious that more could be done. Because the students had a diskette with the workbook in HTML, we added more to the disk without requiring additional resources or training for the student.
Additions to the diskette or linking through cyberspace displayed all readings for the course, plus a glossary. HTML forms contained all homework and a written survey that students could submit to the instructor via e-mail by simply clicking on the submit button at the bottom of the form. HTML forms also contained quizzes and the final examination. Each quiz appeared on the classroom's computers' hard drives immediately before the class was to take it and was removed afterwards. The final exam appeared on a published Web page for the students to find, along with examples of prior work by students and the instructor's grading comments.
Layout of students' diskettes. Click here to view a shortened version of the diskette. This homepage shows a large cell on the left, a column of smaller cells on the right and a single cell across the bottom.
The large cell shortcutted to several search engines to facilitate the students' use of search engines. Each engine had a textbox. All the user had to do was enter the desired search strategy and click on the button with the engine's name--no need to go to the search engine's homepage first.
The column of smaller cells to the right were labeled and led to Web pages containing the syllabus, each class day's lecture outline (including readings, homework, examples of earlier graded homework, and links to the glossary and workbook), WSU's homepage, and a mailto box to the instructor. The bottom cell was a link to graded examples of earlier papers and group projects.
Most of the readings were cyberdocuments. Use of simple links provided the readings in two different ways. The students could click on the link and visit the cyberdocument. Of course, that document did not have a return link, so students had to learn to use the back-button or the Go-drop menu features of Netscape.
Some of the readings were on the diskette itself. All these readings were available in full-text databases that would have required at least some additional work by the students to access. In these cases either the article was e-mailed to the instructor's account and then downloaded, or the article was directly downloaded, whichever was easier. The downloaded articles became HTML documents and appeared on the diskette. This seemed permissible, and fair use. There was only one copy per student, distributed in a classroom setting, and collected at the end of the course. Since these articles were of a fugitive nature, there will be new ones for the next offering. These articles were equipped with links back to the day's outline and to the LS100 homepage.
Homework and the workbook. One of the homework assignments and most of the workbook focused on OPAC usage. They created two frames on the student's desktop. The upper frame was a telnet link to the library's catalog, Minnesota State University System's Project on Automated Library Systems (PALS). PALS created a Web page in September 1996 called WebPals. WebPals is actually a Web page that translates commands into the traditional command line forms, sends those commands to PALS, receives the response from PALS, and presents it in a WWW form. It may no longer be necessary for the LS100 Web pages to create the telnet session; instead the upper frame could link directly to WebPals. However, this will result in slightly to moderately slower response times (depending upon the Internet at the time of use).
The lower frame consisted of questions. In the workbook, the questions were multiple choice, each a link to another Web page. These Web pages told the student if the answer was correct or incorrect and if incorrect, why. The Web pages received random numbers as names so that the students could not guess which answers were correct based solely upon the URL. The responses to the answers also contained another question. Students who accumulated enough wrong answers in a particular section of the workbook received a form to fill out, to arrange an individual instructional session with the instructor on that topic. Students who accumulated enough right answers in a particular section received a form to notify the instructor of that success and could then move on to the next concept in the workbook. The instructor could therefore follow students' success or failure and get a sense of the overall progress of the class, to more effectively budget time.
Using HTML as an authoring tool. This required careful attention to proper organization of the links, especially to create the monitoring effect to direct students to the instructor when they encountered difficulties. Visualizing the path followed by correctly answering all questions as a straight line from the top of a lined page to the bottom, the path followed by an incorrect answer is a kind of branch extending to the side (right or left). The first line would be the question. The second line would be the correct answer(s) and incorrect answer(s). The third line would be a question. A correct answer following a correct answer would continue in a straight line down. An incorrect answer would deviate. If a correct answer followed an incorrect answer, the line would start to converge on the straight line again. This would continue through several iterations, usually five, so that the student could demonstrate mastery. After successfully reaching the end of the questions, students received a mailto form to notify the instructor of their success and could start another concept. Figure 1 below shows three iterations.
Having two frames allowed the students to test their answers in PALS before selecting an answer, to compare the results, and then to select the appropriate answer. The pre-written responses to the students could also direct them in using PALS more effectively. The homework arrangement was somewhat different from the workbook. Only some of the questions were multiple choice. These questions were later used in class to illustrate the range of approaches to problem solving and to provide examples for logical and linguistic analysis to maximize search strategies. The rest of the homework questions were short answer using a textbox. The intent of the questions was to make the students consider either their personal approaches to problem solving or the impact certain organizational principles could have upon problem solving behavior. Students' answers were automatically e-mailed to the instructor by a click on the "Submit" button at the bottom of the form. For OPAC questions, the upper frame was still a telnet connection to PALS so that students could test their answers in PALS before submitting their answers.
Technology Difficulties Encountered During the Course
Most of the user difficulties were expected. None of the non-user difficulties were expected. However because this was a short course, there was a several-week grace period between the end of the course and the end of the quarter. This helped some students who needed to catch up, although not all of them used the break to advantage.
E-mail addresses in Web browsers. Most students had problems individualizing the Netscape Navigator program to permit e-mail. Because most of the work was done in computer laboratories, and students could not be sure that the program was set correctly, most did not bother to check before doing their work.
This had two results, one that the instructor had to cope with, one that the students had to cope with. For example, if the program had been set up to allow e-mail, the name and address of someone else was probably entered. If the students did not change that before doing the work, their homework was submitted under someone else's name and address. This was anticipated. All materials to be submitted via HTML forms had a field for the student's name and another for the student's e-mail address, allowing the instructor to ignore the false headers.
A more serious problem for the students: If the Web browser had not been set up with a name and e-mail address, the forms could not be transmitted. Students would not discover this until they attempted to submit the work. This produced a high level of frustration. This was also anticipated. The instructor had pointed this problem out during the introductory session on how to use the program, and during the first class period students had to insert their name and e-mail address into the program. Nonetheless, between the first and the second class period, the majority of the class had problems caused by their not doing this. Some students consistently failed to check beforehand and were therefore very dissatisfied with the program. These students seemed to think that network computers should be able to recognize users and adjust to them. They did not grasp the client-server concepts and implications, even though we covered these in class.
Mac vs. IBM. Although Library Science 100's catalog description states "Windows environment" and the advertising for the course states that IBM clones will be used, as in previous offerings, about a third of the class had never used anything but a Mac. This was expected. However an unanticipated difficulty arose. Many of the Mac users attempted to use the IBM diskette in a Mac, sometimes damaging the diskette; then they had to locate IBM computers, which entailed moving to a different room or building. WSU is primarily an IBM-based university.
Change in VAX e-mail addresses. A totally unanticipated problem arose when WSU's Computer Services Department moved a small number of administrators, staff, and faculty from the Eudora e-mail system to the Beyond Mail system. (Computer Services intends to move all employees to this system eventually.) I was one of the faculty moved to Beyond Mail. A part of the change, which involved using a shorter e-mail address (from email@example.com to firstname.lastname@example.org), took effect just before the start of the quarter, but after the course materials had been created. Computer Services had put in forwarding so that old addresses would still work. The Beyond Mail system worked extremely well for the class at first. However, shortly before the end of the quarter, the mail forwarding from the old address to the new address expired. Although the class should have been finished with their work, not all the students were done. Several students found that they could no longer submit homework, nor would the homepage mailto nor the workbook mailto addresses work. The WSU Library's homepage mailto links did not work either.
The result was that I had to correct each student's diskette as students contacted me. E-mail sent to Computer Services about this problem were unanswered. The Library's homepage mailto links were not corrected until about six weeks after the change.
Different editions of Netscape Navigator. An anticipated, but underrated problem was the use of old versions of Web browsers. During the planning stage, this did not seem to be a problem, because most students would be using computer laboratories. The instructor did warn students during the first class period that Navigator 2.0 was the oldest version that could be used due to the use of frames and forms on the diskette. Students discovered that not all computers in some computer laboratories were using the same version of the Web browser. In one lab the range was from version 1.0 to Gold 3.0 beta. There was at least one lab with version 1.2 as the latest version. Several of the students were faculty with networked computers in their offices. Others had computers with Web browsers at home. Sometimes these students had Macs, not IBMs. Often these students did not have a current version of the program.
This caused intense problems for some students; they did not understand the cause of the problem. The normal reaction of the student was that a program was a program, no matter what version. The faculty understood it better, given the comparison of computer versions to book editions. By the end of the course, all students were checking to be sure they were using at least version 2.0.
Not understanding programs. One student had severe problems with this class. This student did not seem to be able to grasp the concept of what a program was or how to use a computer. Entirely inappropriate choices were commonly made (e.g., selecting MS PowerPoint instead of Netscape Navigator to view the diskette, running a virus checker instead of using Telnet, trying phone to connect to the on-line catalog, and attempting to reformat the hard drive for no apparent reason). This student dropped the course.
Viruses. One student accidentally ran a virus checker during the second class meeting. She discovered the Stealth B Virus on the computer she was using. During the class break, the instructor checked all the computers and all the students' diskettes. Two computers and two diskettes were infected. The two students with the virus had been using other computer labs on campus. The instructor removed the virus and reported it to other faculty using the lab and to Computer Services. Thereafter the instructor ran a virus checker on each computer before and after class. No additional viruses were found.
Effects of Student Demographics
Enrollment characteristics. During the period this course has existed, student demographics have changed drastically. In earlier offerings, such as the Fall 1995 class, enrollment was about one half juniors and seniors, about one quarter non-traditional students, and one quarter freshmen. Most of the students were experienced with computers and Windows, although there were two non-traditional students without any computer skills at all. One of these students dropped the course after the first meeting; the other arranged tutoring sessions with the instructor to get the necessary background skills. All Fall 1995 students except one persevered and completed the course.
The Fall 1996 class was very different. About one third of the class were faculty, who primarily wanted to learn how to integrate this technology into their classes or do on-line literature research for their dissertation. Faculty attendance was poor during this course, partially due to their having more important demands. Not everyone completed the course.
Another third were upperclassmen and graduate students. The final third of the class were freshmen and sophomore women who wanted to learn the social-tools of the Internet--e-mail, phone, and chat. Most of the second group dropped the course when they discovered that this would not be covered and that they were expected to actually do academic work in the course. I offered to tutor them in those tools outside of class. A session was arranged, but only two of the students showed up and were satisfied with that one-hour session.
Computer ownership. Another difference was in computer ownership. In Fall 1995 about one half of the class owned computers, and only about half of these students had modems. Students used their computers primarily for word-processing. Those students who used a wider range of programs, and especially network tools, tended to use computer laboratories on campus. In Fall 1996 about half of the class had a computer, but this was mainly due to faculty's having university computers in their offices. Every student with a computer had it networked or owned and used a modem. All but one of the students in Fall 1996 rated their usage lower that the mean of the students in the Fall 1995 class.
Education levels. The range of students' educational levels made it difficult to teach the class, although at times the range provided unexpected enlightenment and humor. One student was reporting on his search of Web space for information on speakers. The faculty in the class assumed this referred to guest speakers--for a class, conference, or a special event. The student was actually looking for information on automobile audiosystem speakers. This provided an excellent illustration of the ambiguity of language and the complexities this causes in searching databases with uncontrolled vocabulary.
Differing student expectations. This is closely linked to the student demographics. The faculty in the class wanted to discuss how to introduce more networked technology into their courses. This was definitely not of interest to the rest of the class, who quickly became restless. One member of class wanted to learn HTML, or at least have the instructor walk him through building a homepage.
Student's reactions. The students' reactions to the course were mixed, but on the whole seemed to reflect the demographic breakdown noted above. The students interested in the social tools were not happy with the content or structure of the course and eventually dropped the class. These students had many problems with the HTML materials, and it appears they did not do the readings nor use the workbook. They did not seem committed to learning the course content and so were not interested in the format in which that content was presented.
Some faculty taking the course also seemed dissatisfied with the content and the manner in which it was presented. Some of them also had problems with the type of computer and Web browser version (e.g., one tried to use the IBM diskette in her Mac, used a word-processing program instead of a Web browser, and was using a 1.2 version of her Navigator program). Such technical difficulties, plus the fact that faculty had a higher frustration level in searching Web space due to a demand for high quality scholarly information, resulted in dissatisfaction among some.
Upperclassmen and graduate students were interested and appreciative of the materials on diskette. Their comments were positive and encouraging. They enjoyed the homework on forms so that it could be automatically e-mailed. They found the ease of use and lack of paper very attractive.
There was an element of excitement about the course format, and this excitement continued throughout. There was also a large amount of disappointment with the failure of the students to follow what I thought were simple instructions.
The technical side of the course worked very well, when it worked, which was most of the time. Most errors were user errors with simple solutions. The problems that were not user errors (e.g., the virus and the change in e-mail address) were also simple to correct.
The really vexing problem was the wide range in students' expectations from the course. The student demographics of the Fall 1996 class were very different from what was expected or experienced in the past. It was too difficult to address that range. I felt that no one was satisfied. If the students had been split into two sections, faculty and nonfaculty, the course might have been more successful.
Overall, the students' response to the course format was mixed and so not very encouraging. It was possible to solve both anticipated and unanticipated technical difficulties. I liked how well the method actually worked, but was disappointed with the student response. Given the change in demographics for this course, the student reaction may not be typical. In terms of increased learning, I did not see evidence of this in the overall quality of work submitted as compared with earlier offerings.
The amount of work required to establish the course materials on HTML for this course was large. Even allowing for this being my first attempt at using HTML as an authoring tool, the amount of work was too large for a one-credit course. In WSU, this course is taught each quarter, so the amount of work can be viewed as an investment to be amortized over a couple of years; however, this method is not recommended for a single-time, low credit course offering.
There is also obvious potential for using this method of delivering course materials in distance education. The lecture portion of the course could be delivered by videotape, as could the behavior modeling done by the instructor. E-mail could ensure students' timely progress. The group project would be much more difficult to conduct. Testing in distance education remains a problem to educators who insist on positive verification of the identity of the person taking the test. Quizzes and tests would have to be coordinated between the instructor and students, providing the actual quizzes and tests via Web pages and restricting amounts of time. For distance education, Look at Me is a recommended program, which would permit easier consultation.
The method I described in this chapter has both advantages and disadvantages. It is not appropriate for many courses. It should be tested in a course with more credit hours and a course with more students. It should also be tested in a course sequence, that is, a series of courses that build upon each other.