In this report we describe two methods used to present an introductory computing class. One version used conventional scheduled lectures and laboratories; the second version was based on independent study (IS), with students working on their own. Two sections of the IS course have been offered to date. The first section compared a control group of comparable students from the lecture mode class with IS students, in terms of performance outcome. Results based on pre-testing and post-testing of both groups show that the IS students performed as well as those in the lecture course. Two quarters later, the same instructor offered a second IS course at the same time that another instructor offered a lecture section. These courses showed similar results. Analysis of the use of multimedia resources suggests a need to adapt linear lecture material to a more dynamic, user-controlled format for multimedia presentation in IS courses.
Independent study courses offer a potentially attractive alternative
to scheduled lecture courses for many students. Enhanced by interactive
electronic communication, this approach allows instructors and
teaching assistants to use their time in responding to student
questions, whether electronically or by other means. Furthermore,
IS courses can reach audiences not otherwise able to take such
courses, owing to scheduling, location, financial or other factors.
An earlier paper (Walters, 1994) describes an introductory computing
course, "General Education" (ECS 15), for non-computer
majors. Several hundred students enroll each year; it's always
filled to capacity, with a waiting list. In the Fall quarter
of 1995, in addition to the regular ECS 15 class, there was a
first offering of ECS 15AT (Autotutorial), an IS ECS 15 class.
There was an additional section of ECS 15 AT (the IS course) in
the spring quarter 1996. The first author of this report taught
all three sections.
We present a comparison of the two versions (ECS 15 and ECS 15AT)
offered in fall quarter 1995 and the additional section of the
IS version (ECS 15AT) offered in Spring 1996. We describe the
courses, their resources, the delivery of both lecture and IS
sections; we also present the outcome of each section. The paper
concludes with recommendations for improving the IS offering.
Introduction to Computers
ECS 15 is a general education course using conventional lectures
and scheduled laboratories. The same course, taught in the IS
mode, ECS 15AT, uses a student-instructor interaction named Remote
Technical Assistance (RTA), videotaped lectures, a laboratory
manual with detailed exercises for each session, and other materials
for use in self-study mode. ECS15 and ECS15IS include nine laboratory
exercises covering the use of MSDOS, a word processor, use of
the Internet and the Web, a spreadsheet, and four exercises in
a high-level computer programming language (Mumps or Scheme).
For three hours per week, groups of approximately 25 students
meet in the lab. The lab contains 30 fast PCs with Internet connections
and a variety of pre-loaded software. At least two instructors/assistants
are available. In addition to the laboratory exercises, there
is one midterm, a final, and a term paper written on the use of
computers in some field of interest to the student. The course
is offered every quarter to approximately 100 students in lecture/scheduled
laboratory mode. The enrollment is limited by hours available
for computer laboratory use and funding for instructors/assistants/graders.
The demand for the class always exceeds the available space.
Administration of the Scheduled Versus the IS Offerings
We decided to test the IS version with a small group of students--25
or fewer--and to offer two sections simultaneously, one lecture
mode, the other IS (we use the term IS to refer to the latter
The lecture mode section was open to 100 students using lectures,
scheduled laboratories, and strict deadlines for all assignments.
The second section was open to juniors or seniors; they were not
permitted to attend lectures or scheduled laboratory sessions,
relying instead on the printed and electronic resources made available
to them, but including access to the instructor and TA during
office hours and the learning materials described earlier. Both
groups had the same deadlines, and possibilities of different
exposure to instructional resources were minimized.
Twenty-five students signed up for the IS version, with 20 completing
the course in Fall 1995. Forty-six of the students in the lecture
section were at the equivalent academic level and served as the
control group for the study. In Spring 1996, the IS section had
an enrollment of 25, with 21 students completing the course.
Recognizing that this paired offering represented an excellent
opportunity to study the effects of distance learning, we performed
an analysis of the relative merits of the two courses. The support
paid for a research assistant, for data collection and analysis,
and for additional student instructional staff.
Our pre-test to give us baseline information on the students taking
both versions of the course included several types of questions:
term identification, programming concepts, computer components,
and the relative costs of hardware and software. We also surveyed
both groups to determine general background, including educational
level of parents, high school and college GPA, and the degree
to which they made use of computers for word processing or other
applications. The registrar provided cumulative college gradepoint
average (GPA) for students in both groups.
Learning resources. Learning resources included the instructional
staff, who were available during scheduled laboratories (for lecture
mode students) and during office hours; handouts included the
laboratory manual, lecture notes, term paper guidelines, a list
of references used by former students, and auxiliary material
on the World Wide Web (WWW). Included on-line were: guides to
the use of software; lecture notes in HTML format; previous examinations;
solutions posted following each week's laboratory exercises; sample
term papers; a keyword-indexed list of over 1,000 references used
in previous classes; searchable on-line by keyword; course syllabus
and all handouts; a list of videotapes including those described
below and ones purchased commercially for the class; and a glossary
of computer terms.
We videotaped the majority of the 16 lectures given in the conventional
class as well as 9 laboratory introductions, recording them and
using the instructional media facilities available at Davis. Tapes,
which ranged in length from 9 minutes to 45 minutes, included
extensive use of special graphics, live shots of such scenes as
computerized traffic signals, barcode scanning in grocery stores,
and computer workstations of several different types. We used
some of these tapes in the first part of scheduled lecture hours,
supplementing them with a discussion period at their conclusion.
The tapes were also available for playback in a campus facility.
Arrangements for video check-out are being explored. We have not
yet concluded arrangements to broadcast the tapes overnight via
the local cable service, but negotiations are underway.
A concept under development at Davis, the software package called
Remote Technical Assistance
(RTA), provides students with three methods of communicating with
instructional staff and access to auxiliary resources. The first,
an advanced form of messaging significantly improved over e-mail,
includes the ability to attach screen snapshots, files (e.g., programs,
term paper drafts, spreadsheets), and even audio clips in store-and-forward
mode. The second mode permits live interaction with a member of the instructional
staff, including shared screen annotation of snapshots and other images, and
multimedia file transfer to enhance the interactive dialog. The third component
is an "expert system," described in Fonseca and Reed
(1996). The expert TA (ETA) is based on resources prepared for
the course and augmented by responses to frequently asked questions
as they occur in other modes of RTA use.
Deadlines. Both IS and lecture mode students had strict
deadlines for submission of laboratory exercises and term paper-related
assignments. However, students taking the IS course were allowed
to complete assignments early (including taking examinations when
they felt prepared).
Student performance measures. To provide a consistent base,
the same individuals did the grading in all three sections, using
absolute measures; there was no grading "on a curve."
Laboratory exercises turned in on time received full credit.
As a further incentive to in-depth learning, students earned a
maximum 7 points extra credit by completing advanced portions
of the laboratory exercises and by turning in newspaper clippings
related to computer uses in today's society. Letter grades assigned
were 88, 78, and 70 as lower limits for grades of A-, B-, C-,
respectively. The weighting of points assigned to different portions
of the class were: laboratory exercises: 40%; Term Paper: 25%;
Midterm 15%; and Final: 20%.
Analysis for Comparison of Student Performance
Table 1 below shows two performance outcomes along with the differences
from the baseline measure. Because the final exam was not cumulative,
the average of exams (midterm and final weighted in the same ratio
as for the course) became the exam measure (expressed as a percentage).
The pre-test most closely matched the exam questions in type of
material. The total course score became the second measure. Each
outcome is compared to the pretest scores for the group.
The pretest scores were not significantly different, although
the average for the Spring IS section was more than 10 points
higher than the averages for the other sections. One explanation
is that by Spring quarter, all students have had at least two
quarters to familiarize themselves with the computers on campus
at Davis. During Fall quarter, freshman and transfer students
are completely new to campus.
The outcome results of the three groups were nearly identical
in exam totals and course totals as shown by the group averages
in Table 1. The percentage of improvement from baseline (difference
from pre-test) measures were smaller for the spring section (although
not significantly), which appears to be due to the higher pre-test
average in that section.
The difference from baseline was 45% or more in all sections.
That is, students achieved almost exactly the same final scores
in all groups, and almost identical improvements from the pretest
to the exams. One student in the Spring section did well on both
exams, but did not complete a term paper, therefore receiving
a score of 0 for that portion of the course (25%). Therefore the
student's total score was 25 points lower (of 100), which could
explain the smaller difference in course total compared to exam
We next analyzed the basic profiles of students in the three groups.
A commonly used criterion of achievement is overall GPA. The GPA
at UC Davis is on a scale of 0 to 4.0 with A
receiving a 4.0, B receiving a 3.0, etc. A plus or minus on
a grade adds or subtracts 0.3 of a grade point, respectively,
with the exception that an A+ also receives a 4.0.
The two groups had no significant differences in this measure
(see Table 2). Female to male students were approximately 2:1
and 3:1 in the Fall lecture and IS sections, respectively, and
were approximately 1:1 in the Spring IS section.
A greater standard deviation in the IS students for both quarters
reflected a proportionately greater number of students at the
top and bottom GPAs in the IS groups, whereas lecture students
were more tightly grouped near the average. A comparison of the
histograms of GPA scores for each section further verified this.
The histogram for the lecture section shows a much larger group
of students in the center than the ones for the IS sections, although
there are approximately the same number of students spaced at
both "ends" of all the distributions. This finding lends
support to the hypothesis that both the fast learners and the
slow learners are more likely to self-select IS instruction than
those in the middle of a class (in order to move faster or receive
more individualized attention, respectively).
Almost all students (including those in both IS groups) completed
final course evaluations, which sought (a) an overall evaluation
of the course effectiveness; (b) estimates of the use of different
learning resources and opinion of their value; and (c) reasons
for selection of this format and degree of satisfaction with the
approach and the various auxiliary learning resources. These results,
summarized below, do not appear in table form for reasons
Overall course satisfaction was high, averaging 8.7 in Fall 1995
(81 responses of 135 enrolled students) and 8.1 in Spring 1996
(8 responses of 26 enrolled) on a rating scale of 1-10. Unfortunately,
the numerical evaluations for Fall 1995 were not separated by
group (lecture vs. IS), so it was not possible to get direct comparisons,
but the positive response from the additional questions asked
of IS students suggests that their satisfaction equaled that of
lecture students. Eighteen of 20 IS students in Fall 1995 and
12 of 15 IS students in Spring 1996 responded that the course
met their expectations (one of the negative responses in Fall
1995 was tempered by the comment, "but the instructor made
up for it"). Fifteen of 20 IS students in Fall 1995 and 13
of 15 in Spring 1996 expressed satisfaction with the IS format;
one commented, "It is empowering to realize I can teach myself."
The greatest frustration with this mode of instruction was lack
of immediate answers to questions. As noted in the following sections,
we have addressed some of the problems mentioned and are working
To the question, "Would you take another IS course?": 11 students
in Fall 1995 and 13 students in Spring 1996 said yes, 4 said no
in Fall 1995; there were no negative responses in Spring 1996.
The remainder said it would depend on the course content.
Use of learning resources. Because this class has made extensive
use of electronic and personal support for several years, both
lecture and IS students had equal access to e-mail, videotape
libraries, keyword-indexed term paper reference listings (both
electronic and hard copy), electronic lecture notes and other
WWW resources, and electronic solutions to laboratory pre- and
post-tests, practice and actual midterm, practice final, and a
variety of other items.
As our survey showed, however, there were significant differences
in the use made of these resources, based on self-reporting and
instructor data. The results are summarized in the Table 3.
E-mail use decreased with both groups during Fall 1995 in the
final weeks of the term (based on separate statistics collected
by the instructor). Office visits were higher for both IS sections
compared to the lecture section. The visits tended to increase
at the end due in large part to an especially difficult laboratory
exercise at the end of the term. IS students reported spending
over twice as much time on this last laboratory exercise on average
(11.4 hrs for IS students in Fall 1995 and 4.9 hours in Spring
1996, vs. 5.5 for the control group).
Hardly any students used the videotapes. It may be significant,
however, that the one IS student who saw almost all tapes also
received the top grade in the final and was in the top three students
in overall performance for the course. Students seemed to feel
that the lecture notes were sufficient, and that going to the
playback center to view the tapes was too inconvenient. A tape
check-out procedure, or broadcasting over the local cable TV channel
would facilitate the viewing of tapes by distance learners and
on-campus students with more time or location restrictions. Also,
the linear format (30 minutes or so per tape) does not seem to
appeal to students outside the lecture hall.
Reasons for selecting the IS format. When the IS students
were asked what their reasons were for selecting this format (multiple reasons
were encouraged), eight Fall 1995 and five Spring 1996 students
reported schedule conflicts with job or other preferred activities.
Six fall and five Spring students wished to move at a faster pace.
Three students in each fall and spring section enrolled because
the lecture mode was closed out. Three in each IS section stated
they felt they had the self-discipline to undertake a course in
the IS format. Two in each Fall and Spring section honestly reported
that they wanted to avoid lectures. One student in the spring
noted that they were experimenting to see if they had the required
These answers suggest that the IS mode is not for all students. Many prefer the tried and true methods of instruction. Some reported that competition with other scheduled classes led to difficult choices in the use of time. Several lecture-mode students attended relatively few lectures, evidently content with just a passing grade and exhibiting minimal interest in the course. These students might have done even worse in an IS mode course. One IS student in the spring reported attendance at over 2/3 of the lectures given by another instructor that quarter. (That student finished with an excellent course total, approximately 95%).
Resources generated for ECS 15 include a series of lecture notes, a laboratory manual, video tapes, and electronically stored resources. In addition, the instructor has compiled both a hard copy and an electronic bibliography, indexed by keywords, of references used by previous students taking the course.
The total cost of producing the videotapes, exclusive of the instructor's time scripting and taping, was just under $50,000, or an average of slightly less than $2,000 per tape. Videotaping of lectures and laboratory introductions was the most expensive item prepared for the IS version. These tapes required careful scripting, animation sequences, field trips for shots of special sites (e.g., bar-code scanning in a grocery store), taping with director, technician and recorder plus instructor at each session, editing/revision, and some update or replacement.
Students in the lecture mode class used all of the resources.
Whereas only selected videotapes are shown each quarter, they
are also used in lectures as a better means of bringing exhibits
to the classroom and of using animation and other techniques.
Because the instructor makes a practice of preparing lecture notes
and laboratory exercises for other courses not at present taught
in the IS mode, it is difficult to assign a cost specific to IS
instruction in this case.
Cost associated with delivery of the course. It is not
clear what costs can be specifically ascribed to the IS class.
To obtain more precise information, in a second IS section during
Spring 1996 only the instructor and one reader responded to e-mail,
holding office hours, and having one assistant available during
one three hour lab section per week. The second author of this
report taught a concurrent lecture section of ECS 15 in Spring
1996, with slightly different course content. The IS students
did take advantage of office hours with their instructor and assistant,
as well as the scheduled lab time. In addition to student contact,
the same amount of time is necessary to grade the assignments,
exams, and term papers of IS students.
Conclusions and Recommendations
Based on student comments from Fall 1995 IS section, we scheduled laboratories during Spring 1996 each week (three hours) with one assistant attending to answer questions. This was greatly appreciated by the students; they could get immediate help from the lab assistants, resulting in greatly reduced time spent by students.
As discussed in some detail by Laurillard (1993), the use of technology
in instruction requires training and careful attention to the
course objectives during design and construction of these resources.
Electronic resources and other instructional support seem to have
been adequate for this course. Videotapes, although acceptable
replacements for some lectures in lecture mode, do not fit the
IS learning mode. We need to revise the content, convert to a
non-linear, hypertext form of presentation of short clips of those
sequences (animation, live action) where motion is important,
and convert other material to fixed images with audio or written
Undoubtedly the greatest disappointment to the instructor, TAs,
and students, was the fact that RTA did not get to operational
status in time for it to be used in this class. It is clear that
the interactive dialog mode would have received significant use
had it been available in a form convenient to students.
One conclusion that seems inescapable: resources for courses of
this type will require constant updating and revision. This is
not different from the needs of conventional courses, but it may
require a little more planning ahead.
These results strongly support the idea that IS courses can succeed,
given the right motivation, instructional materials, and support.
Although students taking each IS course were on campus during
some portion of each week, the results could be extrapolated to
distance learning, where student-instructor interaction would
have to be via electronic means. The RTA concept would be a major
benefit for that type of independent instruction, also.
Additional investigations should produce more information about
alternative forms of instruction, including Distance Learning.
Although many questions remain unanswered at this time, we hope
that this study will provide a stimulus for future work of a similar
Fonseca, S. P. & Reed N. E. (1996, August). Integration of
an expert teaching assistant with distance learning software.
Proceedings of the Thirteenth National Conference on Artificial
Intelligence 2, 1388.
Laurillard, D. (1993). Rethinking university teaching: A framework
for the effective use of educational technology. London: Routledge
Walters, R.F. (1994). An introductory course on M worth exporting.
M Computing 2(1), 13-19.