Open Online Assignment Submission: First Year Students' Behavior and Views

Article excerpt

Introduction

Homework assignments are a necessary and usual component of learning at all education levels. They are intended to promote assimilation and application of principles and procedures presented in class. Homework assignments can engage students at different cognitive levels from simple knowledge recognition up to synthesis and evaluation (Bloom, 1956). They are intended to enable students to evaluate their knowledge and to provide necessary instructional information to the teacher.

In higher education, homework assignments are particularly important as the knowledge input during lectures is very concentrated, leaving little time for elaboration or practice. Students in higher education (especially at the freshmen stage) may find homework assignments very difficult, difficult, with difficulties stemming from a variety of sources (Ronen & Langley, 2004). The growing sense of accountability for student learning in institutions of higher education necessitates finding ways of providing appropriate scaffolds to support students in performing learning assignments. The search for innovative support methods may require a reconceptualization of learning away from the solitary, individual, teacher-student path and leading towards the socio-cultural learning paradigm, where students are seen as belonging to a group and interacting with each other through multiple channels.

The advancements in ICT during the past decade have afforded new effective communication channels which can be utilized in different ways to promote students' ability to undertake and successfully perform serious homework assignments. One such channel is the discussion group within a course site. The discussion group is a "public forum" where each student can read and post messages. The Open Assignment Submission (OAS) regime we are proposing means that students will post their completed assignments in a designated assignment forum during a fixed period. Posted assignments can be viewed by all group members as well as the instructor, revised versions can be added until the set deadline and comments of all kinds can be added by peers.

The notion of employing an open forum for homework submission may seem at first sight as promoting plagiarism and dependence. Our previous research (Ronen & Langley, 2004) has led us to believe that, given appropriate homework assignments, the potential advantages far outweigh the drawbacks. The current case-study is framed within the Academic Literacy course in the department of Instructional Systems Technologies (Langley, 2007). Within this study we shall attempt to describe and evaluate how first year students act, and the attitudes they express after initial implementation of the OAS regime.

For an extensive survey of the literature regarding learning from worked examples and from peer examples, as well as the social-cultural approach to learning and instruction especially in an information technology environment, we refer the interested reader to our previous study (Ronen & Langley, 2004).

The Study

Context and Subjects

The current study deals with student behaviour and views regarding Open Assignment Submission (OAS) in the assignment forum, during the first weeks of the semester. The research subjects were 55 first year students attending the Academic Literacy course, scheduled as a weekly 90 minute lesson. The students were divided into two groups with separate lessons and access to separate course sites: grpR (32 full-time students who study two semesters a year) and grpG (23 working students who study three semesters a year).

The study consists of two parts: The first analyzes students' responses to a self report questionnaire, dealing with behavior and views regarding OAS, and the second part analyzes data retrieved from the learning management system concerning events of viewing peer work in the assignment submission forum.

The Assignments

The first two assignments of the course were both related to reading material dealing with the multiple effects of advancement in information technology (Appendix A). Assignment1 dealt with the effects of the invention of the printing press as described in "The day the universe changed" (Burke, 1985) and Assignment2 dealt with the effects on society of the rapid advancement in ICT during the past two decades and the resulting profile of the "desired graduate" according to "Education in the information age" (Salomon, 2000). The reading material for both assignments was in the students' native language, while later assignments were based on reading material in English. Each student in each of the groups was randomly assigned one of two assignments. The submission period for each assignment was 6 days. Each of the assignments gave students some choice in deciding which issues to address and required invoking personal knowledge and experience beyond the appointed reading material. For instance, Assignment1 required students to describe the impact of the invention of the printing press on three, personally selected, areas and to explain their views concerning the necessity of memorization in the information age. Assignment2 required students to describe personal strengths and weaknesses with respect to features of the "desirable graduate" defined in the text. This personalized nature of the assignment meant that any outright plagiarism would be easily detectable.

Students' Self Report on OAS

Research instruments and methodology

We prepared a 9 item questionnaire (Appendix B) dealing with issues related to OAS: students' practice in viewing peer work, possible effects of such viewing and student views concerning OAS. The questionnaire was an adaptation of a similar instrument employed in our previous research (Ronen & Langley, 2004).

The questionnaire was administered during a regular lesson at the beginning of week 6 of the first semester. Students were not required to identify themselves, but could if they wanted to. The students completed the questionnaire individually within 10 minutes. It is important to note that the students had been exposed to OAS in other concurrent courses, but the questionnaire specifically focused on the Academic Literacy course.

Results and discussion

The questionnaire was completed by 50 students (grpR = 29 and grpG = 21). Gender composition was: 38% male, 48% female. 18 students (38%) volunteered their names and some added personal comments, which indicated an atmosphere of trust. The results are included in Appendix B. We shall divide the discussion of the results into several aspects:

Viewing peer products at different stages of the assignment submission process. The assignment submission process involves several stages: reading and interpreting the assignment, deciding which of the options to select, seeking resources, composing the answers, checking and revising, submitting the assignment and possibly submitting a revised version depending on additional reflection or comments.

[FIGURE 1 OMITTED]

Figure 1 summarizes students' self report on viewing of peer products. A considerable majority of students admitted to some degree of viewing peer work before starting to work on their assignment (74%), and before submitting their own assignment (88%). The data indicate that students considered peer products valuable resources from which one could possibly gain something. This may also indicate novices' lack of confidence. For many (68%), the viewing continued beyond the point of submission.

The preference for random selection of viewable peer-examples may stem from the early stage in class crystallization in which these assignments were framed, when learner-status had not yet been established.

Effects of viewing peer work--revision. About half the students declared that they might change their submissions according to what they read while viewing peer work (Item 3, Appendix B). To check the reliability of student response we calculated the correlation between the responses to the statements: "After viewing submitted assignments I submit my prepared solution regardless of what I've read" and "After viewing submitted assignments I may change my solution according to what I've read". The correlation value r= -0.53 is significant at the p<0.01 level, for dF=35. No significant gender effect was found.

Effects of viewing peer work - providing feedback. Both high and low quality peer work may receive feedback (Figure 2). There was a slightly greater tendency to inform peers of an error than to praise them for their good work. This was probably considered a service from which each student would like to benefit.

[FIGURE 2 OMITTED]

Figure 3 shows that the forum was the preferred communication method both for praise and for corrective remarks, with a small tendency towards less public channels for corrective remarks. The preference of the face to face channel can be ascribed to the abundant opportunity of encounters during the weekly lessons.

[FIGURE 3 OMITTED]

Students' views. As we shall see in the second part of the study, the assignment submission time varied considerably between students. Items 7 and 8 probed student views concerning possible motivation for early or late submission. Students' responses to these items indicate that the declared primary motivation for early submission is natural conscientiousness (96%) coupled with a desire to receive feedback from peers (68%) rather than a desire to impress anyone.

The responses also indicate that there may be two reasons for last minute submission: primarily a tendency towards procrastination (71%) and also wishing to collect ideas from earlier submissions (49%). The selfish motivation of "not wanting to share" was endorsed only by a very small fraction of the sample (2%).

Finally, students held positive views concerning the benefit of the OAS regime, with over 90% endorsing its contribution to learning. However, there was general support for a mixed submission regime (78%) with some assignments submitted individually and others in the open forum.

Students' Activity in the LMS

Research instruments and methodology

This part of the study analyzes data recorded by the Learning Management System (Britannica Knowledge Systems, n.d.) each time a student accesses a forum and each time the student posts a message. The data include student identity, date and exact time. We interpreted access into the assignment forum as a "viewing" event. From the raw data we have calculated for each student how many "viewing" events occurred prior to assignment submission and how many subsequently. There is quite a large amount of uncertainty tied to counting viewing events, as a single access event into the assignment forum may not involve reading anything beyond the identity of those who had already submitted or it may involve reading many peer examples. Thus, the calculated number of viewing events can be considered a lower limit on the number of occasions the student gleaned some information.

Results and discussion

We defined the Submission Index (SI) as the ratio of the period from assignment presentation until assignment submission and the total number of days in the assignment period (Ronen & Langley, 2004). For example, Assignment1 was presented on October 14th and was due by midnight on the 20th. The maximum submission period was 6 days. If a student submitted on the 18th , the submission time was 4 days and the SI would be 4/6= 0.67.

Using the LMS data we were able to visually present the viewing and submission data per student for each assignment and each group. For instance, Figure 4 shows the number of pre and post submission viewing events against the student's SI for Assignment1 in grpG.

[FIGURE 4 OMITTED]

The SI distribution for both assignments is shown in Figure 5. Most students submitted their work on the last one or two days. However, 25-30% submitted their work earlier--providing the others with opportunities for reading, commenting, adopting and adapting format and ideas. Additional data concerning assignment SI and gender composition per group appear in Table 1.

[FIGURE 5 OMITTED]

Effect of assignment nature and order. Each of the assignments was submitted by 27 students. Figure 5 shows the frequency distribution (%) of SI's for each of the assignments. We can easily see that the frequency of later submissions for Assignment2 is higher. Table 1 indicates an increase in SI average and median from the first to second assignment in both groups. A one-tail, unpaired t-test showed that in both groups the difference was significant (p<0.02). This difference can stem from several sources. Since the assignments were randomly distributed in each group the main effect was probably related to the change in the degree of novelty. The students who were assigned Assignment2 had already viewed the submissions of Assignment1 (we shall show evidence of such viewing) and may have realized the advantage of postponing their submission until they had an opportunity to view some peer examples. Likewise, it is possible that increased pressure from other courses during the second week of the semester dictated a later submission of the assignment.

Effect of group characteristics and gender. Although the average SI in grpG are higher than in grpR, we found no significant difference between the groups for either assignments. The gender ratio (male/female) was a little higher in group G(1.27>1.07) and that may have had some marginal effect. No significant effect of gender on SI was found for each assignment separately.

Viewing patterns. A viewing pattern consists of the number and identity of peer-work viewing events a student performs during and after the submission period. Our previous research (Ronen & Langley, 2004) showed a significant relation between student self-concept, viewing patterns and SI. In particular, we claimed that the instructional benefit of the OAS regime depended on the existence of a group of early submitters and on the concept that peer-work could be legitimately employed as a learning resource (given the appropriate type of assignments).

In the current study we wished to determine whether students employed peer examples as a resource and what viewing patterns emerged for different assignments and groups. Since data about the identity of the viewed peer work were not provided by the LMS, we limited our analysis to the number of viewing events. We shall enrich the discussion by adding self-report data concerning viewing patterns of a sample of students who volunteered their names on the self report questionnaires. Each student was required to submit only one of the two assignments for credit, yet the recorded data showed many instances where students accessed the forum of the "other" assignment. Thus we have divided our analysis between viewing patterns for the "assigned task" and viewing events related to the "other task".

Viewing patterns for the "assigned-task". As we have already discovered, the assignment nature and order made a difference to submission time and possibly to viewing patterns. We have also seen that there were no significant between-group differences in SI for each assignment. So we shall concentrate on viewing patterns within each assignment--number of viewing events prior to and after assignment submission. Figure 6 shows the number of pre- and post-submission viewing events ordered by SI.

The average number of viewing events per student was 12.85, (s.d. =8.12) for Assignment1 and 10.2, (s.d.= 5.02) for Assignment2. In Assignment1 there were some extreme examples of 20-30 pre-viewings. The most extreme case's grade was one of the lowest (70%). In Assignment2 there were 4 extreme examples with 12- 20 pre-viewings, who all achieved high grades.

[FIGURE 6 OMITTED]

The viewing-submitting-viewing pattern for the two assignments was similar. Both the early and late submitters viewed few peer examples after submission, while most of the "late" and "last minute" submitters had accessed the forum many times (>5) before submitting their own work.

The average number of viewing events for both assignments (>10) indicates that students availed themselves of the opportunity the OAS provided to view peer work. The considerable variance indicated different student types with respect to personality (e.g. confidence, diligence, effort, procrastination or experience with the medium) as well as external circumstances (work, home or study pressure).

Both the average and standard deviation of viewing events were higher for Assignment1. This can be related to the novelty of the first assignment in the course and the initial exposure to OAS which could lead to lower confidence and the need for increased "peeking" into the forum.

Assignment grades vs. SI and viewing events. Both assignments were graded using assessment rubrics. We grouped the assignment grades into 3 levels: Very Good (>90%), Good (between 76% and 89%) and Poor (<75%). The distribution of grades against SI in Figure 7 shows a wide range of mostly pre-deadline SI for Very Good and Good grade achievers, in contrast to the much narrower range of last-minute or post-deadline SI for Low grade achievers. The distribution of grades against the number of pre-submission viewing events in Figure 7 shows a very wide range of viewing events for Very Good grade achievers in contrast to a much narrower range (with a lower average) for Good and Poor grade achievers. So it would seem that although the lower achieving students submitted their work later than the others (thus having a greater opportunity to view valuable peer-work), they did not use this time to improve their final product.

[FIGURE 7 OMITTED]

Viewing patterns for "other tasks". Scanning the LMS "viewing events" data we encountered instances of access into an assignment forum by students who were not required to submit that assignment for credit. Each student could access both assignment forums of his/her group. Table 2 shows the break-down of these "not for credit" viewings.

The data show that in both groups Assignment1 was viewed more times by students who were required to submit Assignment2 than vice versa. This viewing was performed during and often after the submission deadlines. We also see that the viewing was performed by only 20-30% of the students.

This viewing behaviour can be explained in terms of "intentional learning" (Bereiter & Scardamalia, 1989) afforded by the OAS regime, especially for those students who viewed peer-work after they had already submitted their own "for credit" assignments. This minority of students was acting in accordance with the lecturer's recommendation that all assignments should be addressed by all students, and that the differential submission schedule was a technicality intended to lighten the work load for both lecturer and students. The OAS regime enabled students who had "learning as a goal" at this early stage of the semester to address the extra assignments with less effort. Towards the end of the semester, with the final exam in sight, we may expect many more "not for credit viewings" into these early assignment forums.

Viewing Patterns: Recorded Events vs. Self Report

Some of the students (6 males and 12 females) volunteered their names on the questionnaire. We shall select a sample and cross check data from the questionnaires with viewing data recorded by the LMS. This will serve to check and hopefully validate our interpretation of the recorded data and shed additional light on these students' behaviour and views. The sample we selected is meant to represent different viewing types within each assignment. We divided the number of viewing events into two categories: few (<=5) and many (>5). Table 3 lists for each student in the sample the assignment grade, the number of viewing events recorded in the LMS prior to and after assignment submission and the corresponding View type. An Expected type was composed using data from the self report questionnaire according to the frequency selected in each of the items dealing with viewing peer work ("any" indicates an undeterminable expected frequency).

Data from both sources were compatible for most (60%), but not all, of the selected sample. R07 and R18 claimed to view peer-work after submission while the record showed no such viewing events, R29 and G13 claimed not to view peer work after submission while the record showed that 8-9 such viewing events occurred.

We can suggest several possible explanations for these discrepancies between the self report and recorded data:

* Lapse of memory: The questionnaire was completed more than 4 weeks after submission of the assignments, so the self report might not reflect student behaviour at the relevant period.

* A lack of shared meaning: A viewing event may or may not include actual reading of peer work, while the questionnaire clearly referred to "reading peer work".

* Uncertainty related to interpreting a recorded access event: The number of access events can be far less that reading events, as reading of several examples of peer work can occur during a single access event.

Summary & Implications

The current study can be seen as a case study within the general issue of learning support afforded by peer examples which we investigated in our previous work (Ronen & Langley, 2004). We set out to investigate first year students' behaviour and views related to OAS for the first assignments of the Academic Literacy course in the first semester. The first part of the study employed a self-report questionnaire. The data analysis provided information that students did take the time to read peer work and that this reading sometimes resulted in revision and was also likely to lead to peer-feedback both for praise and corrective remarks. Students ascribed the tendency to submit assignments early or late mainly to character traits (conscientiousness or procrastination) but also to the potential of gleaning ideas from peer examples. In general, students endorsed the OAS regime as a beneficial method, but also supported the more traditional method of individual assignment submission.

The second part of the study analyzed data recorded by the LMS for assignment forums in the course site. The submission patterns were analyzed using the Submission Index (SI) concept. We showed that the assignment nature or order could affect the average SI. We further analyzed the viewing patterns before and after assignment submission and discovered several predominant patterns mainly depending on student characteristics. We were able to show that students did indeed access the assignment forums quite frequently. We were especially pleased with those instances of "not for credit viewing" which indicated that the OAS regime afforded opportunities for students to engage in extra learning with reduced effort. As the OAS regime is suggested as a potential learning support, we analyzed the relation between assignment grades, SI and the frequency of pre-submission viewing of peer work. We found that high grades were related to a wide range of SI and viewing events and that low grades were related to last minute or late submission (SI>=1) and a relatively low average frequency of viewing events. The implications of these results for the effectiveness of the OAS regime as a learning support are that it may be of most help to highly motivated, medium ability students who are willing to exert the effort required to read, interpret, evaluate and possibly adapt multiple examples of peer work.

In the final part of the analysis we cross-checked the recorded LMS data against student self report in the questionnaire for a sample of identifiable students, and found mainly agreement but also some discrepancies. Our main source of uncertainty is related to the fact that the LMS records do not provide information concerning the extent of actual reading of peer work that occurs during a single viewing event.

We find it vital to reiterate that our entire conception of the OAS regime as a potential learning support afforded by modern instructional technology rests on the proviso that appropriate assignments are employed. The hallmark of a suitable assignment is that within a uniform framework it is individualized, in that it requires individual information and personal example seeking from one's own lived experience as well as specific application of studied material. Viewing peer examples for such assignments may help reduce the cognitive load of interpreting the task for some students and inspire superior performance by others, but does not provide a ready-made solution. Plagiarism is easily detected and the public forum acts as a social deterrent against any such attempt.

Finally, we have shown a method of utilizing the research potential of the LMS technology. In addition to supporting student learning, the LMS has provided us with a valuable research tool which rendered visible otherwise invisible student behavior.

Material published as part of this publication, either on-line or in print, is copyrighted by the Informing Science Institute. Permission to make digital or paper copy of part or all of these works for personal or classroom use is granted without fee provided that the copies are not made or distributed for profit or commercial advantage AND that copies 1) bear this notice in full and 2) give the full citation on the first page. It is permissible to abstract these works so long as credit is given. To copy in all other cases or to republish or to post on a server or to redistribute to lists requires specific permission and payment of a fee. Contact Publisher@InformingScience.org to request redistribution permission.

Appendix A

Assignment1

1. Describe the effects of the invention of the printing press on 3 of the following domains: Education, Commerce, Culture and Art, Society, Politics, Natural sciences.

2. Before the invention of the printing press actors, clergymen, teachers and others were obliged to depend on their memory--thus many methods were developed for improving the ability to memorize information. In your opinion is it still important to memorize information by heart in the present age with the great accessibility of information? Explain your opinion and present a relevant example in one area.

Assignment2

Select one of the questions 1-3. Question 4 is compulsory.

1. The author suggests different names for the current age (p 33). Choose one of the names and explain why it fits the current age.

2. The author suggests three characteristic features of the current age (pp. 33-39). Select one of the features and present 2 examples of the way it is manifested in the current age.

3. The author justifies the continued existence of schools using several lines of reasoning (pp. 4143).

Describe the line of reasoning that seems most important to you and explain your position.

4. The author defines the "desired graduate" in three domains (pp. 43-49).

a. Define the domains

b. Of the skills and inclinations described, select one in which you believe you excel and one in which you consider yourself somewhat deficient. Explain how your excellence was developed and what you believe you should do to improve the deficiency.

 
Appendix B 
 
Open Assignment Submission Questionnaire and Results (%), N=50 
 
1. Before starting to do the assignment ... 
 
I check who has already                  True 56    False 44 
submitted in the forum 
 
I read assignments that have             As many as possible 2, 
been submitted                           Selected 24, 
                                         Random 48, None 26 
 
I check to see which questions           True 18    False 82 
others have selected 
 
2. Before submitting my solved assignment.. 
 
I check who has already                  True 60    False 40 
submitted in the forum 
 
I read assignments that have             As many as possible 2, 
been submitted                           Selected 34, Random 54, 
                                         None 10 
 
I check to see which questions           True 24 False 76 
others have selected 
 
3. After viewing submitted assignments.. 
 
I submit my prepared solution            True 49 , False 43 , 
regardless of what I've read             Don't view 8 
 
I may change my solution                 True 53, False 39, 
according to what I've read              Don't view 8 
 
4. After submitting my assignment.. 
 
I continue viewing other                 As many as possible 10, 
submitted assignments                    Selected 32, As many as 
                                         possible 10, Selected 32, 
                                         Random 36, None 22 
 
5. If I read a colleague's well done assignment 
in the forum.. 
 
I will send him positive                 Always 8 Sometimes 80 
feedback The communication               never 12 Forum 47, Email 17, 
method I would prefer                    Phone 7,Orally 30 
 
6. If I find an error in a colleague's assignment 
in the forum.. 
 
I will send him a note                   Always 29, 
                                         Sometimes 63, never 8 
 
The communication method                 Forum 41, Email 31, 
I would prefer                           Phone 7, Orally 17 
 
                                         Totally     Quite   Don't 
                                         agree       agree   agree 
7. Students who submit their assignments a long 
time before the deadline.. 
 
Wish to impress the lecturer             0           22      78 
Wish to impress their colleagues         4           8       88 
Are simply conscientious and eager       58          38      4 
  to complete their work 
Are hoping for feedback from colleagues  8           60      32 
 
8. Students who submit their assignments very near the deadline 
 
Do not wish to share their work          2           14      84 
  with others 
Wish to collect ideas from               4           45      51 
  their colleagues' submissions 
Simply leave everything to               22          49      29 
  the last minute 
Do not wish to share their               2           14      84 
  work with others 
 
9. In my opinion.. 
 
Open forum submission of assignments     62          30      8 
  contributes greatly to learning. 
It is preferable to submit               10          18      72 
  assignments personally. 
Some of the course assignments           27          51      22 
  should be submitted in the open 
  forum and others should be 
  submitted personally. 

References

Bereiter, C., & Scardamalia, M. (1989). Intentional learning as a goal of instruction. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honor of Robert Glaser (pp. 361-392). Hillsdale, NJ: Erlbaum.

Bloom, B. S. (1956). Taxonomy of educational objectives, Handbook I: The cognitive domain. New York: David McKay.

Britannica Knowledge Systems. (n.d.). Education: HighLearn. Available at http://www.britannicaks.com/Solutions/Education.asp

Burke, J. (1985). The day the universe changed. (Hebrew translation, 1988) Tel Aviv, Israel: Ma'ariv Book Guild.

Langley, D. (2007). Treasure hunt in the library: An exercise in academic Literacy. Proceedings of the 2007 Informing Science and IT Education Joint Conference, Ljubljana, Slovenia, 22-25, June, 2007.

Ronen, M. & Langley, D. (2004). Scaffolding complex tasks by open online submission: Emerging patterns and profiles. Journal of Asynchronous Learning Networks (JALN), 8(4), 39-52

Salomon, G. (2000). Education and technology in the age of information. Haifa and Tel Aviv: Haifa University Publishers and Zmora-Bitan

Dorothy Langley, Miky Ronen, and Shlomit ben Shachar

Department of Instructional Systems Technologies,

Holon Institute of Technology, Holon, Israel

Langley@hit.ac.il; mikyr@hit.ac.il; shlomitbensh@gmail.com

Biographies

Dorothy Langley, PhD is a member of the Department of Instructional Systems' Technologies at the Holon Institute of Technology and a member of the Physics group at the Department of Science Teaching at the Weizmann Institute of Science, Rehovot, Israel. She graduated in Physics from the Tel-Aviv University and received her PhD in Science Education from the Weizmann Institute of Science in 2001. Her research focuses on student learning in information technology based environments with a special interest in inquiry activities, teacher education and science teaching. Email: langley@hit.ac.il

Prof. Miky Ronen is the chair of the Instructional Systems Technologies Department at the Holon Institute of Technology. Her research focuses on the instructional design of interactive learning environments and on the incorporation of technology in the teaching and learning process. She has designed and taught numerous online graduate, undergraduate and teacher training courses and faculty training programs. Email: ronen@hit.ac.il

Shlomit Ben Shachar is a recent graduate (cum laudum) of the Instructional Systems' Technologies department at the Holon Institute of Technology. She is studying towards her Masters degree in the department of Education graduate school at the Tel Aviv University, Israel. She is also currently acting as Teaching Assistant in the Academic Literacy and other courses. Her special interests include the implementation of ICT in instruction and learning. Email: Shlomitbensh@gmail.com

 
Table 1: Gender composition and SI average and median values per group 
 
         Gender %            SI Assignment 1     SI Assignment 2 
Group 
          male     female    average   median    average   median 
 
grpR       52        48       0.65      0.67      0.89        1 
grpG       56        44       0.79      0.83      1.01        1 
 
Table 2: Not-for-credit viewing events 
 
       Assignment 1          Assignment 2 
 
       viewings   students   viewings   students 
 
grpR      25         7          13         8 
grpG      23         9          14         8 
 
Table 3: Recorded events vs. self report 
 
student   Grade      View       View       View 
ID                   pre        post       type 
 
Assignment 1 
 
R07       80         6          0          many-0 
R29       80         2          8          few-many 
G21       75         2          10         few-many 
G06       90         19         5          many-many 
G04       85         8          1          many-few 
G08       75         5          1          many-few 
 
Assignment 2 
 
G13       84         9          9          many-many 
G20       91         16         1          many-few 
R18       83         8          0          many-0 
R01       64         9          4          many-few 
 
student   Pre        Pre        Post       Expected 
ID        start      submit     submit     type 
 
Assignment 1 
 
R07       none       selected   selected   few -any 
R29       none       random     none       any- few 
G21       random     random     random     any-any 
G06       selected   selected   selected   any-any 
G04       selected   selected   selected   any-any 
G08       none       random     none       any-few 
 
Assignment 2 
 
G13       random     random     none       any - 0 
G20       random     random     selected   any-any 
R18       selected   selected   random     any-any 
R01       random     random     random     any-any