One aspect of evaluation I've been thinking about is the role of the instructor for a particular application (see Table 2.2 in DuBravac, p. 26). DuBravac notes on page 29 that, "adopted applications should be able to reduce the amount of work for instructors in comparison to what they would do without the technology."
This seems somewhat obvious, but it is not always easy to predict how much time and work a teacher will spend using a particular application. And if an entire program decides to implement an application, some teachers will likely use the technology more efficiently than others. In my L2 composition class, I use several BBLearn features, such as the discussion board and online submission tools. This will be my fourth semester teaching L2 composition, and each semester I move toward a more "digital" format for class. I'm still not sure if this creates more or less work for me, but maybe I'll figure that out by thinking through some pros and cons. Here it goes:
Con: Clicking through a set of homework assignments takes longer than flipping through a stack of papers. For the most part, if I'm going to ask students to take the time to write, reflect, research, or peer review something, I need to take the time to offer feedback (qualitative and/or quantitative), and for me, BBLearn submission takes more time for short assignments.
Pro: There are no disagreements about whether or not an assignment is late or whether or not a teacher lost something. BBLearn lists the deadline as well as the student's submission time. Students can see all their missing assignments. And, students can still submit even if they're too sick to come to class. This places more responsibility on the student to keep track of deadlines and missing assignments and saves me from digging through stacks of papers when a student thinks I've lost an assignment.
Con: "I tried to submit, but.... BBLearn wasn't working!" This usually means I have to make a new homework assignment, or extend the deadline on BBLearn, or download homework assignments in my e-mail inbox. Technology is not always reliable.
Pro: Students can do peer review at home using the discussion board. They don't have to come to class at 7:55am, and neither do I. Everyone wins.
Con: I make up for it when I have to untangle all the threads upon threads of discussion board responses. It gets messy. And, although we go over the peer review directions in class the day before, many students do their own thing. Or review the wrong person's paper. Maybe they have questions about peer review, but they can't ask me on the spot. Because I wasn't monitoring the peer review process as it was happening, I now have to decide how to fairly grade the variety of peer review formats I've received.
Pro: Students appreciate opportunities to become familiar with BBLearn. This class serves as their "bridge" to the university, and students say they feel better about using BBLearn after taking this class. I am also (slowly but surely) learning the do's and don'ts of BBLearn.
Although this list is not exhaustive, I think the final "pro" listed is one of the most important. It fits with our class goal of serving as a bridge and preparing students for the university. And while the discussion board feedback may be messy from a grading perspective, it's a reality of "composing in electronic environments," which is one of the WPA outcomes for the class. The time an instructor spends using technology must be considered alongside the skills that may be gained.