July 3, 2012
DoIT News Moving to myUMBC Groups
As of today, the DoIT News will be published through the DoIT Group on myUMBC, which in turn feeds the new DoIT site at doit.umbc.edu. As such, we will no longer be maintaining this Movable Type blog. However, it will remain for archive purposes.
FYI to DoIT Group members:
If you have questions, please contact the Technology Support Center (TSC) located on the first floor of the library next to the RLC or call 410.455.3838.
July 11, 2011
UMBC Featured in BbWorld Developer's Conference Keynote
UMBC's Check My Activity (CMA) feedback tool for students featured prominently in a keynote talk by John Fritz at the Blackboard World Developer's conference today.
February 24, 2011
IHE Journal Publishes Article on Check My Activity Tool for Students
The Internet and Higher Education has published an article about UMBC's Check My Activity (CMA) tool for students in its special issue on web mining in education. For more information, see the following:
Fritz, J., Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers, Internet and Higher Education, Volume 14, Issue 2, March 2011, Pages 89-97.
February 4, 2011
Educause Publishes Video Demo of UMBC "Check My Activity" Tool for Students
The final 2010 issue of EDUCAUSE Quarterly includes a brief (5 minute) video demo of UMBC's "Check My Activity" (CMA) tool for students. The CMA demo was part of EQ's special focus on student retention, and includes new information comparing how students use Blackboard by final grade distribution in all 1,461 Spring 2010 Blackboard courses. Specifically, D & F students used it 47 percent less than students earning a C or better, which is similar to results from previous semesters based on much smaller samples. A similar analysis is being conducted on all Blackboard courses for Fall 2010.
May 19, 2010
A Case for Using The Blackboard Grade Book
In 2008, a national study of more than 30,000 seniors and freshmen at more than 100 colleges and universities (including UMBC) reported that students value the ability to check their grades far more than any other function in a course management system (CMS) like Blackboard. Yet, less than half of UMBC's Bb courses used the grade book this past fall and spring. We understand why: it takes a little time to master the Bb grade center, faculty may prefer to use Excel functions in an "offline" mode, and there may be a reluctance to put a grade on everything students do. But apart from keeping students from constantly pestering faculty, an active Bb grade book can be "leveraged" in some interesting ways.
|UMBC Bb activity by grade distribution, 2007-2009. Detail table|
As some may recall, DoIT has been researching how students use Blackboard. Specifically, over two years in a sample of 110 courses, D & F students tend to use Blackboard 39 percent less than students earning better grades. To be clear, we are not suggesting or interested in how Blackboard "makes" good students. Instead, we are interested in how good students use Blackboard, and how displaying this information (albeit anonymously within courses) might help raise awareness of their under performing peers.
This is where faculty can help, by voluntarily adding to the sample of courses so we can study the Bb activity of students by final grade distribution. To do so, simply add a column called "GRADE" (quotes omitted) to your Blackboard grade book with your students' final letter grades. You can even hide the column from student display if you only want them to see your final grades in SA. But after final grades are due on May 26, if your Bb grade book contains a GRADE column, we will run a script that analyzes your student grades AND Bb activity to see if and how the usage trend does or does not persist. For more information, visit our project blog at www.umbc.edu/blackboard/reports.
If faculty use the grade book, our "Check My Activity" (CMA) tool beneath the "My Grades" tool lets students see an anonymous summary of Bb activity for students who earned the same, higher or lower grade they earned on any assignment. From prior surveys about the CMA, nearly 30 percent of students say they are "surprised" how their own Bb activity compared to peers (another 42 percent say they'd have to use the CMA more to determine its usefulness, but they appear to be intrigued). In addition, 54 percent say they would be "more inclined" to use the CMA's grade distribution report to review their Bb activity on past assignments--before future assignments are due. In other words, the CMA may leverage or amplify the feedback effect of grades that faculty are going to give students any way, but it can only do so if they use the grade book.
Note: After initial promotional campaigns within Blackboard each weekend in October and March of this year, the CMA currently generates more than 1,300 weekly visits, 80 percent of which are by return users who are spending more than a minute per session viewing the 2-3 custom reports they can generate themselves. While we don't completely know who is using the CMA or why (the reports can only be viewed with a myUMBC login), we plan to study usage further and conduct focus groups and interviews with students. For more information, including a brief, online video demo of the CMA, visit www.umbc.edu/blackboard/reports.
|Hardy's presentation begins at 00:28:34.|
Using Blackboard's "adaptive release" function, faculty can create "pre-conditions" that students have to meet before they can even gain access to other functions or content in a Bb site. For example, you could require that students pass a syllabus or academic integrity quiz at the start of the semester, before they can turn in any work for credit. But to do so, you'll need a "grade" for the quiz that adaptive release can use as a condition for allowing students to even see the web link to an electronic assignment, subsequent quiz or discussion board.
Tim Hardy in Economics has used adaptive release so effectively, his ECON122 Bb course was UMBC's most active undergraduate Bb course this past fall and spring. Also, students in his hybrid course have consistently scored 20 percent higher on his department's required, common final exam, largely (he says) because the Bb course structure makes them apply concepts in assignments that prepare them for it. "Initially, students hate it, but after they pass the final exam, they tell me it worked," says Hardy, who also participated in the Alternate Delivery Program to re-design his course for hybrid delivery last summer and fall. For more information, see Hardy's 3/10/10 Brown Bag workshop with Katie Morris from Social Work, "Developing and Assessing Quality Learning in Online and Hybrid Courses."
From our position in supporting faculty, we typically see three kinds of Blackboard usage:
1. User & Document management (Pulling students to the site through mandatory use)
- Password-protected class & group space
- Attach or copy/paste documents such as syllabi, presentations, notes.
2. Communications (Pushing content to them and facilitating interaction)
- Chat & Discussions (especially if "email subscription" of new posts is enabled)
3. Assessments (Pushing & Pulling to facilitate self-paced learning)
- Electronic assignment delivery & collection
- Quizzing, Surveys, use of Grade Book
- Adaptive release to structure student responsibility and self-awareness
To be sure, most faculty are doing #1 (maybe even #2), but students would like to see more faculty doing #3. Depending on where you see yourself now, I'd like to encourage you to consult your peers who are actively using Blackboard, or let us know if and how we can help you help your students take more responsibility for their own learning.
Asst. Vice President
Instructional Technology & New Media
UMBC Div. of Information Technology
410.455.6596 | email@example.com | www.umbc.edu/~fritz
February 13, 2010
24/7 Bb Support Update: January Stats & Known Issues
Last month, UMBC began its first full semester with Presidium Learning as the provider of our new 24/7 Blackboard support service. As a result, 169 students and 83 faculty sought direct, start-of-semester help (266 total). In addition, the related Bb self-support portal recorded 755 visits, 167 knowledge base article accesses and a total of 1,766 page views. This does not include 195 Blackboard "Request Tracker" (RT) tickets resolved by DoIT staff directly. A more detailed summary of January's 24/7 Bb support is available here, and future monthly reports will be announced in this DoIT News blog each month during this one-year pilot that was announced last fall.
While it's too soon to say what our 24/7 Bb support usage reports mean, or if and how Blackboard support may improve given staffing challenges reported last fall, a few specific issues emerged last month that need to be clarified or improved:
1. UMBC course creation and auto enrollment issues cannot be solved by 24/7 Bb Support.
As part of this pilot, UMBC elected not to give Presidium access to the Student Administration (SA) system, which is needed to troubleshoot course and enrollment mismatches between SA and Blackboard. A DoIT workgroup is meeting regularly to address the course and enrollment mismatch issues that emerged again this semester, with a focus on how to improve our current process before a planned upgrade to Blackboard version 9 in Fall 2010. While DoIT initially announced that Presidium would have to refer such "UMBC-specific uses of Blackboard back to the DoIT Help Desk," including course enrollment issues, some students and faculty sought support for these issues from Presidium, which could not solve them.
For now, DoIT asks students and faculty to submit an RT ticket via myUMBC help or directly at http://rt.umbc.edu for any course creation or enrollment mismatches between SA and Blackboard. Please include the specific course name, and userIDs of students or instructors who are officially enrolled in or eligible to teach the course according to the SA system. We will also continue to ask Presidium to escalate these requests to DoIT on behalf of UMBC students and faculty who end up contacting Presidium first.
|Tips for getting & improving UMBC Bb help (4:02 min)|
2. Presidium should facilitate a "warm transfer" of issues they cannot solve or that are out of scope (e.g. UMBC Bb course creation and enrollment).
By mutual agreement, this means the Presidium Customer Service Representatives (CSRs) should explain to the UMBC student, faculty or staff member that they will need to transfer his or her call to the DoIT Help Desk during our business hours, or create an escalated ticket in DoIT's RT system after hours. Unfortunately, this was not done consistently in the first two weeks of January, resulting in some students and faculty contacting 24/7 support only to be told they should hang up and call the DoIT Help Desk directly. This is understandably frustrating and not acceptable. DoIT worked with Presidium's account manager, and the UMBC-specific knowledgebase that guides Presidium CSRs now reflects this "warm transfer" procedure.
3. UMBC's course copy process is actually Blackboard's export/import function.
When UMBC first started using Blackboard in 2000, DoIT made a deliberate choice to disable a delivered function called course copy, which literally allows a faculty member to make a new, exact copy of an old course. The problem is that it also requires faculty to create a unique and permanent courseID that follows a consistent naming convention we knew we'd need for any kind of auto-enrollment process to work. Instead, like a number of other institutions, we disabled course copy, and opted to create empty course shells each semester (with the unique and consistent courseID embedded in the new shell for auto-enrollment to work). As we never imagined giving true "course copy" functionality to faculty in the future--and had no third party support entity like Presidium now--we continued to use the phrase "copying an old course to a new one" for what is actually the export/import function in Blackboard now. This simply became a matter where IT and the faculty we've supported for the last 10 years came to understand and thus redefine what "course copy" meant in using Blackboard at UMBC.
Unfortunately, our understanding differed from Presidium's, which has clients who do allow faculty to use the "course copy" function, resulting in Presidium advising UMBC faculty to do something they can't. When we became aware that Presidium's "course copy" advice to UMBC faculty conflicted with our long-standing recommended export/import practice, we decided it would be easier to train Presidium CSRs than change a term and task UMBC faculty had been using for several years--especially right before the semester started. Unfortunately, some Presidium CSRs reverted to their common practice and terminology, which led to unsatisfactory support for some faculty. Again, we altered the UMBC-specific knowledge base that Presidium CSRs use in supporting our faculty, but it has come at the expense of some faculty understandably being upset with the advice they received from our 24/7 Bb support.
Going forward, as part of our plans to upgrade to version 9 in Fall 2010, DoIT is exploring a change in our current processes to make course creation and enrollment less problematic for UMBC students and instructors, as well as DoIT and Presidium.
Finally, I want to thank UMBC students and faculty for their patience as Presidium learns how we operate and vice versa. Eventually, this pilot may show us that 24/7 support is not needed or desirable, and that we should try to staff up if and when budget issues ease, to meet the support needs of ever growing Blackboard usage. However, like January's summary report, I have noticed a similar pattern in November and December that students used the 24/7 support twice as much as faculty. At a minimum, I'm hopeful this may mean we can reduce the default tech support role many faculty find themselves in their current Blackboard courses. In turn, this may free up DoIT staff to support more advanced pedagogical needs of faculty, if Presidium is handling basic Blackboard support needs.
For now, I'd like to issue an open call to any UMBC faculty member to contact me or Karin Readel (firstname.lastname@example.org), Director of Instructional Technology, if Presidium is not "learning UMBC" or supporting faculty the way they should. In addition, if you open a ticket on the Bb support portal, please provide feedback when the ticket is resolved, if your support experience could have been improved. We can't help Presidium improve if we don't have specific details of how and when they've stumbled (e.g., date/time of the incident, name of course, name of CSR). To date, DoIT has received 12 such reports from faculty since we started using Presidium in November. Presidium has also expressed appreciation for feedback on what they can do better, and I have appreciated them providing access to the monthly reports I'm sharing in this and future updates about our 24/7 Bb support.
Together, I still believe we can partner with Presidium to provide quality 24/7 Blackboard support to UMBC students and faculty. If you have questions, concerns or would like to discuss this further, please post a comment to this entry or contact me directly.
Asst. VP, Instructional Technology & New Media
UMBC Division of Information Technology
410.455.6596 or email@example.com
January 16, 2010
UMBC Blackboard Usage Sets All Time Record in FA2009
Based on UMBC's most recent Blackboard Report for Fall 2009, students and faculty used the course management system (CMS) in more classes than ever before. Specifically, 1,430 courses used Blackboard in Fall 2009, compared to 1,014 a year ago (a 30 percent increase). In addition, 215 courses included one or more sections, which means at least 1,645 sections (or 65 percent of the university's approximately 2,500 sections) used Blackboard.
While it's hard to pinpoint specific reasons for the increase, the preparation for academic continuity in case of an H1N1 outbreak may have been a factor.
|UMBC Blackboard activity by grade distribution, 2007-2009. Detail table|
We know hits alone are no measure of quality teaching or learning. Also, the sample needs to be expanded and the demographics of students needs to be studied further. But does the pattern hold true throughout the semester? If so, how might students’ self-awareness, motivation and performance change if they could know how their CMS usage activity compares to more successful peers, earlier in the semester? If not, how and when does the pattern break down? And is it significant enough to dilute student awareness and motivation to seek or accept help from an instructor or the Learning Resources Center (LRC)?
While we can not yet fully answer these questions, DoIT continues to explore ways to provide this information to students sooner. And they are finding it. During the three-month period from September 1 to December 1, our custom Check My Activity (CMA) tool recorded more than 15,000 visits and 52,000 page views (see Google Analytics report). The CMA lets student compare their own activity against an anonymous summary of their peers, based on average hits per user. This is also how UMBC's most active Blackboard courses are determined.
In addition, if faculty post a grade for any assignment in the Blackboard grade book, students can view an anonymous summary of the Bb activity by students who earned the same, lower or higher grade as their own grade. Nearly half of the FA2009 Bb courses (658) contained active grade books with at least one recorded grade, and past surveys of specific UMBC classes have shown more than 50 percent of students would be inclined to use the CMA for past assignments before future assignments are due--if they have grades to check. This confirms a national study by the Educause Center for Applied Research (ECAR) that shows students value checking their grade more than any other function in an online CMS like Blackboard.
While we would like to better understand how and why students are using the CMA, and what it tells them, DoIT has made the CMA easier for students to find by creating a custom "building block" that links directly to it from within any UMBC Blackboard course (tools-->"My Activity").
For more information, visit the UMBC Blackboard Reports project blog at www.umbc.edu/blackboard/reports.
December 19, 2009
Faculty Request: Participate in Final Grade Distribution Report by Bb Actvity
As announced last spring, the Division of Information Technology (DoIT), would like to encourage faculty to post their FA2009 final grades in the Blackboard grade book using “GRADE” as the column heading (please omit quotes and make the column heading ALL CAPS). This is totally voluntary and does not constitute official submission of final grades, which is due Tuesday, January 5, 2010.
December 4, 2009
Student "Check My Activity" Tool Now Available in Bb Courses
|Show & Tell Video|
October 9, 2009
UMBC "Check My Activity" Reports for Students Now Available Inside Blackboard
Based on user response to a system-wide announcement in all UMBC Blackboard courses this past week, the Division of Information Technology (DoIT) has made a permanent link to the self-service "Check My Activity" (CMA) reporting tools for students on the myBlackboard tools menu.
Specifically, all students now have an easy way to find the "Check My Activity" (CMA) and "Grade Distribution Report" (GDR) tools for comparing their own activity against an anonymous summary of their course peers. Using this same myBlackboard tools menu link, faculty and staff have access to similar reports, though staff are unlikely to have need for the GDR tool, which is only valuable if a grade has been entered in a Bb grade book.
As reported previously by DoIT and The Retriever Weekly, students have had access to their own CMA and GDR tools since Spring of 2009. However, adoption has been slow, probably because the tools don't reside inside Blackboard. So, a link to both reports was posted on Friday, October 2, at 5 p.m. and expired a week later at the same time on Friday, October 9.
|CMA & GDR Demo|
- In just one week, CMA & GDR usage activity increased more than 1,000 percent to 6,051 visits compared to 391 visits the entire previous month (Sept. 1 to Oct. 1).
- The total number of page views increased to 20,008 from 537 the previous month. Average page views per visit also increased to 3.31 compared to 1.37.
- The average time spent on the CMA & GDR reports jumped to 1 minute, 18 seconds, compared to just 13 seconds the previous month.
- Also, returning visitors accounted for 83 percent of all page views, and they spent more time on the site (1 min, 26 secs) compared to new visitors (58 secs). This means the the CMA & GDR tools constitute a "sticky site," in that once visitors discover it, they come back again and again.
- Is is worth noting that the Grade Distribution Report (GDR) was by far the most popular tool, accounting for 73 percent of all visits to the UMBC Blackboard Reports site that contains all "self service" tools and list of Most Active Courses for each semester over the last two years.
While user activity steadily declined as the week progressed, it may be because there were fewer new grades to compare user activity against. This strong interest in grades and related Blackboard activity confirms key findings of a national study showing students value the ability to check grades and gain access to practice quizzes and sample exercises as the most valuable functions in a course management system (CMS) like Blackboard.
July 13, 2009
UMBC Releases Blackboard Reporting Code at BbWorld09
While UMBC is not responsible for supporting any school's use of this software, we welcome any suggestions for improvement through comments to this DoIT news blog post.
FYI . . .
The UMBC code release site also contains a brief "code walk through" video created by Jeffrey Berman, a former graduate assistant in Information Systems, who is now a senior web application developer at Drexel University.
May 8, 2009
Faculty Request: Show Each Other How Good Students Use Blackboard
To help DoIT’s ongoing study of how good students use Blackboard, the Division of Information Technology (DoIT) would like to encourage faculty to post their final grades in the Blackboard grade book using “GRADE” as the column heading (please omit quotes and make the column heading ALL CAPS).
This is a totally voluntary action on your part that will allow DoIT to run a script that determines the average hits per student by grade distribution. DoIT has been reporting final grade distribution by Blackboard activity for the past two years, and we will include SP2009 results after final grades are officially submitted. If you do not want your Bb final grades to be included in the overall, anonymous grade distribution report summary, then do nothing.
As a matter of convenience to students, we know posting of final grades in Blackboard is a wide-spread, informal practice among many faculty now, and should not be construed as a replacement for “official” submission of final grades. But until we can link Blackboard activity and official sources of final grades in the new Student Administration, this voluntary action by faculty is the only way we can begin to see what relationship, if any, exists between student activity and performance.
Note: DoIT is NOT suggesting there is a correlation between Bb activity and grades. However, we are interested in Bb activity as AN indicator of student engagement. As such, we would like to see how good students (as measured by final grades) tend to use Blackboard. We are also interested in seeing whether feedback to all students – by showing their grades against an anonymous summary of their peers – can be effective in helping them be more aware about their own performance during the semester, when they might be able to do something about it.
For more information, visit http://www.umbc.edu/blackboard/reports.
April 7, 2009
UMBC's Retriever Weekly Features Blackboard Reporting Project
January 13, 2009
Chronicle of Higher Ed Features UMBC Blackboard Reports
This week's Chronicle of Higher Education contains a feature story that looks at the "average hits per user" approach behind UMBC's Most Active Blackboard Courses reports, which are published after the last day of classes each semester.
In "A Wired Way to Rate Professors," Senior Writer Jeffrey Young quotes or mentions three UMBC faculty about their "Top 50" rankings as teachers of UMBC's Fall 2008 most active Blackboard courses--across the university or within their disciplines.
While the Chronicle's article focuses on "rankings" of faculty based on Bb activity data (which has drawn mixed reviews from commenters on a companion blog called "The Wired Campus"), it also acknowledges what DoIT staff have maintained since the project began:
Hits alone are no measure of course or instructor quality, but by publishing the activity data each semester, faculty and students can more easily seek each other out about what works or doesn't in using Blackboard.
One thing UMBC faculty and students may still be learning is how an "average hits per user" approach can also shed light on student learning. For example, an examination of 2007-08 Bb-activity based Grade Distribution Reports (GDRs) showed students earning a final grade of D or F tended to use Blackboard 35 percent less than students earning a final grade of C or higher.
This trend--and the tools DoIT staff have been developing to help students and faculty better understand and apply it in specific UMBC Bb courses--will be the focus of "Showing How Good Students Use Blackboard," the first Teaching, Learning and Technology (TLT) Brown Bag Workshop on Thursday, February 12.
January 7, 2009
2/12 Brown Bag to Show How Good Students Use Blackboard
UPDATE: Video Archive Now Available
Consistent with a 2008 national study showing students value checking grades more than ANY single function in a course management system like Blackboard, UMBC's first Spring 2008 Teaching, Learning and Technology (TLT) "Brown Bag" workshop will show how and why UMBC instructors may want to influence AND leverage this obsessive "status checking" behavior.
Specifically, Suzanne Braunschweig and John Fritz will show how she used new myUMBC & Blackboard reporting tools in her Fall 2008 course (SCI100 "Water: An Interdisciplinary Study"), and why the Division of Information Technology developed them.
They will also share results of a SCI100 student survey showing that students are much more inclined to check their own Bb activity before future assignments are due if instructors will post an anonymous Grade Distribution Report (GDR) by Bb activity for past assignments. To view a short, informal video screencast Fritz created to show Braunschweig how to create a GDR by Bb activity, click here.
Note: An examination of 2007-08 voluntary instructor GDRs showed students earning a final grade of D or F tended to use Blackboard 35 percent less than students earning a final grade of C or higher.
This Brown Bag workshop will be held on Thursday, February 12, at noon. For more information and to RSVP, visit www.umbc.edu/brownbag. Light refreshments will be provided to registered participants.
December 15, 2008
DoIT Publishes FA2008 Most Active Bb Courses Reports
DoIT has again published UMBC's "Most Active Blackboard" courses reports for the Fall 2008 semester, based on an "average hits per user" approach. While activity alone is not a measure of quality, DoIT publishes these reports so faculty can seek each other out about what does (or doesn't ) work in using Blackboard.
FA2008 Highlights (based on student activity ONLY):
Most Active Graduate Course: EHS 640 "Fundamentals of High Performance Mobile Health Systems" taught by Stephen Dean, Richard Bissell and Brian Maguire (avg. hits per student: 2,235).
Most Active Undergraduate Course: PSYC 100 "Introduction to Psychology" taught by Linda Jones, Laura Rose and Karen Freiberg (avg. hits per student: 1,365).
Note: Two other sections of PSYC 100 also came in a very close second and third most active of all undergraduate Bb courses.
Note: The "Disciplines" report now allows anyone to sort a department's Bb activity by avg. hits per students across all of its active Bb courses OR by its total number active Bb courses.
Most Active Community: Project Lead the Way managed by Mechanical Engineering Professor Anne Spence.
Total Number of Blackboard Courses (including those with multiple sections): 1,014
For more information about the Blackboard Reports project, contact John Fritz at 410.455.6596 or firstname.lastname@example.org.
August 21, 2008
New SU2008 Bb Most Active Courses (and Disciplines) Report
DoIT has again published UMBC's "Most Active Blackboard" courses reports for the Summer 2008 semester, based on an "average hits per user" approach. While activity alone is not a measure of quality, DoIT publishes these reports so faculty can seek each other out about what does (or doesn't ) work in using Blackboard.
SU2008 Highlights (based on student activity ONLY):
Most Active Graduate Course: EDUC 605 "The Adult Learner" taught by Ciaran Lesikar, Greg Williams, Jeffrey Boham (avg. hits per student: 1,692).
Most Active Undergraduate Course: SCI 100 "Water; An Interdisciplinary Study" taught by Karin Readel (avg. hits per student: 1,135).
Note: The "Disciplines" report now allows anyone to sort a department's Bb activity by avg. hits per students across all of its active Bb courses OR by its total number active Bb courses.
Most Active Community: College of Engineering and Information Technologies Business Manager Search managed by Karen Mattingly, James Milani and Karen Rose.
Total Number of Blackboard Courses (including those with multiple sections): 156
FYI: faculty may also want to try out the new "Average Hits Per User by Final Grade Distribution" reports announced on March 6.
For more information about the Blackboard Reports project, contact John Fritz at 410.455.6596 or email@example.com.
July 21, 2008
DoIT Staff Present Reporting Project at Blackboard World Conference
Last week, staff from UMBC's Division of Information Technology (DoIT) presented "Showing How Good Students Use Blackboard" at the Blackboard Users World Conference in Las Vegas, NV.
John Fritz, asst. vp for instructional technology, and Jeffrey Berman, a former graduate assistant who is now a senior web applications developer at Drexel University, shared the latest development in UMBC's effort to better understand how and why students and faculty use Blackboard.
Specifically, they showed the new Check My Activity (CMA) tool that allows UMBC students to compare their own Blackboard activity against an anonymous summary of their course peers. Combined with a new tool that allows instructors to generate (and optionally publish) a Grade Distribution Report (GDR) for any assignment in the Bb gradebook, it is now possible for everyone in the UMBC community to see how good students are using Blackboard.
Reminder: DoIT does not believe or encourage that using Blackboard produces good students. Instead, the CMA is merely designed to give students another self-assessment tool they may use and act on as they see fit.
For the BbWorld08 presentation (attended by more than 60 people), Fritz and Berman were joined by Deborah Everhart, principal architect of Blackboard's product development team, which has shown interest in the UMBC reporting project.
Next steps will be to study if and how students use the Blackboard Check My Activity tool this fall.
For more information, visit www.umbc.edu/blackboard/reports
May 20, 2008
OIT Publishes SP2008 Most Active Blackboard Courses
OIT has again published UMBC's "Most Active Blackboard" courses reports for the Spring 2008 semester, based on an "average hits per user" approach. For the second straight semester, one department (Information Systems) and one class (PSYC 100 "Introduction to Psychology) have taken the top spot for most active discipline and undergraduate course, respectively. For more information, visit www.umbc.edu/blackboard/reports.
Highlights (based on student activity ONLY):
Most Active Graduate Course: IS 634 "Structured Systems Analysis and Design" taught by Heather Holden, Carlton Crabtree, Carolyn Seaman (avg hits per student: 2,685).
Most Active Undergraduate Course: PSYC 100 "Introduction to Psychology" taught by Linda Jones, Eileen O'Brien, Brian Jobe (avg hits per student: 1,632).
Note: PSYC 100 was also the most active undergraduate Bb course for Fall 2007.
Most Active Discipline: Information Systems (84 Blackboard courses).
Note: Information Systems was also the most active discipline for Fall 2007.
Most Active Community: Geography and Environmental Systems Department managed by several GES faculty (avg hits per user: 533).
Total Number of Blackboard Courses (including those with multiple sections): 1,034
While activity alone is not a measure of quality, OIT publishes these reports so faculty can seek each other out about what does (or doesn't ) work in using Blackboard. Toward this end, faculty may also want to try out the new "Average Hits Per User by Final Grade Distribution" reports announced on March 6.
For more information about the Blackboard Reports project, contact John Fritz at 410.455.6596 or firstname.lastname@example.org.
April 29, 2008
Students Provide Insight Through Fall Blackboard Survey
As we've done in the past, OIT again conducted it's Undergraduate student survey in Fall 2007 with 759 students participating. The survey was divided into several sections including general demographics, an open-ended section for students to recommend instructors that they perceive to be strong users of BlackBoard, tool usage and functionality, and suggestions for improvements.
The most original comments were received when students were asked to recommend an instructor who they viewed as a strong user of Blackboard. Respondents listed numerous faculty members with the following faculty receiving the highest number of positive comments:
- Phil Sokolove (Biology)
- Lili Cui (Physics)
- Clayton Laurie (History)
- Tara Carpenter (Chemistry)
- Eileen O’Brien (Psychology)
Lili Cui and Clayton Laurie have also been highlighted in the Interviews section of the Blackboard Best Practices site as faculty who use Blackboard well.
Students reported the most frequently used tools in Blackboard included:
- Posting Course Content (91%)
- Announcements (80%)
- E-mail (54%)
- Discussion boards (47%)
Major advantages to the use of Blackboard were:
- 24/7 access to course content (78%)
- Prompt visibility of posted grades (56%)
The biggest disadvantage to using Blackboard was the reliance on technology (53%) and the most needed improvements were server reliability and performance (59% combined). Overall, the respondents scored faculty as “Good” (59%) when it came to the use of Blackboard.
Students provided a number of suggestions for improving Blackboard including:
- More online student help
- Mandatory usage of Blackboard for all faculty
- Improved design and functionality
- Improve performance and reliability
Respondents provided a number of suggestions in how to improve Blackboard that included providing more online student help and mandatory usage of Blackboard for all faculty. The respondents overwhelmingly supported Blackboard as a great tool despite its occasional problems. Other input included the suggestion for Blackboard.com to improve its design and functionality and the most received comment was the need to improve performance and reliability. Respondents noted that the system always seems to be down.
From this sampling of undergraduate students, it appears that Blackboard is perceived as an asset to the educational process at UMBC. Faculty that incorporated the use of Blackboard into the classroom setting received appreciation from undergraduate students. OIT needs to continue to address issues that impact usage in order to increase end-user acceptance of this online course management tool.
Complete survey results can be found online.
March 6, 2008
New myUMBC Tools Show How Good Students Use Blackboard
Specifically, students can "Check My Blackboard Activity" to see a comparison of their own activity against an anonymous summary of other students in their classes. In addition, faculty can now run their own "Grade Distribution" and "Tool Usage" reports showing their students' Blackboard activity by any item in the Blackboard grade book (including final grade), or by overall tool usage within the course.
While the trend needs further study, initial findings suggest that students who earn higher grades tend to use Blackboard more than students earning lower grades. It is NOT the case that using Blackboard alone will produce higher grades, simply by "gaming" the system.
But if students know how their activity compares to an anonymous summary of peers, at any time during the semester, this timely and personal feedback may allow them to monitor and change their own behavior as it pertains to engaging with course concepts, materials, instructors or each other. In essence, students can now draw their own conclusions about the quantity AND quality of their Blackboard activity and the impact this may or may not have on their learning.
Since the "Check Grade" tool in Blackboard allows students to view statistical differences between their grades and the class average for any visible item in the grade book, OIT will eventually publish student reports showing activity by grade distribution as well. In the meantime, faculty who wish to can provide further insight to students, by viewing, printing and publishing (as PDF files in their Bb course site) their current or past student activity GDRs for their specific courses.
A year ago, OIT began reporting on UMBC's "Most Active Blackboard Courses" based on a simple "average hits per user" approach. In this way, "rankings" don't favor large enrollment sites over smaller ones. To date, we now have reports for the Spring, Summer and Fall 2007 semesters.
After OIT announced the Fall 2007 Blackboard reports and availability of the "self service" faculty Grade Distribution Reports (GDR), 16 faculty used their GDRs, which collectively showed overall Blackboard usage was higher by students earning higher grades. The self service GDRs complemented earlier pilots with two faculty from Geography and Environmental Systems (Karin Readel and Chris Swan), which also showed higher Blackboard usage among students earning higher grades.
While there is no statistically significant difference among students earning As, Bs and Cs (numerically there is: students earning higher grades tend to use it more), there is a significant difference between this group and those students who earn Ds and Fs. So far, the pattern has held true in the courses mentioned above, and OIT is working on a way to load final grades into a table that looks at all Bb courses to see if the trend is broadly generalizable.
OIT recognizes that hits alone are no endorsement (or indictment) of course quality. But looking at and publishing user activity is one way to help faculty and students identify and consult each other about effective teaching and learning practices. By adopting an attitude of transparency about Blackboard usage, the entire UMBC community (and not just OIT system administrators) gains a "birds eye view" of how this Course Management System (CMS) is being used.
December 14, 2007
OIT Publishes FA2007 Most Active Blackboard Courses Reports
OIT has again published UMBC's "Most Active Blackboard" courses reports for the Fall 2007 semester, based on an "average hits per user" approach. To learn more, visit www.umbc.edu/blackboard/reports.
Highlights include the following (based ONLY on student activity):
Most Active Graduate Course: EDUC 688 "Methodology of Teaching English as a Second or Foreign Language" taught by George Chinnery, Eunju Chen, Margaret Wilson (avg hits per student: 2,075)
Most Active Undergraduate Course: PSYC 100 "Introduction to Psychology" taught by Kelly Forys, Brian Jobe, Linda Jones (avg hits per student: 927).
Most Active Department: Information Systems (90 Blackboard courses).
Most Active Community: "Project Lead The Way" managed by Anne Spence, Mechanical Engineering (avg hits per user: 1,214).
Total Number of Blackboard Courses (including those with multiple sections): 1,074
While activity alone is not a measure of quality, OIT publishes these reports so faculty can seek each other out about what does (or doesn't ) work in using Blackboard. Toward this end, faculty may also want to try out the new "Average Hits Per User by Final Grade Distribution" "self service" report announced on November 30, 2007.
November 27, 2007
OIT Pilots New Report Showing Student Activity by Final Grade Distribution
As announced earlier this year, OIT has been publishing new reports and videos about how students and faculty actually use Blackboard. Now, after piloting with two faculty from Geography and Environmental Systems, OIT is extending the pilot to all faculty who want to see their students’ activity by final grade distribution.
In three of four courses taught by Karin Readel and Chris Swan, “A” students used Blackboard more than “B” students who used it more than “C” students and so on. OIT is making these “self service” grade distribution reports available to faculty to see if there is a similar trend in their Blackboard courses.
These “self service” reports query a “replica” of the main Blackboard server that is updated weekly. Only the instructor of record can query (and view) his or her course’s student activity by grade distribution, if there is a grade book column called GRADE and the grade type is set to “text” display using only A, B, C, D or F.
Note: If you use these reports, they may be added to an anonymous, cumulative summary of all UMBC Bb courses to show student activity by final grade distribution. Unless you give OIT permission to do so, we will NOT publish your specific course’s final grade distribution in the list of most active Blackboard courses.
Future plans include developing a “Check My Activity” link faculty can enable for students that provides contextual feedback on how each student's activity compares to an anonymous summary of all other students in the course at any point during the semester. Students might also be able to “opt in” to receive email, rss or txt updates if their activity falls below a specific (or desired) level.
Reminder: The FA2007 Most Active Blackboard Courses reports will be run on Dec. 12, the day after classes end.
October 16, 2007
GES Faculty Show and Tell How and Why They Use Blackboard
In addition, OIT has published pilot reports for some of Readel's and Swan's recent courses showing student activity by final grade distribution:
While this kind of individual course report is not yet available (faculty can do it manually), OIT intends to provide this service to faculty who wish to view it privately (or eventually post it inside a Bb course for future students to monitor and benchmark their own activity).
OIT will also be publishing a similar, public report that summarizes student activity and tool use by grade distribution in the top 25, 50, 75 and 100 percentile ranges of all UMBC Blackboard courses. The goal is to see what difference, if any, exists in student grade distribution across a range of Bb courses and activity levels.
After watching Readel and Swan show how they use Blackboard in their actual course sites, faculty may want to check out why they do so in two new "Q & A" video interviews on UMBC's iTunesU service (for more information about UMBC on iTunes, see http://itunes.umbc.edu).
OIT plans to publish more "Show & Tell" (how) and "Q & A" (why) videos from faculty teaching active Blackboard courses in other disciplines, but if you or a colleague has an effective practice or insight you'd be willing to share, send email to email@example.com. Please include a short description of the pedagogical problem that is solved or the new learning opportunity that is created in using Blackboard.
Note: To protect the work and identity of students who may appear in the "Show & Tell" videos, only UMBC faculty can access them with their myUMBC userid & password. These videos are intended for collegial, professional development only, so all faculty are reminded that any medium containing identifiable student academic information constitutes an "educational record" that is protected by the Family Education Rights and Privacy Act (FERPA).
October 15, 2007
OIT Staff Present Blackboard Reporting Project at MDBUG Conference
OIT staff members John Fritz, director of Instructional Technology & New Media (ITNM), and Jeffrey Berman (an ITNM graduate assistant) presented the UMBC "Most Active Blackboard Courses" Reports project at the Maryland Blackboard Users Group (MDBUG) conference on Tuesday, October 2, at the UMBC Technology Center.
Proceedings are not yet available on the conference website, but you can watch Fritz and Berman's presentation, "Why and How UMBC Publishes Its Most Active Blackboard Courses Reports," on the Blackboard reports site (Fritz handles the "why" and Berman explains the "how").
Other UMBC presenters included:
Anne Rubin (History): "Child Labor in the American South: Using a Bb Wiki for Historical Research"
Matthias Gobbert (Math/Stats): "Screen Capture of Mathematics with Voice Over using a Tablet Laptop"
Katie Morris (Social Work): "Social Work & Technology: An Unlikely Pair?"
Bev Bickel & Adriana Val (MLL): "Multiple Voices from Online EFL Teacher Education"
For more information about the Maryland Blackboard Users Group, including the opt-in email listserve, visit www.umbc.edu/mdbug.
September 13, 2007
SU2007 Blackboard Reports Now Include Tool Usage
OIT has published the Summer 2007 "Most Active Blackboard Courses" reports for all categories of usage (e.g., all courses, graduate courses, undergraduate courses, communities, etc.).
In addition, we have published the specific "tool usage" metrics for the top 50 courses in all categories. Future plans include providing this for all courses.
Reminder: Course activity alone is not a measure or endorsement of quality. These reports are merely provided as a way for faculty to seek each other out about what works (or doesn't) in using Blackboard.
April 29, 2007
New Report Lists Most Active Blackboard Courses and Communities
Today, OIT is publishing a new set of reports that, for the first time, identifies the Top 50 most active Blackboard courses and communities by a simple “average hits per user” methodology. While there is no implication of “quality” based on activity alone—after all, students could be struggling to find directions for an assignment—these new reports rank activity across all Blackboard courses (e.g., undergraduate, graduate, by discipline) and by all users (e.g., faculty, students). As such, they allow faculty and students to network with one another about what works or doesn’t work in using technology in teaching and learning. The new Blackboard reports are available on the MyBlackboard tab and directly at http://www.umbc.edu/oit/newmedia/blackboard/stats (login required).
If you have any questions, contact John Fritz at 5-6596 or firstname.lastname@example.org.