SCC LMS Evaluation Team

Desire2Learn - is the LMS selection

noreply@blogger.com (Melissa) — Mon, 11 Feb 2008 18:38:00 +0000

Report Brief Regarding The Work of The LMS Task Force and Its Recommendation To District Educational Technology Committee:
February, 2008
Marsha Leeman-Conley and Greg Beyrer (co-chairs)

Background:
Both college and district strategic plans have identified distance education and especially online education as important components of our services to students and the community. To achieve the vision expressed in these plans, we must provide a consistent and reliable online learning environment.

History of Choosing a Learning Management System:
In Spring 2001, the district undertook an evaluation of available alternatives for providing a district-wide Learning Management System (LMS) and the Ed Tech Committee recommended that the district to purchase a license for Blackboard. Blackboard was installed on one server array located at the district office and shared by all colleges. In Spring 2004, another review of LMS alternatives was undertaken and the recommendation to the Ed Tech Committee was to continue to renew the Blackboard contract.

Growth in Online and Web-Enhanced Courses:
Access to the Blackboard LMS has resulted in very rapid growth in fully online classes and in the number of web-enhanced classes. To summarize this growth, the district has gone:

From 31 course sections in Spring 2001 to over 2,300 course sections in Spring 2008.
From 8,950 student enrollments in Fall 2001 to over 73,000 student enrollments using the system in Spring 2008.

Blackboard Performance Problems:
Performance issues on the Blackboard system have been intermittent over the years and have varied on different versions of the software. In recent years, it became clear that the poorly designed software and database architecture becomes non-functional and unresponsive under high user demand. Other large institutions have reported similar problems. Another issue has been the poor quality of tech support provided by Blackboard. These problems have become more frequent over the past year and have resulted in unacceptable performance and unscheduled system outages. While continuing with Blackboard in our current configuration and hoping that things will improve remained an option, it also seemed prudent to investigate other options.

LMS Evaluation Process:
To start the process of identifying LMS options, the Educational Technology Committee asked a small group:

to investigate which LMS options met our criteria for Technical Viability,
to propose a process for further evaluation and a timeline for moving forward,
to propose a set of criteria for use in the subsequent evaluation process, and
report back to Ed Tech at first meeting in Sept. for further action.

The process used by the group over the summer of 2007 was:

to locate other large educational institutions,
to identify which LMS they were currently using,
to determine if their level of utilization was comparable to Los Rios use of Blackboard,
to gather information from them on a variety of criteria of relevance to our decision.

This process resulted in a large amount of useful information from other large institutions and identified six possible options for further investigation. A set of criteria for evaluation was prepared by the group along with a process and timeline for continuing to evaluate LMS options.

These options included:

Blackboard status quo
Blackboard ASP (hosted by Blackboard rather than operated locally at Los Rios)
Angel Learning – A vendor licensed package of products designed specifically for large enterprise institutions.
Desire2Learn - a vendor licensed package of products in use by many large institutions.
eCollege - a vendor licensed package of products in use by many large institutions
Moodle – an open source product used by large and small institutions
Sakai – an open source product developed by a consortium of educational institutions

Process Used by the LMS Task Force for Evaluating Learning Management Systems:
(during fall semester 2007)

A process for evaluation of alternatives and a set of criteria to use for evaluation were developed.
Each college identified potential members of LMS Task Force (faculty, staff and administrators) and these names were forwarded to Academic Senate and Classified Senate for appointment to LMS Evaluation Task Force. College Senates endorsed names OR selected others members to serve on the Task Force.
LMS Task Force began meeting in October and elected co-chairs (Marsha Leeman-Conley and Greg Beyrer)
A sample class was developed in Blackboard and archived so that each LMS option could be presented using the same content.

This sample class would be recreated in each of the other LMS options.
If there was a conversion tool available for each LMS, then it was used to convert the same Blackboard archive into the new system.
Details of the steps need to either convert the archive or recreate the content was documented for each option.

A generic “sandbox or playground” site was also set up in each LMS option so that Task Force members could try out the system without impacting the sample class site.
During the same period, access to the demo sites on each system were provided to all faculty so that they could also provide simultaneous online evaluation of each system.
A survey form was created to capture this feedback which would be provided to Task Force members before they make their recommendation.

Recommendations of the Task Force:
On January 25, 2008, the last meeting of the LMS Task Force was conducted and the group prepared a recommendation and report to the district Ed Tech Committee for their meeting on January 31. The recommendations included:

That the district cease using Blackboard as its primary LMS as of June 2009.
That the district contract with Desire2Learn as its primary LMS.
That the ETC and the district discuss a way for faculty to explore the innovations possible using open source LMS platforms, such as Sakai and/or Moodle in the future.

Proposed Timeline for Transition from Blackboard to Desire2Learn:
Throughout Spring semester 2008

On-going support for Blackboard faculty and students
Acquisition of recommended system (or changes in existing system)
Configuration of new system for pilot courses in Summer 2008.
Identification of faculty willing to participate in early use of new system
(ideally, a mix of online, hybrid and web-enhanced classes)
Begin Train the Trainers workshops at each college
Preparation of training materials and support materials for new system
Training of pilot faculty through accelerated program of face-to-face workshops at each college
Training of Help Desk and college support staff
Conversion of existing Blackboard archives to new system course sites
Online and web-conference training also conducted at regular intervals to train additional faculty

Summer 2008

Intensive training of faculty during summer via all delivery methods
(workshops, online and self-paced)
Summer Intensive training sessions before summer semester begins and perhaps during summer.
Data and feedback gathered from summer faculty using the new system

Fall 2008

Majority of faculty have converted to new system
New system switched to largest server array, Blackboard given secondary resources
Communication to remaining Blackboard faculty about need to participate in training on new system during Fall 2008.

Spring 2009

Last semester when faculty can still use Blackboard rather than new system
Identification and training of remaining faculty using Blackboard

Summer 2009
1) Only new system available
2) Blackboard contract dropped

Direct Comparison of Blackboard and Desire2Learn:
(as of January 2008)

**Key Differences between Blackboard and Desire2Learn**
	Blackboard	Desire2Learn
High volume performance data	Very poor performance for large volume clients - documented at Los Rios and many other institutions due to software architecture and code issues	Many high volume large institutions use D2L – many that are much bigger than Los Rios
Client Satisfaction	1) Many large colleges choosing other options, 2)Many former Web-CT switching at end of license instead of converting to Bb	1)No institution has ever installed D2L and then switched to another LMS since founded in 1999, 2) All current customers we contacted were planning to renew.
Customer service and support	Very poor record in Los Rios and many other colleges report similar problems.	1)Best of those we evaluated, 2)Recently received award for Customer Service
Conversion from Blackboard	None required	Best process of all options – approx. 30 minutes to convert a sample class from Bb to D2L
User Interface	Clean and some customizing possible	Clean and very customizable
User Interface for Content:	Folders-style	Has both folder and outline views
Online Quizzes/Tests	1) Many problems with performance – Students kicked out if click wrong or if system overloaded, 2) Instructors cannot customize security options	1)Can handle very high volume without problems 2) Student interface clearer, 3) Quiz Security optional – set by instructor
Forum Display Options	Limited options – always collapsed, always oldest first	Flexible options – instructor can choose collapsed, threaded, newest or oldest first, can “pin” a posting at the top
Forum Grading	Good – can see posting and replies for grading	Good - can see posting and replies for grading – can link to rubric or other assignments also
Timed Release	Good – can choose dates and times	Better – can choose dates and times but can also interface with calendar and gradebook easily
Conditional Release	Good – but many instructors get confused when creating “rules” for release and have problems	Better - easier interface to choose options, can link to rubrics or SLOs
Browser Check	None	Built in
Gradebook Display Options	OK	Good – easier to collapse across categories, can repeat names and assignments across screen
Gradebook – usability	Poor – limited options	Good – more options such as drop lowest quiz, better choices for grading schemes and grade weighting options
Assignment Submission	Good	Better – easier for students & instructors to use.
Assignment Grading/Comments	Good	Good
SLO Tools	None	Good – built in customizable rubrics and SLO options
Multiple File Upload Process	Only single files can be uploaded – instructors have to Zip content and then unzip in Bb	Folder upload – no zipping needed

Desire2Learn Pros and Cons:
Desire2Learn Pros:
1. Scalability - used by large institutions based on stats has good performance,
2. Customer Satisfaction - Happy clients - current large higher- education customers contacted are happy and planning to renew
3. Customer Service - Vendor provides responsive support and set-up assistance as part of licensing
a. Recently won awards for customer service and innovation in eLearning
4. Conversion from Blackboard - Bb archives conversion very easy – 30 minutes to be ready with Sample Class
5. Student interface - clean and user-friendly, outline view or collapsed
6. Instructor Interface – intuitive with efficient workflow for common tasks
7. User Home page – information on what’s new available before enter a course site
8. Built in browser and system check before users login
9. Content Availability – timed and conditional release features easy to use,
10. Quiz performance -- Good Data on high-volume utilization without problems
11. Quiz display - easy for students to see where they are in quiz
12. Quiz security features optional –
a. can use Respondus lock down browser
b. instructor can choose which security options to apply
c. print and other features can be disabled
13. Discussion Boards have more display options
a. Can choose preferred view of forums – collapsed, threaded, reading
b. Can hide by date OR lock by date
c. Can “pin a message” in top position – useful for “Read me first” posting by instructor even if board is displayed different ways or sorted
d. Can rate postings, can moderate forums, can move postings
e. Can include original message in reply
f. Can "collect" for print view which displays in order of posting
g. Can view message history and also search and sort
h. Students can save drafts of messages before posting
14. Forum grading available - shows multiple responses can grade directly from forum
15. Communication - Instant Messaging and email both built-in,
16. Chat tool and whiteboard built-in but LiveRoom (Virtual Classroom) with many more features is well-designed but extra cost
17. Assignment submission - well-designed for instructors and students
a. (automatically adds student info to file during upload)
b. Can sort by those who haven't submitted yet and send email direct from Folder to individual students or groups about assignments –
i. can’t do this in Blackboard
ii. can do this in Moodle
iii. can do it in Angel but can also automate this process as in Angel
18. SLO Tools - customizable rubrics and SLO tools integrated at no additional cost
19. Multiple section sites possible - can have separate gradebooks
20. Switch roles – instructor can switch to student role or even impersonate a particular student in the class to see exactly what they are seeing
21. Gradebook very flexible but not overly complex
a. Instructor has ability to customize display options
b. Organized by categories so gradebook is more easily organized and viewable (for students and instructors)
c. Can repeat user names across the screen and item names on long student list so easier to navigate and know where you are (you can choose how frequently to display each
d. Can show running total without non-graded items then switch to automatically add zeros for un-graded at end of semester
e. More complex grading options available (auto-delete lowest or multiples) and options for grading schemes available
22. File Management – can upload multiple files using WebDav direct from desktop computer
23. Groups features – can quickly create groups and randomly assign set number of students to groups or can quickly manually enroll them or switch them from one group to another
24. Broken Link Viewer – shows any links in course that are not working
25. WIKIs, Blogs, RSS, podcasting - built in to Desire2Learn and not Blackboard
26. Content Management (Learning Objects Repository) integrated and available for additional cost
27. Statistics Tool - Data for user statistics stored off of production database but still accessible for reporting without slowing down production system
28. No reports of performance problems even with all options turned on and all tools enabled
29. e-Portfolio available at extra cost but then is fully integrated
Desire2Learn Cons:
1. Turnitin integration won’t be available until April 08
2. Some interface anomalies in Firefox browser (at times can't see buttons needed, text doesn't wrap appropriately). Vendor reports these problems fixed now.
3. Cost - Yearly license (higher price than Blackboard but comparable to Angel)
Demo site for D2L: http://82showcase.desire2learn.com

Blackboard - Pros and Cons

Blackboard Pros:
1. Conversion process from Blackboard – none required
2. Student interface - clean and user-friendly –folders instead of outline view like Moodle or D2L – means more clicks to see what is inside folders
3. Communication – email with multiple attachments – also see below under Cons
4. Quiz Issues – Not all question types in Blackboard available in other options -- only Blackboard has HotSpot questions – also see Cons below
5. Discussion Boards: - also see Cons below
A. options for display – only collapsed and collect – instructor can’t set preferred
B. can search across postings and across forums
C. draft feature not well designed – hidden even from authors
D. subscribe feature still doesn’t work correctly
6. Forum grading available – can see multiple postings (threads and replies) and grade easily
7. Content Release by date and time
8. Conditional Release for content and assignments by user, group or conditions
9. Assignment submission easy to use for instructors and students
A. Can do in-line comments on assignments or download in groups
10. Turnitin integration available
11. User Home page – information on what’s new available before enter a course site (if each user enables What’s New manually) but no way to enable for all users
12. ePortfolio available at extra cost

Blackboard Cons:
1. Scalability – significant performance problems experienced at Los Rios and reported by many other institutions during high volume of transactions
2. Customer Satisfaction - many large higher-education customers switching to other systems – most former WebCT customers switching rather than staying with Blackboard
3. Customer Service – many institutions reporting poor quality support on Bb listserve
4. Quiz Issues:
A. security features can’t be disabled and “wrong” actions by students mean they are kicked out of quizzes and instructor must reset quiz and student must start over
B. Exams – NOT possible to have calculated math questions where actual numbers differ for each student
5. Gradebook - very limited in options, clunky interface, easy to misuse weighting of grades
6. Communication - internal message feature badly designed and no Instant Message tools
7. Virtual classroom has very limited tools – no presentation options, no document or application sharing
8. File Management - Instructors can only upload one file at a time so must zip folders into archives and then unzip after upload (time consuming and error prone)
9. Content Management System only available at very high extra cost
10. Calendar feature – poorly designed and time consuming to use – not integrated with other tools
11. SLO tools - Not available for our version and only at extra cost for newer versions
12. Wikis, Blogs, RSS and podcasting – NOT built in
13. Time zone -- students cannot set if outside of our area
14. Statistics tracking of student activities slows performance so must limit data retained

Taskforce Members

noreply@blogger.com (Melissa) — Fri, 09 Nov 2007 23:38:00 +0000

SCC LMS Review Taskforce Members:

Melissa Green, Instructional Development Coordinator
Jory Hadsell, Distance Education Coordinator
Chris Seddon, IT Committee Faculty Tri-Chair; Mac/CAD Lab Coordinator
Ken Fitzpatrick, Engineering Design Technology faculty
Andranik Divanyan, LRC/IT staff
Rhonda Rios-Kravitz, LRC Dean
Elaine Ader, IT Dean

District Team Members - Draft

noreply@blogger.com (Melissa) — Tue, 09 Oct 2007 02:04:00 +0000

SCC: Chris Seddon and Ken Fitzpatrick

FLC: Jennifer Kramer and Gordon Lam

ARC: Andrea Garvey and Rob Diamond

CRC: TBA

District Representative: TBA

The College Blackboard Coordinators and/or DE Coordinators since they are appointed by virtue of job title.

Acrobat - Delivers Online Content

noreply@blogger.com (Melissa) — Tue, 25 Sep 2007 18:29:00 +0000

No matter what LMS software is used, Acrobat can deliver all types of documents as one download instead of many file types each with it's own viewer. Acrobat can also embed multimedia files. Acrobat Reader can now save what users type on forms and participate in shared reviews of documents.

Converting files to Adobe Acrobat before posting online has several advantages:
Instructors keep the layout of the printed page

multiple source files combined into one pdf file
full color graphics
web site links
interactive features
full text search

Students can interact with the document

by adding comments & stamps, drawing or markup
reading & sorting comments from other students
clicking buttons to show hidden content
filling-out & submitting forms to email or databases

Resources hosted by Adobe:

Delivering on the Promise of eLearning by Ellen Wagner, Adobe Systems
Acrobat User Community
Acrobat in Education Blog

Other university examples:

Staff Development - Acrobat Pro 7 Training at Virginia Tech

Local examples of Acrobat documents include:

Counseling Educational Plan - form used by all counselors on campus and at outreach centers
LRCCD Paycheck Description - highlighted text with detailed comments
SCC Bb Course Request - online form sends details as email & to a database

LMS Evaluation Team Volunteers Needed

noreply@blogger.com (Melissa) — Tue, 25 Sep 2007 16:31:00 +0000

Interested SCC Nominees Need to Volunteer for the Learning Management System Evaluation Team.

Over 450 Sacramento City course sections with over 14,200 student enrollments use the Blackboard Academic System in support of online, hybrid and web-enhanced classes - currently, 109 online sections have 4100+ enrollments.

Every computing system should have periodic review with consideration for growth or alternative approaches. The Los Rios District has begun the evaluation of Blackboard and other alternative LMS options. This detailed review is motivated by dissatisfaction from some of our experienced faculty teaching online and hybrid courses and student frustration.

The LMS Evaluation Team, assisted by District Office Information Technology staff, will work through October and November. At this time, meetings are planned on Fridays with additional time required to complete the evaluation tasks. Participating faculty will need to make a commitment to evaluate all options and attend all meetings.

The evaluation team will recommend renewal of the Blackboard Academic System contract or select an alternative LMS product. Part of the evaluation is to consider how current course content will be copied to a new system and if publisher content will be available. The final report and recommendation from this team will be forwarded to the district-wide Educational Technology Committee and then to Chancellor Harris for action.

Yes, this is a fast-track project and an important opportunity for you to actively influence the next Los Rios learning management system.

Faculty who do not serve on the LMS Evaluation Team will have opportunities to view demonstration course sites for each product and to submit feedback to the team.

Interested faculty should contact Linda Stroh as an SCC nominee. The list of nominees from all four colleges will be reviewed and appointments made in consultation with campus Academic Senate Presidents.

Partial list of Learning Management Systems for review include the following (links are to the product descriptions): Blackboard, ANGEL, Desire2Learn, Sakai and Moodle.

Watch for the SCC LMS Features Survey to distinguish between courses requiring LMS features and courses that want to deliver content online.

(Announcement previewed by the SCC Academic Senate on September 18, 2007 and SCC Information Technology Committee on September 14, 2007)

-- Melissa Green Instructional Development Coordinator
-- Jory Hadsell, Distance Education Coordinator