Section V: D2L Evaluation Post Major Outages

Last Updated: 08/08/2013

Overview

OIT has evaluated several factors related to self-hosting D2L versus continuing to host the system with the vendor. This section of the report contains details regarding evaluation activities related to this. Specifically, this section includes details regarding the Faculty-led OLE Review Committee's activities, interviews with other universities, hardware evaluations, risk analyses and information related to an audit of D2L.

In February 2013, Aisha Jackson, CU-Boulder OIT Program Manager for Teaching and Learning Applications, convened a group of faculty to conduct a faculty-led review of the D2L OLE. The Faculty-led OLE Review Committee comprised members of faculty, staff as well as students from CU-Boulder campus.

Committee Charge

The committee was charged with recommending whether to:

  • Host D2L here at CU-Boulder;
  • Continue to have D2L host;
  • Implement a hybrid hosting solution (if possible); or
  • Migrate to another OLE.

The members of this technical evaluation team included:

  • Faculty representatives from:  the Integrative Physiology Department; the Department of Communications; the LEEDS School of Business; the Engineering Management Program; the Department of History; The Department of Electrical, Computer and Energy Engineering; the Department of English.
  • Graduate student representation from: Telecommunications.
  • Representatives from OIT:
    • Jon Giltner, Director of Enterprise Services
    • David Hamrick, Director of Information Technology Operations
    • Aisha Jackson, Program Manager – Teaching and Learning Applications
    • Larry Levine, Associate Vice Chancellor for Information Technology and Chief Information Officer
    • Kimberly Parker, Program Manager – Continuous Improvement
    • Marin Stanek, Director of Communications and Support

Committee Member Selection Process

Members of the committee were invited to be part of the committee based on their participation in the Faculty IT Advisory Committee; as a faculty participant in the D2L evaluation and selection process; or as a faculty member who participated in one of the recent D2L focus groups.

Information Gathering Activities

After a review of the charge and a discussion about decisions that had to be made, OIT was tasked with collecting information about the risks associated with continuing to host with D2L and self-hosting. The sections below describe the activities OIT undertook to gather this information.

*Note: It should be noted that the OLE Review Committee asked OIT to determine whether a hybrid option – hosting some components of D2L while having some redundancy on the CU-Boulder campus – was possible. During consultation with D2L, it was determined that a hybrid option would cause database integrity issues which would result in course, student, and other data not being in sync.

Interviews with Other Schools that Self-Host with D2L

OIT reviewed the experience of schools that host D2L. Three schools that self-host were interviewed and it included two large, comparably sized state universities who use D2L for both hybrid and online courses and one university that deliveries courses primarily online.

The interview covered questions about self-hosting, which included:

General Information

  • Size of student population
  • Each university’s current version of D2L
  • Whether they have a hosted instance of D2L, and if they ever migrated from a hosted to a self-hosted environment
  • How long they have used D2L
  • The previous OLE if the university had one
  • How many courses they have available in D2L
  • What availability and stability has been like
  • Whether the university has a scheduled maintenance window

Infrastructure

  • The university’s current infrastructure
  • Whether D2L has a standard supported configuration for self-hosting
  • Implementation time from when hardware arrived to having D2L in production
  • Storage
  • Database transactions and size

Resources

  • Number of full-time staff to support infrastructure
  • Training needs
  • Professional services for setup
  • Additional up-front costs and unexpected costs
  • Support from D2L

Conclusions of this evaluation of other schools that self host:

  1. D2L provides universities that are interested in self-hosting with a suggested infrastructure environment. All the self-hosting schools that were interviewed chose to follow D2L’s recommendations.
  2. Although CU-Boulder may choose to self-host in order to have more control over their management of the OLE, D2L would still have access to the system in order to install the application, deploy service packs and hot fixes, and run scripts. Additionally, D2L would need access to troubleshoot the system should need arise.
  3. With respect to support, given the experience of the schools interviewed, if we choose to self-host, we should consider hiring a Technical Account Manager from D2L to facilitate communication and escalation of incidents and requests at D2L.

Server Hardware Recommendations for Self-Hosting

OIT also evaluated server hardware recommendations for self-hosting. They did so in partnership with the D2L vendor. Following is a summary of the recommended server hardware needs to self-host D2L on campus.

D2L provides guidance to its clients that are considering self-hosting on potential server hardware, server software, storage requirements and hosting infrastructure specifications that can be used to host the D2L platform.

For server hardware configurations, D2L indicates these are the key decision-making factors in determining what implement:

  • The number of users: The larger the number of users regularly or concurrently using the system, the more powerful the servers must be. More users typically require an increase in load balanced application servers and a larger database server. Storage requirements for the database and file servers are also increased along with network bandwidth.
  • The level of availability and reliability required: Production systems typically require a higher level of availability than test and development servers, which typically leads to a higher degree of redundancy for the hardware and supporting systems design.
  • Budget availability: Budget is always a determining factor in hardware decisions. For most production systems a reasonable storage, CPU and RAM buffer should be built in, and at least one level of redundancy is recommended for all hardware components. Budgetary constraints may reduce the extent of the buffers, and the level of redundancy that can be acquired.
  • Existing infrastructure: Existing infrastructure, such as storage arrays, virtual servers, enterprise SQL instances, and other technologies may significantly affect the hardware decisions for a D2L installation. As these situations can be unbounded, D2L recommended that OIT discuss these circumstances further with a D2L Consultant.

D2L notes that server configurations can range from standalone server options to fully redundant options. Prior to the procurement of any server hardware, D2L indicates that it is imperative for clients to review with D2L any proposed hardware purchase, and subsequent software and system configuration, to ensure it will meet the needs of the specific installation. CU-Boulder would follow this guidance in the event we select to self-host D2L.

Risk Analysis of Self-Hosting versus D2L Hosting

In order to help inform the Faculty-led OLE Review Committee's decision as to whether CU-Boulder should continue hosting with D2L or move to self-hosting, OIT conducted a risk analysis of each option.

Individuals who currently support D2L, from the Teaching and Learning Applications Program and Communications and Support groups, participated alongside individuals from Network Operations, Shared Infrastructure and IT Security who played a role in D2L’s selection or supported our previous OLE. Additionally, a representative from the Arts and Sciences Support of Educational Through Technology (ASSETT) program attended.

Brent Phillips, CU-Boulder OIT Associate Director for Project and Portfolio Management, led this group through an activity to aid in determining key risks and potential mitigations for continued hosting with D2L and moving to self-hosting. Participants were asked to brainstorm what they felt were the key risks for each option. After risks were identified, they were consolidated, categorized, and duplicates were removed.

The group then discussed each risk in order to identify 5 to 10 key risks to be considered for OIT’s internal discussions and for review with the Faculty Review Committee. Once the key risks were selected, they were ranked according to the probability of the risk occurring (i.e., low, medium, high) and the impact if the risk realizes (i.e., low, medium, high). After the risk analysis meetings, the probability and risk levels were used to identify the appropriate severity level. Risk strategies and suggested actions were determined for risks at the Critical and High levels.

Below is a summary of the findings of the risk analysis:

  • Risks related to D2L continuing to host the campus’ instance of the D2L software include:
Issue Suggested Action to Address Post-Audit Update
Concerns regarding confidence in D2L’s reliability Communicate to the campus how D2L would avoid a future, similar, outage and how OIT will hold D2L to that commitment. During the audit, D2L indicated they had assessed their disaster recovery and business continuity capabilities and are augmenting them through technical improvements; this is a good first step, however if we continue to have them host D2L additional actions would be taken.
The need for D2L to increase robustness of their incident management processes and change management procedures Better integrate D2L’s change management procedures into CU-Boulder OIT’s and enhance review and approval processes between the two organizations. In addition, further clarify and strengthen the overall change management processes and enforcement of it. Post-audit, OIT determined that D2L would need to adjust their change management risk categories to account for customer impact and make additional improvements, including better customer notification processes.
Our dependence on D2L to resolve outages As with any vendor-hosted software, this is an inherent risk that is a factor regardless of whether D2L hosts the software for the campus or if we host D2L onsite  
  • Risk related to self-hosting D2L on the CU-Boulder campus include:
Issue Suggested Action to Address
The potential for issues during migration from D2L hosted software to self-hosting Clarify migration procedures and the personnel needed to support the transition and subsequent system hosting, including tools, training, etc. so that performance is not affected by such a transition; in addition, address any issues related to system changes that D2L makes that would affect our instance of the system.
The potential for significant increases in the costs to self-host (versus D2L continuing to host) Comprehensively estimate the costs for self-hosting vs. continuing to have D2L host the software
Potential challenges with in-house expertise to address service issues Comprehensively determine the personnel and knowledge needed to support the system at all levels
Our dependence on D2L to resolve outages As with any vendor-hosted software, this is an inherent risk that is a factor regardless of whether D2L hosts the software for the campus or if we host D2L onsite
Limited in-house time and resources to support the software on-site Initiate an official transition project to conduct the transition in a planned manner in accordance with OIT project management methodology

D2L Audit and On-Site Visit by OIT Technical Staff

During the latter half of April 2013, OIT technical experts visited D2L to audit the D2L vendor’s services at their site. This additional information was used to further advise the Faculty-led OLE Review Committee in determining a path forward for D2L for the campus.

As a part of the effort to rebuild faculty and student trust in the campus online learning environment, OIT conducted a review of the services provided by D2L. The review focused on an evaluation of information technology processes based on guidelines developed by the Information Systems Audit and Control Association (ISACA). The assessment involved documentation review and discussions with Desire2Learn management staff with authority for specific processes being evaluated. A walk-through of the disaster recovery processes was also conducted. 

The intent of the assessment was two fold; evaluate contractual compliance and help the University understand the risk of continuing to host the online learning environment (OLE) with Desire2Learn relative to self-hosting.

Overall, the maturity of Desire2Learn processes for hosting an OLE environment meet or exceed existing OIT processes.  As such, the University will not reduce risk by self-hosting the OLE.  D2L is making significant investments to mature processes through increased staff, increased training, and other changes focused on enabling continuous improvement.   D2L efforts to mature processes appear to have begun in mid 2012.  It should be noted that if OIT were to decide to self-host, similar investments in staff and training would be required to similarly reduce the risk of service issues.  Because D2L is already positioning to further mitigate existing risk factors, a decision by OIT to host the OLE would require a ramp up period that would initially increase risk for a period of time and then would likely mature behind or in parallel to the processes improving at D2L, and it is not likely that developing our own D2L hosting environment would be cost or time effective.

The information gathered during the D2L evaluation was presented to the Faculty-led OLE Review Committee on July 2, 2013 and shared in the Path Forward section of this report.