Moodle Pilot Focus Group Winter 2009

Yesterday our Moodle Pilot faculty participated in a focus group with the CMS steering committee.We started by identifying the top 3 pros and cons of using Moodle and spent some time discussing those items. The faculty voted on the items, and the these 6 items came to the top:

Pros:

1. Look & Feel
2. Stability
3. Ease of Use

Cons:

1. Grade book was cumbersome
2. Lack of a selective release function
3. Forum tool lacked some functionality

The faculty seemed to universally like the layout, look & feel and organization of Moodle and felt the consistency was a benefit for both students and faculty. Several commented that they had fewer questions about locating course items, and one faculty noted that in his role reviewing courses, Moodle would make it much easier to both design and review Quality Matters courses. When asked if the consistency of the look and feel outweight the limited flexibility, the consensus was yes, especially for students' sake.

Not much needs to be said about stability other than use of CE8 has turned it in to a non-standard feature, but we should point out that there was one brief outage over the weekend that lasted less than 2 hours. Faculty did not themselves experience any issues with being kicked out while working in the LMS: a common occurrence in Blackboard CE8. No students reported this to them either.

Ease of use was also popular. The faculty were "prepared" with a crash-course style training and then left to some extent to their own devices to update and run their courses. Comments from students were similar and that there was little additional support of the LMS needed.

On the negative side, several faculty have had and continue to face challenges working with the grade book. The problems tended to be around set-up and management, though some of those problems were mitigated by switching to a different theme. Another specific item is working with a running grade that shows students their current grade based on the work they've done. Even with staff support, we've had several challenges setting this up.

Faculty also missed the release criteria from CE8. While the hide tool worked fairly well (in some cases better than in CE8), it still required human interaction to release and hide content, and there weren't the same flexible options for releasing content to specific individuals.

The forum seemed to be less polished than the discussion tool in CE8, especially with regards to grading discussion topics, which can be an essential component to many courses. One faculty member even started using a sheet of paper to help tabulate who had done which posts for grading. Another faculty mentioned that there wasn't a comparable option to "create printable view" in CE8 that allowed her to view multiple posts and reply to each individually in-line.

There were a few follow-up questions and discussions that I've summarized.

• Several people used the in-line help and found it useful.
• Grading assignments was greatly improved (even "Awesome")
• Working with quizzes was great, and stability paid off here for students, though there were some issues with the "secure" option for a few students.
• Breadcrumbs were a bit of a nuisance because they didn't include the activity or topic so you often had to go several steps back instead of one or two steps. (note: this has been fixed in the admin settings)
• The lack of a mail tool was an early cause for concern but didn't top the list in the discussion, possibly because we've enabled the eMail module for Spring. The pilot faculty felt that an internal mail module was essential for
• Andy will be contacting new pilot instructors to give them the names of folks from the winter pilot for info.
• Media in courses worked well (maybe better?) with the exception of a single embedded raido player.
• Browser usage was 4 on Firefox, 1 on Safari.
• Superfresh theme is available, one instructor has been using it.
• Several folks used Respondus, but one instructor used Aiken format to import quizzes (and was happy)

Moodle Pilot Evaluations for Winter 2010

We've completed the evaluations from our Winter 2010 Moodle Pilot. Here is a summary of the results from both the student and faculty evaluations. More details from each can be found in links at the bottom.

Winter Moodle Pilot Summary
Summary of Student Feedback
208 submitted, 31 responded (15% response)

Overall, the student response suggested a medium to strong preference for Moodle over Blackboard CE8.  When compared as a whole, there were no categories which students preferred Blackboard over Moodle, though some preferred specific functionality of the Discussion Board in BB CE8 over the Forums in Moodle. However, there appears to be a 2/3rds preference for Moodle among students that responded.

The most common complaint was around forum subscription and e-mails that were sent to their MyPCC account.

Full evaluation from winter term students (pdf)


Faculty Summary

6/5 faculty polled, 100% response rate.

The faculty pilot participants felt that Moodle was easier to learn, use, and is more intuitive than Blackboard CE8, but that both were flexible and allowed them to effectively teach online courses. The format of the faculty evaluation differed in that we asked them a series of questions at the beginning of the pilot and at the end, comparing functions in Moodle to Blackboard.

On the whole, the pilot participants felt that Moodle functioned better than Blackboard in all categories except in "setting up the grade book," "ability to customize course layout,"  "Gradebook," and "Mail."

The biggest cons to using Moodle were

• Lack of internal Mail
• Grading and Gradebook
• Forum functionality/Grading
• Breadcrumbs unhelpful
• Lack of selective release

The biggest pros to using Moodle were

• Look and Feel (consistency)
• Stability
• Simplicity / Ease of Use
• LaTeX

Full faculty evaluation for Winter Moodle Pilot (pdf)

Blackboard 9 Test Drive accounts

Faculty and staff who are interested can request a Blackboard 9 test account. We will likely be requesting a bulk group of accounts when we get to the detailed "hands on" evaluation, or sandbox portion of our process.

Banner Integration

At the February Steering Committee meeting, we discussed options for Banner integration both for our current LMS (WebCT CE8) as well with whatever future product we select. The committee agreed that course creation, course assignment, user account creation, user updates, enrollment, and drops should be handled automatically in the new LMS. Grade publishing is also a highly prized functionality, though not essential at the time of implementation.

From this point on, any products we consider will need to cover the basic SIS integration. I will work with TSS to evaluate any needs on our side, and we will start communicating with Student Records about grade exchange.

Vision of distance learning from Desire 2 Learn and Blackboard

During the last two weeks, folks from both Blackboard and Desire 2 Learn have visited PCC to discuss their vision of distance learning in the next five years, what they see as their place in that future, and what they're doing to get there.

We also asked them to try and address the following areas which we've deemed as important for our own program.

• Integration and interoperability
• SIS for account creation, course creation, etc.
• Content packaging and sharing
• APIs for sharing information between applications
• Assessment and Identity Management
• Collaboration and Portfolio
• Accessibility
• Customization for institutional needs
• Ease and breadth of customization
• Flexibility in teaching and course management styles
• Hosting
• Disaster recover/business continuity
• Service Level
• Product and Company Roadmap
• History and Plan for the future of the company
• Mobile Learning
• User testing and feedback (faculty AND students)

Both vendors were able to talk to each of the points and demonstrate a little of their existing product. Sadly, both sessions included non-disclosure agreements, so this is the extent to which we can share.

December Steering Committee Minutes

CMS Steering Committee Meeting – 12/9/09

In attendance: John Sneed, Larry Clausen, Jeff Edwards, Rebecca Robinson, Chris Hughes, Art Schneider, Andy Freed

Agenda:

1. Set dates for visioning
1. D2L
2. Blackboard
2. Review  themes for Visioning
3. Review message to Div. Deans
4. Review Moodle pilot and evaluations
5. Other  - training for Spring pilot participants

1. Desire2Learn will be in town the week of January 18th, so we’d like to offer stakeholders the opportunity to participate in a vision session led by Desire 2Learn. The Committee would like to do the session 1:30-3:30PM, Friday, January 22nd.  We will try to set up a session in ITV rooms at each campus assuming there’s no issue with the Non-Disclosure Agreement.
1. Schedule ITV rooms at each campus/center
2. John will contact Blackboard to try and arrange for a session with them on the following Friday, January 29th.
   
   
   
   
2. We evaluated the Topic list that we’d like the Visioning speakers to address. Here is the final list.
1. Integration and interoperability
1. SIS for account creation, course creation, etc.
2. content packaging and sharing
2. APIs for sharing information between applications
3. Assessment and Identity Management
4. Collaboration and Portfolio
5. Accessibility
6. Customization for institutional needs
1. ease and breadth of customization
7. Flexibility in teaching and course management styles
8. Hosting
1. disaster recover/business continuity
2. Service Level
9. Product and Company Roadmap
1. History and Plan for the future of the company
10. Mobile Learning
11. User testing and feedback (faculty AND students)
    
    
    
    
3. John would like to give Deans some notice that we’d like to include a review of courses based on our current QM process for all DL courses moving to the new LMS which have not passed review in the previous 3 years.
1. John will contact Steve Ward to try and get on the agenda rather than sending a memo
2. May follow up with more details in a memo to pass on to dept. chairs, etc.
   
   
   
   
4. Andy updated committee on progress of the pilot,
1. Training for Winter pilot instructors is on Dec 15th via Ellumiante. Additional resources have been placed in Moodle, including videos and discussion board
2. Students will have 2 opportunities for online training/intro and one on campus option. December 31st at 3pm and January 4th at 7pm. There will also be one in-class orientation at a time/place TBD.
3. Support will be provided via phone and e-mail by Andy, Donna, Peter, Clayton and a student employee named Brandon. We will cover business hours and hope to offer some additional hours.
4. Evaluation:  Faculty participants will be evaluated twice, once prior to teaching, and again after teaching for a term. We hope to see if there in any change in perception of the CMS, opinions about tools, etc. We will also be doing a focus group during the term to get some more in-depth feedback
1. The Steering Committee would like to take part in this focus group
2. Students will receive a single end-of-term evaluation.

Meeting was adjourned when the fire alarm went off in TCB.

November Steering Committee Minutes

CMS Steering Committee Meeting, November 13, 2009

In attendance: John Sneed, Art Schneider, Rebecca Robinson, Dawn Davis, Chris Hughes, Jeff Edwards and Andy Freed

Moodle Pilot
Andy went over the progress of the moodle pilot

• Support issues
• Training issues
• Migration (largely done by DL)
• Faculty volunteers (6 at this point)
• Agreement on compensation for feedback and evaluation
• Memorandum of understanding – approved by committee
• Posed question “what questions would the moodle pilot have to answer for the CMS committee to recommend moodle without going to formal RFP?"
• Consensus seemed to be that basic functionality was available
• Committee wants to know what tools/features might be given up
• Committee would still like to see other products as well, maybe not in a full pilot but in a bakeoff scenario.
• Faculty will communicate pilot involvement to students, link to info from MyPCC
• Question about training regimen for Part 2 of pilot.

Visioning updates.

• We’d like to have representatives from some commercial CMS providers visit and pitch their vision for CMS platform. 
• If the vision is appropriate for PCC, we may invite them back to do a function demo of their wares.  
• Committee would still like to hear from other institutions who have migrated to each platform recently. Would like to hear faculty opinions of the process. 
• Curious if there are any institutions who have left moodle after adopting it. 
• John and Andy will draft a set of topics and plan for presentations, committee will evaluate next month.

Quality Matters review of all courses prior/during migration.

• Contentious issue since there seems to be no ownership of quality evaluations. Courses that do not pass still get taught. Unclear who determines when courses move on. 
• Committee would like to see a set of technical standards as well as quality standards to meet during a migration (project team) 
• Will SAC pick “master shell” to be migrated? Is it possible to review all these courses when migrating? 
• What’s the timeline that the course has to be reviewed during? (3 years, for example) 
• Can DL say “we won’t migrate courses that don’t meet QM?” If so, how? 
  

Training

• Opened issues for discussion, will continue later. Should training be required, should faculty be compensated? Will it address online teaching or just tools?

For December 9th

• Start technical standards for migration with project team 
• Define vision and topic areas to cover for vendors 
• Identify similar schools that have moved to another LMS, try to get info on “success” 
• Look for any institutions that adopted then left moodle.