Answering some Questions about OER

This week I was asked by a reporter at my school newspaper about Open Educational Resources. I wrote up my responses to her, and figured in the spirit of openness, I'd share them here. The reporter's questions are in bold.  For those who don't know, WAMAP is the same platform as MyOpenMath, and the one that people at my school are more familiar with.

Firstly, what made you interested in creating an online textbook, and was it difficult to do?

The first book I did, for Math 107, came about because I was teaching the course online.  I really hated that my students were paying $150 for the book in a terminal course they really didn't want to take anyway, especially since it is a topics course and I was only using about half the book.  Somewhere around 2008 I moved the homework for the course online in WAMAP, so I wasn't even using the text for homework questions anymore, and that cost really seemed ridiculous.  I figured I could write a decent replacement for the chapters I was using, so I sat down and did it.  I can't remember how long that first version took, but it wasn't too horribly long.  I started using it, and figured I might as well share it in case anyone else would find it useful.  Over the years, I've added several more chapters, some totally by myself, and some by building off of work given to me by other faculty at other schools.  I was able to do a refinement and add some more chapters as part of the Open Course Library grant (described better below).

The second book I worked on together with Melonie Rasmussen, for Math 141 and 142, was connected with the Open Course Library grant from the state community college system.  The grant was technically for building a course using existing open resources, not writing a book, but we couldn't find anything existing we liked, and Melonie and I had always wanted to write a free book for that course, so we figured it was a good opportunity to do it.  That grant gave us each 1 course release from teaching for a year.  The first draft for the book for 141 took me about 6 weeks of 2-3 hours a night at Starbucks, and about the same for the 142 book.  Melonie put in a comparable amount of time revising, adding "try it now" problems, etc.  Luckily, we were able to use existing WAMAP questions for homework - we didn't end up adding exercises to the book for several months.

So was it difficult?  Yeah, it was :)  But the effort we put in meant that other faculty at Pierce, Green River CC, Shoreline CC, UCLA, Scottsdale CC, etc. didn't have to put in that same effort; they could simply take what we created, make any changes they needed to fit their college's course, and start using it.  And that's exactly what's happened.   The 141/142 book has saved students around the country at least $500,000, probably quite a bit more.

Also, do you feel that your textbook works just as well as a print book?

To be clear, open doesn't have to be an online vs. print thing.  Many students do buy bound printed copies of our open text, which is sold in the bookstore, or available on Amazon for $15.  And some students assigned commercial texts decide to buy ebook versions of them.  The issue is really commercial vs open:  $100+ print and $60+ ebook, vs ~$15 print and free ebook.

To the question, I certainly do think it works just as well as a commercial text, though obviously I'm a biased source :)   Luckily I have some data to back that up.  For the 107 book, I only have a little data (about 450 students), but it's shown a steady (but not statistically significant) increase in student success for students using that text.

For the 141/142 text, we compared about 5000 students from Pierce, Green River, and Shoreline using the text in 2011-2013 to about 5000 students who had used a commercial text the years prior, and saw no statistically significant difference in success rates.  I'd love to be able to say we saw a huge jump in success, but frankly, I'm fine with no change since we saved those students $300K in the process.

There's a lot of data out there for other math courses and courses outside math, and all of it seems to suggest that students can do just as well, often better, with open resources than with commercial texts.  I think there are two main reasons:
1) Students have access to the materials day 1, so they're not getting behind on their homework or reading while waiting for their financial aid check to come in.
2) The instructors often have collected and customized the materials to exactly target their desired course outcomes.  This also means the instructor is more connected to and excited about the material, since they have some ownership of it.

Finally, do you feel that OER would be possible for all kinds of courses?
Of course it would be possible, but not everyone has the time or ambition to create a book for the fun of it :)  Happily, a lot of progress has been made in existing open resources, but whether they are sufficient for faculty is a different question.

I generally see that there are 3 kinds of instructors:

The first are the makers - the people who like to create unique learning experiences for students.  These folks are really well suited to OER, and are probably perfectly happy to just not use a book at all.  Particularly in disciplines like social sciences and humanities, I know some instructors have chosen to ditch a traditional text altogether, feeling they can create a more meaningful experience for students by combining readings and articles from a variety of sources.

The second group are folks that want a complete textbook for any of a variety of reasons, often driven by a course that needs a strong reference text.  For them, things have come a HUGE way in the last few years.  When I released by first version of my 107 text, it was one of the very few open texts out there (and one of a very few that would be considered mainstream).  Since then, tons of grant-funded projects have released really high quality OER material for a ton of disciplines.  There are now dozens of courses that have high quality complete open textbooks available, and there are dozens more in the pipeline.  Math now has open textbooks for almost every course.

The third group are folks that rely on publisher materials.  Usually when we adopt a commercial book, the publisher will supply us with complete solutions manuals, test banks, powerpoint slides, online homework systems, etc.  Some faculty rely heavily on this kind of material, and as you can imagine, most open textbooks don't come with this kind of stuff.

We've tried to address some of that in math through WAMAP, an open online homework system for math that can be used with open textbooks.

The open textbook organization OpenStax College got enough grant funding for their project that they're trying to build some of those ancillary materials for the books they're creating, and is teaming up with commercial companies for the online exercises.

There are also efforts, like the Open Course Library and the Kaleidoscope Project, that have worked on developing "complete courses" using open resources, in the hopes of filling some of those needs.  The Open Course Library had mixed results, with some courses being very useful, and some not so much.  The Kaleidoscope Project courses were created a bit differently, with a stronger focus on adoption.  Some of them have been quite successful in adoption beyond the original creators, partly because of the services of the company providing support for the project.  (disclosure:  I've been on leave working with that company for the last year).

A Practical Overview of LTI

A Practical Overview of LTI v1.1

Learning Tools Interoperability (LTI) is a standard developed by IMS Global to facilitate integration of learning tools with learning management systems.  At its essence, LTI is a security protocol, built on top of OAuth that provides a method for a secure trusted launch of a learning tool sharing some course and user information from the LMS to the tool.  This post will the basics of working with LTI, as well as explore some practical considerations that arise in various use cases.

Getting Started

At the core of LTI is the consumer key and secret.  These act like a username and password, and are typically agreed on by the LMS and tool via phone, email, or other such process.  When the tool is set up in the LMS, the administrator or teacher will provide the key, secret, and launch URL.  When launched, the LMS sends a POST request to the launch URL containing a number of parameters, including the consumer key and a signature created by OAuth using the parameters and the secret.  When the tool receives the POST, it can use OAuth to verify the signature using the secret, thereby ensuring the parameters were not tampered with or sent from someone not knowing the secret.

The actual process of doing the signatures is laid out well in sample code for several common languages provided by IMS on Google code.

In the POST, the consumer key is sent as oauth_consumer_key

Use Case 1:  LTI as a glorified link

If a tool did not actually need any data from the consumer, and the LTI link is really acting like a glorified link, the tool could completely ignore the key, secret, and POST parameters altogether.

Use Case 2:  Course Level connection

A tool like Tegrity might want to list all the Tegrity recordings for a specific course when launched.  For that, they need to link the LMS course with their local course identifier.  This is done through the launch parameter context_id, which provides a unique identifier from the LMS for the course section.  This parameter is recommended, not required, but is usually sent.  This is often used in combination with context_title, a description or title for the course, and/or context_label, which is a short label or name for the course (like MA151).

To determine if the user should having editing rights, the tool can look at the parameter roles, which provides a list of roles for the user.  Consult the specs or sample code for how to make sense of this field.

If the tool needs to connect LMS users with user accounts on the tool, then the parameter user_id can be used.  Name can sometimes be obtained from lis_person_name_given, lis_person_name_family, and/or lis_person_name_full.   Be aware some LMSs send none of this information depending on link configuration.

Tool Workflow:

• Use oauth_consumer_key and context_id to look up the course in local DB
  • If found, direct the user into the connected tool resource
  • If not found, create a tool resource, and record the connection in local DB
  • If user accounts are need:  Use oauth_consumer_key and user_id to look up user in local DB
    • If found, login that user
    • If not found, create a new user account, record in local DB, and login that user

Use Case 3:  Ephemeral links

In the previous case, there would normally be only one link from the LMS to the tool.  Some tools, like Etherpad, are intended to be used multiple times in a course, with each link taking the user to a separate Etherpad document.  In this case, there is typically no need or desire for the links to persist (point to the same document) if the course on the LMS is copied.  For this purpose, the resource_link_id parameter can be use.  This ID is unique for each placement of the tool in a course, so can be used to connect an LMS link with a unique item on the tool.

Tool Workflow:

• Use oauth_consumer_key and resource_link_id to look up link placement in local DB
  • If found, direct user to the appropriate tool page
  • If not found:
    • If the tool is something like Etherpad, where a new link should point to a new blank resource
      • Create a new blank resource, record the connection in local DB
      • Direct user to blank resource
      • If the tool is something like a content reader, with a set of content items that can be linked to
        • Provide a GUI for a user (with instructor role) to select a content item from a list.
        • On selection, record the connection in local DB
        • Direct user to that resource

Use Case 4:  Persistent content links

A tool like a textbook reader, assessment system, or learning object repository may wish to set up links in the LMS which point to a content item on the tool, for which those links will continue to point to the same item if the course on the LMS is copied.  There are two ways to do this, though the specific approach that can be used depends on the LMS.

Approach 1:  Use query strings on the launch URL to indicate the content item.  This approach works in Canvas, Moodle, Desire2Learn, and BlackBoard (see caveat below). 

Approach 2:  Use custom parameters.  These get sent with POST parameters as custom_{name}.  This approach works in BlackBoard (again, see caveat below).  I’m not sure about D2L and Moodle, but it does not work in Canvas.

BlackBoard caveat:  Bb only allows custom parameters or link-specific URLs to be specified when using the “this link is to a Tool Provider” option for web links, and only when the credentials are set up by the instructor.  When a system-wide tool is set up, the options to specify the link or custom parameters is hidden.  Likewise, if a tool is set up using Bb’s tool provider and tool placement mechanism (which adds the tool to the content building menus), URL and custom fields cannot be specified.  This makes it essentially impossible to have a system-wide tool and have persistent content-specific links.

Canvas addendum:  Canvas has an LTI extension called content selection.  Using this, the instructor creates a new LTI tool placement, the tool is called and provides a way for the instructor to select a content item.  The tool then returns a unique URL to Canvas which gets stored as the launch URL.  This provides a way to have a GUI for setting up persistent content links.

Tool Workflow:  (not including Canvas extension)

• If the tool wants to provide a user-customizable version of the indicated resource, on first launch the tool could make a copy of the indicated resource, and record an ephemeral link placement (with the downside that the customized version will not be retained on an LMS course copy) or some other approach
• Direct user to the indicated resource

Use Case 5:  Grade return

Tools that deliver assessments may want to return a score to the LMS.  The LMS will indicate it’s willing to accept a score by sending lis_result_sourcedid, which is a unique identifier that the LMS can use to determine both the user and link, and lis_outcome_service_url, a URL the tool uses for sending back the score to the LMS.  The details of the format can be pulled from the spec or sample code.

In Canvas, tool placements are only gradable if they are set up as Assignments and a point value is assigned.

In Blackboard, unfortunately if web links are used for tool launches and the tool credentials are set up system-wide, there is no way for an instructor to indicate that a link is gradable, which is frustrating for tools that need persistent content links.

I have not had a chance to test grade return setup in D2L or Moodle.

Tool Workflow:

• Record the lis_result_sourcedid and lis_outcome_service_url when the link is launched in local DB, associated with tool resource and user.
• When the grade needs to be updated, use those fields to return the grade to the LMS

Use Case 6:  Lots of content links

A content reader might want to provide an LTI link for each chapter of the text, or an assessment system might want to set up links for each assignment.  For a teacher to have to create these would be very time consuming and difficult if a list of unique URLs have to essentially be copy-pasted in.

To address this, the Common Cartridge format provides a means for importing LTI links.  The details are beyond what I’m going to go into here, but I will address some of the challenges.

In Canvas, this process works well, though if a system-wide key and secret have already been set up, the instructor will have to delete the tool created during the import process.  Canvas has added extensions to the specs that allow the common cartridge to indicate if an item is intended to be gradable and the points possible.

In Blackboard, this process works fine.  The LTI links come in as web links with the “this link is to a tool provider” option.  Unfortunately, as above, if the tool credentials are set up system-wide, there is no way for an instructor to indicate that a link is gradable.  Even if the tool credentials are set up in the course, there is no way to indicate in the cartridge that the item is gradable or the points possible, so the instructor still will need to manually make those changes for every link.

In Desire2Learn, this process does not appear to work at all.

In Moodle, this process worked when I tried it in Moodle 1.9.  I haven’t tried it in Moodle 2.x yet, nor have I explored the grade return setup.

Using Canvas's API for global search and replace

So, this is a little outside my usual topics for blogging, but I always appreciate it when I find helpful code snippets online, so wanted to share this.

Today, I needed to go through 100 content pages in Instructure's Canvas LMS and make some search-and-replace type changes, but the kind that are best done using regular expressions.  I needed to rewrite image URLs, remove some links, etc.

So in case you ever need to do the same, here's the quick PHP script I threw together.  There is probably a more elegant way to do this, but I was in "get 'er done" mode :)

Why I Do It

Every so often, particularly after working with faculty who are resistant to considering open resources, it is nice to have a reminder of why we do it.

Today I got a greeting card from a class of students at a college in California who have been using my Math in Society open textbook and the associated MyOpenMath exercises and videos I developed for the Open Course Library.  Receiving this kind of response is uplifting, so say the least.  This is the second card I've gotten from students of this instructor, and I secretly hope it's not the last.

View Full Size

A Primer on Creative Commons Licenses for OER

So there have been a TON of overviews written about the Creative Commons open licenses, but I often find myself writing out explanations for faculty of the implications of the various licenses, so figured I'd write it into a post so I can just refer to it in the future.

The Creative Commons (CC) suite of licenses have become the standard set of licenses to use for Open Educational Resources (OER) such as open textbooks, open courseware, videos, etc.  These licenses take copyrighted materials from "all rights reserved" to "some rights reserved"; they explicitly grant users of the contents some basic rights (the four "R"s):

• Reuse:  The right to reuse the material in its original form
• Revise:  Make changes to the material
• Remix:  Combine the material with other materials
• Redistribute:  Share the material with others

For most users, the last is the important one:  That the materials can be freely used and shared, without having to ask permission.   Most of the time this also means free-of-cost, unrestricted access to the materials.

There are three basic "flavors" of CC licenses that are commonly used.  I'll detail each, and it's upsides and downsides.

Creative Commons Attribution license (CC-BY)
This is the most permissive of the licenses.  In addition to the four "R"s above, it gives the user the rights to use the content in any way they'd like, so long as they attribute (give credit to) the original creator.  This license is favored and required by many granting organizations and legislative bills.

On the upside, the content is the most remixable and reusable.  There are no restrictions to what you can do with the content.  However, this also means that a commercial company could take the content, add a bunch of cool stuff to it, and release their new version under a non-open, fully copyrighted license, so long as the attribute the original creators.  Some consider this "encouraging innovation," but most faculty seem to consider it exploitation of their work, which is why I've found faculty rarely choose this license on their own.  As a content creator, though, I love finding CC-BY work, since it means I can easily remix it into my own work without having to worry about license incompatibility.

Creative Commons Attribution-ShareAlike license (CC-BY-SA)
This license adds a restriction to the CC-BY license, stating that any revisions or remixes that are distributed must by under the same CC-BY-SA license as the original.  This license is "viral," forcing derivative works to remain open.  This license is the one used by Wikipedia.  This is similar to the GNU Free Document License, and the GPL (GNU Public License) which many open-source software projects use. 

On the upside, the content is remixable and reusable, so long as you're willing to have the derivative work under the same license.  This license does permit commercial use, so a company could try selling your content for $200, but since the content has to be released open, anyone could buy 1 copy of that work, then turn around and freely and legally redistribute or resell it.  This is enough to dissuade most companies from doing anything profit-mongering with the content, but at the same time discourages commercial innovation around open content.  Personally, I'm OK with that tradeoff.

The downside to this license, in some people's eyes, is its viral nature.  While many, like myself, feel this promotes openness, other argue it stifles freedom since it restricts people's use of the material.  It can certainly cause difficulties when trying to create a new work remixed from content with different licenses.

Creative Commons Attribution-ShareAlike-NonCommercial license (CC-BY-NC-SA)
This license adds a non-commercial use restriction to the license.  This means the four "R"s are still allowed, and you can freely redistribute the original or a derivative work, but you can not do so for commercial gain.  This license is used by MIT OpenCourseWare, and bunches of other materials.

On the surface, most faculty love this license.  They want their work to be free and open, so of course they want to restrict non-commercial use!  A lot of the time, this makes sense.  The NC license is fine for many things, but there can be issues.

For me, the biggest issue has been around printing.  If the content is NC licensed, am I allowed to having a print-on-demand company make copies of that book for my bookstore to sell to students in the bookstore?  I general I personally think this is fine, as my motive in printing it is not for commercial gain, and the printing company and bookstore, while making money off the transaction, are just executing their normal business roles.

The CC license itself is not specific about the definition of non-commercial, and suggests that users contact the content creator for clarification.  Unfortunately, asking permission is exactly what the CC licenses are suppose to help us avoid.  Many content creators are fine having their works printed and sold, but I've talked with other authors who are adamant that non-commercial means that no one should be making money.  Because of this, I steer clear of NC books unless the author themselves has listed it on a print-on-demand site, but even then, my ability to revise or remix the book has been stifled, since I don't know if I can get my remixed version printed legally.

Less of a concern to most faculty, an NC license could restrict the use of OER materials in a course offered by a for-profit company or college.

So, what should I use?
My general recommendation is this:

• If you don't care if someone uses your materials commercially, use the CC-BY license, as it grants the most freedoms, and is the least restrictive.
• If you want to prevent commercialization, use the CC-BY-SA license to discourage commercial use, while avoiding the issues of the NC license.  And, as a bonus prize, your license forces any derivative work to be open.
• If you're hyper-worried about commercial use, use the CC-BY-NC-SA license.  But please consider adding a clarifying statement saying whether you are OK with people doing things like printing your books, and charging enough to recover the costs of printing.  By explicitly stating these things, it saves people like me the headache of guessing or having to contact you.

Arithmetic Content Sprint

I was greatly inspired by the Finnish group of mathematicians which hacked out an open math textbook in 3 days, by Siyavula's content sprints, and by Boundless's recent physics book hackathon. 

So, when some unexpected funds came along, I suggested we hack out an open arithmetic/prealgebra book over a weekend.  I solicited help from faculty involved in the Washington Rethinking Precollege Math project, who I consider to be thought leaders on lesson study, faculty inquiry, persistence, and innovative instructional strategies.  I also called up my friends and colleagues down in the Maricopa college district, who I consider to be leaders in the move to bring open education resources (OER) into the classroom, especially at the developmental level.  They also have been doing a mass re-visioning of the dev math curriculum, based on the CCSS and carefully researched learning trajectories.

After a couple online planning meetings, it became clear that the group was not interested in writing a traditional textbook.  Partly, this was recognition that the books largely serve as instructor guide nowadays, and most student's only use the book narrative as an occasional reference, turning to in-class lecture or online videos when they get confused.  So, we planned instead to attempt to create  a set of resources covering the spectrum of what an instructor and student would need to teach and learn arithmetic:  Topic intros, contextual motivations, concept development, guided practice, interactive reference (like videos and animations), static references (written examples), practice problems, and wrap-up activities.

We wanted to both create materials that students would directly consume, but also create materials for the instructor.  For example, it is good practice to use manipulatives to develop conceptual understanding, and to show alternatives to the standard algorithms for doing calculations, but not all instructors know how to do these things.  Rather than write a text that explains it, we figured videos for the instructor might be more useful.  Some of these could either serve the instructor or the student.

So from Dec 18 - 20, 2012, a group of 4 faculty from Washington, myself included, traveled down to Arizona and worked with a team of faculty from across the Maricopa district.  Predictably, you get a group of very excited and passionate teachers in a room together, and quickly the "work session" turned into a very valuable sharing and exchange session.

But, in addition to the new ideas and inspiration we got from our colleagues, we were able to produce a number of exciting products.  You can find our official collection here, but a lot of people are still working on items, or have large collections of items that didn't make sense to add this this listing site (notably, Scottsdale's workbook), and many of the other items from their SCORE site.

So while we didn't manage to write a book in 3 days, I'm very excited about the foundation we laid towards building quality and innovative open resources for arithmetic and prealgebra.

Comparing modes of video instruction

Derek Muller, creator of the Veritasium science videos, did his dissertation research on videos for physics education.  He describes in his research and in this video that when presented with a clear video explanation, students really liked the video but their scores did not improve from pre-test to post-test.  But, students given videos that address common misconceptions did see improvement, even though they found the videos confusing.

I figured the same was likely true in math, and certainly Derek implies that he believes it is.  I set out, mostly for fun, to try to replicate the experiment with a math topic.  I chose "adding fractions with unlike denominators" because it's a topic that students of all levels struggle with, from my arithmetic students up through my calculus students, and a topic where there are a lot of longstanding misconceptions.  My hope was that by addressing those misconceptions, it would help students more than just showing them the correct method.

While I was at it, I decided to also compare a pure arithmetic approach to one supplemented with a manipulative demonstration - fraction bars in this case.  So, I created three videos, one purely arithmetic, a second that builds on the first with a manipulative demonstration, and a third that builds on the first by addressing two common misconceptions up front, showing why they don't work visually with fraction bars.

What follows is the methodology and results.  The TL;DR is "no significant difference."

I solicited subjects through Twitter and through WAMAP.org and MyOpenMath.com.  The majority of the subjects were community college students whose teachers asked their students to participate, some offering extra credit or other incentives.  The survey began with some demographic questions, then launched into a pre-test consisting of 4 questions adding fractions (details below).  Students were also asked to rate their confidence in their answers.  They were then presented one of the three videos.  After the video, the students took a post-test, again rating their confidence.  The final page of the survey asked them to rate the clarify of the video on a Likert scale, and gave a free response box for leaving feedback.  Order of the pre and post test and video assignment were randomized.

I received about 270 responses.  After filtering out incomplete surveys and participants who clearly didn't watch the video, I was left with 197 useable results.  For each student I computed their improvement from pre-test to post-test (scores out of 4), then computed the mean improvement for each experimental group.  The mean improvements where 0.25, 0.242, and 0.265 respectively, each with standard deviation around 0.8.  Long story short, the data did not provide evidence that the video shown made a significant difference.

I must, of course, admit that my study design is not ideal, and that results might be seen if the study had been done with the assessments in a controlled environment, where students were forced to complete watching the video, or with a more targeted subject group.  While I am disappointed in the results, since I do believe addressing misconceptions is a good idea, it does also raise the question of whether the video was just completely worthless.  If I repeat the study, I may add a control group that is asked to watch some non-math video between the two tests.

For those curious the pre/post tests contained these questions:
Version A:  1/9 + 4/9,  1/4 + 1/8,  1/3 + 1/5,  2/3 + 1/6
Version B:  1/7 + 2/7,  1/2 + 1/4,  1/3 + 1/4,  2/3 + 1/9