Arithmetic Content Sprint

I was greatly inspired by the Finnish group of mathematicians which hacked out an open math textbook in 3 days, by Siyavula's content sprints, and by Boundless's recent physics book hackathon. 

So, when some unexpected funds came along, I suggested we hack out an open arithmetic/prealgebra book over a weekend.  I solicited help from faculty involved in the Washington Rethinking Precollege Math project, who I consider to be thought leaders on lesson study, faculty inquiry, persistence, and innovative instructional strategies.  I also called up my friends and colleagues down in the Maricopa college district, who I consider to be leaders in the move to bring open education resources (OER) into the classroom, especially at the developmental level.  They also have been doing a mass re-visioning of the dev math curriculum, based on the CCSS and carefully researched learning trajectories.

After a couple online planning meetings, it became clear that the group was not interested in writing a traditional textbook.  Partly, this was recognition that the books largely serve as instructor guide nowadays, and most student's only use the book narrative as an occasional reference, turning to in-class lecture or online videos when they get confused.  So, we planned instead to attempt to create  a set of resources covering the spectrum of what an instructor and student would need to teach and learn arithmetic:  Topic intros, contextual motivations, concept development, guided practice, interactive reference (like videos and animations), static references (written examples), practice problems, and wrap-up activities.

We wanted to both create materials that students would directly consume, but also create materials for the instructor.  For example, it is good practice to use manipulatives to develop conceptual understanding, and to show alternatives to the standard algorithms for doing calculations, but not all instructors know how to do these things.  Rather than write a text that explains it, we figured videos for the instructor might be more useful.  Some of these could either serve the instructor or the student.

So from Dec 18 - 20, 2012, a group of 4 faculty from Washington, myself included, traveled down to Arizona and worked with a team of faculty from across the Maricopa district.  Predictably, you get a group of very excited and passionate teachers in a room together, and quickly the "work session" turned into a very valuable sharing and exchange session.

But, in addition to the new ideas and inspiration we got from our colleagues, we were able to produce a number of exciting products.  You can find our official collection here, but a lot of people are still working on items, or have large collections of items that didn't make sense to add this this listing site (notably, Scottsdale's workbook), and many of the other items from their SCORE site.

So while we didn't manage to write a book in 3 days, I'm very excited about the foundation we laid towards building quality and innovative open resources for arithmetic and prealgebra.

Comparing modes of video instruction

Derek Muller, creator of the Veritasium science videos, did his dissertation research on videos for physics education.  He describes in his research and in this video that when presented with a clear video explanation, students really liked the video but their scores did not improve from pre-test to post-test.  But, students given videos that address common misconceptions did see improvement, even though they found the videos confusing.

I figured the same was likely true in math, and certainly Derek implies that he believes it is.  I set out, mostly for fun, to try to replicate the experiment with a math topic.  I chose "adding fractions with unlike denominators" because it's a topic that students of all levels struggle with, from my arithmetic students up through my calculus students, and a topic where there are a lot of longstanding misconceptions.  My hope was that by addressing those misconceptions, it would help students more than just showing them the correct method.

While I was at it, I decided to also compare a pure arithmetic approach to one supplemented with a manipulative demonstration - fraction bars in this case.  So, I created three videos, one purely arithmetic, a second that builds on the first with a manipulative demonstration, and a third that builds on the first by addressing two common misconceptions up front, showing why they don't work visually with fraction bars.

What follows is the methodology and results.  The TL;DR is "no significant difference."

I solicited subjects through Twitter and through WAMAP.org and MyOpenMath.com.  The majority of the subjects were community college students whose teachers asked their students to participate, some offering extra credit or other incentives.  The survey began with some demographic questions, then launched into a pre-test consisting of 4 questions adding fractions (details below).  Students were also asked to rate their confidence in their answers.  They were then presented one of the three videos.  After the video, the students took a post-test, again rating their confidence.  The final page of the survey asked them to rate the clarify of the video on a Likert scale, and gave a free response box for leaving feedback.  Order of the pre and post test and video assignment were randomized.

I received about 270 responses.  After filtering out incomplete surveys and participants who clearly didn't watch the video, I was left with 197 useable results.  For each student I computed their improvement from pre-test to post-test (scores out of 4), then computed the mean improvement for each experimental group.  The mean improvements where 0.25, 0.242, and 0.265 respectively, each with standard deviation around 0.8.  Long story short, the data did not provide evidence that the video shown made a significant difference.

I must, of course, admit that my study design is not ideal, and that results might be seen if the study had been done with the assessments in a controlled environment, where students were forced to complete watching the video, or with a more targeted subject group.  While I am disappointed in the results, since I do believe addressing misconceptions is a good idea, it does also raise the question of whether the video was just completely worthless.  If I repeat the study, I may add a control group that is asked to watch some non-math video between the two tests.

For those curious the pre/post tests contained these questions:
Version A:  1/9 + 4/9,  1/4 + 1/8,  1/3 + 1/5,  2/3 + 1/6
Version B:  1/7 + 2/7,  1/2 + 1/4,  1/3 + 1/4,  2/3 + 1/9

My thoughts on Flat World Knowledge "free to fair"

So as you no doubt already know, Flat World Knowledge announced last month their plans to stop free online access to their books starting January 1, 2013.  At the risk of sounding very cynical, my first thought when I read the news was "told you so."

From early on, it felt like FWK was using openness as a marketing gimmick.  I actually loved the idea of their iTunes-like model, where they try to upsell students on $0.99 add ons like quizzes and worksheets, while giving the book away for free.  But I found it horribly frustrating that they had their open content intentionally set up to be difficult to move off their platform.  Open licenses give four basic rights: reuse, revise, remix, and redistribute.  FWK's platform made redistribution nearly impossible.  Their "Make it your own" platform allowed for remix, but only within the confines of their system.  Any other use required extensive copy-paste or web crawlers.  Legal and allowable, but annoying.

In some ways, I feel that FWK's move to charge for their content is more in line with how they were running things already.  I am concerned about what they are going to do with their license.  As David Wiley noted recently, the area of charging for open content is a "no man's land," since if someone were to pay to access the content, the user would have rights to copy that content and share it outside the paywall.  However, if you read Jacky Hood's comments, featured on FWK's blog, she writes

We also believe that open materials are ‘free to be improved’, not necessarily ‘free of charge.’ [...] While some thought leaders would prefer that all materials be licensed CC BY without additional restrictions (NC, ND) and without cost, Creative Commons licensing allows both restrictions and payments.

While she is right, of course, that CC licenses allows NC restrictions and allows for collection of payments, it does not allow for FWK to prevent distribution outside their paywall.

In their original announcement, FWK said

To most of you, what matters is not how we license our content, but what you can do with it.  [...] there are some licensing issues and options that we are currently discussing with our own advisors, as well as leaders from Creative Commons and other OER organizations.  We will share more details as decisions are made.

Personally, I don't see what all the fuss is about.  If FWK owns the copyright to their works, then they can grant users the right to make changes to it, with or without an open license.  Publishers have been doing this for years, albeit not to the same depth of changes.   Michael Boezi expressed that a dual license had been considered, but this wouldn't prevent faculty from redistributing the content they were granted a CC license to.  I really don't see how they can avoid dropping the CC license at this point.  If anything, putting CC content behind a paywall would just encourage redistribution outside of their system, undercutting their sales opportunities.

Mind you, I'm very happy FWK will continue to offer low cost textbooks - that is a very good thing, and will hopefully have a positive effect on the market.  But I feel kind of bad for all the open textbook advocate groups that worked so hard to make FWK what it is today, who now will have to go purge all those FWK off their open textbook lists.

Building on "Better online math"

In his post "Better Online Math", Dan Meyer shared an example of his vision for improving online math video + exercise sites by adding in a challenge problem at the beginning.  I really like the idea, and I like his implementation, but it felt like the video was missing interactivity.  This was echoed in some of the comments on his post, and even in Dan's own reflection:

There were at least five different moments over that five-minute lecture where I wanted to stop, pose a question, or have students work for awhile.

As I shared a few days ago, I've been playing around with cuing questions based on videos, and this seemed like a great way to extend both what I have been working on, and what Dan had been working on.  I took his video, edited in a few short pauses to make timing the questions a little easier, and built some questions that felt natural based on what was happening in the video.  There probably would be room for more.

Try it out.

At this point, the "5 of your classmates' answers" data is all bogus, but it's certainly something that could be added in down the road without too much trouble.

Making real-world problems "real" with video modeling

While at the AMATYC conference a few weeks ago, I saw a nice talk on teaching a linked chemistry and algebra course.  It got me thinking more about an idea I've had for a while, to bring physical experiments into my math classroom.  I used to do this for linear regression in precalculus, and have been inspired a lot by Frank Noschese and other physics "180" blogs.  One of my concerns was the difficulty and time involved with setting up an experiment in class.

Then it occurred to me - what a great use for video!  In many ways, it's really similar to Dan Meyer's "three-acts" - using a video for motivating a question, then using the same video to provide the Act 2 data.  So, I shot a few of these and made a document suggesting a line of questioning that could go along with each. 

But I also figured this was a great way to test out the video-cued assessment feature I recently added to MyOpenMath, so I created a series of assessments that pop up questions during the video.  One uses Tracker to track a dropped ball and form a model, and eventually use that to predict how high a balcony is by dropping a ball off it.  If you want to try some out, visit MyOpenMath, login with username: guest, and open any of the items in the Video-based modeling folder.

I also create a series of single questions that use videos to provide visual real-world context to some basic questions.  It's amazing how much it changes the feel of the question.  It's one thing to say that a chain has a volume of 5 cm^3 and a mass of 38 grams, and ask the student to calculate the density, but it's quite different to show the volume measurement then ask the question.

Next goal:  figure out a way to adapt the "give an estimate then look at the class's estimates" approach that Dan Meyer, Dave Majors, and Riley Lark are exploring to work in MyOpenMath.  My initial concern is how to address the first few students, before much data has been collected.  Perhaps seeding the pool with a handful of responses would be enough to make it work.

Engaging students with interactive videos

Dan Meyer recently wrote a great post about online math websites.  Most of MyOpenMath's courses with videos also follow the model of video + exercises, or more often, exercises + video (where the video is there for the student to watch if they can't figure out the exercise).  I've never felt too bad about this, since our focus is on supplementing a face-to-face classroom, not replacing instruction.

That said, with more teachers exploring flipped class, hybrid, and fully online models, I want to explore options for making online video-based instruction the best it possibly can.  I really like Dan's idea of opening with an initial challenge.  But I also wanted to find some way to making the actual video instruction more engaging.

I recently took a few "MOOC" courses through Coursera and Udacity.  I noticed that I found the Coursera courses, which often have long blocks of video broken up with fairly stupid multiple-choice questions, fairly boring.  In contrast, I found Udacity's courses more interesting.  In those, the instructor explains a general concept and asks a question, usually before showing any examples.  The student is then expected to take a crack at it.  If they can't get it, it's not a big deal, since the next video goes over the solution.  I found this general approach much more engaging, and prompts a higher level of thinking vs just replication.  It also helps that each video segment was short.

So, I sought out trying to replicate this idea on MyOpenMath.  Happily, Class2go, Stanford's new open-source MOOC platform, had some great code written for interacting with YouTube's API.  I was able to use this as a starting point to get something working.  See here for a a video of an example, or  to try it live, visit MyOpenMath, login with username: guest, and open the Demo 1 item.  In this example, I am assuming we just finished talking about writing algebraic expressions (so the first question is review), and that I'm introducing students to graphing.  Like Udacity, I try to follow the pattern of explaining what we want to do, then ask the student to try it before really detailing how to do the problem.

Next goal:  To expand this idea, adding in some of Dan Meyer's "start with a challenge" approach.