Streamlining the logistics of MBT

Happy 2017! I escaped the snow and ice of the Atlanta Joint Mathematics Meetings for the wind and cold (and snow) of northwest Iowa. While at JMM, I had a nice discussion with a few Gold ’14 dots about their implementations of MBT, and as a result, I decided to write a post about how I manage the logistical side of things in my MBT (and often specifications graded) courses, and share some of the resources I’ve developed.

My goal in managing the logistics has always been to make things as simple as possible for me. That might sound selfish, but it’s really more about self-preservation. Your mileage may vary when implementing any of the ideas/tools below, and that’s okay! You need to find the system that works for you and the students in your classes at your institution. Here are some general principles and ideas that have worked for me.

Do not create custom versions of exams and quizzes

I learned this one the hard way. The first semester I implemented MBT in a calculus course, I started offering students the opportunity to take mastery quizzes approximately halfway through the semester. Quizzes were administered one day/week, and students were to sign up for up to two mastery objectives by the night before, so I had time to print off the correct number of copies of each objective. Thus, if only two people wanted to do objective 3, I only made two copies of a page with an objective 3 problem, and space for a solution. If no one wanted to do objective 1, I would make no copies.
There are lots of issues with this approach to logistics, but let me highlight the two that haunt me to this day:
  1. Sorting and copying custom quizzes is overwhelming. My system, at least, was to print the (hopefully!) correct number of copies of each problem, write the students’ names on them, and sort by student before going to class. This was to make sure that everyone got the problems they signed up for, and ensure that we didn’t spend 10 minutes of class time making sure everyone got only the problems they’d signed up for. But this took me at least 20-30 minutes every week, and was a real pain to manage.
  2. Not all students remembered to sign up. Now, this was certainly the student’s fault, not mine, but it did prevent the students who forgot to sign up from showing whether or not they had learned any calculus since the last mastery opportunity. If I had extra copies of problems relevant to the students who didn’t sign up, I would sometimes offer them, but what if there were multiple students who had forgotten? Who gets the benefit despite not following the instructions?
Instead, make your exams/quizzes a just a list of problems tied to the mastery objectives (see an example from Austin Mohr). Include all the problems students have available for mastery, and let them choose the one(s) they need to complete. Leave no space for solutions, but hand out blank paper and ask that students complete their work on the blank paper (making sure to label the problems and turn them in in order).

Spreadsheets are your friend 

I spoke at JMM on implementing specifications grading in abstract algebra, and one of the questions I got (which I’ve gotten in previous MBT/specs grading talks) is how well I think the system would scale to a course with N students in it, where N >> 10. One of the main concerns seem to be tracking student performance on revisions/repeated attempts at the same problem.
My answer is always that it scales really well. To assist in the task of tracking student performance on repeated mastery opportunities, I’ve created an Excel spreadsheet. The instructions are in the file, but the idea is that each assessment gets its own worksheet in which you enter numbers corresponding the students’ performance. Suppose that cell B8 corresponds to Student A’s performance on objective 1; this will be the case in every worksheet corresponding to an assessment. There is then a master worksheet that takes the maximum value of B8 over all the worksheets corresponding to the assessments. The master sheet then counts the number of times a student achieves a passing designation over all the objectives (and also counts the number of ‘high passes’, if your system has such a thing), which you can then plug into an appropriate formula to determine the student’s final grade.


Put a price on out-of-class revision opportunities

Along the lines of the first suggestion, make things easier on yourself by limiting out-of-class revision opportunities. In fact, through Fall 2016, I have not allowed students any out-of-class revision opportunities. In Spring 2017, I will be allowing some in-office revision opportunities, but with a price. As a devotee of specifications grading, my students receive a limited number of tokens each semester (they usually start with 5 or 7, and can sometimes earn more by doing additional optional work). My Spring 2017 calculus students will receive 7 tokens, and can cash in two tokens for one additional mastery opportunity, taken orally in my office. The goal is to the number of such instances (I have 50 students) and to encourage the students to prepare well before coming to my office. I’ll let you know how it goes!
Anyway, these are a few of the things I’ve learned about streamlining and managing the logistical side of running an MBT course. What have you come up with?

One thought on “Streamlining the logistics of MBT

  1. So I too have found ways to help organize my classes. In my Real Analysis class, I rarely give retests outside of their exams, but I have found that for classes like Calculus (and Applied Linear Algebra -coming up!), I do need to give extra retesting chances beyond the exams. Now, I don’t have the luxury to have much time to include retesting exams with in-class weekly quizzes -maybe someday. What I do is a retesting week in between each of the 4 Mastery Exams and then one retesting week during final exams. During these retesting weeks, students can retest any concept, but can only retest a concept once during the week. For example, a student can retest concepts 3 and 5 on Monday and concept 4 on Tuesday, but they couldn’t retest 3 on Tuesday.

    Here is how I manage the retesting:

    1) I make many versions of each exam question -I usually try to do this early when I am creating the concept question for the exam. I keep them in a folder and labeled for particular testing weeks. I make sure there is one concept per page so students don’t see other concepts when they retest.

    2) I send out a doodle link with times in which students can come to my office or the Math Study Tables to retest. Most of the times are during the Math Study Tables and I have math tutors who will proctor the exams (so far I haven’t had issues with this and it is much easier than having them try to find space near my office). I tell students that if no one signs up for a timeslot, I may not be there.

    3) I print off a decent number of copies (more versions of the concepts I know people haven’t missed) and give them to the tutors in the math study tables (they have a guarded folder along with Gateway Integral Tests so it is easy to add these to the folder). The tutors will email me if they are running low -or find me in my office.

    4) The tutors drop the completed exams off at my office or under my door. I collect them and grade them at the end of the week. I keep track of them and the total number of concepts in Bb.

    So far this has worked well for classes between 20-48. I have heard of some instructors who have allowed retests at any time, but I try to make sure students still have to focus because there is only a finite number of chances the get to retest. It is also easier for me to keep track of!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s