One test prep tutor's take on how analytics play a major role in understanding your student's academic needs and shaping a suitable test prep curriculum

In 2005, my life took a turn.

It was in that year that the SAT saw its first major overhaul in more than ten years. Some of the changes included: the end of analogies and quantitative comparison questions, new grammar multiple choice questions, and, most notably, an altered scoring scheme in which the max score was shifted from 1600 to 2400.

At that particular moment, I’d been tutoring for a few years, but had only dabbled in test prep. Amazing as it now seems, there wasn’t a robust demand for one-on-one test prep tutoring. Or rather, I hadn’t found my entry point into that world. Almost overnight, that changed. The revised SAT had parents frantically searching for people to help make sense of the new test, and as a young, hungry tutor who was excited to tackle new material, I was well-positioned to jump in and grow my practice.

Shortly before they administered the first round of the new test, the College Board released a few sample exams. And of course there were prep books touting New Strategies for the NEW SAT. But most people (myself included) didn’t really know what they were doing. Tutors, students, and test prep companies all fundamentally needed kids to try the new types of questions in order for the industry to learn how to beat the test. It was then that I began tracking my students answers to practice questions in Microsoft Excel.

While I would love to claim I saw the long-term potential in embracing technology in this way, that would be a lie. My chief interest was automating the intricate scoring process, yet another aspect of the exam which had been altered (read: complicated) in the test overhaul.

As my Excel skills grew, I came to understand a critical tech lesson: processes you’re certain to repeat (particularly the more tedious and time-consuming ones) are the low-hanging fruit of innovation. That is, streamlining those actions with tech can produce instantaneous rewards in the form of time saved. And more often than not, those small tech choices give rise to other, more advanced innovations that can save even more time.

In 2005, I was quickly rewarded for my automation. Not only could I quickly and easily score tests, but I was also able to instantly identify students’ errors, which made me a more efficient in-session tutor. Even more crucially, my tech venture represented a marketing tool for prospective clients who wanted to know what differentiated my tutoring from other independent tutors’. Suddenly, I had an actual data-driven system to improve outcomes.

My first scorer looked something like this:

Since then, my journey has been toward more advanced analytics over a wider array of tests including the SAT, ACT, SAT II’s, AP exams, Regents, GMAT, and more. Nowadays, the tech output looks like this:

Let’s be clear: there are pros and cons to incorporating technology into tutoring.

Chief among the cons is the idea that the interpersonal interaction between tutors and student will get short-changed amidst the tech. When tutors try to deploy a poorly designed tech tool within tutoring sessions, this is a very real possibility.

In my experience, however, the benefits of incorporating tech far outweigh the costs. And this is especially true within test prep -- an undertaking that is fundamentally about a final score, and thus a process for which score projections and progress tracking are essential. If tech can help tutors bring about better score outcomes, then customers (both kids and their parents) stay happy and those tutors get referrals. In other words, everyone wins.

With all that in mind, I’d like to talk about the two phases of data and test prep that I currently employ and why I’ve found each to be so important.

NOTE: If you’re already in the test prep game, this will be immediately relevant. If you’re not currently tutoring students for standardized tests, but you’re curious, there might be some valuable insights in here. If you’re a parent who is currently (or may one day be) looking for a high quality test prep tutor, then perhaps you can use some of the following to inform your search.

Pre-tutoring assessment

For most independent tutors, referrals are king. When someone calls or emails mentioning another client you’ve worked with, there is a strong temptation to get the tutoring going as quickly as possible. For 10+ years, this was me.

But when it comes to test prep, I’ve learned the hard way that it’s worth taking a breath before diving headlong into a new tutoring gig. Here’s why: many students/parents have unreasonable expectations of score improvement. And if you don’t address this early on, it's a matter of time before you have an extremely angry client on your hands. Put simply, the work itself with those clients is likely to be frustrating and unfulfilling.

For this reason, I insist that every prospective student complete a self-proctored mock exam before we discuss rate, schedule, and general logistics.

This accomplishes several things:

1. I get a read on where the student currently sits score-wise.
2. I learn something based on how long it takes the student to turn around the assessment.

Students who take weeks to complete the pre-tutoring work tend to have issues completing the homework I assign them, which will absolutely eat away at their final score. Any honest test prep tutor will acknowledge that the single biggest determinant of score is homework output.

The analytics I now run on the pre-tutoring assessment allow me to differentiate between careless errors and expected misses, as well as between lucky guesses and expected answers. All of the work before tutoring has begun helps me set reasonable score goals and chart a highly customized path for each specific student.

Practice Test Analytics as a Component of a Test Prep Curriculum

Once the tutoring begins, it’s essential that the student tackle several full-length practice tests. Because these tend to be unpleasant for most kids, I usually beginning with worksheets and smaller homework assignments.

As soon as I feel the student is ready for a full length practice test, I assign, score, and analyze it. In my case, the engine for the analysis is Excel, and the key question I’m after is how that exam performance aligns (or doesn't) with my pre-tutoring assessment data. Running the numbers on a student’s proficiencies across sections, question types, and test position lets me know if my instruction is getting through. If it isn't, I’ll tweak my methodology so that I can put that student in a position be even more successful on the real thing.

It can be humbling when you realize a student hasn't grasped a concept you feel you've explained well. But sticking with your explanation/pedagogy/system when it clearly isn’t working does nothing for anyone. And without tech, test prep is largely high-stakes guesswork: an awful pairing.

--

With the pre-tutoring assessment and a data set on each student that's constantly updating, I can home in on the content that most needs attention and which promises the greatest potential score improvement.

Above all, tech allows me to customize my work to the unique needs of each student in a way that would be virtually impossible by hand. And customization is the unstated promise of one-on-one tutoring.