Introducing “Beyond the Test”

Lightly edited for readability. Beyond the Test is a conversation, so please check the audio for exact wording.

Cold open, an excerpt from later in the conversation:

Matthew Pietrafetta: But there had been a longstanding trend from, like, 1990 to, say, 2020, in that 30-year period, the average high school GPA, I think, went from 2.7 to 3.2. So it went up half a point, right? But at the same time, if you look at the corresponding data of ACT scores, trends and SAT score trends and NAEP, you know, the nation's report card test, those performances for those same students, right? Either were flat or declining. So how do you how do you correlate the two?

The full conversation begins here.

Andrew Ferguson: So as Academic Approach is celebrating its twenty-fifth year, and we're also starting this podcast, I thought it would be best if the audience got a chance to know you, Matthew. And we're gonna make this a little bit of an interview and a little bit of a conversation between two guys who've worked together now for twenty years. And so I guess the best first question would be take us back to 2001. What like what made you want to start a test prep company. Was there something you were trying to fix or solve? Or was it or what was the beginning of it all?

Matthew Pietrafetta: Well, I appreciate the question and I appreciate you asking it because to your earlier point, the last twenty of these 25 years of Academic Approach have been spent with you as a great colleague and friend. And it's been great, you know, the work we've done together. So that's where we got started 20 years ago. I did found it in July of 2001. But I got my start in test prep five years before that as a graduate student, 1996, 97 in New York City at Columbia. And I was teaching in the core curriculum there while earning a PhD in English. And the core curriculum at Columbia was like you know, introductory courses for the students to kind of boot camp them through like skills based instruction, logic, rhetoric, grammar, get them ready to write for the tenured faculty and build their skills. So I was teaching those classes, designing. Those lessons. So that was really fun to start lesson planning and engaging young people with skills-based curriculum. At the same time, I got my start sort of a side job as a tutor for an Upper East Side test prep group. And I was working with high school students. So I was working with, you know, sophomores and juniors preparing for the SAT and subject tests and the ACT and so forth, even though the ACT in New York at the time was not a very popular test. And so I was teaching those high school students while also teaching those freshmen in the core curriculum classes at Columbia. And two things really hit me. One, I started to look unlike in when I was 17, the SAT and ACT didn't mean much to me. But as a graduate student teaching and tutoring the tests, I started to look at them and say, wow, a lot of the skills that these tests are assessing, critical reading, reasoning, reading comprehension skills. Flexible math problem solving skills. These are a lot of the skills that I'm the logical skills and the essay composition skills I'm teaching the freshmen of Columbia. So what if you started to build a curriculum around these tests that was a bit more serious? Back in those days, I'd go into Barnes & Noble and look at, you know, “Crack the SAT” books and Barron's books were just coming out then with like shortcuts and tips and tricks. And the thought was, and that's where the name Academic Approach came from, is what if you built skills-based curriculum. Around these tests that was thoughtful, serious, deep. And you started treating those high school sophomores and juniors like they were freshmen at Columbia and say, look, this is important material to learn. It can improve your performance in the classroom in high school. And you need it when you get to college to write and reason and argue at a high level. And that was kind of the birth of the idea. And then just quite simply, I really just fell in love with mentoring teenagers. Just adolescents are so open and willing to be coached. And it's such a great position, the role of that we have with young people because we're not their parent, so we can't ground them. We're not their teacher at school, so we can't give them an F. So we're this like safe audience where they can ask questions, they can be open and available to learn and be curious and not have to worry about being, you know, raising their hand and asking a silly question. So I really fell in love with that, creating that safe space for students to learn and being a coach and being a mentor. And really trying to pull them up to be as successful as they could be academically, with that kind of that bar in mind of what I was teaching Columbia freshmen and what was expected of them, that became kind of the bar for what I wanted to get out of my high school students and preparing them, you know, using the test as a means to that end, get the scores up, but also get the skills up so they could be so they could be as successful as they could be and get off to college and be really prepared to succeed.

Andrew Ferguson: Did you do you did you feel like when you made that transition from New York to just Chicago that the same sort of that families and students were kind of were just as worried about the same things and worried you know, worried about test prep in the same way,

Matthew Pietrafetta: Yeah, you know, it's so interesting and to think about to think back to the students that I was coaching, you know, 30 years ago and then five and then 25 years ago, and now you know, students I'll meet with today. It's kind of just a universal through line that cuts through it all, which is adolescents and as a father of adolescents and you know, 22 year old, 17 year old, seven year old, just kids in development, they're looking for coaches and mentors to help them answer questions. Help them understand themselves, help them self-reflect and self-correct and improve in a safe environment. And that has not changed a bit. Now the Barnes & Noble tips and tricks, the “Crack the SAT” books, have become ChatGPT, which is a totally different landscape to operate in, as you well know, in terms of how students are like looking for support materials and getting answers. But the adolescent brain, the adolescent, you know, mentality, the psychology, it has not changed. It's still adolescents need mentors, variety of mentors to come in and you know, give them the opportunity to learn in a safe space, to self reflect, to ask questions, to be pushed and coached and have high expectations for them to kind of reach that bar. And I think of it now, like the teachers in school with my own kids who have taken the time to get to know them, who've developed a personal relationship with them, personalized the instruction to them, who give them great feedback that they can learn from, who challenge them. Like you value that so much as a parent. And it's that's always been the case. And I think that's one of the reasons why to the extent that we've been able to keep doing the work that we're doing at a high level, We do continue to do it because we care about serving that role in the lives of adolescents. And parents deeply care about people who will do that for their children. Because you know, I've often joked, you've heard me say this million times, if adolescents listen to their parents, we wouldn't have jobs, right? Because this is a certain point you don't want to hear it from mom and dad. You wanna but you do you are looking for trusted mentors who can push you, teach you, support you. And that's the role we play. I think that was 30 years ago, 25 years ago, it's today. It's kind of the same thing that students need. Also the function of high school will as wonderful as your school is, it will invariably leave gaps in learning. And because you can't you're gonna not necessarily connect with a certain teacher or not certainly connect with a certain syllabus of the coursework, and you're gonna end up with some gaps in skills. And that's not an indictment of the school or the teachers. It's just a function of like a one-to-many model in schools where not all students in a classroom will receive the same learning and like retention of material and understanding of material from that class. So there's always a need for supplemental education. And that was the same thing that I observed 30 years ago, is that when I would assess my students. Regardless of the schools they went to, there were always gaps in learning. They didn't know what a semicolon was versus a colon. They didn't really know how to craft a thesis statement in a really, you know, argumentative and interesting way. They there were kind of they had gaps in their math curriculum because they didn't really connect to geometry or the geometry teacher. So there was always something to supplement. So I think

Andrew Ferguson: It's always geometry. It's the one. It's always geometry.

Matthew Pietrafetta: It's always geometry. In fact, I'm gonna be teaching some geometry lessons to some juniors later today. As they get ready for the ACT and they've forgotten they learned geometry back freshman year and for have forgotten all of it or didn't really learn it you know comprehensively the first time around. So that I think the need for mentors and mentoring the adolescents benefit from along with the kind of systemic need for supplemental educational services for students in high schools because of the fact that they will never retain all of those things or absorb it a hundred percent without some gaps. That has created the conditions of I think consistency are kind of the through line of the work that we do.

Andrew Ferguson: Yeah, I it's the mentoring piece, like I you know, I've been doing this with you for twenty years and it's and I think to myself, like I could never no matter how busy we get doing other things, I could never walk away from the tutoring piece because of how much I just it that's the part I love the most is the part of I you know, as I know you do as well, but hearing from students that you tutored a decade ago and that just the success that you know the success stories and the fun and the all the things they've you know accomplished since I last saw them and they were just you know a kid struggling with geometry or whatever it may have been, those are the that's those are the best parts. And so yeah.

Matthew Pietrafetta: Some of those students, some of whom specifically you've worked with and I've worked with, are now back as tutors at academic approach. Or they're doing like amazing things and we're following their careers and keeping up to date with what they're doing in other industries. Or they're bringing their own kids back now to be tutored, which is a wild thing. when we're getting them in middle school. So that first cohort of students that I tutored like circa you know 2001, 2002. They have grown children now who need tutoring and are coming to us. So that's a cycle of goodwill and just investment in those relationships and care and concern for mentoring that's coming back to us now, these years later, with the a next generation of students from that first cohort is pretty wild and pretty exciting. But it does, you know, indicate I think the impact, the positive impact of the mentoring it's goodwill how it cycles back to us even generate generationally now, which is really amazing.

Andrew Ferguson: Well you did bring up something interesting and it kind of goes hand in hand with my next question of you mentioned the ChatGPT and the AI, but I kind of want to make that a bigger question of because it's part of the answer, but if in the last 25 years, the mentoring and the coaching you know has pretty much stayed constant, but the learning process I imagine has changed. What have you seen in students that is in their own personal learning processes that have changed, and do your same skill building, your same, do your same, you know, approaches work or have you had to kind of come up with some of your own new tricks?

Matthew Pietrafetta: Yeah. Well on one level y yes, on another level, no. So the yes first, which is like thirty years ago, 25 years ago, I was teaching like analogies on the SAT, right? Like those were those are the what you know, the skills that I was teaching. They cause the tests have changed so much, they've been under such consistent revision, right? The SAT has been through three major overhauls four since I first started. And the ACT just underwent a huge one. So the actual skills and strategies and content that we're that we've been working on over the years has indeed changed. And we have to remain innovative and open-minded and constantly self-reflective and self-corrective in the way that we kind of encourage our students to be to get better at teaching. And you can always get better at teaching. That's another part about the work that's interesting is you're never as good as you can be and will be the next year. And so you have to re remain open to change, read more you know, more current research, compare notes with other tutors and educators and constantly imp improve and change. So on that level, yes, instructional strategies, they get deeper, they get better, they change to the content of the test. As far as student learning goes, I believe in the same kind of pedagogical model that I believed in thirty years ago when I originally first researched it, which is you know, based on 20th century learning theory from Vygotsky about the zone of proximal development or the productive struggle, it's sometimes called now in contemporary literature, which is one of the most important things we're teaching when we're teaching problem solving, when we're coaching, is the flexibility and resilience, right? To be able to persist through a challenge and take a couple of different approaches to it until you crack it, to you crack the problem. And to have there's a lot that goes into that. So yes, it's prepping for a test, right? And you're trying to get the score up, but you're also teaching a student how to problem solve, how to persist, how to be resilient, how to be flexible. And now you get into the real area where learning growth takes place, which is in that different that kind of postponement of satisfaction, where you build all those skills, the executive function skills, to plan, to execute, to be flexible, to s self manage and self monitor your emotions because you're getting a little scared or frustrated or nervous, but you learn to manage that while you persist in the problem solving and see it through successfully. That's where the real educational development is taking place on those two levels. And so that's what I've always believed in. And that is the kind of the you know, the derivative of the work that we do. And we hear that back from students: “Thank you, I learned so much and I'm applying it in my classes, but I also learned how to plan and problem solve and be more persistent. It grew my confidence, and now I feel I can do more things.” But this is where, back to ChatGPT, where I think that the largest threat. From ChatGPT comes. There's so many ways to talk about AI and its possible risks to education or workforce, all that kind of stuff. But if you just want to talk about that learning model, what chat does, and there's been tons of research from MIT and other big research universities on this, is it cuts down the period of productive struggle. Because instead of looking at that math problem and thinking, how can I approach it? Maybe I'll approach it this way. Well, that didn't work. Let me try it that way. That didn't work. Let me try a third way and build all that resilience and that flexibility. You pop it into chat, the answer comes up. It just comes up with like one way to do it. And it doesn't give you three or four different ways to do it, just gives you one. And most students accept that. Sometimes when we look at process, they just look at the answer and move on. So now what you've done is you've limited the that you've cut down on the productive struggle, you've truncated that, that, that. Really critical time and space for learning. And you've outsourced that process to technology. And when you outsource that process to technology, then the brain is not being activated in the same way. And they've done studies on this to show there's like less brain activity, less working memory results from using chat. When you write papers, you can't retrieve what you wrote where you're writing. So you are under utilizing the brain at a critical time of development. So from that learning standpoint, like if you were gonna use SparkNotes or CliffsNotes or something, you're at least still reading something, right? And then you are maybe, you know, getting a plot summary and then from there you go to the novel and you and then you read you know and get your quotes and so forth. If you just ask chat to write your paper for you, none of that process, even like a truncated one that uses other resources, none of that's taking place. So that to me is the is that learning process I still believe in and it's being cut short. It's being outsourced in a kind of undisciplined and I think not really well thought through use of chat. And so one of the one of the things I tr I've tried to do, and I've done presentations on this, is encourage parents and students just to understand what are you outsourcing? Like it's one thing to use ways to figure out how to get around in a new city and not like get a map and read it. Like understand outsourcing that as a convenience. But when you outsource Critical learning tasks that involve productive struggle and investment of time and critical thinking and executive function development. Like now you're possibly outsourcing developmental opportunities for your brain.

Andrew Ferguson: Yeah, I was meeting with a student last night about for a chemistry test that they have coming up and they the student was so frustrated because the he's like the textbook just won't get to the point. It won't explain. I'm like, well the textbook's job is to kind of give you all of it. As it I think you're a little used to just maybe typing something in and getting just exactly what you want. That's the kind of world we're living in right now. And I the textbook's job is to teach you all the pieces that go into complete you know creating that answer. He's like, yeah, I don't like that. I said, well, yeah, okay. I get it.

Matthew Pietrafetta: Delaying gratification is not a fun thing to do.

Andrew Ferguson: He actually got into more he got into the he's like it's kind of like when you're watching television versus watching Netflix where you can just fast forward through like the to the stuff you're not interested in on Netflix, but TV makes you just sort of have to look at stuff you're not interested in. The book just won't get to the point. It won't give me the answer. I'm like I said it's not giving you the answer, it's giving you the ability to do it with you know, moving forward. It's giving you it's giving you the skills. And he's like, Yeah, I guess. So it's

Matthew Pietrafetta: Yeah, yeah. It's like, you know, just wait a little bit. You'll get two marshmallows. You get two marshmallows. Don't just grab the first marshmallow, right? That's the marshmallow test, and that study has been conducted again and again under different contexts, but the same idea persists, which is that the delay, like learning the delay of gratification, right? And trusted mentors are helpful in this too. They've done the experiment before with like you know, researchers who are not trusted by the children versus those who are. And if you're a trusted researcher, then you the child's more likely to delay gratification, wait for the payoff of more marshmallows later rather than grabbing that first marshmallow. And I think that's part of the role that great teachers serve is to invest students with a sense of purpose as to why should you do this? Like why should you care about taking the time to do this authentically and for yourself? What does it do for you for your brain? What does it do for your self-confidence?

Andrew Ferguson: Right.

Matthew Pietrafetta: And teachers are getting more savvy about this too, because a lot of what I'm seeing more from the students we work with and my own daughter who's a junior in high school is that you know, teachers are doing more in-class spontaneous writing assessments. So you can't be using chat when you're writing in class or writing on paper sometimes. Or you're in a lockdown browser and you're typing so you can't get to other, you know, you can't get to anything else to help you. So they're trying to find ways to get more authentic assessments of what students are doing and how they're problem solving, which I think is really important. And to tell our students that we're really preparing for you're gonna have lots of academic tasks, you know, which you cannot do what you're doing right now late at night when you're doing your homework. You cannot do that when you're sitting in a three-hour exam in college without any you know technology resources to help you. Or take the AP exams, like the students who just took them: you're writing two hours of essays, spontaneous writing with no technology to support you. You've got to prepare for tasks like that because you're going to be assessed that way. And so the more you rely on the shortcuts, the less you build the skills to deal with those other authentic problem solving tasks that you are going to be given and are going to be important not only for your performance in school, but also for the development of those skills and of your brain and of your own planning and organizing skills and your own resilience and your own you know your own mindset basically.

Andrew Ferguson: With academic approach, kind of the way we teach, and we are not we are the epitome of not giving shortcuts, right? We want you to learn. And do you think that's the key to what's made us relevant for so long? And that it's not just it's we're not just giving you, you know, all you know, what you want, when you want it, where we're gonna make you learn and we're gonna make you be able to transfer that knowledge or and it and what's separated us from our competitors that haven't necessarily lasted 25 years. Is that the key or is there something else?

Matthew Pietrafetta: I really think the key is in mo mostly it's people. Mostly it's you. They like you know, you and the kind of impact you've had on the company, the relationships you've built, the way you make students feel and learn, right? It's that it's the people. And we have some excellent we've been able to work with I mean the fact that we could work together for 20 years. We've been able to assemble a group of colleagues that I know we both value greatly. Who are really, really talented and get better every year in their role and share a common mission with us. So the fact that we've been able to attract excellent people who really care about, you know, impacting you know, students positively and their families positively and the school leaders that work positively, that's number one. The second part is definitely the instructional service, which goes to the instructional approaches that we are trying to really you know, it's we're trying to be really flexible and creative in the way that we teach. Yes, to give multiple approaches so there's not just one way to do it. And we do want to accommodate it and personalize it always to the student. So if there is a shorter way to do it that they understand, we'll most certainly promote that. But we'll teach them multiple ways you could do it this way, that way. And maybe you want to choose the shorter way because it's more efficient under time pressure, and that's cool. But we want you to know all these ways. So there's that thoroughness of it and there's that personalization. And choice that we want to promote. And then you have to also say the thing is it's people, it's the instructional service, you know, the way that we teach, the way that we personalize it. And then third, which is just absolutely necessary in business, particularly in the landscape that we're working, is innovation. Like we've had to change an awful lot. We've developed, you know, our own curriculum and technology and revised it and then rebuilt it a million times because we need to be able to have really high quality resources. To do our instructional work well and to train instructors to have uniform approaches to doing the work so that we can feel that the very best lessons that we have and the very best content that we have is being used in a similar way to the way you and I would use it, even though we're veterans. We want all of our instructors, even newer ones, to have excellent resources and training to be able to do things well. That requires curriculum and technology and innovation. And we've had to which has been a great opportunity for us. We've had to change so many times as the assessments changed. But we have our curriculum team and we have our technology team and our own proprietary technology. So we can spin up a new assessment quickly. We can change our platform for administering it. Like, I mean, there's another way that you can think about it from Barnes & Noble to ChatGPT, but also from like pencil and paper analogies to Desmos calculator now built into SAT and ACT digital, like it is a rapidly, you know, innovating curriculum and technology landscape. And so you gotta as an organization, you've got be heavily invested in being responsive and adaptive to those changes quickly. Or you would absolutely lose relevance.

Andrew Ferguson: So that's a great transition into my final question. We've been looking back over these last 25 years. Now I kind of want to look forward and say, like, what do you see in this sort of ever changing landscape of test optional and the tests changing as much as they have over the last couple of years? What how where do you see the future of standardized testing and even college admissions going? And where will test prep and college admissions be in let's say 10 years?

Matthew Pietrafetta: Yeah. Jeez, it's hard to think that far ahead. It's hard to think back on 25 years. Give me the hard questions today, man. But I mean there's one trend that I've been looking at. You know, and it's making the news obviously what Harvard did with you know, a cap on A's on the college level. But the trend of grade inflation has been really interesting to study and there's been lots research and written on it in the past few years. And it kind of went through the roof with COVID, as everyone knows. But there had been a longstanding trend from, like, 1990 to, say, 2020, in that 30-year period, the average high school GPA, I think, went from 2.7 to 3.2. So it went up half a point, right? But at the same time, if you look at the corresponding data of ACT scores, trends and SAT score trends and NAEP, you know, the Nation's Report Card test, those performances for those same students, right, either were flat or declining. So how do you correlate the two? So there's like a general flatness or decline in skills as assessed, right, through these core curriculum assessments, but yet the GPAs keep going up. So I think what that means as far as a standardized test goes, is if you are a college you know admissions you know officer you're kind of curious does that what is that what does that a mean from this school versus an A from that school is it that the grading is a little bit more rigorous over there and less rigorous over there so those two A's don't really equal each other. I'd love to have a common instrument like a you know something that's nationally normed that I could also just get another data point please on where the student is in terms of their college readiness skills. And that's where a standardized test comes in because it can be that nationally normed, national percentile based instrument to give you something that you know the those two students had maybe very different English teachers, maybe very different English courses. They both walk away with an A, but how do they do on the English section or the reading of the ACT or the reading and writing section of the SAT? That's not a be all end all metric, but it is a contextual it contextualizes. Things for the admissions officers see, well, there's a huge discrepancy here between them while they both had A's. That that could be meaningful that maybe one is a little bit more prepared in terms of their college readiness skills, their kind of grasp of those core curriculum skills that I was teaching students at Columbia years ago, that we really wanted to really reinforce for those Columbia freshmen before we sent them forth into the to write and work with tenured faculty because we wanted them to succeed and not be ill prepared. So that's in part how you want to think about it. Is are they going to come into my school and be able to handle the curriculum? Are they, you know, prepared enough to do it? And I really want to avoid a false positive on that, because then I might be admitting them into my school and setting them up to fail. And then they may not graduate, they might not make it their freshman year. So that's why I think that there is still a need for the you know, a standardized testing instrument, you know, in that assessment and that thinking. And that's where we come in, I think, doing the work that we do on both levels, right? That's why it's important because what we're trying to do is get the scores up, yes, but get them up in a meaningful, really authentic way. Because the student really developed some lasting transferable skills that don't just equal the higher score on the test, but transfer directly into readiness, like in the classroom when they get to college. They will do better, they will feel better. And I remember, you know. Gosh, this is probably 2003. A wonderful student I had worked with who worked really, really hard and had a goal of getting into Cornell and worked so hard on academically in school and on the ACT, did well enough to have competitive score to get in, but also had really worked with me a lot on skill building and areas where she really needed it. And she wrote me a letter with she actually sent me it's like a an envelope with a copy of the first paper she wrote for her like intro freshman seminar class and she received an A, and then she put a big smiley face on it and said, I learned a lot of these skills that I applied here in this paper from you. And so that is that moment of what we call the both-ands: you increase both your score and your college readiness skills. So you wanna think about the two together because that's what it's all about ultimately.

Andrew Ferguson: Yeah. That's a great note to end this conversation on. I could talk about this last topic, you know, and we'll and future episodes perhaps we'll dive in further. But that was that's that kind of story is the perfect example of why we do what we do and from so many different angles. The fact that, you know, you're still staying in touch with that student to the fact that they took the skills that in the moment at, you know, 4:30 on a Tuesday you met with that kid and you're covering you know misplaced modifiers or whatever it may have been and it felt like I'm so tired and but that those skills moving those into the college level and her being the one that acknowledges it and how and she's telling you know look at how far we've come that's a that's an awesome story. That's 25 years, man. All that you've done. It is fun.

Matthew Pietrafetta: Yeah. And it's why it's fun. It's meaningful and it's very rewarding work and it's very fun to do it with you.

Andrew Ferguson: Well my man, I can't thank you enough. You've given me more opportunities than pretty much anyone. So I thank you. And this is gonna be a fun journey. We get to talk about we get to tell stories and talk about all kinds of fun things. And I'm looking forward to hearing more details. Think about all the topics we talked about today, from AI and learning to just to the grade inflation topic to, you know, how the skills that these that this is not just so you can get through some test that you are gonna use this stuff at the next level. I'm excited.

Matthew Pietrafetta: Yeah. Yeah, I hope it, you know, I think we'll make it useful to people too, to the families and students who listen in to give them practical information along the way, points of view and context with other folks that we'll end up hanging out with on this podcast. And you know, we've developed along the way over these 25 years a lot of great colleagues in this space, you know, from you know, educational psychologists to college counselors that can enrich the dialogue here. You know, without between us and among them, and really, you know, be a useful service to the families that we love working with.

Andrew Ferguson: Absolutely. Awesome. Thank you, Matthew. Talk soon.

Matthew Pietrafetta: All right. Thank you, Andrew.