How to Prepare for Multiple Mini-Interviews, Part 2

By Jeremiah Fleenor, MD, MBA

In part 1 of this two part series we looked at some of the reasons why ADCOMs (admissions committes) are searching for a new way to assess an applicant’s personality. The correlation between an applicant’s GPA and their future success in the didactical components of medical school is well established. The new frontier is a more fair and predictive way to evaluate an applicant’s character, ethics, and communication skills. That evaluation tool seems to be found in the multiple mini-interview (MMI). Continue reading “How to Prepare for Multiple Mini-Interviews, Part 2”

How to Prepare for Multiple Mini-Interviews, Part 1

By Jeremiah Fleenor, MD, MBA

In recent months and years, the popular press and scientific journals have seen an increased interest in the multiple mini-interview (MMI) with regards to medical school admissions. It seems to be the new buzz word in the admissions circles. As one would expect, there is a bit of skepticism from the applicants and a touch of intrigue regarding the new format. This is very appropriate for any new process, especially one that plays such a big part in your future as a physician. Continue reading “How to Prepare for Multiple Mini-Interviews, Part 1”

What Is Your Weakness?

By Samir P. Desai, MD and Rajani Katta, MD

An excerpt from Medical School Interview: Winning Strategies from Admissions Faculty

In 2011, the AAMC published a survey that evaluated the importance of 12 variables on medical school admissions decisions. These variables included total MCAT scores, science and math GPA, and the interview. The interview was rated the most important factor. “High grades and/or MCAT scores alone are never enough,” writes the LSU Shreveport School of Medicine. “For those interviewed, impressions from the personal interview are exceedingly important.” Continue reading “What Is Your Weakness?”

How to Prepare for Multiple Mini-Interviews, Part 2

By Jeremiah Fleenor, MD, MBA

In part 1 of this two part series we looked at some of the reasons why ADCOMs (admissions committes) are searching for a new way to assess an applicant’s personality. The correlation between an applicant’s GPA and their future success in the didactical components of medical school is well established. The new frontier is a more fair and predictive way to evaluate an applicant’s character, ethics, and communication skills. That evaluation tool seems to be found in the multiple mini-interview (MMI).

The Multiple Mini-Interview

Some logistical orientation of the MMI is necessary to better appreciate the data that drives its increasing popularity. Briefly, the MMI is a type of admissions assessment tool where an applicant is shuttled through many (between 6-12) different stations or rooms where he or she is asked to perform a certain task. The task might be to break some bad news to a friend or to role play telling a patient you made a mistake while you were operating on him, discuss a current event that has a strong ethical element, or even engage in a traditional medical school type of interview. Each station may last 6-10 minutes. Once the applicant has completed all of the stations, individual scores are assigned and a composite score may be given based on all of the evaluators’ input. This then becomes your interview score and is weighed according to the university’s policies along with your academic record and other factors.

Before getting into any more of the specifics of the MMI or how to prepare for this type of interview I’d like to go into the thinking of ADCOMs to even entertain a new type of interview format. This will shed some light on the motives of admissions committees and give you some inside information.

Reliability, Context, and Validity

There are many reasons why the traditional medical school interview does such a poor job of assessing personality and future clinical success. Let’s briefly look at each.

Reliability

When speaking about the reliability of the medical school interview the issue at hand can be summed up with the question, “Would two interviewers give similar scores after interviewing the same applicant (presumably one right after the other)?”. Reliability for the medical school interview has been reported to range from .14 – .95. That’s a huge range. It’s been hypothesized that this is due to an interviewer having access to the applicant’s academic record and the other interviewer not having that information, and/or power differences between members of the ADCOM and their resultant influences and/or the interviewer’s background and expectations.

One study showed that 56% of the difference between interview ratings has to do with interviewer variability. Meaning, it has nothing to do with the applicant. Not only is that frustrating for the applicant, but it’s unethical as well.

Context

Much research has shown that success or the ability to communicate effectively in one area does not mean that the same person can do so in another area. Just because you can talk about the effects of the printing press on Western civilization doesn’t mean that you can speak clearly about the role of the magnetic compass on sea travel. Meaning, you may communicate very well during interviews and convince the ADCOMs of your great personality but this doesn’t mean that you will communicate and behave accordingly in future clinical practice.

The converse of this would be true as well. You may not excel in the pressure cooker setting of a medical school interview but in time would be able to show yourself to be a fine physician who’s caring, compassionate, and very conscientious. However, if you’re not freed from the bias of context, you may never have the chance to prove the interviewer wrong.

Validity

This was discussed in part 1 of this series but deserves some repeating. The traditional medical school interview is about as good as flipping a coin when it comes to predicting how well an applicant will do on future clinical tests. When sticking to the strict measures of validity, the traditional interview is nearly useless.

After reviewing all the data, it’s easy to see why more and more schools are looking for better ways to assess an applicant’s personality.

Can I Get Some Help?

I think any admissions committee member who was totally honest with himself wishes that he had a “personality GPA” for each applicant. ADCOMs hope to have a single measurable data point that correlates with success in the non-cognitive areas of medical education just like the real GPA correlates with success in the cognitive/didactic areas of medical education. Up to this point such a thing hasn’t existed but the MMI is showing some signs of providing the kind of predictive power that ADCOM’s are looking for.

In the same study just mentioned, the investigators found that the MMI was, by far, the best predictor of the applicants’ future success on a clinical examination. It was a better predictor of non-cognitive success than the medical school interview, the applicant’s personal statement or his or her GPA.

The good news is that this wasn’t just some random paper. The data has been confirmed in other publications.

A 2007 study showed that the MMI, when compared to the traditional interview, personal statement and GPA, best predicted higher scores on the major clinical tests during the medical students’ training. This same study showed that the MMI also best predicted success on the “non-cognitive domains” of Part 1 of the Canadian qualifying exam. This is a test that medical students must pass to become licensed physicians in Canada. It is similar to the United States Medical Licensure Exam (USMLE).

A 2009 study showed that a correlation between the MMI score and the number of stations passed in a structured clinical exam (part of the Canadian qualifying exam) was 0.43. The value rose to 0.65 when the sub-score of patient interaction was evaluated.

Substitution, Please

When looking at all of this data one wonders why all U.S. medical schools haven’t bitten on the MMI hook, line and sinker. Yet when you count up how many U.S. schools are using the MMI it’s shockingly low. According to the MSAR, only 9 U.S. schools list the “MMI” as their chosen interview format. Considering there are approximately 167 U.S. medical schools (MD and DO) that means only a little over 5% are using the MMI.

Why such a low number?

While this is only speculation on my part, I suspect there are several reasons for the low but rising use of the MMI.

First, the ground breaking paper on the use of the MMI for medical school admissions was published in 2004. That means it’s been less than a decade since this idea was brought to the attention of medical school admissions committees. To give you some perspective, the first U.S. medical school was opened in 1765. So eight years vs. the 247 years that medical education has been around in the United States isn’t actually that long. I would just give it some more time and I think the MMI’s use in medical schools will continue to rise.

Second, the majority of the research being done on the MMI is happening in other countries, namely Canada, the U.K. and Australia. For example, 11 of 17 Canadian schools recognized by the Association of American Medical Colleges (AAMC) use the MMI. I found very few U.S. based papers that were studying the MMI at their institutions. I suspect this too will change as more and more U.S. medical schools adopt the MMI and want to substantiate its validity here.

Third, there is always risk when changing from one format to another. There’s no guarantee that what has worked elsewhere will work here and I think universities are wrestling with this issue. In addition to the risk is the matter of cost. There is cost in testing a new interview format and then implementing it. In these tight economic times, expense issues weigh heavier and heavier. Furthermore, institutional cultures are hard to change. This is true in business as well as government and education. Cultural inertia is hard to overcome but I think increasing data and U.S. acceptance of the MMI will help turn the more entrenched universities.

Conclusion

Although the MMI is far from perfect, there seems to be more and more data that this type of admissions tool can help ADCOMs better predict how a medical student will perform when it comes to the non-cognitive or more personality based tests. This is significant in that these aspects of a physician’s overall nature significantly determine how he or she will behave and succeed in future clinical practice.

Certainly there are limitations to what the MMI can predict and how far out the predictions can safely be assumed to hold true. Thus far, it appears that the reliability and validity of the MMI to better predict success during medical school is solid. How well this admissions tool can predict success as a practicing clinician has yet to be determined. Regardless, the MMI is likely to be welcomed by more ADCOM’s that must make the difficult decision of who to admit into the competitive field of medicine.

If you have any questions please email Dr. Fleenorn, author of The Medical School Interview: Secrets and a System for Success 2nd ed., at [email protected].

References
Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Adv Health Sci Educ 2001; 6:159-75.

Eva K, et al. The Ability of the Multiple Mini-Interview to Predict Preclerkship Performance in Medical School. , Vol 79, No 10. October 2004.

Kulatungo-Moruzi C, Morman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14:34-42.

Eva K, et al. An Admissions OSCE: the multiple mini-interview. Medical Education 2004: 38: 314-326.

Reiter H, et al. Multiple Mini-Interviews predict clerkship and licensing examination performance Medical Education 2007; 41: 378-384.

Eva K, et al. Predictive validity of the multiple mini-interview for selection medical trainees. Medical Education 2009; 43: 767-775.

MSAR: Getting Started 2013-2014. Association of American Medical Colleges. Copyright 2012. Washington, D.C.

Medical School Admission Requirements (MSAR). Association of American Medical Colleges. Copyright 2011. Washington, D.C.

Harasym PH, et al. Reliability and validity of interviewers’ judgments of medical school candidates. Acad Med (Suppl) 1996; 71: 40-2.

– See more at: http://www.studentdoctor.net/2013/02/how-to-prepare-for-multiple-mini-interviews-part-2/#sthash.rOgywrKq.dpuf

This article was originally published on StudentDoctor.net on February 6, 2013.

How to Prepare for Multiple Mini-Interviews, Part 1

By Jeremiah Fleenor, MD, MBA

In recent months and years, the popular press and scientific journals have seen an increased interest in the multiple mini-interview (MMI) with regards to medical school admissions. It seems to be the new buzz word in the admissions circles. As one would expect, there is a bit of skepticism from the applicants and a touch of intrigue regarding the new format. This is very appropriate for any new process, especially one that plays such a big part in your future as a physician.

The emergence of the MMI brings with it many new questions:

-Why are medical school admissions committees (ADCOM) switching to this new format?
-What was wrong with the old way they did interviews?
-What are the admissions committees looking for?
-Why are some schools using the MMI and others not?

I hope to answer these questions in this two part series. The information is vital for your success and will likely provide you with a leg up on your fellow applicants. It will also provide some welcomed peace of mind.

In The Beginning
It’s not hard to understand that admissions committees want two general skills from the physicians they graduate: to be smart and personable. This after all is the stereotypical view of a physician: an intelligent, caring person. Now I’m using these two categories very broadly, especially personable, but let’s look at each one in more detail to gain a better understanding.

Smarts
It is no secret that patients expect the doctors caring for them to be smart. The field of medicine is a demanding one that requires a certain cognitive ability from physicians in order for them to be competent and successful. A doctor needs to have a certain ability to process and critically analyze information to functionally do his or her job. We’re not breaking any new ground with these statements.

The shocker may come when you learn just how smart medical schools expect you to be. According to the Medical School Admission Requirements (MSAR), the average GPA and MCAT scores for the 2011 admissions cycle were 3.5 – 3.75 and 30, respectively. Those are some high marks and they just seem to climb every year.

Admissions committees’ love of applicants with good grades is not likely to change either. They have solid data backing up their strong commitment to only let in applicants with high marks. Multiple studies have shown that “preadmission GPA” is the best predictor of success in medical school with regards to the didactic component of training. In other words, the higher the GPA, the more likely the applicant will successfully complete/pass the courses and certification exams that focus on the didactic (non-clinical) components of medical school. It’s just that simple.

Personality
Even more broad than the category of a physician’s mental capacity is that of his/her personality. From the admission committee’s point of view your personality consists of your ability to communicate, your attitude, how you interact with others, your ability to empathize and your ethical tone. These traits and characteristics are what I mean when I use the term “non-cognitive.” Of note, non-cognitive doesn’t mean that whatever is being assessed doesn’t require thought. When you are going through a multiple mini-interview you will be thinking a great deal. It simply is meant to communicate that the evaluation is assessing the components of an applicant’s overall personality and not their cognitive function.

Why do admissions committees even care about your personality? To put it bluntly and simply, once you have crossed the threshold of sufficient cognitive ability, nearly the entirety of your success as a physician is determined by these above factors. In other words, your “bedside manner,” conscientiousness and ability to get along with co-workers, determines how successful you will be as a clinician; both in the clinical years of your medical education and, I would argue, throughout your career as a practicing physician. That’s rather significant and why admissions committees try so hard to determine the personality of each applicant.

Old Faithful
To date, admission committees largely have used the medical school interview to help them determine the strengths and weaknesses of an applicant’s personality. This usually consists of two, approximately 20-45 minute long interviews with members of the admissions committee. These people are usually physicians, researchers or current medical students associated with the university. Once the interview is complete, the interviewers try to communicate their assessment of the applicant’s personality to the ADCOM. The admissions committee then comes to a consensus and makes a decision on the applicant.

This all seems well and good but the rub comes when you start to look at how accurate the interviewers are at predicting the personality, and in turn, success, of any given applicant. The truth of the matter is, they’re not very good at all. With regards to the scoring of an applicant by an interviewer, here’s how disparagingly one journal article puts it, “…despite acceptable interrater reliability in some cases, a candidate’s score may still be attributable, in large part, to chance”.

One of the ways in which researchers attempt to study personality with regards to medical students is via an Objective Structured Clinical Examination (OSCE). This is a type of test that measures clinical skills and takes into account your ability to communicate and interact with patients and co-workers. In some ways an OSCE score is a proxy for personality. It’s not a perfect way to assess these aspects of a person but does a reasonable job and is the preferred format to date.

A study published in 2004, showed that the personal interview was able to predict the mean performance on an OSCE at .06. To understand this more, a score of 1 would mean that for every unit increase in the interview score would result in the same unit increase on the OSCE score. A score of -1 would essentially mean the opposite or that the interview score and the OSCE score were inversely related. And a score of 0 would mean that the interview can’t tell you one way or another how a student will perform. So a score of 0.06 means that the interview was little better than the toss of a coin in predicting how well an applicant would perform in clinical areas.

Chance?! A coin toss?! I sure wouldn’t want my future hopes and dreams of becoming a physician coming down to a roll of the dice or a 50/50 proposition and I suspect you feel the same way. The good news is that more and more U.S. medical schools are starting to feel this way as well and are adopting a more ethical and predictive way to assess applicants.

Enter the MMI
Please check back later for Part 2 of this series on the MMI. We will cover the logistics of an MMI and look at the data that supports its use. This will provide a look behind the scenes of the ADCOMs and shed some light as to why the MMI is gaining such increased acceptance abroad and in the U.S.

If you have any questions please email Dr. Fleenorn, author of The Medical School Interview: Secrets and a System for Success 2nd ed., at [email protected].

Part one of a two part series. Return next week for part two.

This article was originally published on StudentDoctor.net on January 30, 2013

References
Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Adv Health Sci Educ 2001; 6:159-75.

Eva K, et al. The Ability of the Multiple Mini-Interview to Predict Preclerkship Performance in Medical School. , Vol 79, No 10. October 2004.

Kulatungo-Moruzi C, Morman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14:34-42.

Eva K, et al. An Admissions OSCE: the multiple mini-interview. Medical Education 2004: 38: 314-326.

Reiter H, et al. Multiple Mini-Interviews predict clerkship and licensing examination performance Medical Education 2007; 41: 378-384.

Eva K, et al. Predictive validity of the multiple mini-interview for selection medical trainees. Medical Education 2009; 43: 767-775.

MSAR: Getting Started 2013-2014. Association of American Medical Colleges. Copyright 2012. Washington, D.C.

Medical School Admission Requirements (MSAR). Association of American Medical Colleges. Copyright 2011. Washington, D.C.

Harasym PH, et al. Reliability and validity of interviewers’ judgments of medical school candidates. Acad Med (Suppl) 1996; 71: 40-2.

– See more at: http://www.studentdoctor.net/2013/01/how-to-prepare-for-multiple-mini-interviews-part-1/#sthash.G8vMdkI2.dpuf

Medical School Interviews: 6 Common Mistakes That Admissions Officers Hate

By Joel Butterly

Medical school interviews come in all different shapes and sizes. Some schools interview one-on-one, some have multiple interviewers, some have multiple-mini-interviews (MMI). Some schools use students, others use faculty, and some use alumni.

In preparing this article, I interviewed admissions officers from the top medical schools – Harvard, Yale, Stanford, Penn, UCSF, and others. What I found was that their advice on med school interviews varied tremendously.

However, nearly every person I interviewed commented on several common mistakes that applicants are constantly making. These aren’t always deal-breaking mistakes, but for an applicant who is right on the cusp of admission, avoiding these mistakes can be the difference between a fat envelope and a thin rejection letter.

1. Don’t assume that the interviewer knows how serious you are about becoming a doctor.
In your interview, you must demonstrate that you are fully committed to this field, and convey a clear sense of purpose in your professional goals. Never assume that applying and showing up for an interview is sufficient evidence of your commitment. Here’s why:

Admissions officers have a job. For medical school admissions officers, that job includes vetting and selecting the best possible students for their program. One of the most important criteria by which these admissions officers are judged is the dropout rate. If 20% of the students that they admit end up dropping out midway through medical school, it reflects terribly on the admissions officers who selected them.

Moreover, medical students are extremely expensive. Medical schools pour hundreds of thousands – if not millions – of dollars into training their students. Every time a student drops out, that money is wasted. Even the wealthiest medical schools cannot afford too many dropouts.

Long story short, admissions committees are extremely sensitive to indications that an applicant will “go the distance.” You would do well to convey this to your interviewer. Similarly, admissions committees are extremely sensitive to indications that an applicant is not 100% sure that medical school is right for them.

The most common example cited by admissions officers was that applicants, when asked why they wanted to attend medical school, would routinely respond that their parents are doctors. This is never a good strategy. To an admissions officer, this is like saying that you are going to move to Bhutan because your friend did it and seemed to enjoy it. Of course, you can mention that your parents are physicians, but leading with this fact is a surefire way of indicating that you haven’t thought your decision through.

Moreover, when asked why you wish to become a doctor, make your answer specific. Don’t say, “I’ve always wanted to help people.” Say how you want to help people. Better yet, give a short story that is representative of what you want to do, and why. The more specific and concrete, the more likely you are to seem like a safe bet to admissions officers.

2. Understand how difficult medical school and practicing medicine can be – and be able to explain why this isn’t a problem for you
Admissions officers want to make absolutely certain that applicants know what they are getting themselves into. The reality is that very few, if any applicants truly understand the commitment it takes to see themselves through medical school and residency.

Nonetheless, you might consider talking with some current residents or recent residency graduates to figure out what the absolute worst parts about becoming a doctor are. You should probably know this for your own sake, but for the purposes of the interview, being well-informed about the downsides of medical school and practicing medicine is a huge asset.

If your interviewer doesn’t think you know what you are getting yourself into, they may reject you because they don’t think you’ll be able to stomach the challenges. It is crucial that you show them otherwise.

Spend some time thinking about how to address these “downsides” in an interview. You could rephrase them as a positive feature: “I love intense working environments, and I think working late-nights in the hospital when there are a million things to do, and not enough people to do them, would be really exhilarating.” Obviously, you don’t want to be too over-the-top, or the interviewer will know you’re full of it.

You might also explain why these downsides don’t matter, in light of other considerations. Maybe you are so determined and passionate about a certain aspect of medicine, that these things are a small price to pay.

Bottom line: make sure that you take a moment in your interview – if at all possible – to show that you know something about the challenges you will face, and that you are fully committed to facing them head-on.

3. Compassion is key – don’t forget to show that you’re human 
Shockingly, many students go into the interview too scared or too hyped-up to demonstrate a critical characteristic: compassion. Without compassion – and a lot of it – practicing medicine is nearly impossible.

Make sure that you don’t ignore the human element of your candidacy. Interviewers know your test scores, your GPA, they’ve seen your resume…now show them that there is a compassionate, amiable human being behind it all.

Admissions committees (at least for many of the top-ranked schools) sit and debate whether an applicant should be admitted. Your interviewer will probably speak or write on your behalf during these deliberations. If you have failed to convey your compassion and amiability during your interview, chances are good that your interviewer will not fight too hard if other members of the committee decide to pass on your application. Thus, you cannot give the impression that you lack compassion; if you do, your allies on the admissions committee will be dramatically reduced.

4. A Student Interviewer is not an excuse to relax
Most applicants breathe a small sigh of relief when they find out that they are being interviewed by a medical student. This is a mistake for two reasons.

First, these student interviewers are more likely to be an “interviewer-from-hell” than their senior counterparts. They are eager to prove how rigorous and devoted they are, and that might translate to a much more aggressive and rigorous interview style.

Second, admissions committees do not send in student interviewers with the expectation that they will perform a different job than any other interviewer. They are assessing the same things as the more seasoned interviewers, and so you should maintain the same level of composure and professionalism as you would for any other interview.

A number of admissions officers and student interviewers have told me that many applicants seem to think that they can be more casual when a student is interviewing them. The truth is that while this is sometimes the case, most applicants go way too far. This is an interview, not a social interaction, and trying to act like you’re talking with a friend on the street will only make you seem unprofessional and unqualified.

Bottom line: keep it professional.

5. Know your application
Many interviewers will have your application right in front of them. Sometimes, they will start the interview asking you questions about different parts of your application. You need to know your application(s) by heart. If you put something in the activities/experiences section of the AMCAS application, and can’t remember what you wrote when your interviewer asks about it, you are going to look foolish, if not dishonest. This happens surprisingly often.

Bottom line: You should be able to recall every part of your application, so that you can talk about it fluently if and when you are asked.

6. Be honest about your weaknesses 
Many interviewers ask applicants about their weaknesses. When discussing your weaknesses, honesty and candor can make the difference between a great response and one which will, at best, do nothing for your chances of admission.

Your biggest weakness is not that you work too hard, care too much, take on too many challenges, are too hard on yourself, or any other strength-disguised-as-a-weakness. As soon as you give one of these self-serving responses, you have grouped yourself into the roughly 50% of applicants (according to one source) who believe that offering one of their positive characteristics as their “biggest weakness” is an effective strategy. It isn’t. Admissions officers see right through this, and are likely to feel that you are either too immature to be self-reflective, or too dishonest to give a straightforward answer.

This doesn’t mean that you should go nuts criticizing yourself. Don’t say that you are a heavy drinker, a drug addict, a criminal, or a psychopath; you’re right to keep those thing to yourself. However, you can and should talk about things that you really do struggle with, and how you’re working to improve on those things (this latter component is very important).

For example, maybe you are easily frustrated, have difficulty taking orders, or struggle to memorize long lists of details. What is important is that you recognize these things, and explain that (and how) you are working on improving them. This shows honesty, integrity, and maturity – all things which medical schools want to see in their students.

Bottom line: you have weaknesses which are undesirable. Pick one of them, work on it, and then tell your interviewer about it with a straight face.

Joel Butterly is a JD Student at Yale University and received his AB in Government & International Studies from Dartmouth University.

This article was originally published on StudentDoctor.net on December 22, 2013.