B-School Admission Criteria: A Critique, Part Two

Continuing from where I left off yesterday.

THE PROFILE

Now, the profile weightage, which is the winner of the most flawed of all criteria. This includes weightage for the following parameters:
(a) 10^th standard marks
(b) 12^th standard marks
(c) Graduation marks
(d) Post-graduation marks, if applicable
(e) Work experience
(f) Extra-curricular activities
(g) Achievements

What is the rationale behind this?
To quote IIMB, “IIMB has found over the years that students who perform well in the academic program are typically those who have a consistently good academic record during their school, high school and graduation level, besides exhibiting sufficiently high aptitude as measured by the CAT. Therefore IIMB uses multiple parameters, namely academic performance in school, high school and graduation programs as well as candidates’ scores in CAT to judge the suitability of candidates for the PGP program.”

And, “Evaluation by multiple criteria is also consistent with empirical research on recruitment and selection that shows greater efficacy of recruitment processes that use multiple criteria. Multiple criteria are used to arrive at a composite score for every candidate, which is used to select candidates for the subsequent stage.”

That’s the rationale in a nutshell. There are quite a few holes in it. I’ll start with the basics.

1. What do marks have to do with an MBA program?
An MBA program is not one in which you write exams, get marks and graduate. It is supposed to prepare you for the business world. Businesses don’t use marks as a test; they judge performance. They don’t want you to write solutions to hypothetical problems with inadequate data for making the right decisions; they expect you to actually solve real ones.

Marks are awarded by an examiner based on his judgment of your answer (i.e. subjective evaluation). In business, the mark equivalents, bonuses and incentives, are awarded based on the tangible value added to the company through your work (i.e. objective evaluation). I add that these are general statements. There are exceptions, but they don’t significantly affect the purpose of my observations.

In sum, if we accept that an MBA program is supposed to prepare a student for a corporate career (an implicit assumption) then marks have no relevance whatsoever.

2. The problem with empirical research
Empirical research is research based on observations or experiments. The problem with empirical research is the problem of induction. I’ll introduce it in an easy to understand manner by using The Black Swan Theory.

The Black Swan Theory
Let’s say you have observed 4000 swans and found all of them to be white in color. You publish the result of your empirical study as: All swans are white.

All it takes to refute your conclusion is for me to show you a black coloured swan. If I can successfully do that, then I can confidently say “Not all swans are white.” This finding of mine would totally invalidate the conclusions you formed from your empirical research. Incidentally, black swans do exist.

Basing anything on empirical research alone without logic is like building a house without a foundation. It looks fine, till it collapses.

If the empirical research on which IIMB bases its selection criteria is someday invalidated by a new study, they would have been ‘unfair’ in their selection process. But that’s not the worst of it. The worst problem is that they would have created a self-fulfilling prophecy, which we shall see in a while.

Also, even if using multiple criteria makes the process more efficacious, the question of what criteria to use still remains. And here, we get into the criteria of marks, and why it is flawed selection criteria.

3. What do marks project?
I studied in a really good CBSE school. My school marks have always been average among my school peers. I never did learn by rote (save for Hindi in 10^th standard). I used to understand and analyze concepts and their implications. I have sometimes even gone beyond what is ‘required’ in order to get marks out of my curiosity. I used to love studying some subjects, though I didn’t score greatly in them.

Do my 10^th and 12^th standard scores reflect my academic development? Not performance in useless tests, but real intellectual development. The answer is an emphatic NO. What do they actually reflect? The mood of the examiner, the presence of certain ‘keywords’ in each answer, my level of boredom at that time and in a small way, my knowledge about the questions asked. As a judge of my caliber, they are merely some meaningless numbers.

College was even worse. I could graduate with a B. Tech. in Biotechnology without knowing anything relevant whatsoever about biotechnology. How? Because I just had to write some answers to some questions on an exam paper and clear the cut-off, and I am now an engineer with a specialization in biotechnology.

I don’t have a ‘consistently good academic record’, so IIMB concludes that I won’t perform well in their academic program. I may have, I may not have. They’ll never know. But I can say this with certainty: if I did do well, it would not be ‘in spite of’ my poor academic record. And if I didn’t do well, it would not be ‘because of’ my poor academic record.

Doesn’t past performance matter?

Monkeys on a typewriter
If you have a billion monkeys typing away on a billion typewriters for a period of a billion years, by sheer chance, one among them may produce the complete works of Shakespeare. Now, let us take this impressively performing monkey and tell it to produce the complete works of Homer next. What are the odds you would give it?

Clearly, past performance does not matter when things happen by sheer chance.

But IIMB claims to have found a correlation between their selection criteria and student performance at IIMB. To again quote their selection document, “IIMB has found over the years that students who perform well in the academic program are typically those who have a consistently good academic record during their school, high school and graduation level, besides exhibiting sufficiently high aptitude as measured by the CAT. Therefore IIMB uses multiple parameters, namely academic performance in school, high school and graduation programs as well as candidates’ scores in CAT to judge the suitability of candidates for the PGP program.” So why not use those criteria for selection?

From these statements, we can see that IIMB has made a basic logic error which any scientist, statistician, academic researcher, or real economist knows he should avoid: correlation does not imply causation.

First, the correlation
If, when A happens, B happens, can we say B and A are correlated? For example: It rains on election days. Are elections and rains correlated? Should rains be regarded more highly when they occur on election days?

Now let us look at this statement:
“IIMB has found over the years that students who perform well in the academic program are typically those who have a consistently good academic record.”

Does this imply that good academic performance at IIMB and past academic record are correlated? Should past academic performance be regarded highly? (There is actually a correlation, but not for the reasons you may think. I’ll address it in a while).

Second, implying causation
A happens. Then, B happens. A caused B. This is called post hoc ergo propter hoc logic. B happened after A, therefore B happened because of A. Example: The performance of bank stocks is correlated with interest rates. RBI hiked repo rates. The BSE Bankex (Banking index) fell 1%. Therefore, the banking index fell 1% because RBI hiked repo rates. If you think that statement looks correct, try the reverse: The BSE Bankex (Banking index) fell 1%. RBI hiked repo rates. Therefore, RBI hiked repo rates because the banking index fell 1%. Now you’ll see the absurdity of this logic much more clearly.

As we see from their selection document:
“IIMB has found over the years that students who perform well in the academic program are typically those who have a consistently good academic record. Therefore IIMB uses multiple parameters, namely academic performance in school, high school and graduation programs as well as candidates’ scores in CAT to judge the suitability of candidates for the PGP program.”

First, they assume they have found a correlation between past academic performance and IIMB performance. Second, they determine there is a causal link between the two i.e. good track records in school and graduation implies good performance at IIMB (if they didn’t think so, they wouldn’t try to get more students with a good track record). And finally, they reward consistently good academic records, thus ensuring they get more students who have it. They have successfully created a self-fulfilling prophecy.

Self-Fulfilling Prophecy
Let’s understand this concept with an example. I use one loosely based on one from the book Outliers by Malcolm Gladwell. An anthropologist found that most Canadian hockey players were born during the first three months of the calendar year. Upon researching, he found that the selection criteria employed an age cut-off date of Jan 1.

What this means is that if you are born on January 2^nd 2005, you are eligible for the hockey tryouts in the Under-6 category in 2011. You’ll be the oldest in the group, and much more physically mature than someone born on December 31^st 2005 (who is over 5 but under 6 on that arbitrary cut-off date and so competes in the same category).

The extra physical development the January born player has passes off as talent. At that age, size really matters. So even if the January and December born are equally talented, or the December born is slightly more talented, the January born gets selected.

Selected players are given rigorous training, due to which they actually become better. If the coach then measures that January born player against that December born player, he will think his selection is validated because that January born player is much better than that December born. Thus, the slight initial advantage of that January born player due to his being born just after that random cut-off date, turns into an actual advantage because of the training he gets.

The coach then observes over the years that people who perform well in the rink are typically those who were born in the month of January, besides exhibiting sufficiently high aptitude for hockey as measured by the selection tryouts. Therefore, he decides to give people born in January an additional profile weightage in the selection tryouts, which boosts the initial advantage they got by being born in January, after that random cut-off date of Jan 1^st. This profile weightage will make even the less talented January born players to appear more talented than they actually are.

With the odds so markedly stacked against the February to December born players, they eventually stop trying, and even when they do try, they lose out unless they have a really massive skill advantage and they get the opportunity to exhibit it during the tryouts.

While the coach thinks that he is picking the best among the players, he is actually only picking the best among the January born players! But, all his observations still validate his selection criteria! Such is the power of the self-fulfilling prophecy.

Won’t some really talented February or March born players surmount their handicap and make it through the tryouts? Yes, that’s a possibility. We’ll see how the coach handles that.

If typically only the January borns play good hockey, but I show the coach a February born player on his team who excels, won’t that make the coach change his mind about his selection criteria? No, it wouldn’t. He would say that player is statistically insignificant. If he employs smoothing techniques, that player would be the noise he would eliminate from his data set.

The mind only sees that which it wishes to see. If you think only January borns make good hockey players, you’ll dismiss all data to the contrary as ‘noise’, ‘outliers’ or ‘statistically insignificant’. This is called the confirmation bias: seeing only the information which supports your preconceived notions, regardless of whether that information is true or not.

I’ll now proceed to relating this analogy to IIMB profile criteria.

Having compared the arbitrary January 1 cut-off date to ‘consistently good academic record in school, high school and graduation’, I have to show why this criteria is also arbitrary i.e. has no basis in logic.

My personal example won’t suffice to prove this. In fact, attempting to use it as a ‘proof’ would be a fallacy. Why? Because I’ll then be guilty of using a local example to dispute a global criteria.

For example, if Chennai has a spell of cold weather for two years (it actually being a hot city) I cannot cite that as a ‘proof’ that global warming is a myth. Nor can I cite it as a proof that there is now global cooling. It is obvious how ridiculous such a statement would be.

(It is ironic that we accept the reverse to be a proof: If Chennai has an unusually warm climate for two years, it is ‘because of’ global warming, and if Chennai has a cold spell, then Oh! It is because global warming caused local cooling due to certain natural phenomena. Warming or cooling, global warming can never be proved false if we accept such proofs. Alas, the mind sees only what it wants to see.

Here, it should be noted that some global warming zealots try to use logic to prove their beliefs. They say: global events have local consequences but local events do not have global consequences so the second example above is true while the first one isn’t. My answer is: you see it that way because you have already accepted the ‘global event’ i.e. global warming and you now see Chennai becoming warmer (or even colder) as the ‘local consequence’. You assume A precedes B, and so when B happens you say it happened because of A, applying the fallacious post hoc ergo propter hoc type of reasoning. Another example of the mind seeing only what it wants to see!)

I had to digress to address the critiques of that example, since failing to do so would have given an entry point for the global warming cult to attack my logic. They never rest in their bid to unleash some eco-fascism couched in the name of science and utilitarianism.

I won’t use my personal example to show why ‘consistently good academic record’ is not an objective criterion. Rather, I’m going to introduce another concept here to create doubt on the efficacy of the criterion.

The Survivorship Bias
I’ll explain the concept with an example related to the criteria we’re analyzing: marks.

First, I’ll assume that people who do well in school (as measured by marks) do so because:
1. Their parents force them to (most likely)
2. They love to score high marks since it gives them an identity or sense of worth or respect (somewhat likely)
3. They love their studies, and also do well in tests (least likely)

All these categories of students move on to college and do well there in terms of marks. After this stage, most of category 1 students move on in life. Some are forced into preparing for CAT. Some among category 2 see it as the ultimate exam, and revel in the challenge so they too prepare for it. Some from category 3 love the learning opportunity and they too prepare for CAT.

In the end, 375 of them make it to IIMB, helped along by lots of luck as we saw earlier. By analyzing these candidates’ profiles, what can we conclude?

IIMB likes to conclude that their consistently good academic performance is linked to their performance at IIMB. But what is IIMB actually doing? It is fooled by the survivorship bias.

It sees only the students who make it in, not the entire sample of students from which these students emerged.

If they cared to see the entire sample, they’d find students with the same ‘consistently good academic performance’ but yet not in IIMB. Since they only look at the qualities of the survivors (i.e. the students actually in IIMB), they miss the point that there are candidates with the same qualities as the survivors outside of IIMB, and that it is not these qualities of the survivors which matter for their academic performance at IIMB.

If the people both inside and outside IIMB have the same qualities, then how do these qualities predict someone’s performance there? It cannot.

I’ll make it simpler. All students have a pen. IIMB selects some students, who then write great essays. Can IIMB then say that it is because their students have a pen that they write great essays? Could it not be the case that they write great essays because IIMB trains them to write great essays? As we saw with the coach example, the initial advantage becomes a real advantage due to training. Looking for explanations elsewhere misses the whole point. When you’re fooled by the survivorship bias, you tend to attribute success to certain qualities of the survivors, missing the point that it is not these qualities which mattered for their success but something else altogether. If only these qualities mattered, then everyone, not just these survivors, would be as successful.

It’s a little tricky to understand, but well worth the effort. When you get stuck in understanding certain logic, draw a parallel. For instance, this statement of mine: If the people both inside and outside IIMB have the same qualities, then how do these qualities predict someone’s performance there? A parallel would be: If everyone in the world had a great pair of lungs, then how does this predict someone’s sprinting ability? Yes, it is a necessary, but not a sufficient condition, and definitely not the predictor of a sprinter’s ability. (I have addressed why consistently good academic performance is not a necessary condition earlier).

This is the survivorship bias. You tend to notice the qualities of the achievers and erroneously conclude that it is these factors which made the achievers what they are. If I find the same qualities among non-achievers, your conclusion goes out the window. This is why statistics without logic is dangerous. You tend to get fooled by what you see. Retro-fitting logic to statistics doesn’t work so well either, but it can be helpful at times.

I have also shown how school and graduation marks have no relevance to an MBA program. Even if marks objectively measured your subject knowledge, my knowledge of physics and engineering has no relevance to my management potential. A good doctor need not be a good actuary. The exception is math, but since CAT tests your current level of math, why go back to scores from your history?

Thus, we have seen that the rationale behind using marks as a criterion is sufficiently flawed.

4. The final nail in the coffin
Even if marks mattered, how will they be judged? Not all universities are the same. Not all boards are the same. Marks over years vary due to the difficulty level of the paper, the type of correction, the test-taking ability of the students (on aggregate) that year etc. Apart from that, a student who did very well in an easy paper may do equally well in a tough paper. We cannot judge his ability based on the difficulty level of the paper.

The solution? More statistics using student data accumulated over the years! Let’s commit more of the same mistakes!

Logic without statistics is fine, but statistics without logic is dangerous. There is no logic behind academic profile weightage. However good the statistical tools employed, the final results are bound to be flawed.

THE WORK EX MYTH
MBA education at top B-schools abroad is prohibitively expensive. Only working executives can afford it. Not wanting dissatisfied freshers (students with no work ex) with huge loans on their backs bringing down the B-school’s image, they started to enroll only working people who could afford the fee or the loan burden.

Indian B-schools only saw that foreign B-schools demanded work ex. Without thinking through the logic behind it, they simply aped the west, and concocted their own logic: that work experience matters for getting the most of an MBA program. Some went a step further and admitted only those with work ex (and ironically, also charged prohibitively high fees), completely fooled by the trend abroad.

And so the myth of work ex was born. Just like the myth that engineering plus MBA is a great combination (which sprouted up because engineers who hated engineering jumped into the non-technical field of management and did well there because a failure to do so would have meant a career as an engineer; and then people applied induction to deduce that engineers make good managers), a new myth was created: work experience is essential for an MBA.

Aside from the myth’s impact on a fresher’s chances of admission to an MBA program, most of the impact is social.

A 22-year old female engineer works for 2 years (2 years is a ‘magic number’ for work ex) in a software company, does her MBA by the age of 26 and spends 1.5 grueling years as a management trainee, only after which she gets married and contemplates starting a family. The full-time residential requirement at MBA programs affects personal life. Late marriage becomes the norm. The same is true for men, but the social impact is most distinctly felt for women.

The other problems with the work ex criterion

1. Work ex is measured in units of time
A year of data entry work is apparently ‘more valuable’ in terms of MBA admission than 10 months of BPO tech support work, which in turn is ‘more valuable’ than 8 months of Project Engineer work at an IT company, which in turn is ‘more valuable’ than 6 months of managerial work ex at a tech startup, and so on. You get the point. It is not the quality of the work ex which gets measured, but the quantity.

IIMC had a unique criterion this year: rewarding work ex in IT and Telecom sector more. Is it because mainly IT and Telecom sector employees populate B-schools? Seems like they want to firmly plonk their self-fulfilling prophecy in place: more IT and Telecom employees do MBA, they do well because they come with IT and Telecom background, so let’s reward IT and Telecom work ex more. This may not be the actual reason (for instance, recruiters may actually prefer candidates with work ex in these sectors) but this whole thing sounds fishy to me.

2. Informal sector work ex is not counted
Let’s say my friend makes around Rs. 1 lakh/- a year doing Internet Marketing part-time, while still pursuing his under-graduation. He finishes his studies and gets into Internet Marketing full-time, increasing his earnings as well as his learning in this new field. He is an entrepreneur, with no company to his name. He has no joining letter, no salary sheet, no visiting card, nothing. All he can show is a screenshot with his earnings and a bank balance. According to the proof of work ex criteria, he is unemployed.

If he does this business for 2-3 years, he can’t show it in his B-school application form and he will get summarily rejected by all B-schools simply because a person who doesn’t seek employment in the formal sector for such a long period of time is not ‘serious enough about his life’ to be considered for an MBA program. The same fate awaits successful day traders, vendors and all the self-employed who don’t have a ‘proof’ of their work ex.

I say the proof of the pudding is in the eating. Joining letters, pay slips etc. are mere bureaucratic red tape. (On that issue, one of the most annoying parts of the admission process is asking for attestation for mark sheets, admit card, score card etc.) Judge the self-employed on their terms.

3. Quality of work ex is not measurable
This is why B-schools measure quantity. No one can objectively measure the quality of an intangible attribute. However, ignoring things just because they are not measurable is a fatal flaw, since these things do have a measurable impact. (Incidentally, this is a flaw of neo-classical economics). The best way to measure quality of work ex, if it must be measured, is during the interview. Subjective evaluations have their place there, since the whole interview is in itself a subjective evaluation of a candidate’s entire life in 5-30 minutes.

4. Incentives to cheat
Now let me make this clear before I start: I am not criticizing the B-schools for this. Incentives to take the easy way out always exist. This criterion of work ex just adds to the already existing incentives. How?

I took a break from formal academics after graduating in May 2010. I didn’t seek employment in the formal sector. This practice is generally frowned upon, since only ‘not-so-serious’ people ‘waste their time’ away from college and workplace. I like the implicit assumption here: that college and workplace don’t ‘waste your time’ but doing something on your own does.

The ultimate purpose of formal education is to make money. If you don’t accept that, I challenge you to opt out of placements in your engineering/MBA (and to prove me wrong, all students have to opt out of placements). We make money so that we can live a life of our choosing. And we may also make money by living the life of our choosing. The point is that a job or education is not an end in itself to be looked upon so highly: the lifestyle you choose is. If I say no to the means (work) because I get the ends directly (leisure or my choice of lifestyle), can I be faulted for it?

But B-schools apparently value conformity (much like socialist bureaucrats) more than individual choice: You study in school and college, you go to work and you come here to do your MBA. We don’t want people who do things differently; because we are so trained in conforming that we cannot look beyond it. It is the same with the classical music snobs. All those who prefer rock and roll are tasteless, because they don’t conform to these snobs’ tastes. Just to be clear, I have nothing against either type of music, just people’s attitudes.

Some of those who took a break from the formal sector take the easy way out: fake some work ex. This is only because there is an incentive to cheat.

But all said and done, this enforcement of conformity, though subtle, plays havoc with the students’ minds: chase your dreams or conform. Most choose the latter.

What’s the link between extra-curricular activities and MBA? Nothing. Some say it is just a way to start the interview, by making the candidates comfortable by asking questions in their areas of interest. I’m all for making candidates feel comfortable. The problem is that it actually doesn’t. People who have no extra-curricular activities feel threatened. People who do have them feel like they have to know all about it since they have mentioned that they like it. It’s akin to the principal of a school saying attendance at his Saturday afternoon speech is optional. He may really mean it, but that’s not the way the teachers and students see it. Can we blame the B-schools for that? I don’t know.

And seriously, what do your achievements in sports or music have to do with your admission into an MBA program? How does an ‘achievement’ give the impression that you’re talented, and that you’ll apply this talent in a different field? It really doesn’t. As I mentioned earlier, a good surgeon need not be a good actuary.

With so many flaws in profile based selection, why do B-schools even choose to adopt it? I can think of two reasons: they’re ignorant and just want to continue business as usual, or they want students who conform to their tastes. Both aren’t objective selection measures.

The entrance test and the profile weightage are the supposedly objective selection criteria, which I have addressed so far. In my next article, I’ll examine the subjective criteria: the interview, group task, group discussion, essay writing, etc. I’m taking it up separately because it is my subjective evaluation of these subjective criteria.