Related Posts with Thumbnails

Thursday, January 28, 2016

Qualitative VS Quantitative Assessment/Research

This post is just some random musings about Ong Ye Kung's call for a higher degree of qualitative assessment by the government. I am just so happy that Ong Ye Kung stuck around after his 2011 failure to get voted in. It seems to me that he is clear what he believes in and much of what he says he believes in, resonates with me.

When Quantitative Research Was King
When Francis Galton devised the first intelligence tests, he single-handedly invented the branch of psychology called Psychometrics. That was in the late 1800s.

Alfred Binet (who developed the Binet-Simon scale), Wilhelm Stern (who developed the first formula to calculate the Intelligence Quotient) and Lewis Terman (who further advanced the quantitative wok of Dalton, Binet and Stern in the Unites States from whence it spread to all over the world) all studied psychology using numbers. The study of psychology is inserted within the wider domain of social science research. In effect, ever since the late 1800s, social science research has been enamoured of the quantitative approach.

Historically, social science research itself has never had the same kind of prestige as research in hard sciences. For a long time, in order to prove themselves scientists, people who studied social phenomenon adapted hard science methods of study to social phenomenon. These were quantitative methods of the sort most people are familiar with - control groups and statistical significance.

Not surprisingly then, qualitative research was considered second class (by the larger proportion of social scientists themselves). It would appear now that social science research has now come of age. Indeed, Singapore has set up a Social Science Research Council after all these years of focusing on the hard sciences.

The Rise of Qualitative Research
By coming of age, the social sciences have taken on the confidence to be themselves and to develop methods consistent with the material being studied. Hence, along with this newfound confidence came the rising star of Qualitative Research.

I have observed that since the early 1990s, the voices of qualitative approach researchers have become ever louder in top tier social science research journals. Myself, I am only familiar with the top tier business research journals (such as The Academy of Management Journal, The Journal of Applied Psychology and Administrative Science Quarterly) but I would imagine that the spirit of Qualitative Research has grown in stature amongst researchers in every social science.

The Advantages of Qualitative Research
Proponents of Qualitative Research developed these qualitative research methods because they experienced the inadequacies of Quantitative Research in elucidating the whys and wherefores of social phenomenon. For example, quantitative research informs us that 1st class Honours graduates earn more (on average) than a non-grad. This is true when you look at mass tendancies. The majority of 1st class Honours grads (at age 23) will evolve earn more at age 50 than a non-grad (at age 23), even if the non-grad eventually becomes a grad by age 50.

However, there are people like these 12 Singaporeans Who Prove You Don't Need A Degree to Succeed. Suppose you want to understand why these non-grads make it big (whilst other non-grads don't), you would need to study them and each of their paths individually. You really could not get at the richness of the reasons why they are successful, with a mere Likert scale survey.

In general too, illiterate people don't earn much. Yet, David Gan is illiterate and he earns more than many graduates. In this case, there is only one David Gan. How do you propose to study the whys and wherefores of his success using statistical techniques and the bell curve?

Similarly, qualitative researchers like to cite the case of Phineas Gage, in support of the usefulness and importance of qualitative research. Phineas Gage was a railroad construction foreman. He sustained a traumatic brain injury. He was a hardworking and pleasant man before his accident. A 43 inch rod with a diameter of 1.25 inches entered and exited his brain. After his accident, he became surly and temperamental.

Clearly, you cannot go about shooting 43 inch rods into and out of the brains of 1000 research participants along the exact same trajectory through the brain, in order to later apply quantitative statistical techniques to explain the change post-accident in Phineas Gage's personality. A qualitative study of Phineas Gage's singular brain did teach neuroscientists a great deal about the brain.

Reducing social phenomenon to bell curves fails to explain individually rare phenomenon. Bell curves fail to explain people and events that are infinitely valuable to the human condition simply because they are so few (perhaps only one). Measures do not exist when there is only ONE case. If there is only one, you CANNOT have control groups. You cannot have bell curves and statistical significance.

You cannot have a t-score.

What Is Wrong With the PSLE T-Score (our national quantitative measure of a child's worth?)
The t-score is a proxy measure of 3 things...
- child's IQ
- child's motivation
- child's privileged access to quality tuition or parent coaching

The t-score is a measure of competence in...
- English
- Math
- Science
- Mother Tongue

The t-score is a very very very narrow measure of a child's worth.

Actually, no one has ever explicitly said that the t-score is a measure of a child's worth. However, the fact that it is used (almost exclusively) to decide entrance into secondary schools means that it is a yardstick by which a schools judge whether or not a child is worthy to enter this or that school.

What it really measures is a very narrow part of the whole child (mastery in 4 subjects). The t-score does not show up rare and emerging skills (e.g., Bill Gates' ability to code before coding became a recognisable skill)... a child's sense of humour... a child's ability to influence his peers (primary school prefects, without their official status, may not be good at persuading their peers)... a child's kindness... a child's resourcefulness when caught in a bind... a child's passion ... a child's innate sense of social responsibility... a child's determination to rise out of poverty...

Surely, these qualities are important in tomorrow's top talent, and deserve consideration when it comes to selection into secondary schools.

Even more importantly, the t-score fails to pin down those who will succeed in the future because they are different than anything the t-score currently measures. Their worth to the world of the future is in areas that the t-score does not capture. T-scores do not capture the X-factor. A qualitative assessment however, can surface that special X-factor that is the child's very own gifting.

We now recognise that the t-score is an insufficient measure of talent. Yet, we are afraid to step into the unknown of qualitative assessment for fear of subjectivity and cronyism.

All My Students Have Worth
Though trained to do both quantitative and qualitative research, my personal preference is qualitative research.

You don't leave behind years of training. So, I cannot help it but see something special in every child I teach. This one will grow up into a Michelin star chef. That one will be a world-renowned singer. That one there will break the mould in painting. This one here is a go-getter. That one there is a mediator. Yet that one will one day make it to CEO of a multi-national. I can see that other one as Entrepreneur of the Year.

All that latent potential that even their parents do not see because parents are blinded by the t-score (English, Math, Science, Chinese).

I teach them English but when they come for class, I see super-imposed upon their faces the ghostly images of people of the future that escapes being defined by their A* in English, or their t-scores. T-scores do not capture specialness, unless at the very top score... and even then, the child is only special in that he/she is the best in the country in English, Math, Science and Mother Tongue.

Here in Dr. Pet's Enrichment, I do not compute class averages. I do not calculate percentiles. I could think of many statistical operations to do with the grades given by the schools and the weekly grades I assign.

I don't.

Instead, I think about EACH child as an individual that cannot be compared with another numerically. I observe EACH child and make qualitative observations and construct special teaching moments needed by that child only. It is great fun to work like that.


Anonymous said...

Yup, just like the movie X-men ...

Petunia Lee said...

How so like the movie X-men?

Anonymous said...

Dr Pet
i guess another possible reason why ppl used to "swear by" quantitative research is because they somehow believe that stats and numbers do not and cannot lie - they are objective.

lies, damned lies and statistics ...