To the Edtech World,
Stop showing pre-test and post-test statistics as metric of success!
I was advising an edtech company on their upcoming product. Their user testing entailed conducting a pre test, using the product and then a post test. And then they showed glowing results.
This is not the first time I am seeing this. Pedagogical interventions in general have umpteen research studies based on pre test and post test.
Unfortunately, that's not the right way of measuring success. It doesn't tell you anything - oh sorry, let me rephrase - it can never tell you if the product is good, it can only say if it's worse.
In a classroom where students are accustomed to a certain mundane pedagogy, any decent intervention will yield a positive outcome irrespective of whether the new pedagogy is better than the previous one or otherwise.
That's just one example. Objective testing in general is problematic, we need to really look beyond the age old pre tests and post tests.
PS: The only place you will find some relevance of pre test and post test is when you're generating longitudinal data, but still will not recommend.
Comentarios