Bait for the MedWeb2Skeptic
Gah, @amcunningham beat me to a proper look at this paper on web2 use in medical education. To be fair, I was on night shift at the time, so wasn't really in the right frame of mind to write anything longer than 140chars. Still, feeling quite chuffed that I got in there early with the critique, even if it was a little... concise.
Anyway, there isn't a massive amount to add to Anne-Marie's skewering of this survey-based paper on use of Web2 tools in medical/nursing education - she rightly critiques the low response rate, confusion & conflation of web2 / social media tools, and the authors' rather bold conclusions (subsequently echoed around the twittersphere).
The authors do acknowledge one of the paper's weaknesses when they state:
"...given the small sample size, it is difficult to predict whether the use of Web 2.0 tools portends a growing trend in education or merely represents a passing fad"
But although they note the small sample size, they still accept their findings as significant, albeit perhaps transient. To be honest, in this paper, the future of web2 use in medical education is not 'difficult to predict', it's completely outwith any of the conclusions that could possibly be drawn from the data.
But just a few more points...
1. A survey of web2 usage by medical/nursing institutions by a fairly open-access survey, with a very poor response rate means that any conclusions must be interpreted with a degree of caution. But it's not just the low response rate that sounds a note of caution. One also has to question why those particular people bothered to respond (selection bias). It's easy to hypothesize that survey recipients who'd never heard of Moodle etc would just delete the email, while those who were evangelical about using wikis and youtube would reply in their droves. So the sample biases itself.
2. I think there's two other ways to do this kind of research - either spend some time identifying IT/education leads at medical schools and send them a better-designed survey asking questions about overall web2 tool use in medical school, or survey a large number of medical students from several medical schools with a very short survey to ask what tools they actually use on a regular basis.
3. As Anne-Marie mentioned, the qualitative data isn't mentioned. My guess is that there wasn't very much of it. The question is too broad and vague "please briefly describe how these tools are used in your institution". This makes it difficult to answer (therefore most respondents probably don't bother) and unlikely to identify any common themes, as the responses given are likely to be highly heterogenous. If you've ever tried to get useful qualitative responses from questionnaires, you learn this lesson pretty quickly. I did, and I was doing an MSF in my spare time.
So, having kicked the corpse a bit, what's the real issue here? I think it's this - apart from generating headlines, what use is this kind of research anyway? So 45% of medical/nursing schools use web2 tools. Big woop. Who uses them? What for? How? How often? And most importantly, why? If a web2 tool can deliver a better educational outcome (or an equivalent one more cheaply / easily / quickly) than a conventional teaching method, that's a good thing. But just using web2 education tools isn't important - it's what you do with them that counts.
Anyway, there isn't a massive amount to add to Anne-Marie's skewering of this survey-based paper on use of Web2 tools in medical/nursing education - she rightly critiques the low response rate, confusion & conflation of web2 / social media tools, and the authors' rather bold conclusions (subsequently echoed around the twittersphere).
The authors do acknowledge one of the paper's weaknesses when they state:
"...given the small sample size, it is difficult to predict whether the use of Web 2.0 tools portends a growing trend in education or merely represents a passing fad"
But although they note the small sample size, they still accept their findings as significant, albeit perhaps transient. To be honest, in this paper, the future of web2 use in medical education is not 'difficult to predict', it's completely outwith any of the conclusions that could possibly be drawn from the data.
But just a few more points...
1. A survey of web2 usage by medical/nursing institutions by a fairly open-access survey, with a very poor response rate means that any conclusions must be interpreted with a degree of caution. But it's not just the low response rate that sounds a note of caution. One also has to question why those particular people bothered to respond (selection bias). It's easy to hypothesize that survey recipients who'd never heard of Moodle etc would just delete the email, while those who were evangelical about using wikis and youtube would reply in their droves. So the sample biases itself.
2. I think there's two other ways to do this kind of research - either spend some time identifying IT/education leads at medical schools and send them a better-designed survey asking questions about overall web2 tool use in medical school, or survey a large number of medical students from several medical schools with a very short survey to ask what tools they actually use on a regular basis.
3. As Anne-Marie mentioned, the qualitative data isn't mentioned. My guess is that there wasn't very much of it. The question is too broad and vague "please briefly describe how these tools are used in your institution". This makes it difficult to answer (therefore most respondents probably don't bother) and unlikely to identify any common themes, as the responses given are likely to be highly heterogenous. If you've ever tried to get useful qualitative responses from questionnaires, you learn this lesson pretty quickly. I did, and I was doing an MSF in my spare time.
So, having kicked the corpse a bit, what's the real issue here? I think it's this - apart from generating headlines, what use is this kind of research anyway? So 45% of medical/nursing schools use web2 tools. Big woop. Who uses them? What for? How? How often? And most importantly, why? If a web2 tool can deliver a better educational outcome (or an equivalent one more cheaply / easily / quickly) than a conventional teaching method, that's a good thing. But just using web2 education tools isn't important - it's what you do with them that counts.
No comments:
Post a Comment