The intuitive answer to those of us involved in ICT is “of course it does”. However, the evidence from research is not conclusive. I think the reason is that it’s actually very difficult to carry out robust research in this area. As the impact of ICT has been a topic for discussion recently in the Naace and Mirandanet mailing lists, I thought it might be useful to try and clarify the issues as I see them.

The question “Does ICT improve learning?” naturally leads on to a set of other questions that need to be addressed:

What ICT?

The question as stated is too broad. A computer is not the same as a suite of computers. It’s not even the same as a laptop, which is not the same as a handheld device. Software is not the same as hardware, and generic software, such as a spreadsheet, is not the same as specific applications, such as maths tuition software.

What other factors are present?

ICT doesn’t happen in a vacuum. What is the environment in which the technology is being used? How is the lesson being conducted? What is the level of technical expertise of the teacher? What is the level of teaching expertise of the teacher? These and other factors mentioned in this article are not stand-alone either: they interact with each other to produce a complex set of circumstances.

What is the ICT being used for?

What is being taught? There is some evidence to suggest that computers are used for low-level and boring tasks like word processing, in which case comparing technology-“rich” lessons with non-technology-rich lessons is not comparing like with like. On the other hand, technology can be, and often is, used to facilitate exploration and discussion. Since these are educationally-beneficial techniques in their own right, the matter of validity needs to be scrutinised (see below).

How is the impact of the ICT being evaluated?

There are several ways in which this might be done, each with their own advantages and disadvantages. For example, in-depth case studies yield rich data but may be difficult to generalise from. Also, there are three other problems. One is that it is difficult to conduct experiments using a suitable control group, because no teacher wishes to try something which may disadvantage a particular group of students. Another is the so-called “starry night” effect, in which case studies focus (naturally) on the successful projects whilst ignoring all the ones which either failed or were not believed to have deliver the same level of benefits. Finally, there is the danger of all kinds of evaluation study, that the methodology itself may affect the outcome.

What exactly is being measured?

This is the issue of validity, already touched upon. Are we measuring the ability of a teacher to conduct a technology-rich lesson, in which case it’s the effectiveness of the teacher rather than the ICT that is being weighed up? By implication, it may be the quality and quantity of professional development which is being measured. It may be students’ home environments that are inadvertently being evaluated, or student-staff relationships.

How much is ICT being used?

I suggest there may be a difference between schools in which ICT is being used more or less everywhere, and those in which it’s hardly being used at all. In the former, presumably both teachers and students would be accustomed to using it, there would be a good explicit support structure in the form of technical support and professional development, and a sound hidden support structure in the form of being able to discuss ideas with colleagues over lunch or a cup of coffee.

Is there an experimenter effect going on?

This is the phenomenon whereby the results of a study confirm or tie in with the expectations of the people or organisation responsible for the study. This is an unconscious process, not a deliberate attempt to cheat. I’ve explained it in my article called Is Plagiarism Really a Problem?

Conclusion

My own feeling – backed up by experience --  is that in the right set of circumstances, the use of ICT can lead to profound learning gains. However, rather than falling into the trap of arguing whether ICT is “good” or “bad”, we need to move the debate onto a much sounder intellectual basis.

Further reading

I’d highly recommend Rachel M. Pilkington, “Measuring the Impact of Information Technology on Students’ Learning”, in The International Handbook of Information Technology in Primary and Secondary Education, Springer, 2008, USA.

What are the big issues facing ICT (Ed Tech) leaders? Please take a very short survey to help us find out.