I was driving home on Friday evening last week when a discussion on Radio 4 caught my attention. It concerned the famed Hawthorne Experiment. This was a set of experiments conducted in the 1920's on workers, it centered around making changes in work place conditions and studying the effects on worker output.

The results of this research have been constantly cited ever since by academics and consultants referencing the key findings. The Hawthorne Effect is neatly summed up by the definition in the Oxford English Dictionary (OED) as meaning “an improvement in the performance of workers resulting from a change in their working conditions, and caused either by their response to innovation or by the feeling that they are being accorded some attention.”

However, it seems the approach in the experimentation was flawed and the data incorrectly analysed and interpreted. Here are two references, the first quick and easy to read; an article in the Economist and the second the full blown and damming analysis conducted by the people who discovered the lost data from these experiments. I have extracted quotes which follow on from each url giving a flavour of their analysis.

http://www.economist.com/node/13788427

"It turns out that idiosyncrasies in the way the experiments were conducted may have led to misleading interpretations of what happened. For example, lighting was always changed on a Sunday, when the plant was closed. When it reopened on Monday, output duly rose compared with Saturday, the last working day before the change, and continued to rise for the next couple of days. But a comparison with data for weeks when there was no experimentation showed that output always went up on Mondays. Workers tended to beaver away for the first few days of the working week in any case, before hitting a plateau and then slackening off."

http://home.uchicago.edu/~jlist/papers/Was%20There%20Really%20a%20Hawthorne%20Effect%20at%20the%20Hawthorne%20Plant_An%20Analysis%20of%20the%20Original%20Illumination%20Experiments.pdf

"The “Hawthorne effect” draws its name from a landmark set of studies conducted at the Hawthorne plant in the 1920s. The data from the first and most influential of these studies, the “Illumination Experiment,” were never formally analyzed and were thought to have been destroyed. Our research has uncovered these data. Existing descriptions of supposedly remarkable data patterns prove to be entirely fictional."

"The illumination studies have been hailed as being among the most important social science experiments of all time, but an honest appraisal of this experiment reveals that the experimental design was not strong, the manner in which the studies were carried out was lacking, and the results were mixed at best. Perhaps fittingly, a meta-analysis of the research testing the enormous body of research into Hawthorne effects triggered by this initial study yields equally mixed results (Adair, Sharpe, and Hyunh 1989).

Perhaps the most important lesson to be learned from the original Hawthorne experiments is the power of a good story. The mythology surrounding the Hawthorne experiments arose largely absent careful data analysis, and has persisted for decades even in the face of strong evidence against it generated by Franke and Kaul (1978) and Jones (1992). While our research is probably no more likely than the previous papers to put an end to such myths, at a minimum it raises the costs of propagating these stories among those who are concerned with scientific accuracy."