Let the facts speak for themselves
One of the things I love about science is that everything happens without human intervention: measurements were made, errors were introduced, the data shows, ... as near as you can tell, the experiment, the document, the peer review all happened spontaneously without human intervention.
The language is carefully chosen to remove perception of bias – after all, who can argue with data?
Well, I can. The data shows nothing. In fact, data is meaningless. Data only acquires meaning within a context. And a context has bias.
You don't believe me? Here is some data: 1 1 2 3 5 8 13 21.
Some of you probably recognized it as the first 8 terms of the Fibonacci sequence. Is it the number of petals on flowers, the seeds in a sunflower, the curvature of a nautilus shell? Without context, it has no meaning.
The data in a scientific paper has no intrinsic meaning. It acquires meaning only in the context in which it is presented and, despite the detached language used, the context is constructed to give meaning to the data.
One of my favourite examples is a piece of medical diagnostic software I got my hands on (this used to be a competitor of the company I worked for, but after a couple of purchases, we were one big happy family). The software boasted minimum accuracy of 75% (documents submitted to the FDA). When I ran the software on their test cases, I got very shocking results on one file - there were almost 1400 false positives in a file with only 4 real events. This is a very high noise to signal ratio - 1400 results to identify the 4 real ones? Yet, they claimed with a straight face they met the 75% requirement because they correctly identified 3 out of 4 real events. Here is the context, "How many real events were correctly identified?"
When someone tells you to let the data speak for itself, they are really telling you to interpret the data in the context they have provided. Nobody ever spends time explaining the data, they spend time explaining the context. And the bias is in the context.
The language is carefully chosen to remove perception of bias – after all, who can argue with data?
Well, I can. The data shows nothing. In fact, data is meaningless. Data only acquires meaning within a context. And a context has bias.
You don't believe me? Here is some data: 1 1 2 3 5 8 13 21.
Some of you probably recognized it as the first 8 terms of the Fibonacci sequence. Is it the number of petals on flowers, the seeds in a sunflower, the curvature of a nautilus shell? Without context, it has no meaning.
The data in a scientific paper has no intrinsic meaning. It acquires meaning only in the context in which it is presented and, despite the detached language used, the context is constructed to give meaning to the data.
One of my favourite examples is a piece of medical diagnostic software I got my hands on (this used to be a competitor of the company I worked for, but after a couple of purchases, we were one big happy family). The software boasted minimum accuracy of 75% (documents submitted to the FDA). When I ran the software on their test cases, I got very shocking results on one file - there were almost 1400 false positives in a file with only 4 real events. This is a very high noise to signal ratio - 1400 results to identify the 4 real ones? Yet, they claimed with a straight face they met the 75% requirement because they correctly identified 3 out of 4 real events. Here is the context, "How many real events were correctly identified?"
When someone tells you to let the data speak for itself, they are really telling you to interpret the data in the context they have provided. Nobody ever spends time explaining the data, they spend time explaining the context. And the bias is in the context.
Comments
Have a great Canada Day weekend! Must be lots going on in Ottawa. We're just staying here and may catch Fireworks in out town.
We had a BBQ over at my dad's place Sunday afternoon and returned to Montreal early enough to participate in some Canada Day festivities in the evening and see some fireworks. The kids liked the fireworks. It was a relatively small crowd where we were and the fireworks were quite nearby (I took some picture - they look fine on the little LCD screen, but I suspect they might look awfully shaky on the big screen).