Where's the science?

After a long delay, I have finally found time to update the blog (probably due to the knee replacement surgery I had done and I am getting bored laying around). The topic of this posting has its origins from multiple sources. The first being my attending the AAFS conference and sitting through several presentations in the Digital & Multimedia Sciences Section [full disclosure – myself and a student presented 2 papers]. The second source of motivation was the excellent book by Dr. Ben Goldacre "Bad Science". Both of these got me thinking about where the science is in digital forensic science? We seem to have plenty of case study presentations, tools being developed, and novel investigative protocols being proffered. What appears to be missing is any real empirical research!

Very few of the manuscripts I review report any type of hypothesis testing, statistical analysis, or at the very least error rates or reliability estimates. When these oversights get brought up, the typical refrain is that we are an applied science, not basic research. This rings hollow with me. The term applied science should and is not synonymous with a lack of proper scientific analysis, data reporting, validation or replication of findings. It is almost as if we in the community have an inferiority complex and some believe that our field is not worthy of scientific rigor.

In the context of the National Academy of Sciences report to congress on forensic sciences and the pending bills being floated around the Whitehouse (e.g. Senator Leahy's), we need to step up and step back to cast a critical eye on the science of forensic science across all of the fields, ours being no exception. I have commented before how there seems to be a lack of scientists actually involved in charting the direction of digital forensic science, a fatal mistake in my opinion.

It should be very interesting to see if external bodies such as the proposed Office of Forensic Science and the Forensic Science Board will push us in the direction of being more scientific or if they will be the typical political lame ducks and produce only the illusion of science. Unfortunately based on the historical record I predict the latter will happen. Therefore it is up to we in the community to push for better accountability and research based on proper scientific methods (even a focus on reproducibility would be a giant leap in the right direction).

Here is an interesting interview with Ben Goldacre on the booming age of pseudo-science:


Pseudo Science

Ben Goldacre