Does Digital Forensics Suffer from Physics Envy?

For over a decade we have been fighting to have digitial forensics recognized as a science. Committees have been struck, organizations created and a great deal of blood sweat and tears has gone into having the American Academy of Forensic Sciences recognize us as a new section - Digital and Multimedia Sciences. We have also written numerous papers, conducted workshops and heavily marketed for the covetted recognition as a "SCIENCE".

What if we got it wrong? Can we really make a solid case for digital forensics being a science? The goal of science is the pursuit of knowledge. This is accomplished by using the scientific method or process. Theories are derived, hypotheses created and experiments designed to test these educated guesses. The interpretation of the findings are supposed to be value free and the results reproducable.

The goal of technology on the other hand is to meet the needs of some applied problem, focusing on some short term solution. The major processes include design, implementation, and testing. The deteremination of the success or failure is value ladden and reproducability of findings not necessarily criticial.

If we limit our discussion to the current state of digital forensics which category do we more easily fit into? It is really a no brainer - we are a technology that may at some point in the future move to a science, but we are not their yet.

The next important issue to contemplate is whether we actually have to become a science. Can we still serve our purpose and mandate (as well as the courts') by remaining a technology? Maybe we just have a case of Physics envy.

Who watches the watchers?

We are reaching what could be termed a tipping point in the development/ maturation of digital forensics. We are starting to see an increasing demand for certifications specific to digital forensics. This parallels the historical development of information security and assurance. As the demand for more secure networks and personnel trained to test and defend these systems increased, so to did the need for industry to gauge who was at least minimally qualified to claim they were professionals in this field. This demand spurred on a industry dedicated to certify professional information security professionals. Unfortunately a conflict of interest soon arose. The same companies that were certifying the professionals through some type of examination, were also selling the training and study guides for their tests. One of the cardinal rules of accreditation and certification was broken – no independent accreditation body was setup to provide oversight for the training and testing companies. The fox was and is guarding the hen house.

Digital forensics now has numerous certifying bodies which in fact are thinly disguised training and testing companies. Come take our training, read our guide, take our test and low and behold you are certified in digital forensics. The arguments put forward for using this model yet again is that it if it is good enough for information security then its good enough for us. But wait, we can claim a direct lineage to the forensic sciences. Given this context we can look to the world of forensics and determine what our sister sciences have done. The forensic sciences seem to be very sensitive to the issues of conflicts of interest, whether real or perceived. As such the Forensic Specialities Accreditation Board (FSAB) was developed. The mission of the FSAB is as follows:

"The goal of this program is to establish a mechanism whereby the forensic community can assess, recognize and monitor organizations or professional boards that certify individual forensic scientists or other forensic specialists. This program has been established with the support and grant assistance of the American Academy of Forensic Sciences (AAFS), the National Forensic Science Technology Center (NFSTC) and theNational Institute of Justice (NIJ)."

Currently there about 16 accrediting boards recognized by the FSAB. Unfortunately there are no recognized boards or bodies related to digital forensics. This presents us with a very real problem. The FSAB standards and criteria cover the management structure of the body, examination and certification standards, competency of evaluation personnel, recertification etc. The standards also states that no certified members can be "grand fathered":

"Grandfathering is not an acceptable method of certification. 5.1.4.1 Certification bodies that used grand fathering and which were established prior to February 17, 2001, may apply for accreditation if not more than 50% of its certificants were grandfathered. Note: An individual is considered “grandfathered” if the person was issued a certificate without having taken and attained a satisfactory score in an examination designed to assess the knowledge, skills and abilities in the stated field of certification. 5.1.4.1.1 Any grandfathered certificants must be subjected to the same examination and competency assessment as new applicants (as defined in 5.3 of these standards) no later than the regularly scheduled recertification for that individual, not to exceed a period of five years.5.1.4.2 No certification body established after Feb 17, 2001, may apply for accreditation until all its certificates have been issued according to the standards as defined section 5.1.4 of these standards."

Grand fathering has been a popular method in the Information Security field in order to instantly populate a new credential with a critical mass of members. It would seem that this practice is off the table with forensics.

The lack of any recognized accreditation board for digital forensics (at least by the FSAB, AAFS, NIJ) translates into certifications that may have little if any value. This is not to say that there aren't some first rate digital forensics professional certifications out there, it just means that none are technically recognized by the same standard as other forensic sciences. This fact cannot be lost on opposing counsels and will certainly find its way into the court room in the not so distant future.

More upcoming Pubs

I am proud to announce that I have been working with the folks at Chelsea House (James Chambers in particular) on a series of edited books for young people ages 12-18 yrs old. The objective of the series is to provide young people with a frank discussion related to "Cybersafety". As the Chief Technical Editor I had the pleasure of working with some of the leading authors and researchers in the areas of cyber stalking, cyber bullying, online addictions, cyber predators, identity theft etc.

The series should make an excellent resource for middle and secondary schools. If anyone is interested in being involved in the second edition of this series, please just let me know.

http://www.amazon.com/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=cybersafety+Rogers&rh=i%3Aaps%2Ck%3Acybersafety+Rogers&ajr=3

Response to Where's the Science

I appreciate the opportunity to discuss this topic on your blog.  I find that my opinion would vary from yours to some degree.

While science and experiments are a vital and necessary part of digital forensics a large part of the evidence uncovered during an exam does not necessarily require an application of science or an experiment.  The people who originated the term “Computer Forensics” could have picked a better phrase to define the discipline.  I prefer to think of most of the work I do as a “forensic search” of a piece of evidence.  I preserve the evidence, the image, in such a way that it is not altered and anyone can duplicate my work from that image and then I search it just as a detective would search a house for a gun, narcotics etc. 

The vast majority of evidence that I have located in exams has come from allocated space.  In my report I document the file, its dates and times and its location on the storage medium.  As an example I had a case where I was requested to examine a Blackberry for evidence which might link a given suspect to a bank robbery.  On the micro SD card in the phone I recovered a photograph of the suspect holding a version of the MAC 10 machine gun.  Statements from the witnesses and the video show one of the suspects holding a MAC 10.  The prosecutor and the jury found the photograph amusing.  My testimony consisted of stating that I found the graphic on the micro SD card.  I did not testify that it was a real gun or to any other fact other then I found the picture and where.

Using the image of the SD card the graphic could be located by any forensic tool available.  Since the report provides the location of the file I could even clone the image to another SD card.  The card could be put in a media reader and anyone computer literate could navigate to the same evidence I found.  It was nothing special and anyone could have done the same.

I read the report from the National Academy of Sciences and came out with a slightly different idea of what they found distressing.  I believe the central complaint in that report is not that the science in the disciplines is lacking but that the opinions expressed by the experts in court tend to go beyond their literal/explicit findings assuming they bother to do an exam at all.

In a recent discussion in a college class I asked the question if a computer examiner could state that passwords held any evidentiary value.  My position was that they hold none except in very limited circumstances.  As an examiner I can determine that a password exists or does not exist and I may even be able to determine what it is but that is it.  From an exam I cannot tell who in the home or office that the device was recovered knew the password.  I cannot tell if someone walked away from their computer etc.  In short I cannot testify as a computer examiner to what other people know.  It was interesting that many of the students insisted that you might be able to associate a password with a user if they used something personal like a date of birth or used the same password for multiple things.  They completely over looked the fact that they would be testifying to information not in their forensic exam but their personal opinion of someone’s computer habits.

Ego can be a problem when testifying.  As an “expert” there is a temptation to be willing to provide an opinion to whatever question you are asked on the stand.  Judges and attorneys often do not help as they frequently regard anyone with computer knowledge as an all knowing expert on all subjects involving computers.  It is up to the examiner to let the attorney/judge know that he or she does not have that knowledge or expertise.  It is very difficult to tell people looking up to you as the source of all knowledge that you don’t know.  I would say that the science is not lacking in forensics but the willingness of experts to provide opinions outside of the literal results of their findings is the actual problem.

Please don’t get the impression that I believe that science and research have limited value in digital forensics.  I do believe that experiments and research are of use and necessary in digital forensics.  I have engaged in them from time to time in particular when dealing with files recovered from unallocated space or fragments of files.  Determining what program generated the file or fragment, determining the evidentiary value or lack there of, has most of the time required experiments and research.  I do believe that much of my work is simply a search which identifies information of value.

Posted by:

Sgt. Kevin Stenger
Orange County Sheriffs Office
Orlando Florida 

Request for Authors

I am proud to announce that the "Encyclopedia of Information Assurance": SBN-10: 142006620X ISBN-13: 978-1420066203 is now out.

Encyclopedia of Information Assurance

We are currently working on the second edition and are actively soliciting authors for this edition. If you are interested in working on the second edition please contact me directly at: rogersmk@purdue.edu or Rich O'Hanley <rich.ohanley@taylorandfrancis.com>.

Where's the science?

After a long delay, I have finally found time to update the blog (probably due to the knee replacement surgery I had done and I am getting bored laying around). The topic of this posting has its origins from multiple sources. The first being my attending the AAFS conference and sitting through several presentations in the Digital & Multimedia Sciences Section [full disclosure – myself and a student presented 2 papers]. The second source of motivation was the excellent book by Dr. Ben Goldacre "Bad Science". Both of these got me thinking about where the science is in digital forensic science? We seem to have plenty of case study presentations, tools being developed, and novel investigative protocols being proffered. What appears to be missing is any real empirical research!

Very few of the manuscripts I review report any type of hypothesis testing, statistical analysis, or at the very least error rates or reliability estimates. When these oversights get brought up, the typical refrain is that we are an applied science, not basic research. This rings hollow with me. The term applied science should and is not synonymous with a lack of proper scientific analysis, data reporting, validation or replication of findings. It is almost as if we in the community have an inferiority complex and some believe that our field is not worthy of scientific rigor.

In the context of the National Academy of Sciences report to congress on forensic sciences and the pending bills being floated around the Whitehouse (e.g. Senator Leahy's), we need to step up and step back to cast a critical eye on the science of forensic science across all of the fields, ours being no exception. I have commented before how there seems to be a lack of scientists actually involved in charting the direction of digital forensic science, a fatal mistake in my opinion.

It should be very interesting to see if external bodies such as the proposed Office of Forensic Science and the Forensic Science Board will push us in the direction of being more scientific or if they will be the typical political lame ducks and produce only the illusion of science. Unfortunately based on the historical record I predict the latter will happen. Therefore it is up to we in the community to push for better accountability and research based on proper scientific methods (even a focus on reproducibility would be a giant leap in the right direction).

Here is an interesting interview with Ben Goldacre on the booming age of pseudo-science:


Pseudo Science

Ben Goldacre

The Coming Storm - Cloud Computing and Digital Investigations

By now we all heard how cloud computing will revolutionize the Internet and be the next best thing to happen to online businesses, consumers, education and the world at large. But we haven't heard much of what investigative concerns the so-called cloud brings with it. As most of us realize, the concept of cloud computing is nothing new. Technically we have been living with this "cloud" since the inception of the Internet and the World Wide Web. What this new cloud concept seems to add to the equation, is the ability to have various levels of distributed storage and application services.

While there are numerous security concerns being discussed by various cyber security "Czars," there seems to be little if any discussion about how the cloud will affect digital forensic investigations. Just off the top of my head I can think of several concerns that are generic to the concept of cloud computing to say nothing of specific concerns related to specific implementations or hardware and software applications.

Some basic questions are related to:

a) Jurisdiction - which sovereign nation or nations has/have authority?

b) Ownership - who actually owns the data in question?

c) Expectations of privacy - what will be the standard for reasonable expectations of privacy in the cloud?

d) Location of evidence - where do we even begin to look for data that may be classified as evidence for the investigation?

e) International cooperation - will countries housing/storing the data be willing to cooperate during an investigation?

f) Localized evidence - what artifacts will be left on the client machine?

To me these seem like obvious questions/concerns that we need to think about, debate and start working toward some answers. As I stated in the opening paragraph, the cloud is being touted as the greatest thing since "sliced bread," whether this is actually the case or not.

We as investigators will soon find ourselves truly immersed in the world of "virtual" evidence; a very sobering thought. One can only imagine how a judiciary who has trouble wrapping its mind around the concept of e-mail, will be able to keep up with the various technical solutions that make up the concept of cloud computing.

It behooves the digital forensics community to weigh in on discussions related to cloud computing and provide input as to what this latest technology savior will eventually become.