Response to Where's the Science

I appreciate the opportunity to discuss this topic on your blog.  I find that my opinion would vary from yours to some degree.

While science and experiments are a vital and necessary part of digital forensics a large part of the evidence uncovered during an exam does not necessarily require an application of science or an experiment.  The people who originated the term “Computer Forensics” could have picked a better phrase to define the discipline.  I prefer to think of most of the work I do as a “forensic search” of a piece of evidence.  I preserve the evidence, the image, in such a way that it is not altered and anyone can duplicate my work from that image and then I search it just as a detective would search a house for a gun, narcotics etc. 

The vast majority of evidence that I have located in exams has come from allocated space.  In my report I document the file, its dates and times and its location on the storage medium.  As an example I had a case where I was requested to examine a Blackberry for evidence which might link a given suspect to a bank robbery.  On the micro SD card in the phone I recovered a photograph of the suspect holding a version of the MAC 10 machine gun.  Statements from the witnesses and the video show one of the suspects holding a MAC 10.  The prosecutor and the jury found the photograph amusing.  My testimony consisted of stating that I found the graphic on the micro SD card.  I did not testify that it was a real gun or to any other fact other then I found the picture and where.

Using the image of the SD card the graphic could be located by any forensic tool available.  Since the report provides the location of the file I could even clone the image to another SD card.  The card could be put in a media reader and anyone computer literate could navigate to the same evidence I found.  It was nothing special and anyone could have done the same.

I read the report from the National Academy of Sciences and came out with a slightly different idea of what they found distressing.  I believe the central complaint in that report is not that the science in the disciplines is lacking but that the opinions expressed by the experts in court tend to go beyond their literal/explicit findings assuming they bother to do an exam at all.

In a recent discussion in a college class I asked the question if a computer examiner could state that passwords held any evidentiary value.  My position was that they hold none except in very limited circumstances.  As an examiner I can determine that a password exists or does not exist and I may even be able to determine what it is but that is it.  From an exam I cannot tell who in the home or office that the device was recovered knew the password.  I cannot tell if someone walked away from their computer etc.  In short I cannot testify as a computer examiner to what other people know.  It was interesting that many of the students insisted that you might be able to associate a password with a user if they used something personal like a date of birth or used the same password for multiple things.  They completely over looked the fact that they would be testifying to information not in their forensic exam but their personal opinion of someone’s computer habits.

Ego can be a problem when testifying.  As an “expert” there is a temptation to be willing to provide an opinion to whatever question you are asked on the stand.  Judges and attorneys often do not help as they frequently regard anyone with computer knowledge as an all knowing expert on all subjects involving computers.  It is up to the examiner to let the attorney/judge know that he or she does not have that knowledge or expertise.  It is very difficult to tell people looking up to you as the source of all knowledge that you don’t know.  I would say that the science is not lacking in forensics but the willingness of experts to provide opinions outside of the literal results of their findings is the actual problem.

Please don’t get the impression that I believe that science and research have limited value in digital forensics.  I do believe that experiments and research are of use and necessary in digital forensics.  I have engaged in them from time to time in particular when dealing with files recovered from unallocated space or fragments of files.  Determining what program generated the file or fragment, determining the evidentiary value or lack there of, has most of the time required experiments and research.  I do believe that much of my work is simply a search which identifies information of value.

Posted by:

Sgt. Kevin Stenger
Orange County Sheriffs Office
Orlando Florida 

Request for Authors

I am proud to announce that the "Encyclopedia of Information Assurance": SBN-10: 142006620X ISBN-13: 978-1420066203 is now out.

Encyclopedia of Information Assurance

We are currently working on the second edition and are actively soliciting authors for this edition. If you are interested in working on the second edition please contact me directly at: rogersmk@purdue.edu or Rich O'Hanley <rich.ohanley@taylorandfrancis.com>.

Where's the science?

After a long delay, I have finally found time to update the blog (probably due to the knee replacement surgery I had done and I am getting bored laying around). The topic of this posting has its origins from multiple sources. The first being my attending the AAFS conference and sitting through several presentations in the Digital & Multimedia Sciences Section [full disclosure – myself and a student presented 2 papers]. The second source of motivation was the excellent book by Dr. Ben Goldacre "Bad Science". Both of these got me thinking about where the science is in digital forensic science? We seem to have plenty of case study presentations, tools being developed, and novel investigative protocols being proffered. What appears to be missing is any real empirical research!

Very few of the manuscripts I review report any type of hypothesis testing, statistical analysis, or at the very least error rates or reliability estimates. When these oversights get brought up, the typical refrain is that we are an applied science, not basic research. This rings hollow with me. The term applied science should and is not synonymous with a lack of proper scientific analysis, data reporting, validation or replication of findings. It is almost as if we in the community have an inferiority complex and some believe that our field is not worthy of scientific rigor.

In the context of the National Academy of Sciences report to congress on forensic sciences and the pending bills being floated around the Whitehouse (e.g. Senator Leahy's), we need to step up and step back to cast a critical eye on the science of forensic science across all of the fields, ours being no exception. I have commented before how there seems to be a lack of scientists actually involved in charting the direction of digital forensic science, a fatal mistake in my opinion.

It should be very interesting to see if external bodies such as the proposed Office of Forensic Science and the Forensic Science Board will push us in the direction of being more scientific or if they will be the typical political lame ducks and produce only the illusion of science. Unfortunately based on the historical record I predict the latter will happen. Therefore it is up to we in the community to push for better accountability and research based on proper scientific methods (even a focus on reproducibility would be a giant leap in the right direction).

Here is an interesting interview with Ben Goldacre on the booming age of pseudo-science:


Pseudo Science

Ben Goldacre

The Coming Storm - Cloud Computing and Digital Investigations

By now we all heard how cloud computing will revolutionize the Internet and be the next best thing to happen to online businesses, consumers, education and the world at large. But we haven't heard much of what investigative concerns the so-called cloud brings with it. As most of us realize, the concept of cloud computing is nothing new. Technically we have been living with this "cloud" since the inception of the Internet and the World Wide Web. What this new cloud concept seems to add to the equation, is the ability to have various levels of distributed storage and application services.

While there are numerous security concerns being discussed by various cyber security "Czars," there seems to be little if any discussion about how the cloud will affect digital forensic investigations. Just off the top of my head I can think of several concerns that are generic to the concept of cloud computing to say nothing of specific concerns related to specific implementations or hardware and software applications.

Some basic questions are related to:

a) Jurisdiction - which sovereign nation or nations has/have authority?

b) Ownership - who actually owns the data in question?

c) Expectations of privacy - what will be the standard for reasonable expectations of privacy in the cloud?

d) Location of evidence - where do we even begin to look for data that may be classified as evidence for the investigation?

e) International cooperation - will countries housing/storing the data be willing to cooperate during an investigation?

f) Localized evidence - what artifacts will be left on the client machine?

To me these seem like obvious questions/concerns that we need to think about, debate and start working toward some answers. As I stated in the opening paragraph, the cloud is being touted as the greatest thing since "sliced bread," whether this is actually the case or not.

We as investigators will soon find ourselves truly immersed in the world of "virtual" evidence; a very sobering thought. One can only imagine how a judiciary who has trouble wrapping its mind around the concept of e-mail, will be able to keep up with the various technical solutions that make up the concept of cloud computing.

It behooves the digital forensics community to weigh in on discussions related to cloud computing and provide input as to what this latest technology savior will eventually become.

Reactions to the NAS report on the State of Forensic Sciences

As we get ready for the upcoming American Academy of Forensic Sciences conference in Seattle February 2010, I am struck by a rather interesting debate that is coming to a head in both the forensic sciences and legal communities. As many are aware, the national academies of sciences report to Congress on the state of forensic sciences really shook the forensic sciences discipline and legal community at its very core. Most commentators have focused on the negative components of the report, but few if any have really looked at the positives and or the gaps in knowledge of those drafting the report.

During a recent discussion with several colleagues who are at the forefront of international and national standards and credentialing, we were struck at the lack of mention both in the report and the follow-up conversations by the different government and quasi-government agencies, of any of the ongoing work by the numerous forensic sciences bodies that were initiated long before the report was tabled.

I think I will leave the discussion regarding the knowledge gaps that appeared in the report for another day. A corollary issue is the heated debate over the role government has in the regulating of forensic sciences. Some post-report camps wholeheartedly support the notion that state, local, tribal and federal governments should be more closely involved in the regulation, standardization and funding of the forensic sciences. A second camp is diametrically opposed to this recommendation. The gist of this camp's argument is that by including government in a regulatory and standardization role, we will end up with an even more fragmented forensic community. It would appear that these folks endorse more of the free market economy approach and believe the scientific community will correct itself albeit under the direction of the legal justice system.

To be honest I have mixed feelings about this issue. Being both a forensic scientist and member of international and national bodies attempting to draft a universal code of ethics, nationally recognized credentials and standards etc., I see little if any real positive development by the scientific community if left to its own devices. Part of this lack of development, or probably more appropriately dysfunctional development, is the result of the interference by the vendor community and other private-sector interests who in fact often have goals contrary to the altruistic goal of developing "good science." Yet I have also seen how completely dysfunctional and self-serving government interference can be in the leadership of the forensic sciences.

Still others would argue that government interference in this domain is no different from what has been historically done. While I agree with this assertion, just because it has been done historically, doesn't mean that it has been successful or should be continued in the future. I believe a more pragmatic solution falls within the realm of what could be termed a "centrist approach". By this I mean a combination of government oversight as it relates to funding and nationally/internationally standardizing the forensic sciences and the introduction of a non-governmental agency who has ultimate oversight of the scientific community; free from influence and interference from both the government and the private sector. I fully realize that such an idea is rather utopian.

The last thing the forensic sciences community needs at this juncture is to become fragmented and bogged down in petty disputes and knee-jerk reactions to an as of yet un-acted upon NAS report. Given the current and near-term economic conditions, it is doubtful that any of the major recommendations of the report (e.g., the creation of the National Institute for Forensic Sciences) will come to fruition. I personally believe that if we look at the bigger picture we soon realize that the "moral of the story" here is that if the forensic sciences community does not get its collective house in order, we will have far less than perfect solutions thrust upon us from external bodies that more than likely will only been given a limited or, agenda biased, view of the domain in question.

If history is any indication, we will likely find ourselves in a situation where the NAS report, while garnering media attention currently, will soon be forgotten, archived, and never acted upon, as has been the fate of other forensic sciences reports that have preceding this one. Only time will tell, but regardless, this should make for a very interesting meeting in Seattle.

ISSUES IN DIGITAL EVIDENCE INVESTIGATION

Cyber crime is an illegal electronic operation that targets the security of computer systems and data processed by them. Hacking, cyber fraud, phishing, identity and data theft come under cyber crime. Bank accounts can be hacked and credit card details can be stolen. When such cyber crimes are committed, we need digital evidence investigators to catch the culprits. Though cyber forensics is doing a great deal to find out who is responsible for misusing computer systems, it faces many issues that have to be handled with care. Listed below are some issues in cyber forensics.

1. A digital evidence investigator must keep in mind the privacy and secrecy of the clients’ data and information while performing the investigation. But in some cases when the information has to be produced as evidence in the court of law to prove a crime, it is not possible for the cyber forensics expert to maintain the secrecy and privacy of the clients’ information.
2. Sensitive data and information that are very important to the client maybe lost or damaged while finding evidence. But it is the duty of the expert to take additional care to ensure that the possible evidence is not destroyed or damaged. Typically this involves making a forensic image or forensic copy of the original media, and conducting the analysis on the copy versus the original.  
3. While the investigations are on, it is possible that some malicious computer programs or computer viruses are released into the computer system. These viruses may corrupt the existing software and they may have the potential to damage the hardware system too. It maybe necessary to use high quality anti-virus software before the investigation is commenced.
4. Once the evidence is found, it must be preserved very carefully. It must be protected against any kind of mechanical and electro-magnetic damage. Any evidence found relevant to the situation at hand will need to be extracted from the working copy media and then typically saved to another form of media as well as printed out. The information that is obtained as evidence is the responsibility of the computer forensic team.
5. When the case is on, the evidence information maybe stored in court and, in some cases, the concerned partied may not be able to use that information. This may affect the business operations. In order to avoid causing any inconvenience and loss to the parties involved, the digital evidence investigator must make sure that justice is delivered as soon as possible.
6. Whatever is done during the analysis has to be documented along with the findings. The findings and reports need to be based on proven techniques and methodology, and any other competent investigator should be able to duplicate and reproduce the results. It is also important that the information acquired during the analysis is ethically and legally respected.
7. The operations cost of digital evidence investigations may some cases exceed regular investigations.

In spite of all these issues, cyber forensics or digital evidence investigation has gained a lot of importance in today’s computer world largely due to its vast application in varied situations.

By-line:

This post was contributed by Holly McCarthy, who writes on the subject of forensic science careers. She invites your feedback at hollymccarthy12 at gmail dot com

Digital Evidence Investigators Required to be Licensed PI's!

We are witnessing a very interesting and disturbing trend in the digital evidence domain. Many states are enacting or amending legislation that will require anyone conducting any type of an "investigation" where a computer is involved to be licensed as a Private Investigator – Michigan being one of the latest examples. This is interesting as it was predicted several years ago that, unless the digital evidence community came up with some sort of gold standard/professional designation with a professional code of ethics, the ability to censure unethical professionals etc. the government would intercede with a less than perfect knee jerk reaction in order to protect consumers of these services.

The American Bar Association has taken a stand on this issue and the Science & Technology Law Section has issued a resolution arguing against this requirement:

AMERICAN BAR ASSOCIATION ADOPTED BY THE HOUSE OF DELEGATES AUGUST 11-12, 2008

RECOMMENDATION

RESOLVED, That the American Bar Association urges State, local and territorial legislatures, State regulatory agencies, and other relevant government agencies or entities, to refrain from requiring private investigator licenses for persons engaged in:

computer or digital forensic services or in the acquisition, review, or analysis of digital or computer-based information, whether for purposes of obtaining or furnishing information for evidentiary or other purposes, or for providing expert testimony before a court; or

network or system vulnerability testing, including network scans and risk assessment and analysis of computers connected to a network.

FURTHER RESOLVED, That the American Bar Association supports efforts to establish professional certification or competency requirements for such activities based upon the current state of technology and science.

Unfortunately it appears that most states are ignoring the advise of the scientific and legal community. The cynical side of my nature wonders whether the motivation for moving toward the PI License requirement is driven primarily by an economic motive (It appears that the PI community has a strong lobbying presence in many of the states that have already passed these requirements) as opposed to any real concern over an unregulated "industry" and consumer protection.

This issue is shaping up to be a watershed event for the digital evidence community and the final outcome will have a long lasting impact on this maturing field.

In case you were wondering, there is a concerted effort underway to address the issue of a neutral, board like certification for digital evidence professionals supported by the forensic science accreditation board. The Digital Forensics Certification Board (www.DFCB.org) housed at the University of Central Florida's National Center for Forensic Science will offer its certification exam early in the spring of this year. This non-partisan body represents the collective effort of law enforcement, private sector, government, military and academia. For the sake of full disclosure, yes I am part of this effort.

More information about this effort will be presented at the Digital Sciences & Multimedia Section of American Academy of Forensic Sciences Annual Meeting in Colorado this February.