Wednesday, February 3, 2010

How accurate is feedback?

What is in a statistics? Let me admit that Kahn Consulting has done many surveys over the years and I have analyzed data provided by many companies responding to survey questions. We labored over how to write them to get “useful” feedback. Yet, “at the end of the day”, “in the final analysis”, “after thinking outside the box”, I was left wondering about the veracity of the input—was the respondent “in the know” and did we get “the straight dope”. Loath to be trite, I wonder about stats all the time. For example, according to an Economist Intelligence Unit Report on Information Governance, “…nearly 73% of respondents report that their company’s overall ability to provide access to critical business information when needed is good or very good”. HOGWASH!!!!—I think? I have no way to verify the report or determine independently whether or not the feedback was indeed accurate. I am not being critical of the report, but rather questioning the feedback. Is it true or not? But in the my heart of hearts, it doesn’t feel right. News reports about business failure after business failure remind me daily that managing information is not nearly as good as it should be. Take the Christmas day terrorist bomber for example. Information about the terrorist existed, in various data bases, from various intelligence agencies and units, in various forms, but the dots were not able to be connected. So is it failure or success even if each of the databases “owners” would say that their “overall ability to provide access to critical business information when needed is good or very good”. I am not convinced we are doing very well at governing or managing information. How many exposures of personal identifiable information (PII) do we need to experience before we conclude something is broken. When my gut says something smells fishy I tend to listen. Are You Kidding Me?

0 comments: