大数据项目遭遇失败的八个理由：管理层阻力，回避问题

大数据目前已经成为万众瞩目的焦点，已经有众多企业在拼命把自己的数据投付使用、希望借此为重要决策提供支持。尽管大数据宣传与炒作可谓如火如荼，但仍有92%的企业始终保持中立态度，即计划在“合适的时间”着手实施或者表示不打算接触大数据项目。而在那些已经亲身实践大数据项目的企业中，多数遭遇失败、而且往往是掉进了同样的几个陷阱当中。

取得大数据项目成功的关键在于构建一套迭代型方案，鼓励现有员工参与并使用，从而在一系列无关紧要的失败中学习知识并积累经验。

从众心理

大数据绝对是项转折性的伟大技术成果。根据Gartner公司的调查，2013年中64%的受访企业表示已经购买或者正计划在大数据系统领域进行投资，这一比例高于2012年调查中的58%。越来越多的企业开始深入探索自己的数据，尝试利用蕴藏在其中的信息最大程度减少客户流失、分析财务风险并改善客户体验。

在这64%认同大数据思路的受访者中，又有30%已经在大数据技术方面投入资金、19%计划在未来一年中进行投资、另外15%则计划在未来两年内进行投资。不过在Gartner的全部720位调查对象中，只有不到8%已经实际部署了大数据技术方案。

这样的结果实在很糟糕，不过造成项目失败的理由明显更加糟糕：大多数企业根本不知道自己在迈入大数据领域后应该做些什么。

难怪现在有那么多企业开出可观的薪酬数字来招徕并雇用数据科学家，目前其平均收入已经达到每年12万3千美元。

八种导致失败的理由

由于众多企业在探索自有数据的过程中完全是在胡打误撞，因此在意识到这一点后、他们决定向能带来更具可预测性方案的专业人士求援(包括认为数据科学家能够奇迹般地随手化解他们面临的现实难题，甚至还有不少更夸张的预期)。Gartnerwngr Svetlana Sicular为我们汇总出八种导致大数据项目失败的常见原因，它们分别是：

·管理层阻力。尽管数据当中包含大量重要信息，但Fortune Knowledge公司发现有62%的企业领导者仍然倾向于相信自己的直觉，更有61%的受访者认为领导者的实际洞察力在决策过程中拥有高于数据分析结论的优先参考价值。

·选择错误的使用方法。企业往往会犯下两种错误，要么构建起一套过分激进、自己根本无法驾驭的大数据项目，要么尝试利用传统数据技术处理大数据问题。无论是哪种情况，都很有可能导致项目陷入困境。

·提出错误的问题。数据科学非常复杂，其中包含专业知识门类(需要深入了解银行、零售或者其它行业的实际业务状况);数学与统计学经验以及编程技能等等。很多企业所雇用的数据科学家只了解数学与编程方面的知识，却欠缺最重要的技能组成部分：对相关行业的了解。Sicular的观点很对，她表示大家最好能从企业内部出发寻找数据科学家，因为“学习Hadoop比学习相关行业的知识更简单”。

·缺乏必要的技能组合。这项理由与“提出错误的问题”紧密相关。很多大数据项目之所以陷入困境甚至最终失败，正是因为不具备必要的相关技能。通常负责此类项目的都是IT技术人员——而他们往往无法向数据提出足以指导决策的正确问题。

·在大数据技术之外遇到了其它意外状况。数据分析仅仅是大数据项目当中的组成部分之一，访问并处理数据的能力同样重要。除此之外，常常被忽略的因素还有网络传输能力限制与人员培训等等。

· 与企业战略存在冲突。要让大数据项目获得成功，大家必须摆脱将其作为单一“项目”的思路、真正把它当成企业使用数据的核心方式。问题在于，其它部门的价值或者战略目标有可能在优先级方面高于大数据，这种冲突往往会令我们有力无处使。

·大数据孤岛。大数据供应商总爱谈论“数据湖”或者“数据中枢”，但事实上很多企业建立起来的只能算是“数据水坑儿”，各个水坑儿之间存在着明显的边界——例如市场营销数据水坑儿与制造数据水坑儿等等。需要强调的是，只有尽量缓和不同部门之间的隔阂并将各方的数据流汇总起来，大数据才能真正发挥自身价值。

·回避问题。有时候我们可以肯定或者怀疑数据会迫使自身做出一些原本希望尽量避免的运营举措，例如制药行业之所以如此排斥情感分析机制、是因为他们不希望将不良副作用报告给美国食品药品管理局并承担随之而来的法律责任。

在这份理由清单中，大家可能已经发现了一个共同的主题：无论我们如何高度关注数据本身，都会有人为因素介入进来。即使我们努力希望获取对数据的全面控制权，大数据处理流程最终还是由人来打理的，其中包括众多初始决策——例如选择哪些数据进行收集与分析、向分析结论提出哪些问题等等。

通过迭代实现创新

由于很多企业似乎根本无力建立起自己的大数据项目，再加上大多数大数据项目往往最终遭遇失败，因此将迭代机制引入大数据是非常必要的。这不会迫使企业向咨询企业或者供应商支付大量费用，大家最好能构建起由内部员工参与的免费数据实验方案。

鉴于几乎所有主要大数据技术都属于开源成果，因此建立起一套“初始规模较小、能够快速发现问题”的方案其实完全可行。更重要的是，很多平台都能像云服务那样立即起效且成本低廉，从而进一步降低了进行项目实验与发现错误的资金投入。

大数据的关注重点在于提出正确的问题，这也是让企业内部员工参与项目如此重要的理由。但即使拥有卓越的相关行业知识，如果根本无法开始提出问题的流程、企业仍然无法收集到正确的数据。这类问题也应该被纳入预期并作好相应准备。

解决问题的关键在于使用灵活而开放的数据基础设施，保证其允许企业员工不断调整实际方案、直到他们的努力获得理想的回馈。通过这种方式，企业能够消除恐惧并最终以迭代为武器顺利迈向大数据有效使用的胜利彼岸。

英语原文：

8 Reasons Big Data Projects Fail

Big data is all the rage, and many organizations are hell bent on putting their data to use. Despite the big data hype, however, 92% of organizations are still stuck in neutral, either planning to get started “some day” or avoiding big data projects altogether. For those that do kick off big data projects, most fail, and frequently for the same reasons.

The key to big data success is to take an iterative approach that relies on existing employees to start small and learn by failing early and often.

Herd mentality

Big data is a big deal. According to Gartner, 64% of organizations surveyed in 2013 had already purchased or were planning to invest in big data systems, compared with 58% of those surveyed in 2012. More and more companies are diving into their data, trying to put it to use to minimize customer churn, analyze financial risk, and improve the customer experience. Of that 64%, 30% have already invested in big data technology, 19% plan to invest within the next year, and another 15% plan to invest within two years. Less than 8% of Gartner’s 720 respondents, however, have actually deployed big data technology.

That’s bad, but the reason for the failure to launch is worse: Most companies simply don’t know what they’re doing when it comes to big data. It’s no wonder that so many companies are spending a small fortune to recruit and hire data scientists, with salaries currently averaging $123,000.

8 ways to fail

Because so many organizations are flying blind with their data, they stumble in predictable ways (including thinking that a data scientist will magically solve all their problems, but more on that below). Gartner’s Svetlana Sicular has catalogued eight common causes of big data project failures, including:

Management resistance. Despite what data might tell us, Fortune Knowledge Group found that 62% of business leaders said they tend to trust their gut, and 61% said real-world insight tops hard analytics when making decisions.

Selecting the wrong uses. Companies either start with an overly ambitious project that they’re not yet ready to tackle, or they attempt to solve big data problems using traditional data technologies. In either case, failure is the usual result.

Asking the wrong questions. Data science is a complex blend of domain knowledge (the deep understanding of banking, retail, or another industry); math and statistics expertise; and programming skills. Too many organizations hire data scientists who might be math and programming geniuses but who lack the most important component: domain knowledge. Sicular is right when she advises that it’s best to look for data scientists from within, as “learning Hadoop is easier than learning the business.”

Lacking the right skills. This one is closely related to “asking the wrong questions.” Too many big data projects stall or fail due to the insufficient skills of those involved. Usually the people involved come from IT — and those are not the people most qualified to ask the right questions of the data.

Unanticipated problems beyond big data technology. Analyzing data is just one component of a big data project. Being able to access and process the data is critical, but that can be thwarted by such things as network congestion, training of personnel, and more.

Disagreement on enterprise strategy. Big data projects succeed when they’re not really isolated “projects” at all but rather core to how a company uses its data. The problem is exacerbated if different groups value cloud or other strategic priorities more highly than big data

Big data silos. Big data vendors are fond of talking about “data lakes” and “data hubs,” but the reality is that many businesses attempt to build the equivalent of data puddles, with sharp boundaries between the marketing data puddle, the manufacturing data puddle, and so on. Big data is more valuable to an organization if the walls between groups come down and their data flows together. Politics or policies often stymie this promise.

Problem avoidance. Sometimes we know or suspect the data will require us to take action that we don’t really want to do, like the pharmaceutical industry not running sentiment analysis because it wants to avoid the subsequent legal obligation to report adverse side effects to the U.S. Food and Drug Administration.

Throughout this list, one common theme emerges: As much as we might want to focus on data, people keep getting in the way. As much as we might want to be ruled by data, people ultimately rule the big data process, including making the initial decisions as to which data to collect and keep, and which questions to ask of it.

Innovate by iterating

Because so many organizations seem hamstrung in their attempts to start a big data project, coupled with the likelihood that most big data projects will fail, it’s imperative to take an iterative approach to big data. Rather than starting with a hefty payment to a consultant or vendor, organizations should look for ways to set their own employees free to experiment with data.

A “start small, fail fast” approach is made possible, in part, by the fact that nearly all significant big data technology is open source. What’s more, many platforms are immediately and affordably accessible as cloud services, further lowering the bar to trial-and-error.

Big data is all about asking the right questions, which is why it’s so important to rely on existing employees. But even with superior domain knowledge, organizations still will fail to collect the right data and they’ll fail to ask pertinent questions at the start. Such failures should be expected and accepted.

The key is to use flexible, open-data infrastructure that allows an organization’s employees to continually tweak their approach until their efforts bear real fruit. In this way, organizations can eliminate the fear and iterate toward effective use of big data.

via:IT168,核子可乐