skip to main content
Close Icon We use cookies to improve your website experience.  To learn about our use of cookies and how you can manage your cookie settings, please see our Cookie Policy.  By continuing to use the website, you consent to our use of cookies.
Global Search Configuration

Ovum view

Summary

The Apache Hadoop framework, with its open source methodology and thriving ecosystem of supported projects, is often the preferred platform for analyzing large volumes of diverse data for business insight. The recent Big Data Everywhere San Francisco conference underscored two of Hadoop's most prominent virtues: its flexibility and speed. While the theme of the conference was analytics that can be performed using Hadoop, the concerns over real-time analytics pertain to any fast data platform or target capable of ingesting, transforming, and analyzing data in real time. Organizations within regulated industries, for example, need to exercise caution with applications or use cases that leverage nearly instantaneous insight. For fast data analytics and insight, an updated and flexible policy framework is needed to accommodate the discovery of potentially noncompliant trends and information.

Real-time analytics needs supervision

Today, the need for compliance-focused supervision over fast data analytics projects is emerging as a new business concern not only for regulated organizations, but also organizations that harbor concerns about inappropriate access to data or misuse of analytics. Data can be ingested, analyzed, and visualized in near-real time – leaving little chance for compliance practitioners to interpret or react to trends revealed by analysis. Instead of increasing dependence on compliance tools and technologies, regulated firms would be best advised to focus on developing a compliance policy framework that can scale with data. Realistically, this means establishing channels of communication and routine opportunities for compliance teams to review analytics initiatives, the data they are based on, and the insights they produce.

This is relevant today due to a timely convergence of factors: propagation of unstructured data sources, sophistication of natural language processing, consumerization of analytics tools, and the legal expectation that businesses should be held accountable for their data, regardless of where it resides. This means that compliance "red flags" might go completely unseen by compliance personnel. For instance, a marketing department using a Hadoop-based system to analyze social media may uncover a trend of public complaints regarding a dangerous product failure. If the compliance team is not made aware of this and appropriate actions are not taken in a timely manner, the business may be exposed to expensive civil litigation and/or settlements.

The modern technology landscape, and particularly open source technology platforms like Hadoop, makes it possible to analyze data and implement automated decision-making at a speed that far outpaces the human ability to interpret the information. Therein lies the problem for risk management and compliance teams, whose authority lies in their ability to override and enforce the policy settings which are often carried out with automated technology. Traditionally, unstructured data types subject to compliance monitoring could be captured and handled by point tools, such as those designed for email. But with the proliferation of data types and the push to cross-analyze content from different sources, there is now the risk that "red flag" patterns – of particular interest to compliance managers – may now exist outside of the systems originally designed to enforce regulatory requirements.

Appendix

Author

Paige Bartley, Senior Analyst, Information Management

paige.bartley@ovum.com

Recommended Articles

  • Service Provider Markets, Consumer & Entertainment Services,...

    MWC 2018 Highlights

    By Ronan De Renesse 27 Feb 2018

    Over 20 of our senior Ovum analysts and consultants attended this year’s Mobile World Congress in Barcelona at the end of February. In between meetings, briefings and presentations, our analyst team were blogging and tweeting about key developments, trends and rumors. Have a look through our daily MWC 2018 Highlights to find out what happened.

    Topics 5G AI IoT Cloud Payments SDN/NFV Smart home

  • Consumer & Entertainment Services

    US pay TV: Is it facing an existential threat?

    By Adam Thomas 28 Mar 2018

    With US pay TV having endured the worst year in its history, thoughts have inevitably turned to the future. The likelihood remains that the immediate future will remain highly uncomfortable for everyone except the scaled multinational digital platforms.

  • Enterprise Decision Maker, Enterprise IT Strategy and Select...

    2017 Trends to Watch: Big Data

    By Tony Baer 21 Nov 2016

    The breakout use case for big data will be fast data. The Internet of Things (IoT) is increasing the urgency for enterprises to embrace real-time streaming analytics, as use cases from mobile devices and sensors become compelling to a wide range of industry sectors.

    Topics Big data and analytics IoT

;

Have any questions? Speak to a Specialist

Europe, Middle East & Africa team - +44 (0) 207 017 7700


Asia-Pacific team - +61 (0)3 960 16700

US team - +1 646 957 8878

Email us at ClientServices@ovum.com

You can also contact your named/allocated Client Services Executive using their direct dial.
PR enquiries - Call us at +44 788 597 5160 or email us at pr@ovum.com

Contact marketing - 
marketingdepartment@ovum.com

Already an Ovum client? Login to the Knowledge Center now