Tomer Shiran, CEO, and co-founder of analytics startup Dremio, a driving force behind the open source Apache Arrow project, predicts that enterprises will see the need for a new role: the data curator.
The data curator, Shiran says, sits between data consumers (analysts and data scientists who use tools like Tableau and Python to answer important questions with data) and data engineers (the people who move and transform data between systems using scripting languages, Spark, Hive, and MapReduce). To be successful, data curators must understand the meaning of the data as well as the technologies that are applied to the data.
“The data curator is responsible for understanding the types of analysis that need to be performed by different groups across the organization, what datasets are well suited for this work, and the steps involved in taking the data from its raw state to the shape and form needed for the job a data consumer will perform,” Shiran says. “The data curator uses systems such as self-service data platforms to accelerate the end-to-end process of providing data consumers access to essential datasets without making endless copies of data.”
Data governance strategies will be key themes for all C-level executives
The European Union’s General Data Protection Regulation (GDPR) is set to go into effect on May 25, 2018, and it looks like a specter over the analytics field, though not all enterprises are prepared.
The GDPR will apply directly in all EU member states, and it radically changes how companies must seek consent to collect and process the data of EU citizens, explain lawyers from Morrison & Foerster’s Global Privacy + Data Security Group: Miriam Wugmeister, Global Privacy co-chair; Lokke Moerel, European Privacy Expert; and John Carlin, Global Risk and Crisis Management chair (and former Assistant Attorney General for the U.S. Department of Justice’s National Security Division).
“Companies that rely on consent for all their processing operations will no longer be able to do so, and will need other legal bases (i.e., contractual necessity and legitimate interest),” they explain. “Companies will need to implement a whole new ecosystem for notice and consents.”
Even though GDPR fines are potentially massive — the administrative fines can be up to 20 million Euros or 4 percent of annual global turnover, whichever is highest — many enterprises, particularly in the U.S., are not prepared.
“When the Y2K boom came around, everyone was preparing for odds that they may or may not face,” says Scott Gnau, CTO of Hortonworks. “Today, it seems that barely anyone is properly preparing for the GDPR being enforced in May 2018. Why not? We’re currently in a phase where every organization is not only trying to deal with ‘what’s next,’ but they’re struggling to maintain and deal with issues that need solving now. Many organizations are likely relying on chief security officers to define the rules, systems, parameters, etc., to help their global system integrators figure out the best course of action. That is not a realistic expectation to put on one individual’s role.”
To enforce GDPR properly requires the C-suite be informed, prepared, and communicative with all facets of their organization, Gnau says. Organizations will need a better handle on the overall governance of their data assets. But large breaches, like the Equifax breach that came to light in 2017, means they will struggle to balance providing self-service access to data for employees while protecting that same data from prospective threats.
As a result, Gnau predicts data governance will be a focal point for all organizations in 2018.
“A key goal should be developing a system that balances democratization of data, access, self-service analytics, and regulation,” Gnau says. “The way we architect data safely going forward will have an impact on everyone — customers in the U.S. and overseas, the media, your partners, and more.”
Zachary Bosin, director of solution marketing for multi-cloud data management specialist Veritas Technologies, predicts a U.S. company will be one of the first to be fined under the GDPR.
“Despite the impending deadline, only 31 percent of companies surveyed by Veritas worldwide believe they are GDPR-compliant,” Bosin says. “Penalties for non-compliance are steep, and this regulation will impact every and any company that deals with EU citizens.”
Read More – eScan Blog
2 Comments
Prosperity
It’s high time they integrate security protocols as with the Facebook Cambridge analytica scandal
Pingback: Rise of the data curator and security concerns | Blog eScan - Secure Signal NYC