Big Data Applications Now ODPi Interoperable, Provide Increased Simplicity To Hadoop Ecosystem - Linux Foundation

New York City, N.Y. – Strata + Hadoop World, September 27, 2016 — ODPi, a nonprofit organization accelerating the open ecosystem of big data solutions, today announced that solution and application providers DataTorrent, IBM, Pivotal, SAS, Syncsort, WANdisco, and Xavient have committed to the ODPi Interoperable Compliance Program. This makes it easier for enterprises to choose and adopt big data technologies and ensures these applications are interoperable across a wider range of commercial Apache™ Hadoop® Platforms.

“The implementation of complex environments is a major obstacle to organizations implementing big data. The new ODPi Interoperable Compliance Program is a big step in the right direction to help organizations mitigate the risk associated with Hadoop implementations,” said John L. Myers, Managing Research Director of Business Intelligence at Enterprise Management Associates. “Adding Apache Hive and Hadoop Compatible File System (HCFS) to the Runtime Specification 2.0 will enable the initiative to make continued progress toward the interoperability of these components and it will make Hadoop more appealing to the organizations looking to make the leap into big data initiatives.”

A dozen ODPi Interoperable applications have been tested to successfully run on multiple ODPi Runtime Compliant Platforms. Currently, Altiscale, ArenaData, Hortonworks, IBM, and Infosys Apache Hadoop Platforms are ODPi Runtime Compliant, offering greater simplicity for big data solution and application providers. ODPi programs give end-users greater choice and flexibility by fostering an open big data ecosystem that transcends traditional vendor alliances.

Additionally, ODPi announced today that the ODPi Runtime Specification 2.0 will add Apache Hive and Hadoop Compatible File System support (HCFS). These components join Yarn, MapReduce and HDFS from ODPi Runtime Specification 1.0. By including Apache Hive, which read, write, and manage large datasets residing in distributed storage, ODPi will reduce SQL query inconsistencies across Hadoop Platforms.  ODPi will base its work on Hive version 1.2 and ensure there is core functionality that will continue to behave in a standard way for future versions of Apache Hive. HCFS support will enable storage and cloud vendors to leverage ODPi standards, letting them leverage their native storage solutions as part of an ODPi Runtime Compliant Hadoop Platform and reduce the incompatibilities that end-users face.

“As our work continues to complement the Apache Software Foundation, ODPi is helping the Hadoop ecosystem become more valuable to those who are testing and building big data applications. Through a common specification, we are enabling developers to easily write applications that sit on top of big data stacks, lowering the costs of interoperability across systems. These compliant applications should need little to no re-engineering to run on other ODPi Runtime Compliant platforms,” said John Mertic, Director of ODPi. “For enterprises, the benefits are increased efficiency, flexibility and smoother maintenance.”

ODPi Interoperability Guidelines and Testing

The ODPi Interoperable Compliance Program is an ongoing commitment between Apache Hadoop platforms, big data solution and application providers, and ODPi to ensure that the best practices outlined by the ODPi Specification Releases are adhered to for the benefit of enterprises; providing them flexibility of deployment and choice of solution. The ODPi Interoperable and ODPi Runtime Compliance programs guarantee that any software or application deemed ODPi Interoperable can run on any of the ODPi Runtime Compliant Apache Hadoop platforms. This makes operating a heterogeneous big data environment easier for enterprises.

Enterprises can easily obtain details regarding which versions of an application are ODPi Interoperable and which Apache Hadoop platforms are compliant by visiting ODPi For End Users.

Current ODPi Interoperable Applications include:

Comments from Members

IBM

“ODPi is providing a common platform to develop big data apps, enabling interoperability across different distributions and application offerings. In that spirit, IBM is ensuring many of our Apache Hadoop related offerings are interoperable including IBM Big SQL, IBM SPSS Analytic Server, IBM Big Replicate, and others. By supporting interoperable applications, the value to clients of ODPi Compliant distributions is being enhanced.” – Ritika Gunnar, Vice President of Offerings, IBM Analytics

Pivotal

“Working with our customers, Pivotal developed Pivotal HDB based on Apache HAWQ, a fully SQL-92 compliant, native SQL engine running on Hadoop. Compatibility and standardization across multiple Hadoop distributions is among the many reasons why we focus on ODPi Compliant offerings, allowing us to more rapidly test, deploy, and innovate.” – Jacque Istok, Head of Data Engineering at Pivotal Software

SAS

“SAS is pleased to announce that we are the first software vendor to achieve the distinction of ODPi Interoperable. By declaring that SAS interfaces with Apache Hadoop in demonstrable, standard ways, we can reduce our customers’ risk, simplify testing complexity and speed time to value for anyone building or deploying SAS applications.” – Craig Rubendall, Vice President of Platform R&D at SAS and ODPi board member

Syncsort

“We are thrilled to be able to provide our customers with Syncsort DMX-h 9.0 as an ODPi Interoperable solution. Interoperability helps to streamline the development and testing process, which, in turn allows us to focus our resources on supporting our customers’ big data requirements.” – Arnie Farrelly, Vice President of Big Data Global Product Support and Services at Syncsort

WANdisco

“ODPi has eliminated significant risk for both developers and end users, by providing a set of standards that makes it possible for an ecosystem of interoperable, enterprise class big data applications to evolve and flourish. WANdisco (LSE:WAND) is extremely pleased to have achieved ODPi Interoperable compliance for Fusion 2.9 and become part of that ecosystem. Our patented replication technology reduces risk even further, by ensuring ODPi Compliant Hadoop Platforms deliver continuous availability and performance, benefitting every application deployed on them.”  – Jim Campigli, COO and Co-Founder at WANdisco

Xavient

“Customer-driven requirements resulted in the creation of DiP(Data Ingestion Platform), which is based on Hadoop and leverages many open source technologies like Kafka, Spark Streaming, Storm and components like HDFS, Hive, Zeppelin and HBase. We are focused on ODPi Compliant offerings because it provides us with a platform to test and deploy DiP on multiple Hadoop versions with an efficient testing process. DiP is a co-dev opportunity, so ODPi gives a structure to test the tool based on customer investment and use case.” – Neeraj Sabharwal, Director Cloud and Big Data at Xavient Information System

Additional Resources

• Participate in the compliance program on ODPi GitHub

• Join the conversation on ODPi Slack channel

• Learn About ODPi Membership

• ODPi Blog

About ODPi

ODPi is a nonprofit organization committed to simplification and standardization of the big data ecosystem with a common reference specification. As a shared industry effort, ODPi members represent big data technology, solution provider and end user organizations focused on promoting and advancing the state of Apache Hadoop® and big data technologies for the enterprise. For more information about ODPi, please visit: http://www.ODPi.org.