Welcome!

Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

News Feed Item

Syncsort Open Sources Technology to Make Mainframe Big Data Available to Apache Spark

Apache Spark Mainframe Connector Now Available on Databricks' Spark Packages Community Site

WOODCLIFF LAKE, N.J., Sept. 1, 2015 /PRNewswire/ -- Syncsort, a global leader in Big Data software, today announced a milestone open source contribution of an IBM z Systems mainframe connector for Apache Spark. The contribution will enable organizations to easily access and get new insights from their critical mainframe data with Apache Spark's advanced analytics capabilities and Spark SQL. The connector is now available on Spark Packages: https://github.com/Syncsort/spark-mainframe-connector.

Syncsort Data Integration Logo.

Spark Packages, launched by Apache Spark pioneer Databricks – the organization founded by the team that created and continues to drive Apache Spark – makes it easy for users to find, discuss, rate and install packages that enhance Spark's capabilities.

"One of the key elements of Spark's continued success is its integration with important data sources," said Matei Zaharia, creator of Apache Spark and co-founder & CTO of Databricks. "We are excited that Syncsort has made this valuable contribution to the Apache Spark community, making mainframe data easily available for use within Spark."  

Spark has emerged as one of the most active big data open source projects, initially as the lightning fast memory-optimized processing engine for machine learning and now as the single compute platform for all types of workloads including real-time data processing, interactive queries, social graph analysis, and much more. Given its success, there is a growing need to securely access data from a diverse set of sources, including mainframes, and to transform the data into a format that is easily understandable by Spark.

"Syncsort's open source contribution makes it easy to get real-time insights from mainframe data using Apache Spark's advanced analytics capabilities and Spark SQL interactive queries," said Tendü Yoğurtçu, general manager of Syncsort's Big Data business. "We believe that Apache Spark will play a critical role in a wide variety of next-generation use cases, including streaming ETL and the Internet of Things. We will continue to contribute to Spark and related Big Data projects to enable a uniform user experience for batch and real-time workloads across all data sources."

Syncsort's mainframe connector for Spark is similar to the Apache Sqoop mainframe connector that Syncsort released as open source last year. Customers simply specify the location of multiple datasets and the associated COBOL copybook metadata and the Spark mainframe connector automatically transfers the datasets in parallel via a secure connection into Spark's DataFrame objects. Users can then manipulate this DataFrame object and join it with their other data sources for further analysis. Syncsort's mainframe connector conforms to Spark's Data Sources API specification, and because of Spark's ability operate on data in memory, the connector will allow queries to access mainframe data without offloading the data first. Mainframe record formats including fixed, variable, sequential and VSAM files are all supported. The connector also handles compressed data transfer, minimizing network bandwidth and optimizing overall elapsed time.

To download the Spark Mainframe Connector on Spark Packages, click here.

About Syncsort

Syncsort provides enterprise software that allows organizations to collect, integrate, sort and distribute more data in less time, with fewer resources and lower costs. Syncsort software provides specialized solutions spanning "Big Iron to Big Data," including next gen analytical platforms such as Hadoop, cloud, and Splunk. For more than 40 years customers have turned to Syncsort's software and expertise to dramatically improve performance of their data processing environments, while reducing hardware and labor costs.  Experience Syncsort at http://www.syncsort.com.

Media Contacts:

Sarah Borup     
SHIFT Communications
Tel: 617-779-1803
[email protected]

Michael Kornspan
Syncsort Incorporated
Director, Corporate Communications
Tel: 201-930-8216
[email protected]

Logo - http://photos.prnewswire.com/prnh/20130520/NY16823LOGO

To view the original version on PR Newswire, visit:http://www.prnewswire.com/news-releases/syncsort-open-sources-technology-to-make-mainframe-big-data-available-to-apache-spark-300135638.html

SOURCE Syncsort

More Stories By PR Newswire

Copyright © 2007 PR Newswire. All rights reserved. Republication or redistribution of PRNewswire content is expressly prohibited without the prior written consent of PRNewswire. PRNewswire shall not be liable for any errors or delays in the content, or for any actions taken in reliance thereon.

IoT & Smart Cities Stories
Disruption, Innovation, Artificial Intelligence and Machine Learning, Leadership and Management hear these words all day every day... lofty goals but how do we make it real? Add to that, that simply put, people don't like change. But what if we could implement and utilize these enterprise tools in a fast and "Non-Disruptive" way, enabling us to glean insights about our business, identify and reduce exposure, risk and liability, and secure business continuity?
DXWorldEXPO LLC announced today that Telecom Reseller has been named "Media Sponsor" of CloudEXPO | DXWorldEXPO 2018 New York, which will take place on November 11-13, 2018 in New York City, NY. Telecom Reseller reports on Unified Communications, UCaaS, BPaaS for enterprise and SMBs. They report extensively on both customer premises based solutions such as IP-PBX as well as cloud based and hosted platforms.
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
In this Women in Technology Power Panel at 15th Cloud Expo, moderated by Anne Plese, Senior Consultant, Cloud Product Marketing at Verizon Enterprise, Esmeralda Swartz, CMO at MetraTech; Evelyn de Souza, Data Privacy and Compliance Strategy Leader at Cisco Systems; Seema Jethani, Director of Product Management at Basho Technologies; Victoria Livschitz, CEO of Qubell Inc.; Anne Hungate, Senior Director of Software Quality at DIRECTV, discussed what path they took to find their spot within the tec...
To Really Work for Enterprises, MultiCloud Adoption Requires Far Better and Inclusive Cloud Monitoring and Cost Management … But How? Overwhelmingly, even as enterprises have adopted cloud computing and are expanding to multi-cloud computing, IT leaders remain concerned about how to monitor, manage and control costs across hybrid and multi-cloud deployments. It’s clear that traditional IT monitoring and management approaches, designed after all for on-premises data centers, are falling short in ...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible. In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...
"Space Monkey by Vivent Smart Home is a product that is a distributed cloud-based edge storage network. Vivent Smart Home, our parent company, is a smart home provider that places a lot of hard drives across homes in North America," explained JT Olds, Director of Engineering, and Brandon Crowfeather, Product Manager, at Vivint Smart Home, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
We are seeing a major migration of enterprises applications to the cloud. As cloud and business use of real time applications accelerate, legacy networks are no longer able to architecturally support cloud adoption and deliver the performance and security required by highly distributed enterprises. These outdated solutions have become more costly and complicated to implement, install, manage, and maintain.SD-WAN offers unlimited capabilities for accessing the benefits of the cloud and Internet. ...
In an era of historic innovation fueled by unprecedented access to data and technology, the low cost and risk of entering new markets has leveled the playing field for business. Today, any ambitious innovator can easily introduce a new application or product that can reinvent business models and transform the client experience. In their Day 2 Keynote at 19th Cloud Expo, Mercer Rowe, IBM Vice President of Strategic Alliances, and Raejeanne Skillern, Intel Vice President of Data Center Group and G...