{"id":67305,"date":"2022-10-05T13:34:02","date_gmt":"2022-10-05T13:34:02","guid":{"rendered":"https:\/\/www.techrepublic.com\/?p=4000142"},"modified":"2022-10-05T13:34:02","modified_gmt":"2022-10-05T13:34:02","slug":"current-2022-confluent-creates-data-pipeline-lifeline","status":"publish","type":"post","link":"https:\/\/cloudnewshub.com\/?p=67305","title":{"rendered":"Current 2022: Confluent creates data pipeline lifeline"},"content":{"rendered":"
\"3d
Image: Mark\/Adobe Stock<\/figcaption><\/figure>\n

Data flows. Even when data comes to rest, gets sent to backup and possibly finds itself in difficult-to-retrieve long-term storage retirement locations, data generally flows from one place to another during its lifetime.<\/p>\n

When data is in motion, it typically moves between applications and their dependent services. But data obviously also moves between applications and operating systems, between application components, containers and microservices \u2014 and, in the always-on era of cloud and the web \u2014 between application programming interfaces<\/a>.<\/p>\n

Today we know that data flows to such an extent that we now talk about data streaming, but what is it and how do we harness this computing principle?<\/p>\n

What is data streaming?<\/h2>\n

A key information paradigm for modern IT stacks, data streaming denotes and describes the movement of data through and between the above-referenced channels in a time-ordered sequence. A close cousin to the concept of computing events, where asynchronous log files are created for every keyboard stroke, mouse click or IoT sensor reading, data streaming oversees system activity related to areas where data flows are generally rich and large.<\/p>\n

SEE: Hiring Kit: Database engineer<\/a> (TechRepublic Premium)<\/strong><\/p>\n

Confluent is a data streaming platform with a self-proclaimed mission to set data in motion. As complex as data streaming engineering sounds, the team has now created Confluent Stream Designer, a visual interface that is said to enable software developers to build streaming data pipelines in minutes.<\/p>\n

Designing data streams<\/h2>\n

Confluent offers a simple point-and-click user interface, but it\u2019s not necessarily a point-and-click UI for you and your favorite auntie or uncle. This is a point-and-click UI that is hoped to be something of an advancement toward making data streams accessible to developers beyond specialized Apache Kafka experts.<\/p>\n

Apache Kafka is an open source distributed event streaming platform created by Confluent co-founder and CEO Jay Kreps and his colleagues Neha Narkhede and Jun Rao while the trio worked at LinkedIn. Confluent offers a cloud-native foundational platform for real-time data streaming from multiple sources designed to be the \u201cintelligent connective tissue\u201d for software-driven backend operations that delivers rich front-end user functions.<\/p>\n

The theory behind Confluent Stream Designer is that with more teams able to rapidly build and iterate streaming pipelines, organizations can quickly connect more data throughout their business for more agile development alongside better and faster in-the-moment decision-making.<\/p>\n

At the Current 2022: The Next Generation of Kafka Summit<\/a> in Texas, there was the opportunity to speak directly with Confluent about its views and ambitions.<\/p>\n

\u201cWe are in the middle of a major technological shift, where data streaming is making real time the new normal, enabling new business models, better customer experiences and more efficient operations,\u201d said Kreps. \u201cWith Stream Designer we want to democratize this movement towards data streaming and make real time the default for all data flow in an organization.\u201d<\/p>\n

Streaming moves in from the edge<\/h2>\n

Kreps and the team further state that streaming technologies that were once at the edges have become the core of critical business functions.<\/p>\n

Because traditional batch processing can no longer keep pace with the growing number of use cases that depend on millisecond updates, Confluent says that more organizations are pivoting to streaming, as their livelihood is defined by the ability to deliver data instantaneously across customer experiences and business operations.<\/p>\n