Press Release Threat Detection & Defense

Conﬂuent Supercharges Apache Flink Offering with New Developer Tools and Enterprise-Ready Security

CSA Editorial September 26, 2024

5 minutes read

Conﬂuent, Inc. (NASDAQ:CFLT), the data streaming pioneer, introduced new capabilities to Conﬂuent Cloud to make stream processing and data streaming more accessible and secure. Conﬂuent’s new support of Table API makes Apache Flink® available to Java and Python developers; Conﬂuent’s private networking for Flink provides enterprise-level protection for use cases with sensitive data; Conﬂuent Extension for Visual Studio Code accelerates the development of real-time use cases; and Client-Side Field Level Encryption encrypts sensitive data for stronger security and privacy.

“The true strength of using Apache Flink for stream processing empowers developers to create applications that instantly analyse and respond to real-time data, signiﬁcantly enhancing responsiveness and user experience,” said Stewart Bond, Research Vice President at IDC. “Managed Apache Flink solutions can eliminate the complexities of infrastructure management while saving time and resources. Businesses must look for a Flink solution that seamlessly integrates with the tools, programming languages, and data formats they’re already using for easy implementation into business workﬂows.”

More businesses are relying on stream processing to build real-time applications and pipelines for various use cases spanning machine learning, predictive maintenance, personalised recommendations and fraud detection. Stream processing lets organisations blend and enrich their data with information across their business.

Apache Flink is the de facto standard for stream processing. However, many teams hit roadblocks with Flink because it’s operationally complex, difﬁcult to secure, and has expensive infrastructure and management costs.

“Thousands of teams worldwide use Apache Flink as their trusted steam processing solution to deliver exceptional customer experiences and streamline operations by shifting processing closer to the source, where data is fresh and clean,” said Shaun Clowes, Chief Product Ofﬁcer at Conﬂuent. “Our latest innovations push the boundaries further, making it easier for developers of all skill levels to harness this powerful technology for even more mission-critical and complex use cases.”

Conﬂuent’s Support for Table API Extends Flink to Teams with Java or Python Experience

Conﬂuent Cloud for Apache Flink offers the SQL API, a powerful and user-friendly tool for processing data streams. While Flink SQL is effective for quickly writing and executing queries, some teams favor programming languages like Java or Python which allow for more control of their applications and data. This can be especially important when developing complex business logic or custom processing tasks.

Adding support for the Table API to Conﬂuent Cloud for Apache Flink enables Java or Python developers to easily create streaming applications using familiar tools. By supporting both Flink SQL and the Table API, Conﬂuent Cloud for Apache Flink lets developers choose the best language for their use cases.

Support for Table API enables companies to:

Enhance language ﬂexibility by enabling developers to use their preferred programming languages, taking advantage of language-speciﬁc features and custom
Streamline the coding process by leveraging customers’ integrated development environment (IDE) of choice featuring auto-completion, refactoring tools, and compile-time checks to ensure higher code quality and minimise runtime issues.
Make debugging easier with an iterative approach to data processing and streamlined CI/CD integration.

“We’re reimagining ﬁnancial services and it’s imperative that we adopt new technologies that not only protect our customers but are also easily used by our teams,” said Shujahat Bashir, Director of Software Engineering at Thrivent. “Data streaming and stream processing can help power real-time fraud detection, and payments processing for an exceptional customer experience. We are looking forward to using Conﬂuent’s new Apache Flink features including Private Networking, Flexible Schema Management, and Table APIs to extend the power of Flink to the languages and development styles we already use today.”

Support for Table API is available in open preview and is available for testing and experimentation purposes. General availability is coming soon.

Enabling Private Networking for Flink Provides a Secure Environment for Data Streaming Workloads

With more data than ever before and more teams using cloud and hybrid solutions, private networking is essential for protecting against unauthorised access and cyber threats. Conﬂuent Cloud now offers private networking support for Flink, providing a critical layer of security for businesses that need to process data within strict regulatory environments.

By enabling private networking for Flink, Conﬂuent users can:

Boost data security and privacy between Flink and Kafka by safeguarding in-transit data and ensuring secure connections between clients and Flink within a private network.
Simplify secure network configuration, making it easier to set up private connections without requiring extensive networking expertise.
Facilitate ﬂexible and secure stream processing by seamlessly joining and processing data across different Kafka clusters, ensuring data accessibility while adhering to strict security protocols.

Private networking support is generally available on AWS for Enterprise and Dedicated clusters. Additional cloud platforms are coming soon.

Conﬂuent Extension for Visual Studio Code Streamlines Workloads And Accelerates Development Cycles

Teams working with real-time data platforms like open-source Apache Kafka often struggle with fragmented tools, clunky workﬂows, and constant switching between environments and other interfaces. This disjointed experience makes it tough to integrate real-time data into applications, slowing down productivity and innovation. Integrated development environments can make this process much easier and 54% of developers globally use Visual Studio Code (VS Code), the most popular choice for code editing. Conﬂuent Extension for Visual Studio Code simpliﬁes the development process by integrating Conﬂuent directly into teams’ preferred integrated development environment.

Conﬂuent Extension for Visual Studio Code enables teams to:

Streamline topic management to easily create, edit, and browse Kafka topics with intuitive tools that simplify debugging and boost efﬁciency.
Code and debug in one place by writing, executing, and debugging Kafka clients, Flink queries, and streaming applications directly in VS Code with enhanced productivity features like code completion.
Seamlessly manage cloud resources to provision and control Conﬂuent Cloud clusters within VS Code, reducing complexity and streamlining cloud

Conﬂuent Extension for Visual Studio Code supports Kafka clients and is available in early access. New Flink capabilities and general availability will be available later in 2024.

Client-Side Field Level Encryption Safeguards Teams’ Most Sensitive Data While Meeting Regulatory Requirements

Data security and privacy are top priorities for organisations in regulated industries, including ﬁnancial services, healthcare, and the public sector. These organisations are subject to compliance rules about how sensitive data, like personally identiﬁable information (PII), can be accessed, moved, and stored.

Client-Side Field Level Encryption helps teams protect their most sensitive data by enabling them to encrypt individual ﬁelds within their data streams for enhanced security and compliance. Complementing existing Conﬂuent security features, Client-Side Field Level Encryption further reduces the risk of unwanted access by encrypting data on the client side so that even system admins and users with highly privileged access cannot view messages in plaintext.

With Client-Side Field Level Encryption teams can:

Improve the security of sensitive data and adhere to strict compliance
Maintain ﬂexible and granular access control which speciﬁc ﬁelds to
Lower total cost of ownership and operational complexity by reducing the need for topic duplication.

Client-Side Field Level Encryption is currently in limited availability and is expected to become generally available soon for all Conﬂuent Cloud users through the company’s Stream Governance suite.