Apache Kafka now available

Platform.sh is pleased to announce the availability of the Apache Kafka streaming data service. Kafka is used to build real-time streaming applications and data pipelines, and enables the storage of streamed data, publisher-subscriber models (similar to a queue or enterprise messaging system), and stream processing in real time.

Kafka is included and available in all your Platform.sh projects. Getting started with Kafka on Platform.sh is as simple as using any other data services, such as Postgres, Elasticsearch, and Redis.

To configure a new Kafka service in your project, simply add this configuration to your project’s .platform/services.yaml file:

mykafka:
  type: kafka:2.1
  disk: 1024

Then in your application’s .platform.app.yaml, set up a relationship that allows your application (in Python, Ruby, etc.) to access the Kafka service:

relationships:
  kafka: "mykafka:kafka"

That’s it! No servers, VMs, or containers to configure or routing to set up.

You can then use the $PLATFORM_RELATIONSHIPS environment variable to access the Kafka service from your code.

Our documentation page on Kafka has details and examples of using Kafka from Python and Ruby, and more language examples are coming soon.

Kafka 2.1 is already available on all Professional plans as a single-node architecture.

Have questions or need assistance? Customers can always open a ticket with our support team, and developers are welcome to join our public Slack channel to chat with our engineering and customer success folks.

Let us know what awesome things you build with this new capability!