Title
#users-public
Jaromir Hamala

Jaromir Hamala

11/09/2022, 1:10 PM
Hello everyone 👋, some time ago I ran a little survey about streaming technologies used together with QuestDB :questdb_new:. The survey shows Apache Kafka is by far the most popular tech. That’s not really surprising, is it? QuestDB has had a guide for Kafka -> QuestDB ingestion for a long time. The guide was based on the generic Kafka Connect JDBC Sink and it relied on QuestDB Postgres compatibility. It worked, but it had some drawbacks. Mainly: Performance was not great due to the combination of JDBC and parallelism always set to 1. Also schema management was somewhat complicated, it required events in topics to always have an explicit schema, etc. I am happy to announce we released the official QuestDB Connector for Kafka! The connector does use the Influx Line Protocol (ILP) under the hood. The ILP is the way to achieve massive ingestion speeds QuestDB is famous for. It can ingest 100 of 1000s rows / second, easily. 🔥 ILP also simplifies schema management, especially during development. The connector is documented at: https://questdb.io/docs/third-party-tools/kafka/questdb-kafka/ If you are using both Kafka and QuestDB then give it a try! As always: Feedback is very much appreciated! Either here or you if you don’t feel like sharing your Kafka - QuestDB story publicly then feel free to send me a private message.