https://questdb.io logo
Title
f

Furkan Küçük

12/06/2022, 7:30 AM
Another question: I have a docker file like this:
FROM ubuntu:latest AS builder
WORKDIR /opt
RUN apt-get update && apt-get install -y curl wget unzip jq
RUN curl -s <https://api.github.com/repos/questdb/kafka-questdb-connector/releases/latest> | jq -r '.assets[]|select(.content_type == "application/zip")|.browser_download_url'|wget -qi -
RUN unzip kafka-questdb-connector-*-bin.zip
FROM confluentinc/cp-kafka-connect:7.2.2
COPY --from=builder /opt/kafka-questdb-connector/*.jar /kafka/connect/questdb-connector/
RUN confluent-hub install --no-prompt confluentinc/kafka-connect-jdbc:10.5.4
USER 1001
However, I am getting this error:
{"error_code":500,"message":"Failed to find any class that implements Connector and which name matches io.questdb.kafka.QuestDBSinkConnector, available connectors are: PluginDesc{klass=class io.confluent.connect.jdbc.JdbcSinkConnector, name='io.confluent.connect.jdbc.JdbcSinkConnector', version='10.5.4', encodedVersion=10.5.4, type=sink, typeName='sink', location='file:/usr/share/confluent-hub-components/confluentinc-kafka-connect-jdbc/'}, PluginDesc{klass=class io.confluent.connect.jdbc.JdbcSourceConnector, name='io.confluent.connect.jdbc.JdbcSourceConnector', version='10.5.4', encodedVersion=10.5.4, type=source, typeName='source', location='file:/usr/share/confluent-hub-components/confluentinc-kafka-connect-jdbc/'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='7.2.2-ccs', encodedVersion=7.2.2-ccs, type=source, typeName='source', location='file:/usr/share/java/kafka/'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='7.2.2-ccs', encodedVersion=7.2.2-ccs, type=source, typeName='source', location='file:/usr/share/java/kafka/'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='7.2.2-ccs', encodedVersion=7.2.2-ccs, type=source, typeName='source', location='file:/usr/share/java/kafka/'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='7.2.2-ccs', encodedVersion=7.2.2-ccs, type=sink, typeName='sink', location='file:/usr/share/java/confluent-control-center/'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='7.2.2-ccs', encodedVersion=7.2.2-ccs, type=source, typeName='source', location='file:/usr/share/java/confluent-control-center/'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='7.2.2-ccs', encodedVersion=7.2.2-ccs, type=source, typeName='source', location='file:/usr/share/java/confluent-control-center/'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='7.2.2-ccs', encodedVersion=7.2.2-ccs, type=sink, typeName='sink', location='file:/usr/share/java/confluent-control-center/'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='7.2.2-ccs', encodedVersion=7.2.2-ccs, type=source, typeName='so
j

Jaromir Hamala

12/06/2022, 8:34 AM
checking
f

Furkan Küçük

12/06/2022, 9:54 AM
@Jaromir Hamala Hey Jaromir, sorry to bother you. Any news?
j

Jaromir Hamala

12/06/2022, 10:03 AM
hello, yes. your base image (
confluentinc/cp-kafka-connect
) uses a different file layout. hence
COPY --from=builder /opt/kafka-questdb-connector/*.jar /kafka/connect/questdb-connector/
should be changed to something like:
COPY --from=builder /opt/kafka-questdb-connector/*.jar /usr/share/java/kafka/
can you give it a try and report back?
the path
/kafka/connect/questdb-connector/
looks like for a Debezium base image. I assume you took it from the “Stocks” code sample? Confluent uses a different file layout in their base images. The “Faker” sample uses a Confluent image: https://github.com/questdb/kafka-questdb-connector/blob/0e4480c948ffdbcd50e0025694[…]84820d/kafka-questdb-connector-samples/faker/Dockerfile-Connect
f

Furkan Küçük

12/06/2022, 10:07 AM
Seems like that was the one. One more question: How can I get all topics by specifying regex?
is it working?
f

Furkan Küçük

12/06/2022, 10:33 AM
@Jaromir Hamala I was aware of that topics.regex thing. I'm just creating dummy redpanda topics and will try it in few minutes. Will certainly let you know how it goes!
j

Jaromir Hamala

12/06/2022, 10:33 AM
oh, you are also using Redpanda. very cool!
f

Furkan Küçük

12/06/2022, 10:34 AM
@Jaromir Hamala Yeah, found out from a blog post at QuestDB 🙂 Not needing zookeper and having even greater performance, redpanda is my go-to kafka variant 🙂