Title
#users-public
p

pok liu

08/24/2022, 4:33 AM
Are there any plans to support connection pooling for questdb ILP client(java)?
Jaromir Hamala

Jaromir Hamala

08/24/2022, 7:29 AM
Hello Pok, there are no plans at this moment. However it should work with a generic object/connection pool. What’s your motivation for pooling?
p

pok liu

08/24/2022, 8:16 AM
Hi @Jaromir Hamala. I have an application that consumes data from MQ and writes to questdb via ILP client. After the network fluctuates during the program running, the ILP client will not automatically reconnect. I wonder if this problem can be solved if connection pooling is supported.
Jaromir Hamala

Jaromir Hamala

08/24/2022, 8:26 AM
that’s a good use-case! a pool would not solve this on its own. this has to be done in concert with health-checking. for example JDBC pools often allow to define “Test SQL” which is executed just before giving a connection to a caller. this ensures the connection is healthy. ILP currently does not have a way to detect a broken connection before using it. so a pool would not have any way to test it. I will think about a possible solution.
p

pok liu

08/24/2022, 11:02 AM
Yeah. An ILP client that automatically reconnects will greatly improve application reliability. Looking forward to your solution!😀
Jaromir Hamala

Jaromir Hamala

08/24/2022, 11:09 AM
Yeah, the idea I’m toying with is two-fold:1. Introduce some kind of a connection health checking. This will likely require ILP protocol changes, but we need to do them for other reasons too. The protocol changes should not affect the end-user API. 2. Introduce a built-in Sender(Connection)Pool. So the Sender configuration would be done on the pool level. The pool would have a method like
get()
which would return a connection from the pool and do an optional connection health check. This way user applications would just obtain an instance of a pool (e.g. via Dependency Injection) and they could get a connection from the pool. without caring about details how exactly is the connection constructed or how to do a health check. Does make sense to you?
Adam Cimarosti

Adam Cimarosti

08/24/2022, 1:27 PM
If you have frequent disconnects you should also check your server logs: The QuestDB server will disconnect a client when the client inserts illegal rows.
1:27 PM
Eg. inserting a string value in a boolean column.
1:28 PM
NB: The ILP protocol does not send errors from the server back to the client.
p

pok liu

08/25/2022, 7:22 AM
@Jaromir Hamala This might be a good solution! But I think it might not be worth it to modify the ILP protocol to do a health check in order to get the socket state. Perhaps another short-term simple way is to make the application code aware of network exceptions, and provide socket reconnection functions in
LineTcpSender
and
LineUdpSender
that can be called by the Sender object.
7:24 AM
Hi @Adam Cimarosti Thanks for your reminder! My application ILP client has not been disconnected from the socket by the QuestDB server due to data errors (the application should verify the data before writing it). The only disconnect so far has been confirmed to be a network factor.