Title
#users-public
Newskooler

Newskooler

09/07/2022, 3:59 PM
Hi QDB team, I am hitting a strange bottleneck here: I am reading data from QDB from port
9000
using
exec
via REST API. For some reason, I am not getting all rows… despite by query being a simple
SELECT * FROM table
Why is that? It works perfect in the UI. Is the REST API having a limit of sorts that is silently triggered?
Jaromir Hamala

Jaromir Hamala

09/07/2022, 6:49 PM
Hello, how many rows you are getting? is it a round number?
6:50 PM
can you open a dev console in your browser and compare the request done by web console (questdb ui) with the request you are sending manually? the web console uses the same REST interface so it should behave the same.
Newskooler

Newskooler

09/08/2022, 11:16 AM
Hi @Jaromir Hamala I did some digging now, and basically, the same request (via curl of via python requests.get) get’s me different results every time I do it. As if it has a time-based timeout and depending on internet speed i get more or less results. In the UI - I get all results, but the UI does not load them fully, right, so it’s not a 1:1 comparison. Do you know what this strange behavior exists?
Jaromir Hamala

Jaromir Hamala

09/08/2022, 11:23 AM
I believe the web UI must fully load query result before it can start displaying anything. Basically it must receive the HTTP response in full. But there could be one difference: The web console includes implicit LIMIT. You can see this
/exec?limit=0,1000
in HTTP requests the console sends. This can make the query faster the execute. How long is the query running? There is a default timeout.
Newskooler

Newskooler

09/08/2022, 11:25 AM
What is the default timeout? THat must be it I guess
Jaromir Hamala

Jaromir Hamala

09/08/2022, 11:28 AM
there is general query timeout which is 60s by default. Is your query time anything close to this?
Newskooler

Newskooler

09/08/2022, 11:33 AM
Yes, that must be it. My query is 69s.
11:34 AM
the second one 80s.
11:34 AM
How can I increase this limit?
Jaromir Hamala

Jaromir Hamala

09/08/2022, 11:37 AM
ok, so that’s probably it. you have 2 options:1. increase the limit 2. make the query faster 🙂 If 60+s is still within your SLA and you are happy then #1 is simpler: open your
server.conf
and search for
query.timeout.sec
. You can increase it. Also make sure the line is not starting with
#
as questdb would ignore it.
Newskooler

Newskooler

09/08/2022, 11:40 AM
OKay, got it. Thank you. I can’t make it faster I guess, if it’s limited by the internet speed 😕
Jaromir Hamala

Jaromir Hamala

09/08/2022, 11:42 AM
ok, so the query itself is fast, but it takes time to fetch all the results. got it.
Newskooler

Newskooler

09/08/2022, 11:45 AM
indeed. Thanks for the help, Jaromir!
Andy

Andy

10/15/2022, 1:33 PM
when select from a bigger table , the qdb collapsed
Jaromir Hamala

Jaromir Hamala

10/15/2022, 1:41 PM
Hi, do you have a reproducer you can share?
Andy

Andy

10/15/2022, 2:35 PM
I delete the table and replace with a backup table then restart , It works !