Title
#users-public
b

Bartosz

09/08/2022, 9:19 AM
Hello guys... Was something change between 6.2 and 6.5.2 with ILP? My service crash after write via open socket connection. I'm using basic Node net library
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 9:22 AM
Hi Bartosz . Many changes went into ILP since 6.2 (https://github.com/questdb/questdb/releases) . Does qdb service crash or clients gets disconnected from qdb ?
b

Bartosz

09/08/2022, 9:23 AM
I can create connection to QuestDB, but on writing it's crash
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 9:24 AM
Does qdb instance crash ? Can you check logs ?
b

Bartosz

09/08/2022, 9:24 AM
No, just connection
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 9:24 AM
Since 6.2.1 ILP errors trigger connection close .
9:25 AM
It depends on
line.tcp.disconnect.on.error
setting, which is true by default .
9:25 AM
Please check logs for details .
9:25 AM
Probably malformed ILP .
b

Bartosz

09/08/2022, 9:29 AM
I've cleanup QuestDB logs and then retried write, but logs are empty... And here is my ILP command
9:29 AM
module_history devid=RHXRXFLXLG,address=P32u0,value=52i,isFloat=false,created_at=1662629292000i 1662629294086000000
Jaromir Hamala

Jaromir Hamala

09/08/2022, 9:47 AM
hi Bartosz, I’m assuming the
devid
and
address
columns are strings. if so then you have to have values in double quotes. ie.:
module_history devid="RHXRXFLXLG",address="P32u0",value=52i,isFloat=false,created_at=1662629292000i 1662629294086000000
alternatively you can use one the language clients to avoid dealing with the ILP protocol directly: https://questdb.io/docs/reference/clients/overview
b

Bartosz

09/08/2022, 9:48 AM
devid and address have type SYMBOL
Jaromir Hamala

Jaromir Hamala

09/08/2022, 9:51 AM
then the line should look like this:
module_history,devid=RHXRXFLXLG,address=P32u0 value=52i,isFloat=false,created_at=1662629292000i 1662629294086000000
b

Bartosz

09/08/2022, 9:51 AM
Here is my structure
Jaromir Hamala

Jaromir Hamala

09/08/2022, 9:54 AM
is
module_history,devid=RHXRXFLXLG,address=P32u0 value=52i,isFloat=false,created_at=1662629292000i 1662629294086000000
working for you?
b

Bartosz

09/08/2022, 9:56 AM
Error: read ECONNRESET at TCP.onStreamRead (internal/stream_base_commons.js:209:20) { errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
Jaromir Hamala

Jaromir Hamala

09/08/2022, 9:56 AM
is there anything in questdb log at all?
9:58 AM
I’ll be AFK for lunch. will be back in about 30 minutes.
b

Bartosz

09/08/2022, 9:58 AM
No nothing, as @Bolek Ziobrowski ask me about QuestDB instance ... it's not crashed... just crash Socket connection to questdb... i'll try with your official liblary for node.js and ILP...
Newskooler

Newskooler

09/08/2022, 10:37 AM
I believe I experience the same issue. Will try to find error logs, but when using docker, it’s a bit harder to dig them out.
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 10:42 AM
With default settings aren't all logs available through docker logs command ?
Jaromir Hamala

Jaromir Hamala

09/08/2022, 10:43 AM
Hello Bartosz, I’m back. that’s weird. on a malformed ILP line the server should log something like this:
2022-09-08T09:57:44.800870Z I tcp-line-server connected [ip=127.0.0.1, fd=9]
2022-09-08T09:57:45.365152Z I i.q.c.l.t.TableUpdateDetails network IO thread using table [workerId=3, tableName=module_history, nNetworkIoWorkers=2]
2022-09-08T09:57:45.365291Z E i.q.c.l.t.LineTcpConnectionContext [9] could not process line data [table=module_history, msg=cast error for line protocol string [columnWriterIndex=0, columnType=SYMBOL], errno=0]
2022-09-08T09:57:45.365295Z E i.q.c.l.t.LineTcpConnectionContext [9] could not parse measurement, NONE at 122, line (may be mangled due to partial parsing): 'module_history devid="RHXRXFLXLG",address="P32u0",value=52i,isFloat=false,created_at=1662629292000000t 1662629294086000000'
2022-09-08T09:57:45.365664Z I tcp-line-server scheduling disconnect [fd=9, reason=0]
2022-09-08T09:57:45.365711Z I tcp-line-server disconnected [ip=127.0.0.1, fd=9, src=queue]
b

Bartosz

09/08/2022, 10:46 AM
logs should be stored in ./questdb/log directory ?
Jaromir Hamala

Jaromir Hamala

09/08/2022, 10:46 AM
how did you install questdb?
b

Bartosz

09/08/2022, 10:48 AM
wget https://github.com/questdb/questdb/releases/download/6.5.2/questdb-6.5.2-rt-linux-amd64.tar.gz then extract, stop old version process and start new from bin directory
Jaromir Hamala

Jaromir Hamala

09/08/2022, 10:51 AM
so by default it should be in
/home/<username>/.questdb/log
b

Bartosz

09/08/2022, 10:51 AM
yes right i've this directory, but i've removed old logs and new logs didin't came
Jaromir Hamala

Jaromir Hamala

09/08/2022, 10:53 AM
when questdb start it should print something like this to sysout:
ubuntu@instance-20210816-0834:~/questdb-6.5-no-jre-bin$ ./questdb.sh start

  ___                  _   ____  ____
 / _ \ _   _  ___  ___| |_|  _ \| __ )
| | | | | | |/ _ \/ __| __| | | |  _ \
| |_| | |_| |  __/\__ \ |_| |_| | |_) |
 \__\_\\__,_|\___||___/\__|____/|____/
                        <http://www.questdb.io|www.questdb.io>

JAVA: /home/ubuntu/.sdkman/candidates/java/current/bin/java
QuestDB server 6.5
Copyright (C) 2014-2022, all rights reserved.

Reading log configuration from /home/ubuntu/.questdb/conf/log.conf
Log configuration loaded from: /home/ubuntu/.questdb/conf/log.conf
10:53 AM
can you see it?
b

Bartosz

09/08/2022, 10:54 AM
Yes QuestDB instance working... i've also access to UI
10:55 AM
when i type questdb.sh status i've
10:55 AM
./questdb.sh status

  ___                  _   ____  ____
 / _ \ _   _  ___  ___| |_|  _ \| __ )
| | | | | | |/ _ \/ __| __| | | |  _ \
| |_| | |_| |  __/\__ \ |_| |_| | |_) |
 \__\_\\__,_|\___||___/\__|____/|____/
                        <http://www.questdb.io|www.questdb.io>

PID: 3855227
10:55 AM
so definitly it's working
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 10:56 AM
You can see open proc files with lsof -p 3855227
b

Bartosz

09/08/2022, 10:59 AM
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 11:04 AM
java    3855227 connect    1w      REG     253,0   387675   8917234 /home/connect/.questdb/log/stdout-2022-09-08T11-21-12.txt (deleted)
b

Bartosz

09/08/2022, 11:05 AM
ok, so i should restart process ?
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 11:05 AM
You've deleted file qdb is appending logs to . You can either restart qdb or tail it as is : tail -f /home/connect/.questdb/log/stdout-2022-09-08T11-21-12.txt
Newskooler

Newskooler

09/08/2022, 11:06 AM
@Bolek Ziobrowski yes, all logs are available, but they are overwhelmingly many since I stream a lot of data to the DB.
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 11:08 AM
There's a number of options you can use to limit output , including filtering with grep
Newskooler

Newskooler

09/08/2022, 11:11 AM
True but I am not sure how to filter for the faulty table.
11:11 AM
Not the faulty table, but the table I save to which causes a reset.
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 11:13 AM
You could try grepping by log level. Errors have E after timestamp .
Newskooler

Newskooler

09/08/2022, 11:14 AM
👍 okay, that’s a good start for me. Thanks!
11:15 AM
is it possible to persist error logs to an attached volume outside of docker?
b

Bartosz

09/08/2022, 11:17 AM
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 11:18 AM
Sure, you can either mount that volume and re-configure log.conf to append errors to files residing on it OR just send all container logs to centralized log manager and split/filter there .
Newskooler

Newskooler

09/08/2022, 11:20 AM
Is there a guide how to do the log mount? I already thought I did it, but apparently I did not.
11:21 AM
is it simply something similar to:
-v /my_server_path/questdb/log:/questdb/log
Bolek Ziobrowski

Bolek Ziobrowski

09/08/2022, 11:23 AM
@Bartosz I think there's a whitespace missing after last symbol column (address): is
module_history,devid=YELZKEGFAE,address=P30v42,value=21i,isFloat=false,created_at=1662635791000i 1662635791981000000
but should be
module_history,devid=YELZKEGFAE,address=P30v42 value=21i,isFloat=false,created_at=1662635791000i 1662635791981000000
11:23 AM
https://questdb.io/docs/reference/api/ilp/overview/ Syntax# table_name,symbolset columnset timestamp\n
b

Bartosz

09/08/2022, 11:29 AM
It's works 🙂 But there is one more question... Until 6.2 sometimes i'm used ILP command without set value so then i've in database record with value null (unset). Is still working so? Null give me information about expire of previous value and there is no new value until new row
Jaromir Hamala

Jaromir Hamala

09/08/2022, 11:30 AM
yes, this is the same. just omit the column from your ILP line and it will be set to null.
b

Bartosz

09/08/2022, 11:32 AM
Thank you guys for help!! I'm grateful!
Jaromir Hamala

Jaromir Hamala

09/08/2022, 11:32 AM
still - I’d recommend migrating to the official node.js client if you can. it saves you from errors like a missing space, it supports TLS, authentication, escape data, it also validates data to make sure you don’t use any illegal chars in a table/column name, etc.