https://questdb.io logo
Title
n

Newskooler

03/10/2023, 12:43 PM
Hi 👋 QDB restarted (for unknown reason). I don’t see any error logs when parsing the log files. However, I did get the following file in my home dir
hs_err_pid146884.log
Does anyone know how can I debug this? 🤔
j

Jaromir Hamala

03/10/2023, 12:44 PM
hi, can you paste the hs_err file here?
Newskooler sent me the file privately and with his permission I’m sharing the most relevant part: segfault in questdb 6.7 at:
---------------  T H R E A D  ---------------

Current thread (0x00007fbb1a0b3000):  JavaThread "questdb-shared-6" [_thread_in_native, id=146920, stack(0x00007fba4fe00000,0x00007fba4ff00000)]

Stack: [0x00007fba4fe00000,0x00007fba4ff00000],  sp=0x00007fba4fefe1d8,  free space=1016k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libquestdb7781528099680187466.so+0xdd37]  Java_io_questdb_std_BitmapIndexUtilsNative_findFirstLastInFrame0+0x1a7

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 12190  io.questdb.std.BitmapIndexUtilsNative.findFirstLastInFrame0(IJJJJJJJJIJJI)I io.questdb@6.7 (0 bytes) @ 0x00007fbb08cf8b65 [0x00007fbb08cf8ac0+0x00000000000000a5]
J 12385 c2 io.questdb.griffin.engine.groupby.SampleByFirstLastRecordCursorFactory$SampleByFirstLastRecordCursor.getNextState(I)I io.questdb@6.7 (830 bytes) @ 0x00007fbb092b93a4 [0x00007fbb092b8e40+0x0000000000000564]
J 30329 c2 io.questdb.griffin.engine.groupby.SampleByFirstLastRecordCursorFactory$SampleByFirstLastRecordCursor.hasNext()Z io.questdb@6.7 (40 bytes) @ 0x00007fbb090527c0 [0x00007fbb09052660+0x0000000000000160]
J 16305 c2 io.questdb.cutlass.http.processors.JsonQueryProcessorState.doRecordFetchLoop(Lio/questdb/cutlass/http/HttpChunkedResponseSocket;I)V io.questdb@6.7 (30 bytes) @ 0x00007fbb08a69900 [0x00007fbb08a68f00+0x0000000000000a00]
J 9334 c1 io.questdb.cutlass.http.processors.JsonQueryProcessorState$$Lambda$536.onResume(Lio/questdb/cutlass/http/HttpChunkedResponseSocket;I)V io.questdb@6.7 (10 bytes) @ 0x00007fbb0153d094 [0x00007fbb0153ccc0+0x00000000000003d4]
J 30327 c2 io.questdb.cutlass.http.processors.JsonQueryProcessor.doResumeSend(Lio/questdb/cutlass/http/processors/JsonQueryProcessorState;Lio/questdb/cutlass/http/HttpConnectionContext;Lio/questdb/griffin/SqlExecutionContext;)V io.questdb@6.7 (118 bytes) @ 0x00007fbb09249450 [0x00007fbb09249340+0x0000000000000110]
J 9332 c1 io.questdb.cutlass.http.processors.JsonQueryProcessor.executeSelect(Lio/questdb/cutlass/http/processors/JsonQueryProcessorState;Lio/questdb/cairo/sql/RecordCursorFactory;Ljava/lang/CharSequence;)V io.questdb@6.7 (76 bytes) @ 0x00007fbb0153db7c [0x00007fbb0153d780+0x00000000000003fc]
J 9006 c1 io.questdb.cutlass.http.processors.JsonQueryProcessor$$Lambda$363.execute(Lio/questdb/cutlass/http/processors/JsonQueryProcessorState;Lio/questdb/griffin/CompiledQuery;Ljava/lang/CharSequence;)V io.questdb@6.7 (11 bytes) @ 0x00007fbb0128a004 [0x00007fbb01289e80+0x0000000000000184]
J 17261 c2 io.questdb.cutlass.http.processors.JsonQueryProcessor.execute0(Lio/questdb/cutlass/http/processors/JsonQueryProcessorState;)V io.questdb@6.7 (426 bytes) @ 0x00007fbb08daf478 [0x00007fbb08dad280+0x00000000000021f8]
J 25741 c2 io.questdb.cutlass.http.processors.JsonQueryProcessor.onRequestComplete(Lio/questdb/cutlass/http/HttpConnectionContext;)V io.questdb@6.7 (89 bytes) @ 0x00007fbb08ce7a8c [0x00007fbb08ce7980+0x000000000000010c]
J 16328 c2 io.questdb.cutlass.http.HttpConnectionContext.handleClientRecv(Lio/questdb/cutlass/http/HttpRequestProcessorSelector;Lio/questdb/cutlass/http/RescheduleContext;)Z io.questdb@6.7 (785 bytes) @ 0x00007fbb091cea98 [0x00007fbb091ce440+0x0000000000000658]
J 12225 c2 io.questdb.network.AbstractIODispatcher.processIOQueue(Lio/questdb/network/IORequestProcessor;)Z io.questdb@6.7 (91 bytes) @ 0x00007fbb0908ee54 [0x00007fbb0908eae0+0x0000000000000374]
J 2943 c2 io.questdb.cutlass.http.HttpServer$1.run(I)Z io.questdb@6.7 (112 bytes) @ 0x00007fbb0887e334 [0x00007fbb0887e260+0x00000000000000d4]
J 2219% c2 io.questdb.mp.Worker.run()V io.questdb@6.7 (553 bytes) @ 0x00007fbb08815788 [0x00007fbb088156c0+0x00000000000000c8]
v  ~StubRoutines::call_stub

siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007fa8d00a6138
b

Bolek Ziobrowski

03/10/2023, 1:24 PM
@Newskooler Could you share schema and query that triggered this crash ?
n

Newskooler

03/10/2023, 1:49 PM
@Tihomir Dimov can you please assist with this? @Jaromir Hamala yes, there is data being streamed to the table(s)
t

Tihomir Dimov

03/10/2023, 1:58 PM
It looks like the problem query was:
SELECT first(timestamp)
FROM table_name
WHERE symbol = Test
SAMPLE BY 1d ALIGN TO CALENDAR
or
SELECT symbol
FROM table_name
LATEST BY symbol
ORDER BY symbol
CREATE TABLE 'table_name' (
  value FLOAT,
  value1 FLOAT,
  value2 LONG,
  symbol SYMBOL capacity 256 CACHE index capacity 256,
  symbol1 SYMBOL capacity 256 CACHE index capacity 256,
  ts TIMESTAMP,
  timestamp TIMESTAMP
) timestamp (timestamp) PARTITION BY DAY;
n

Newskooler

03/11/2023, 11:41 AM
@Jaromir Hamala if it helps, this happens to us all the time, so it seems quite reproducible. I see the same error logs today as well.
@Jaromir Hamala can you please share any info about this once you have some?