Hi.. I am ingesting some data over ILP protocol and database is crashing with below error. I am using

m6a.2xlarge

so I don’t think resource crunch is isssue here. How can I avoid this?

2022-12-30T16:05:08.953802Z E i.q.c.l.t.LineTcpConnectionContext [37227] could not parse measurement, NONE at 1282, line (may be mangled due to partial parsing): 'cdn_logs,ContentId=6346ba504da63b20baf6ecb0,RequestId=F77E08D57C5646C798ED3A53FC33C4AB,JobId=KUOC0ENJS6P5,DistributionType=VOD,FileType=DASH_VIDEO_SEGMENT,Method=GET,CdnType=CloudFront,DistributionId=JJJJC0ENJS6P5,StreamingPartnerId=ABC-ABC,UserCity=London,UserCountry=United Kingdomm,UserDevice=Mobile,UserBrowser=-,UserDeviceBrand=Dangalplay,UserDeviceName=Dangalplay Android Mobilele,UserId=iWGYSC6ZAOR414EBX20,Agent=Special,AgentName=<http://Com.dangalplay.tv|Com.dangalplay.tv>,UserOs=Android,OsClass=Mobile CdnRequestId="-IxVGYSFfWfi_bnXatuj0CLrQ277PqG5ShOSPfLkwFyt3GPmcVwIsw==",UserAgent="<http://com.dangalplay.tv/1.0.10%20(Linux;Android%2010)%20ExoPlayerLib/2.16.1|com.dangalplay.tv/1.0.10%20(Linux;Android%2010)%20ExoPlayerLib/2.16.1>",Referer="",Domain="<http://streams-msp.test.in|streams-msp.test.in>",Ip="1.2.3.4",Extras="",FileName="chunk_ctvideo_ridp0va0br2500000_cs50400000_w1550299147_mpd.m4s",CacheStat="Miss",Resolution="854x480",EdgeLocation="BOM78-P4",FilePath="/ENTERR10/EPISODE/6346ba504da63b20baf6ecb0/dash/854x480/chunk_ctvideo_ridp0va0br2500000_cs50400000_w1550299147_mpd.m4s",AgentVersion="1.0.10",AgentVersionName="<http://Com.dangalplay.tv|Com.dangalplay.tv> 1.0.10",OsVersionName="Android 10",OsVersion="10",isValid=t,SegmentCount=1i,ViewCount=0i,FileSize=263306i,ResponseTime=0.106,ua=1672436108953000t,ca=1672436108953000t 1670508923000000000'^M
2022-12-30T16:05:08.953813Z I tcp-line-server scheduling disconnect [fd=37227, reason=0]^M
2022-12-30T16:05:08.953815Z I tcp-line-server disconnected [ip=10.60.13.143, fd=37227, src=queue]^M
2022-12-30T16:05:08.953969Z I tcp-line-server connected [ip=10.60.13.143, fd=37227]^M
2022-12-30T16:05:08.954072Z I i.q.c.p.PGConnectionContext query cache used [fd=23691]^M
2022-12-30T16:05:08.954122Z I tcp-line-server connected [ip=10.60.13.143, fd=37601]^M
2022-12-30T16:05:08.954343Z I i.q.c.p.PGConnectionContext query cache used [fd=23320]^M
2022-12-30T16:05:08.954628Z I tcp-line-server scheduling disconnect [fd=24098, reason=0]^M
2022-12-30T16:05:08.954630Z I tcp-line-server disconnected [ip=10.60.13.143, fd=24098, src=queue]^M
2022-12-30T16:05:08.954675Z I tcp-line-server connected [ip=10.60.13.143, fd=24098]^M
2022-12-30T16:05:08.954697Z I i.q.c.p.PGConnectionContext query cache used [fd=63343]^M
2022-12-30T16:05:08.955128Z I tcp-line-server connected [ip=10.60.13.143, fd=39064]^M
[thread 10751 also had an error][thread 10750 also had an error]

Compiled method (c1) 209430797 6875       3       io.questdb.cairo.ConcurrentBitmapIndexFwdReader$Cursor::of (368 bytes)
 total in heap  [0x00007f34122a6810,0x00007f34122a9660] = 11856
 relocation     [0x00007f34122a6988,0x00007f34122a6c38] = 688
 main code      [0x00007f34122a6c40,0x00007f34122a87c0] = 7040
 stub code      [0x00007f34122a87c0,0x00007f34122a8970] = 432
 oops           [0x00007f34122a8970,0x00007f34122a8990] = 32
 metadata       [0x00007f34122a8990,0x00007f34122a8a08] = 120
 scopes data    [0x00007f34122a8a08,0x00007f34122a9030] = 1576
 scopes pcs     [0x00007f34122a9030,0x00007f34122a9550] = 1312
 dependencies   [0x00007f34122a9550,0x00007f34122a9560] = 16
 nul chk table  [0x00007f34122a9560,0x00007f34122a9660] = 256
Could not load hsdis-amd64.so; library not loadable; PrintAssembly is disabled
#
# If you would like to submit a bug report, please visit:
#   <https://github.com/AdoptOpenJDK/openjdk-support/issues>

Hi Ajay . This output looks weird - like qdb log mixed with not necessarily crash-related diagnostic messages . Did jvm produce hs_err_pidX.log file ? Could you share it ?

@Bolek Ziobrowski JVM did not produce any such file

Do you have logs suggesting a crash ?

@Bolek Ziobrowski I have only these logs from qdb log file. I am not sure which other logs I can check

hs_err_pid file is usually generated in process work directory - so it depends on how you run qdb . As for qdb logs - you can e.g. grep for "fatal error" .

Besides crash, You are trying to insert an invalid ILP line. Tags like:

UserCountry=United Kingdomm

are incorrect. Same for

CacheStat="Miss"

You cannot use quoted strings as tag values. Use a

\

to escape: space, comma, and double quotes in tag values What client are you using to get the data? We have a reference implementation in different languages ​​to avoid such errors and debugging. https://questdb.io/docs/reference/clients/overview/

If you want to use quoted strings, use fields instead

Can you share any example? I have similar implementation as per refrence docs

You are trying to insert an invalid ILP line.

I am not sure about invalid line as I have already inserted 20 million rows with same approach

hs_err_pid

I am unable to find such file.

As for qdb logs - you can e.g. grep for “fatal error”

Tried grepping for fatal error but no such logs

That means it's not crashing but rejecting malformed ilp messages . Default behavior is to drop connection on error .

@Eugene I am using quest library to insert data. Shouldn’t this library handle spacing in United Kingdom?

sender.table("content_details_test")
                    .symbol("Country", "United Kingdom")
                    .at(System.currentTimeMillis());

LineSender It does escape the common symbols but it can't fix all possible errors .

No, it's a raw tag method, it validates tag/field name only. Use field instead,

sender.table("content_details_test")                    .field("Country", "United Kingdom")
                    .at(System.currentTimeMillis());

it will auto-quote value.

@Eugene for symbols and strings both I can use field?

ilp has no such type as symbol. if you want to use a symbol column, just create the table schema beforehand