https://questdb.io logo
Title
v

vilinski

03/06/2023, 7:04 PM
Hi trying to test the questdb for our needs. Generated test data for a hour and measured used disk space. What I find suspectful - constant, incrementing and random values are resulting in exact same disk size - 867 MB, compared to 19, 226, 261 MB in case of influxdb it also looks as a pretty bad compression What could I do wrong?
a

Alex Pelagenko

03/06/2023, 7:05 PM
maybe nothing. What are the table column types?
v

vilinski

03/06/2023, 7:28 PM
uuid, string, timestamp in ns, and float value
a

Alex Pelagenko

03/06/2023, 7:49 PM
is string length random? Or is it tag / symbol? How many rows you generated?
v

vilinski

03/06/2023, 7:50 PM
uuid and string are symbols, 10000 series of them generated one row per second for a hour 10000 series x 3600 s = 36_000_000 rows
a

Alex Pelagenko

03/06/2023, 8:21 PM
Should be 20 bytes per record.
720mb in total
Plus a bit more for symbols
v

vilinski

03/06/2023, 8:34 PM
so there is no gorilla or lz4 compression?
a

Alex Pelagenko

03/06/2023, 8:34 PM
No
v

vilinski

03/06/2023, 8:35 PM
ok, the numbers are looking realistically then thank you
n

Nicolas Hourcard

03/06/2023, 9:26 PM
hey @vilinski it is possible to use zfs compression (compresses 3x roughly), @Andrey Pechkurov will be able to share some specifics about that
a

Andrey Pechkurov

03/07/2023, 7:12 AM
Hi, You could try using ZFS with enabled lz4 compression. In our tests it provides around 2.5-3x compression ratio
compared to 19, 226, 261 MB in case of influxdb it also looks as a pretty bad compression
We don't have built-in compression, but it's on the roadmap. For now, using more narrow column types and, optionally, ZFS is the way to go
n

Nicolas Hourcard

03/07/2023, 8:41 AM
@vilinski just to make sure you see this! thanks
v

vilinski

03/07/2023, 11:50 AM
👍