Hi there is there a way to use LTTB largest triangle three b | Users-market-data

Channels

users-market-data

users-public

Feed

Channels

Title

#users-market-data

Kevin Grüneberg

07/18/2022, 11:40 AM

Hi there, is there a way to use LTTB (largest triangle three bucket) algorithm to downsample to a given resolution? Let’s say you are trying to fetch 25 years of stock data and you want to downsample it to a maximum of 500 quotes. LTTB is a great algorithm (compared to plain average).

javier ramirez

07/18/2022, 2:42 PM

I can give you a partial solution, not really LTTB. If you want to get a fixed number of samples, you can always precalculate the sampling factor. In your case, you want 500 samples and have 25 years. 25 years have ~25*8760 hours, so 219000 hours. We divide by 500 and we get that we need to sample by 438hours

2:43 PM

Unfortunately the sample for the hour would be the average, so you don’t get the fancy properties of LTTB where you get the best value for visual representation and you always take a value present on the original dataset

Kevin Grüneberg

07/18/2022, 2:43 PM

Yeah, unfortunately that would not work well for us.

javier ramirez

07/18/2022, 2:43 PM

I understand. Just wanted to tackle the half of the problem that you can solve only with questdb 🙂

Kevin Grüneberg

07/18/2022, 2:44 PM

Is it maybe possible to run custom code before returning results?

2:44 PM

That way, we could at least reduce network traffic

javier ramirez

07/18/2022, 2:45 PM

Not right now. Something that has been discussed in the past is the possibility to add server side scripts to run on QuestDB using python, and that is in the future roadmap for QuestDB cloud. But as of today it is not implemented (or even started development)

👌 2 🙏 2

View count: 2

Join thread in Slack