Title
#users-public
j

Joshua Xia

11/20/2022, 7:47 AM
can questdb python client ingest a pandas dataframe? Is there any samples? Thanks
Nicolas Hourcard

Nicolas Hourcard

11/20/2022, 3:59 PM
hey @Joshua Xia, turns out @Adam Cimarosti is working on this now and it should be out there very soon!
j

Joshua Xia

11/21/2022, 3:02 AM
Thanks for your reply, another question: is there plan to support flux language of influxdb? I just plan to migrate from influxdb to QuestDB, Thanks!
Pei

Pei

11/21/2022, 3:16 AM
Hi Joshua, thanks for asking! Currently we don’t plan to support flux. We’re planning to stay closer to SQL and will gradually increase compatibility with PostgreSQL.
j

Joshua Xia

11/21/2022, 5:11 AM
Thanks, I got, I assume that I can do all processing on data by using SQL, right? But I just find there is no “database”, just “tables” in questdb? another problem is that I can use influxdb ILP accroding to samples, but it insert data row by row? Can I insert all rows from pandas dataframe in one line code? I plan to migrate my stocks database from influxdb to questdb, what is the best parameters for questdb in this case? Thanks
5:21 AM
Is there the SOTA config of QuestDB to store all stocks data(OHLCV) and ticket data, forex data in a market? Can QuestDB optimize its performance according to hardware environment automatically or I have to select the best configuration by experience manually? Thanks!
Nicolas Hourcard

Nicolas Hourcard

11/21/2022, 2:40 PM
Hi Joshua - yes we do the processing of data via SQL. you’re right, we do not have the concept of databases, just tables.
2:41 PM
Can I insert all rows from pandas dataframe in one line code? yes this will be possible once we release this functionality soon, @Adam Cimarosti has got the details
2:42 PM
plan to migrate my stocks database from influxdb to questdb, what is the best parameters for questdb in this case?
You could start streaming data via ILP, or do a bulk CSV import via the COPY command to import everything in one go. https://questdb.io/docs/guides/importing-data/
2:43 PM
what do you mean by SOTA config?
2:45 PM
Can QuestDB optimize its performance according to hardware environment automatically or I have to select the best configuration by experience manually
There is a little bit of fine tuning that can be done to reach the best possible performance. The following pages in our doc take you through the concepts that are specific to questdb and specific things you can do to further optimize performance https://questdb.io/docs/get-started/learn-more/ https://questdb.io/docs/operations/design-for-performance/ https://questdb.io/docs/operations/capacity-planning/
2:45 PM
We try to do our best to offer the best performance out of the box, but there will be some cases of fine tuning especially as you increase the data load significantly
j

Joshua Xia

11/21/2022, 11:03 PM
Thanks Nicolas’s help!
Nicolas Hourcard

Nicolas Hourcard

11/22/2022, 9:50 AM
sure thing. Is your project a personal one, or prod within a company?
j

Joshua Xia

11/22/2022, 10:38 AM
currently personal one, one prod within company is using clickhouse, we also investigating questdb to check it meet our prod’s requirements or not, currently we believe it’s performance is best, we wish QuestDB can provide better python client to meet machine learning, data processing requirements conveniently, data visualization console and easy configuration and so on in the future.
Pei

Pei

11/22/2022, 12:31 PM
Hi Joshua, thanks a lot for the feedback and suggestions 🙏 we’d love to learn which features among them are more critical / important for your prod environment and what else might be on your mind. Perhaps we could have a user interview to discuss each item? This will help us a lot on prioritizing the features.
Adam Cimarosti

Adam Cimarosti

11/22/2022, 1:10 PM
We are committed to improving the Python client. If there's specific features you'd find useful we'd very much like to know.
j

Joshua Xia

11/23/2022, 5:58 AM
sure, I am busy recently, I will summarize a feedback list when I migrate database and keep eyes on QuestDB ‘s last update
Pei

Pei

11/23/2022, 7:30 AM
Sounds good, thank you! 👍