Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

https://en.wikipedia.org/wiki/Online_analytical_processing as opposed to https://en.wikipedia.org/wiki/Online_transaction_processing

DuckDB is when you need to do OLAP analysis, and the data fits in a single node (your laptop), but it's too large for plain excel.

technically you can use PG/MySQL/Python+Numpy+Pandas to process those data for that use case as well, but DuckDB does it easier/faster most of the time.



What do frontends for this type of stuff look like these days? I remember one of my first jobs out of college they had wired up an OLAP cube into Excel so that you could import it into a pivot table and arbitrarily slice and dice data by dragging and dropping columns. I thought that was the coolest thing. Is there a modern day non excel equivalent of this? Cube.js?


I'd imagine Looker, PowerBI, Tableau are probably the front-ends you'd most commonly hear about, although I'm not sure DuckDb is supported or acknowledged by the parent companies (yet).


ok That sounds pretty interesting.. i am curious to find out when I would use this over parquet or pandas.


parquet is just a data file format. Both pandas and DuckDB can query parquet files.

It might be confusing that DuckDB does have its own columnar format, but it's more helpful to think it like "just a query engine (or library) for wrangling tables (or data frames)", i.e. same as pandas.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: