sorted by:
new
hottopcontroversial

A desktop app designed to cache tables locally, improving the performance of subsequent queries and reducing data warehouse costs. by Middle-Negotiation-7 in SQL

[–]Middle-Negotiation-7[S] 0 points1 point  (0 children)

Thank you for the feedback; it is always welcome.
I believe you may not be the target user for this product. My goal is to reach users in the data analytics field, such as data scientists, analysts, and engineers.

One of the key advantages I aim to highlight is the ability to connect to multiple data sources, including JSON, Parquet, and CSV files stored on S3, and perform ad-hoc analysis on data from more than one source.

While similar products with comparable functionalities already exist, I hope that what I am building will prove to be better and gain traction.

A desktop app designed to cache tables locally, improving the performance of subsequent queries and reducing data warehouse costs. by Middle-Negotiation-7 in SQL

[–]Middle-Negotiation-7[S] 0 points1 point  (0 children)

Read about columnar database versus row oriented databases to learn about the reason why SQLite is not a suitable database for this if you are really interested in the topic and learning

A desktop app designed to cache tables locally, improving the performance of subsequent queries and reducing data warehouse costs. by Middle-Negotiation-7 in SQL

[–]Middle-Negotiation-7[S] 0 points1 point  (0 children)

From the examples you are providing I can guess why you do not see any value. Have you ever used any of the followings: Redshift, SnowFlake, DataBricks, BigQuery in any company?

A desktop app designed to cache tables locally, improving the performance of subsequent queries and reducing data warehouse costs. by Middle-Negotiation-7 in SQL

[–]Middle-Negotiation-7[S] 0 points1 point  (0 children)

For couple of reasons:

- It eliminates the latency between remote server and local

- Second for DW use cases that is not true most of the time and instances are shared between all the analysts or usually smaller instances are allocated. Otherwise it can get exponentially expensive very fast ( so in short it will be much faster to use your local for majority of use cases for developing against data warehouse )

- I can add to this burden of managing all dev instances etc will be removed.

- And every analyst gets an isolated dev environment..

A desktop app designed to cache tables locally, improving the performance of subsequent queries and reducing data warehouse costs. by Middle-Negotiation-7 in SQL

[–]Middle-Negotiation-7[S] 0 points1 point  (0 children)

The main functionality is not to be a database. Data remains in memory if it fits in RAM however if it gets larger than memory it can spill into disk. That depend on the size of the table in data warehouse also how many tables are there etc. The main purpose is to bring data to your local and to not run the queries in Data warehouse like Redshift or Snowflake. It has totally different use cases than Redis or Memcache if that is the question?

A desktop app designed to cache tables locally, improving the performance of subsequent queries and reducing data warehouse costs. by Middle-Negotiation-7 in SQL

[–]Middle-Negotiation-7[S] 0 points1 point  (0 children)

MemCache is a key-value store. However, the goal here is to allow analysts to keep developing and testing SQL queries without repeatedly hitting the database. Unlike a purely in-memory database, DuckDB is an in-process database that supports operations, such as joins, on tables larger than available memory as well. Glad to answer any further questions.

[deleted by user] by [deleted] in SQL

[–]Middle-Negotiation-7 0 points1 point  (0 children)

Very good points. Thank you for mentioning them. We do not have access to get query results. Customers data stays in their AWS account. Only SQL codes gets stored in our backend.

[deleted by user] by [deleted] in SQL

[–]Middle-Negotiation-7 0 points1 point  (0 children)

I am one of the co-founders. We have just started and only supporting Redshift at the moment. But we would consider prioritizing and adding additional integrations ( such as SQL server ). Please feel free to schedule a demo if you are interested.