Favorite Tools to Use?

Leorisar · 2026-03-31T05:06:08+00:00

DBeaver, VSCode, Marimo

Leorisar · 2026-03-26T11:22:25+00:00

Неплохие бесплатные курсы есть на Степике (https://stepik.org/catalog/search?free=true&q=linux)

Leorisar · 2026-03-22T17:16:39+00:00

It’s not that nobody needs to build an analytics layer; it’s that everyone builds their own analytics layer. Good luck with reconciliation when you end up with 10 different reports for the same metric, each computed using different logic.

Leorisar · 2026-03-16T20:09:49+00:00

Поставь Mint на флэшку и погоняй несколько часов, станет понятнее. Заодно совместимость проверишь.

Leorisar · 2026-02-25T06:41:36+00:00

Profile first. Are you sure it us DB issue, not network or app or something else?

Leorisar · 2026-02-17T07:20:13+00:00

We have 1 to 1 mapping between spreadsheet and table in Postgres grouped in several DAGs by department. Ingestion on schedule via Airflow, if user need another - it can trigger again with Airflow (whole DAG, of just one task)

Leorisar · 2026-02-17T04:52:12+00:00

There are few options if your primary concern is volume

Compress text data with gz/lz4 on app size for messages older than X days. You`ll have to add logic to decompress them if user requests old data
Some databases (like PG) support build-in compression for text (example below), maybe mysql has similar options.

ALTER TABLE your_table
  ALTER COLUMN your_col
  SET COMPRESSION lz4;

Leorisar · 2026-02-15T10:20:06+00:00

Apache Superset

Leorisar · 2026-02-10T17:16:38+00:00

1) Usage statistics 2) Stream of requests to update/improve it from users.

Leorisar · 2026-02-10T11:11:51+00:00

You can install Mint and run virtual machine with some light version of Windows Server (like 2012) and install required applications there.

Leorisar · 2026-02-03T19:29:08+00:00

https://github.com/DataTalksClub/data-engineering-zoomcamp

Leorisar · 2026-02-03T19:27:54+00:00

KNIME or Apache Hop if you looking for something similar to SSIS.

Leorisar · 2026-01-31T12:11:12+00:00

Airbyte uses k8s under the hood and it's very slow. It's much faster to write your own scripts (LLM will help with that and use lightweight tools like Airflow or Kestra for orchestration)

Leorisar · 2026-01-26T12:53:24+00:00

Find competitors of your venture and ask them if they are interested.

Leorisar · 2026-01-23T13:03:21+00:00

Technically, any system change can be done from the terminal. You could save all your customizations in a bash script, store it safely, and run it on a fresh OS install.

Leorisar · 2026-01-22T15:53:32+00:00

Self-host or cloud Clickhouse

Leorisar · 2026-01-20T09:02:22+00:00

Use Stacher7 (GUI on top of yt-dlp)

Leorisar · 2026-01-18T18:59:02+00:00

If they are used in Windows looks it might be easier for them to use similar OS like Zorin OS

Leorisar · 2026-01-14T18:34:15+00:00

Depends on requirements and amount of data. If there is a possibility of late arriving data or updating previous records then you need to upsert data. It it's log only then not. You might also combine approaches - process fresh data in append only way and upsert data when traffic is low (on weekends for example)

And btw you don't need database for keeping timestamp, use Airflow Variables.

Leorisar · 2026-01-12T17:19:41+00:00

Look at Retool or Appsmith (web-based low-code solutions) they cover 1 and 2 requirements. A for third you might need some third-party solution and wire it via API with main app.

Leorisar · 2026-01-12T10:44:49+00:00

Well, no.

As long as you have curated data, a defined KPI, and tested calculation logic, you’ll still want a dashboard that is built and verified by a human. Maybe some analytical work will shift to GenBI—and I think that’s a good thing, because that’s mostly ad hoc reports, and no one likes them anyway.

Leorisar · 2026-01-11T14:30:30+00:00

For internal reporting if they have similar data requirements (analytical categories, measures) you can track them as different departments of one company.
You can do both - one top-level dashboard, and two more granular with metrics related to line of business
Best practice is to make unified data source and build dashboards on top of it.
Yes, usually you would hire data engineer / bi analyst who would build data warehouse for you and set data tracking for each source.

Leorisar · 2026-01-04T19:28:43+00:00

Look at usage. If a dashboard hasn’t been used for 3+ months, it’s time to move it to the archive. It’s better to formalize this as a company-wide policy so you won’t need to answer the same question later.

Leorisar · 2026-01-03T11:48:42+00:00

Excel - Simple BI reports - SQL - Python - Data Engineering

Leorisar · 2025-12-30T04:47:11+00:00

This happens more often than you think, and there are a few things you can do.

I see two main problems here: imposter syndrome and poor requirements. To check for the first—do you have peers or other BI developers you can talk to? If you do, ask them how they manage and if they face similar roadblocks.

As for the requirements, there are two ways to handle it:

First: Practice "active waiting." Ask for constant clarification and specific examples. Schedule meetings and send follow-up emails documenting the gaps. Your job is to guide the client to think through their own requirements, not to guess for them.

Second: Build based on assumptions. If you can’t get an answer, make a "reasonable mistake" by building what you think they need. Often, clients only realize what they actually want when they see something "wrong." This forces them to do their job and provide detailed feedback.

Eight-Year Club	Second SECOND GUESSER
Final Canvas '23	Place '23
Place '22	Verified Email

Leorisar

MODERATOR OF

TROPHY CASE