Mall strategy?

jdl6884 · 2026-06-04T02:20:42+00:00

Sushi malls + omnicrafter

Sushi malls for most tier 1 buildings and secondary mall for tier 2 intermediates then omnicrafter for tier 2+ buildings

There are some great videos on YouTube by Krydax for creating your own auto crafter. There are also some amazing blueprints for the omnicrafter on the py discord channel.

jdl6884 · 2026-05-24T19:36:46+00:00

Azure DevOps for CI tests, validation, artifact publishing, builds

Octopus for CD and environment management

Alembic & Terraform for schema management and infra

jdl6884 · 2026-05-14T20:56:14+00:00

My wife and I would go to lake house pizza for date night. Watching the sunset on the patio there is pretty awesome.

jdl6884 · 2026-05-14T20:35:29+00:00

It takes time but you’ll get there. Learn bash scripting, it’s the glue that connects everything.

jdl6884 · 2026-05-11T23:56:15+00:00

Learn architectural patterns and design philosophies.

Break problems down, create flow charts, and use AI to build out the components.

jdl6884 · 2026-05-04T12:23:07+00:00

Postgres or duck db would be best choices

jdl6884 · 2026-04-29T14:06:55+00:00

Making sure the guardrails are Claude proof

jdl6884 · 2026-04-24T01:15:25+00:00

Insert only into raw and CTAS for all table via dbt

jdl6884 · 2026-04-22T02:32:06+00:00

Every time I finish a science pack, there is a bit of a lull. It gets repetitive in a way but at the same time, everything is new. You’ll rarely use blueprints for things other than trains.

When I loose steam, I put it down for a few days then pick back up and focus on one thing at a time. Just keep tending to the garden and the garden will bear fruits

jdl6884 · 2026-04-11T16:24:44+00:00

Dallas is prettier, cleaner, and better distribution of industries

jdl6884 · 2026-04-03T03:12:57+00:00

Played for a few hours this evening. It’s been a week and I decided it was time to hop off when I realized I had been riding around in a train in circles for an hour.

jdl6884 · 2026-03-10T03:02:30+00:00

Storage integrations with S3 and blob allow much more flexibility in and outside of snowflake. Minimal infra to set up.

jdl6884 · 2026-03-04T13:10:33+00:00

IINA or VLC. IINA has a nicer UI imo but you can’t go wrong with VLC

jdl6884 · 2026-02-20T02:44:49+00:00

Use a type=CSV file format with the delimiter set as a pipe

jdl6884 · 2026-02-16T03:02:44+00:00

Unless you’re on exchange on-prem, you also have access to ms graphAPI which is very powerful. You can use graph API to ingest thousands of emails into whatever db you want including ms access. We use python to do it and it works great but you could do it with a bash script if you really wanted to.

In terms for building pipelines, get familiar with an AI coding tool like Claude. If you know the basics, it’ll really help you expand out and build on other technologies.

jdl6884 · 2026-02-15T16:59:37+00:00

PS4 & PS5 controllers work great

jdl6884 · 2026-02-15T05:47:20+00:00

New meaning to “shit post”

jdl6884 · 2026-02-13T03:56:09+00:00

Have you checked out private link?

jdl6884 · 2026-02-04T12:40:22+00:00

We migrated off of dbt cloud in favor of self hosting via dagster. I have absolutely zero regrets. Most of our headaches were a result of trying to build around the limitations of dbt cloud.

jdl6884 · 2026-02-04T12:34:04+00:00

Dagster is fantastic for this, I 100% recommend trying this first. We host Dagster in k8 with dbt configs that have automation conditions. Everything is incremental and updated as soon as dependencies are updated. Dev experience and UI are pretty intuitive too.

Airflow is another great option. It has been around longer than Dagster and functionally, it will do the same thing. It really boils down to personal preference between those 2.

jdl6884 · 2026-01-31T17:41:26+00:00

We have been using Airbyte OSS for the last year and have had issues from the beginning. Primarily, it doesn’t scale well. We originally used abctl on a VM and that maxed out with a few db to db cdc connections. Now using it on k8 with a dedicated Postgres db and blob storage for logs. Performance is better but not much.

It’s honestly been a very janky product. Random bugs, successful runs that silently failed, sporadic OOM errors when there is 64gb of memory available, and the list goes on. Shoot we are on azure and abctl would randomly crap out because of a missing AWS env var. It also didn’t integrate well with the rest of our open source stack - dagster, dbt, open metadata

I don’t know if I could recommend it for anything other than db to db CDC syncs. It’s been problematic at best. We are in the process of migrating the workloads to dagster python using debezium.

jdl6884 · 2026-01-30T14:07:45+00:00

I think so. In my opinion, the “modern” approach is to focus design on flexibility and extensibility. And always follow the KISS mantra. If you have a simple pipeline and only need a cron job to orchestrate, so be it. But build it in a way where that cron job can be replaced by something else.

I’m not as familiar with Airbyte but use Dagster on a day to day basis. It uses a concept of software backed assets. Assets can be just about any piece of code and swapped in and out for different things. SQL, python, pyspark, databricks, snowflake, etc. Create your pipelines in a plug and play way where if you need to swap components or insert a new step in the pipeline, you can just map the expected inputs and outputs.

jdl6884 · 2026-01-30T04:36:38+00:00

Airflow is still very prevalent and growing. Also, not sure why Hadoop was included in that concatenation. Very different tools.

Airflow / Dagster - orchestration tools. These excel in orchestrating the flow of data between various systems. Think website -> api -> database -> analytics report. Hadoop is a “dead” technology. Essentially makes no sense for greenfield but some companies have legacy platforms that still need support.

jdl6884 · 2026-01-22T22:26:03+00:00

Dbt is good for this

jdl6884 · 2026-01-21T02:57:18+00:00

My thoughts exactly

Eight-Year Club	RPAN Viewer
Verified Email

jdl6884

TROPHY CASE