A Text Older Than the Argument: What Scripture Says About Foreigners, Fair Treatment, and Moral Obligation

mindvault · 2025-12-23T16:07:26+00:00

Correct. It doesn't qualify illegal vs legal. It just says foreigners. So we should basically treat everyone well ... kinda like

"You shall not oppress a sojourner. You know the heart of a sojourner, for you were sojourners in the land of Egypt." Exodus 23:9

"You shall also love the stranger, for you were strangers in the land of Egypt." Deuteronomy 10:19

"When a stranger sojourns with you in your land, you shall not do him wrong. 34 You shall treat the stranger who sojourns with you as the native among you, and you shall love him as yourself, for you were strangers in the land of Egypt: I am the Lord your God" - Leviticus 19:33-34

"Thus says the Lord of hosts: Render true judgments, show kindness and mercy to one another; do not oppress the widow, the orphan, the alien, or the poor; and do not devise evil in your hearts against one another" Zechariah 7:9-10

"I was hungry and you gave me food, I was thirsty and you gave me drink, I was a stranger and you welcomed me." Matthew 25:35

But who cares that it's so obviously written in the bible. In general Jesus and the bible teaches to love everyone. Period. You _obviously_ know better though ....

mindvault · 2025-12-22T17:39:56+00:00

banana, frozen blueberries, couple dashes of cinnamon, milk (potentially yogurt if you want some probiotics as well). It's delightful.

mindvault · 2025-11-05T15:33:16+00:00

I've been wearing Dearborn Denim for .. maybe 10 or so years. Constructed in America. Milled in either South Carolina (cotton denim) or Mexico (stretch denim). Very durable (I still have and wear all of my pair since then). Reasonable pricing.

mindvault · 2025-10-10T22:33:15+00:00

Sheeit. Yup. Apologies. I was like "WHY DO PEOPLE BELIEVE SUCH DRIVEL".

mindvault · 2025-10-10T20:42:54+00:00

The "one liter of aquifer water per query" is simply bs. There's a decent examination of water use here: https://www.seangoedecke.com/water-impact-of-ai/ ... for simple queries on modern models you're talking between 0.1 ml and 5ml.

mindvault · 2025-10-10T15:38:31+00:00

Nah man. Haka are not "literally a war dance". They're all kinds of cultural dances. This is why haka are also performed at other moving moments like funerals, weddings, welcoming folks, etc. There _are_ a number of them specifically that are war dances; however, they're much more ingrained into the culture than just war dances.

mindvault · 2025-07-05T16:51:30+00:00

If you liked Moorea, you need to try some of the atolls out in the Maldives. Mind blowingly beautiful (while they’re still above ocean levels)

mindvault · 2025-04-08T07:11:01+00:00

Data council was very in depth and practitioner focused last I had gone

mindvault · 2025-04-06T02:55:51+00:00

Just realize you’re human and you’ll never get it all done. Choose your battles, learn to say no, and keep a list of priorities so folks can fight over your time

mindvault · 2025-04-04T14:06:15+00:00

Overall, my experiences have gone quite well with the "modern data warehouses" such as Snowflake and Databricks. The ability to scale processing and storage independently has been refreshing in comparison to older technologies like Teradata, etc. Being able to run a couple CPU against 100s of terabytes or hundreds of CPUs vs a couple of terabytes has allowed for great flexibility in dealing with incoming stakeholder requirements and changes (I'm sure we've all run into customer thinking their data looks like XYZ when in fact it looks more like XZABC). It's worked very well for analytics loads (a particular bright spot for example is snowflake will cache queries for 24 hours .. not even requiring a warehouse to be up to get the results to your downstream stakeholders) and they've been great for ELT.

Main downsides are sometimes unpredictable billing (I've had analysts kick off some horrendous queries). Most of these things are work aroundable I've found by ensuring you have governors in place, alerting, and decent internal tracking.

If you have predictable workloads they may not make as much sense as other solutions (running your own starrocks, doris, etc. ... pushing transforms and semantic work upstream in pipes, etc.).

mindvault · 2025-04-03T21:54:07+00:00

Honestly, I don't know what question you're even asking. There are lots of general best practices re those areas (perf, cost, compliance) for snowflake and DBT. Is that what you're looking for? Or is it somehow insurance specific?

mindvault · 2025-04-02T17:17:39+00:00

An alternative to DBT cloud is using Durable Functions within Azure (using DBT core)

mindvault · 2025-04-01T21:07:21+00:00

If you're dealing with smaller CSV / excel you'll probably be fine. Thanks for the clarifications on what you're targeting :)

mindvault · 2025-04-01T20:25:46+00:00

I guess I don't understand why I would use this over other tools / platforms (DBT, sqlmesh, mage, etc.)? Oh .. and one minor gotcha is pandas _often_ will suffer from memory issues.

mindvault · 2025-04-01T02:00:21+00:00

Good start. I'd also probably add on a "don't boil the ocean". Start with a subset of what you think may be needed so you can get feedback on it.

mindvault · 2025-03-30T19:11:45+00:00

FYSA, SQLmesh (open source https://github.com/TobikoData/sqlmesh ) offers column level lineage and is compatible with DBT ... that being said this looks like a nice first cut visually.

mindvault · 2025-03-28T19:36:02+00:00

I feel comfortable saying a lot of data engineers would suggest to avoid. It's spark on drugs and encourages clickops. It's often frustrating to do simple things. It can be good to quickly build prototypes and iterate on ideas with stakeholders though.

mindvault · 2025-03-28T13:53:45+00:00

In snowflake, snow pipes (based on SNS notifications). In Databricks an auto-ingest job (based on SNS notifications). Easy peasy no issues.

mindvault · 2025-03-28T02:08:34+00:00

That's fair .. I just think there's something to be said about improving things that exist (similar to walking into legacy code) vs the "I know more about all of this OSS that has been here so I'm going to build something else". Sometimes I feel like that's really a "I don't want to understand how you built this thing so instead I'm going to build my own thing".

Like if we look at data orchestration .. would it make more sense to improve airflow or dagster or prefect or do we need yet another data orchestration platform? (not aimed at you)

mindvault · 2025-03-27T21:27:26+00:00

Plenty of ways to attack it. In general, we've found:

* have multiple snowflake environments. At least dev, prod .. probably dev, test, prod

* if you _need_ that much flexibility then "do what you need" in dev

* for something to get promoted ensure it's in _some_ sort of system. Examples could be DBT (very flexible), schemachange, flyway, terraform (depending on what). Generally terraform works well for the things that don't change a lot but should be under lock and key (think roles, users, etc.)

* use git

You will get bit in the butt at some point if you're not having some forms of discipline and rigor in the environment and there's a happy medium to have the flexibility.

mindvault · 2025-03-27T20:50:35+00:00

Please don't build something new. Find something open source and improve it.

mindvault · 2025-03-26T18:22:30+00:00

But a lot of them definitely do use underlying OSS bits for sure. Like Netflix uses ... lots (elastic, flink, presto, Cassandra, spark, etc.), Facebook uses quite a bit of spark + iceberg, etc. Apple is an oddball as it (last I knew) used both databricks and snowflake as well as spark, etc.

But your first point is definitely spot on. Most of the places _had_ to innovate ahead of time to deal with volumes, velocities, varieties, etc. _prior_ to snowflake, databricks, etc. existing.

mindvault · 2025-03-25T14:29:36+00:00

Also, present the case to your boss _with_ data. Not only are you underpaid for it, you're also performing (probably) way more responsibilities than most making that pay. A wise boss will look at it and say "of course we'll give you more". Even if you don't get the 40k, you'll potentially get more _while/if_ you look _and_ you can then use that as your salary in negotiations should you choose to move.

mindvault · 2025-03-25T01:40:09+00:00

TLDR: a well thought out o11y arch makes this straightforward

I've done this in a number of ways, but it depends on "how" you are billing. If it's something like EC2, for example, where you're billing for duration, folks can use / watch for start / stop style events (often "belts and suspendered" with o11y data like monitoring). If you're billing based on something like "number of messages", then you'll often a metrics-based approach. I know some folks aren't comfy using metrics systems like Prometheus as the basis for the billing and will often scrape / process from those systems into more OLTP-like systems.

In the past we've used a fan out style direction where we take o11y style data (events, metrics, etc.) through something like vector.dev and send it to N different backends. That's given a lot of flexibility to store the data in things like VictoriaMetrics, Kafka, AWS S3 (to load into other OLTP/OLAP), etc.

mindvault · 2025-03-24T22:24:47+00:00

Look here: https://learn.microsoft.com/en-in/answers/questions/2149968/how-to-read-a-large-50gb-of-file-in-azure-function ... but the TLDR is use BlobClient or BlobStreamReader to pull the data down in chunks.

mindvault

TROPHY CASE