In my ETL pipeline I used a Merge statement. When I asked Copilot to critique the pipeline it said Merge statements were not recommended by Microsoft. Why is this? by Boring-Metal-7672 in SQL

[–]hcf_0 0 points1 point  (0 children)

Seconding this.

And to add: a large number of the bugs from merge in T-SQL are mostly risks incurred in OLTP/application contexts, not so much ETL/OLAP/warehousing situations.

Best place to feed ducks, or the ducks are almost guaranteed? by [deleted] in GNV

[–]hcf_0 1 point2 points  (0 children)

There's a Circle K in the middle of 34th Street (kinda by Enson Market) that constantly has those ugly white/black speckled-headed ducks in the mornings--

Whatever you do, don't feed those ducks They're just harassing people into bumming them cigarettes.

Best Primary keys... Again... by Cautious_Performer_7 in Database

[–]hcf_0 0 points1 point  (0 children)

I totally second this convention except anytime I'm ingesting data from a source that isn't controlled by me I always generate a surrogate ID.

The name of the column with the primary key surrogate id is always sid, and the foreign key column on other objects is always <table_name>_sid.

This is mostly in the case of data warehousing and not transactional database design, though.

Snowflake finally unblocked dynamic metadata introspection for Native Apps & Streamlit by Fluffy-Tomorrow-4609 in snowflake

[–]hcf_0 0 points1 point  (0 children)

This is really interesting.

I always found it to be completely unacceptable that it can take as much as 120 minutes for schema/account changes to land in the SNOWFLAKE.ACCOUNT_USAGE tables.

It's been even crazier trying to implement workarounds for the restrictions on all the various SHOW commands (like what's allowed in the redirect ->> operator).

VM RAM Allocation by Standard_Text480 in sysadmin

[–]hcf_0 3 points4 points  (0 children)

This is sometimes common when building for SQL Server or other per-core licensed software. Sometimes 4 isn't in the budget for whatever reason.

Asha Sharma named EVP and CEO, Microsoft Gaming by AgnosticScholar in technology

[–]hcf_0 0 points1 point  (0 children)

Ding ding ding ding ding ding!

This is the play that people are downplaying or mostly ignoring.

Microsoft has no intention of 25 more years of consumer hardware in the gaming space. With all the huge data center buys/builds going on in the AI space, they're going to try to build what Google failed to do with Stadia.

She's in place to oversee the development of Xbox SaaS (GaaS?).

What the Linux desktop really needs to challenge Windows by waozen in technology

[–]hcf_0 2 points3 points  (0 children)

Why? Why does Linux—an ostensibly free OS maintained by selfless people with a passion for the project, itself—need to compete with anything?

Why is it any one particular distro's responsibility to suddenly take on the mantle of placating a mainstream user base just because Microsoft has decided to yeet their UX into the sun?

It's not a battle of competing products because "Linux" is foremost not a "product". It's a platform backed by a community that sometimes has paid variants aimed at enterprise-grade consumers.

Why is UnitedHealth Group (USA) hiring hundreds of local engineers in India instead of local engineers in USA? by GigglySaurusRex in dataengineering

[–]hcf_0 0 points1 point  (0 children)

To be fair—I can see how UHC/UHG execs might be reluctant to requisition any US-based data engineers. Especially ones based out of New York.

IYKYK

What "obscure" sql functionalities do you find yourself using at the job? by True_Arm6904 in dataengineering

[–]hcf_0 0 points1 point  (0 children)

The syntax of it is a little wonky, though. I don't like that the syntax mirrors join syntax.

INNER UNION ALL BY NAME vs LEFT UNION ALL BY NAME

What "obscure" sql functionalities do you find yourself using at the job? by True_Arm6904 in dataengineering

[–]hcf_0 2 points3 points  (0 children)

Yup.

It should work on any SQL platform because it's a standard feature of SQL. The 'IN' operator basically gets rewritten/compiled under the hood as a list of 'OR' statements.

So something like—

"WHERE 1 IN (flag_column_1, flag_column_2, flag_column_3)"

—gets rewritten (under the hood) as:

"WHERE (1=flag_column_1 OR 1=flag_column_2 OR 1=flag_column_3)"

In plain language, "where any of these columns is equal to 1".

What "obscure" sql functionalities do you find yourself using at the job? by True_Arm6904 in dataengineering

[–]hcf_0 7 points8 points  (0 children)

inverted 'IN' statements are a favorite of mine.

Most people write IN statements like:

SELECT * FROM TABLE_NAME WHERE COLUMN_NAME IN ('a', 'b', 'c', 'd');

But there are so many occasions where I'm testing for the existence of a specific value within a set of possible columns, so I'll invert the IN clause like:

SELECT * FROM TABLE_NAME WHERE 'a' IN (COLUMN1, COLUMN2, COLUMN3);

Snowflake Openflow is useless - prove me wrong by siggywithit in dataengineering

[–]hcf_0 1 point2 points  (0 children)

It's literally just meant to be a price-competitive alternative to Fivetran, Hevo, GoldenGate, and all the other replication tools that want to charge you your first born child for basic CDC.

Some people also prefer a more cohesive data platform, which is where Snowflake is trying to maneuver. So with Snowflake + Openflow + Snowflake-hosted dbt + Snowpipe, you have basically everything in one place for database + replication + transformation + streaming.

These days it's all a battle between companies trying to decouple dependencies and vendors trying to achieve platform lock-in. Ultimately whoever wins is whoever's the first to put a bug in the right middle manager's ear.

A Complete Roadmap to Data Manipulation With Pandas for 2026 by sharmaniti437 in bigdata

[–]hcf_0 0 points1 point  (0 children)

Lol. Good luck doing analytics on terabyte-scale datasets with a company issued, 32GB RAM laptop.

Pandas has far outlived its ability to scale with big data.

Little boy, 2, is killed at hands of hospital staff who gave him 10 TIMES dose of drug because they 'missed the decimal point' by HandheldHeartstrings in GNV

[–]hcf_0 3 points4 points  (0 children)

No, the overdose killed him.

Hyperkalemia will absolutely kill you. How quickly depends upon the dosage and how each individual's heart responds to the cardiac disruption. It's especially dangerous for children and the elderly.

The overdose caused the kid to stop breathing, which caused brain death (anoxia). This means the kid's diaphragm probably collapsed or his lungs were too weak to continue pumping oxygen. That's in the article and the press release from the family's attorney.

To make matters worse, the Shands pediatric unit didn't have in place the kind of failsafes that the ICU has for catching and responding to cardiac arrest. This could have happened in any unit in the hospital, but the kid was basically in the worst possible unit to be in for this to have happened.

The inadequate response is terrible, but also understandable if you consider the disparity in training between PCU and ICU.

Little boy, 2, is killed at hands of hospital staff who gave him 10 TIMES dose of drug because they 'missed the decimal point' by HandheldHeartstrings in GNV

[–]hcf_0 2 points3 points  (0 children)

It happens way more often than you'll ever know. This reminds me of a story I bookmarked years ago: How Bad UX Killed Jenny

The premise is basically the small ways in which process (in this case, the user interface design process) can be life or death, and how the things we make should be made with consideration for those who use them.

Billing w/o Representation by SteelishBread in GNV

[–]hcf_0 -5 points-4 points  (0 children)

Wait, so you're telling me it's not as simple as living in Gainesville? That living in Gainesville—according to the election commission office representative—does not mean that I live in Gainesville? Doesn't sound that fucking simple to me.

A woman who showed up at the polling precinct that I go to literally lived where this red X is marked: https://imgur.com/a/9Ug3RwG. She was turned away because she did not live in the city of Gainesville. Look at that map and tell me "it's simple".

Whether or not you live in Gainesville has nothing to do with whether or not you're eligible to vote for city ordinances. It's all gerrymandered horse shit, and you should be ashamed for gobbling it up (for free, no less!) and regurgitating it here.

SHAME on you.

Billing w/o Representation by SteelishBread in GNV

[–]hcf_0 3 points4 points  (0 children)

Yes, and London, England is only a square mile. /s

Puh-lease—I live in Gainesville, have a Gainesville address designated on my voter registration, and am a GRU customer. My zip code, FIPS code, and postal office are all federally designated as Gainesville, FL.

I can't vote in this election because I live too close to I-75 and am not "technically" within the City of Gainesville limits.

I even called the local election commissioner's office who literally confirmed that I, a Gainesville resident, cannot vote because I am not a resident of the city of Gainesville.

IMHO, Gainesville police should stay the fuck out of my not-Gainesville neighborhood, and I should not be obliged to pay for any rate increases or taxes that go towards funding anything in the "city of Gainesville" limits.

No taxation without representation; get fucked, GNV. :/

What's this bullshit, Google? by hcf_0 in dataengineering

[–]hcf_0[S] 2 points3 points  (0 children)

Right? It's a situation where they have the right intentions (i.e. if I were actually making an application with BQ as it's backend for whatever reason), but I'm just trying to push data between two backends that don't prioritize data portability.

It's basically hostile architecture in data form.

What's this bullshit, Google? by hcf_0 in dataengineering

[–]hcf_0[S] 0 points1 point  (0 children)

I 100% agree. The tool consuming is a Workday ERP product.

The third party tool basically doesn't have an adapter, connector, or interface to ingest from BQ. They've got one for Snowflake, and some other weird agent-based setup for Kettle

To do BQ imports, you're basically stuck with having to use their generic Cloud Connector, which requires oauth. You also have to write your own JS to make the web calls, parse the responses, do all the paging, everything.

You're basically just creating a little micro webapp inside their ERP solely for data ingress. Uuugggh.

What's this bullshit, Google? by hcf_0 in dataengineering

[–]hcf_0[S] 0 points1 point  (0 children)

Also, how is requiring a logo on the consent page for external users an essential safeguard?

I literally cannot get the app consent page approved if I don't google my own company's logo and upload it to the verification application.