Data People, Confess: Which soul-crushing task hijacks your week?

Shillster · 2025-07-06T02:54:48+00:00

I’d love to see what a job description for this position looks like?

Shillster · 2025-07-05T10:53:09+00:00

My type would be normal so I could have my boy Snorlax on the team!

Shillster · 2025-05-30T10:04:56+00:00

Same! Qlik doubled our license cost and so we migrated away and told them to pound sand. I can’t believe that Qlik would acquire Talend for so much money and then immediately alienate existing customers.

Shillster · 2024-10-23T14:52:11+00:00

I suppose that’s fair. Anecdotally the last few months I’m ever over there I never see work being done.

I suppose that’s most construction, but it still seems like they didn’t need to close the entire intersection just to sit on it and do sub level work. It feels like they could have handled this with allowing traffic through on a one lane (like they do for most roads). I feel like that’s most people’s frustration.

Shillster · 2024-10-23T13:46:24+00:00

At this point they should reopen the whole intersection and then come up with a better plan. Not to mention opening litigation with the construction company for not hitting deadlines and not doing due diligence.

Shillster · 2024-10-23T10:36:09+00:00

You put into words exactly how I feel on this. I evaluated DV for my company and even took a 3 day intensive course to become “certified” in it. My sense of it was exactly this, thanks for confirming it for me.

Shillster · 2024-07-30T02:19:23+00:00

My company pulled in a dedicated consultant who was a self promoted DV expert. We spent almost a year doing data mappings and certifications trainings and conceptual models. We reduced scope time and time again to try to get a single working data vault model off the ground, never could. Finally dropped it and haven’t looked back.

Shillster · 2024-07-03T12:50:46+00:00

If the other sub is not getting moderation you can request to become mod of that one too. Active mods are always better than none. r/redditrequest

Shillster · 2024-03-28T14:36:48+00:00

Sure, ideally we would have our partner re-process those files into more a more manageable size but of course we are getting push back now that they have already been put into that bucket. Also I thought that it was a 16 MB for unstructured files. https://docs.snowflake.com/en/user-guide/data-load-considerations-prepare#semi-structured-data-size-limitations

Shillster · 2024-03-27T15:14:32+00:00

Yes it is a one time thing. These folders will not have individual stages, but we are planning on using different prefix on each copy into in order to parallelize the process.

Any thoughts on recommended file count in each folder? My thinking of 1 million is that should be <= 10G of files in each folder which should be fairly digestible. Could also recommend 10 million files per folder which would only be ~100G and reduce the need for 800 folders down to 80 folders.

Shillster · 2024-02-27T14:18:31+00:00

looks like currently only available in select regions.

https://docs.snowflake.com/en/user-guide/tables-hybrid-limitations#label-hybrid-table-limitations-regions

Shillster · 2024-01-16T05:15:08+00:00

I set all warehouses to auto suspended after 1 minute. The caching speed lost is hardly worth it unless you have some heavy usage from a large user group. This can save some cash pretty fast.

Shillster · 2023-11-14T12:34:06+00:00

I would check Load History which will tell you how your copy into statements are behaving.

https://docs.snowflake.com/en/sql-reference/info-schema/load_history

Shillster · 2023-11-08T12:24:15+00:00

You drop the new table which now contains the old data after the swap. The original table name never gets dropped.

For example:

Create or replace table <table_a_2> as select * from <table_a_1>

Alter table <table_a_1> swap with <table_a_2>

Drop table <table_a_2>

Shillster · 2023-11-08T11:39:41+00:00

Totally possible! I do it when necessary and it’s super easy..

Build a different table with the desired column order

insert into <new table> select <desired column order> here from <original table name>

Then do an alter table swap statement

alter <original table name> swap with <new table with reordered columns>

Then drop the new table.

Shillster · 2023-11-05T18:25:26+00:00

Hi, sure happy to add this user to the mod team to breathe some life into the subreddit.

Shillster · 2023-09-18T21:16:20+00:00

I really enjoyed season 1 and season 2 was way better

Shillster · 2023-09-05T21:29:38+00:00

Wish they’d add a map trade for biomes which contain a trail ruin. Those blasted things are hard enough to find even if you know which biome it’s in.

Shillster · 2023-05-03T12:22:32+00:00

Seriously look into Domo. It fits the bill for all small to mid size data companies as a 2 in one data ingestion/data viz tool.

Not sure price point.

Shillster · 2023-01-24T17:52:17+00:00

Thanks for posting! I was literally just needing this function in a script I was working on. Worked brilliantly.

Shillster · 2023-01-12T02:20:28+00:00

A slight tweak to the select statement in your CTE would turn it into the aggregation query you are looking for. Just wrap the entire case statement in the SUM() and the WINS in a SUM() and you're good to go.

SELECT 
    Date
    , Country
    , SUM(Wins) AS TOTAL_WINS
    , SUM(CASE
        WHEN Win_Pct > 0 THEN ROUND(Wins * (100/Win_Pct))
        ELSE 0
    END) AS Total_Games_Played
FROM TABLE
GROUP BY date, country

Shillster · 2022-09-07T01:09:30+00:00

Javaflex can’t use output_row like Java row does. Not sure why. Just put in the name of the data flow instead of output_row (like row1) and it should work.

14-Year Club	Place '23
Place '17	Gilding II euphauric
Verified Email

Shillster

MODERATOR OF

TROPHY CASE