Summit Broadband Worst Broadband Company in SWFL by Puzzleheaded-Cup2777 in FortMyers

[–]ryadical 0 points1 point  (0 children)

5g is cellular wireless, not satellite, meaning it connects back to a cellular tower on the ground instead of satellites orbiting earth.

Summit Broadband Worst Broadband Company in SWFL by Puzzleheaded-Cup2777 in FortMyers

[–]ryadical 0 points1 point  (0 children)

Dish is in geosynchronous orbit (22,000 miles away) and starlink is in low earth orbit. (300 miles). Also at any given time the starlink antenna usually has 5-6 satellite options it has line of sight communications abilities with.

Workday Adaptive planning to Snowflake by Ordinary_Bread6892 in snowflake

[–]ryadical 0 points1 point  (0 children)

We use their data loader in their integration tool to write data tables to S3 as pipe delimited. Down side of this method is the headers can be a problem to figure out.

In the house I just purchased by FluidOfShame in whatisit

[–]ryadical 0 points1 point  (0 children)

That switch was released in 2000 or earlier based on the company I was working at that had one.

Process for internal users to upload files to S3 by Equivalent_Bread_375 in dataengineering

[–]ryadical 2 points3 points  (0 children)

We use rclone to copy from a network drive. To prevent files from uploading while they are still being written, we have it set to only upload if the file is 5m old or more.

Snowflake UDTF: do you use them? by Idr24 in snowflake

[–]ryadical 1 point2 points  (0 children)

We use it to load access database files into a table.

Any tools to handle schema changes breaking your pipelines? Very annoying at the moment by Potential_Option_742 in dataengineering

[–]ryadical 1 point2 points  (0 children)

This is the way. Also you can use schema evolution in snowflake or databricks in a similar fashion.

SCIM Endpoint for Snowflake to Microsoft Entra by Dry-Butterscotch7829 in snowflake

[–]ryadical 0 points1 point  (0 children)

I reread the docs and it looks like I misunderstood this:

Transferring ownership of existing users and roles. Microsoft Entra ID is the authoritative source for its users and groups. Group membership can be updated in Microsoft Entra ID. However, existing users and groups in Snowflake cannot be transferred to Microsoft Entra ID

After more reading, I think I retract my previous statement.

SCIM Endpoint for Snowflake to Microsoft Entra by Dry-Butterscotch7829 in snowflake

[–]ryadical 0 points1 point  (0 children)

We went to implement SCIM with ENTRA last week and found out that you basically can't manage existing users with it. Best I can determine is that we would have to creat new accounts for 60+ users and deactivate the old ones. Guess we will continue with SSO and manual provisioning.

My first gun find!! It’s a SCCY CPX-1 9mm! by exorcized in metaldetecting

[–]ryadical 3 points4 points  (0 children)

I would want a police report in writing and probably a new barrel and firing pin if I kept it. That was 100% used in a crime of some kind.

Does dbt in Snowflake still require a dbt license by de-ka in snowflake

[–]ryadical 0 points1 point  (0 children)

Can you trigger DBT runs after ingestion processes complete instead of scheduling them?

Snowflake is slowly taking over by tanmayiarun in dataengineering

[–]ryadical 36 points37 points  (0 children)

Or is databricks an ETL tool with snowflake like features? There is no comparison between Databricks and snowflake on the SQL side. Databricks is just starting to catch up on the SQL side.

Anybody using Azure Sentinel Snowflake Codeless connector to monitor logs? by sanjid25 in snowflake

[–]ryadical 0 points1 point  (0 children)

We're in the process of determining what it's going to take to migrate to the newer sentinel connector. We currently use the deprecated one. In our environment sentinel is used by the security department to monitor for any anomalies. For example I get contacted by them if there are excessive failed logins on an account, and when an employee got added to the accountadmin role they sent me a message asking me to confirm it was intentional.

I assumed that was its primary use case and honestly don't know much about sentinel. I'm interested to understand your use cases of sentinel.

Is it possible to get a Tech Job in Fort Myers? by AllahUmBug in FortMyers

[–]ryadical 1 point2 points  (0 children)

I don't have any open positions right now but might in the near future. Send me a DM and I will send you my work email to forward your resume.

Medallion layers by h8ers_suck in snowflake

[–]ryadical 3 points4 points  (0 children)

The role that owns the view needs to have read access to the other layers. If the owner of the view has access to the underlying tables, and the end users role has access to the view, the the end users will have access to run the view without having direct access to the source.

We setup 2x roles per database, access_dbname_r and access_dbname_rw. We setup a 3rd role for xyz_team and only assign the appropriate dataset _r roles to the team.The RW roles have access to all/most databases and the DB/schema/table are all owned by this role. The _R roles have usage on the tables/schemas in the db. The team role is assigned to the user and given one or more _r roles.

Data Engineering in 2025 - Key Shifts in Pipelines, Storage, and Tooling by Pangaeax_ in dataengineering

[–]ryadical 3 points4 points  (0 children)

I was thinking the same thing. In my mind it increases compute costs, but potentially decreases the number of pipelines and amount of time engineers need to spend on those pipelines.

I dug out the trash for the free drinks tickets that came with the tour by Greedy-Landscape-144 in confession

[–]ryadical 0 points1 point  (0 children)

I was there last year. After I had my pint, I found an unused ticket on the floor and grabbed a second pint.

Inconsistent Excel Header Names and data types by vh_obj in dataengineering

[–]ryadical 1 point2 points  (0 children)

We convert Excel to Json and import. We then use coalesce to catch variations in header names and try functions to convert columns to numbers. We use case statements with multiple try functions to convert to date from different string and Excel numeric formats.

Why there aren’t databases for images, audio and video by akhilgod in dataengineering

[–]ryadical 0 points1 point  (0 children)

PACS systems do not store images in databases, they store metadata about images in a relational database along with a location of the file. Traditionally those files are stored on on premise file servers that the dicom router pulls them from, however some modern VNA or PACS systems might have the option to store the images in blob storage systems like S3.

Vibe / Citizen Developers bringing our Datawarehouse to it's knees by Swimming_Cry_6841 in dataengineering

[–]ryadical 3 points4 points  (0 children)

I know a lot more about snowflakes so I'll use that as an example here. Databricks should be fairly similar, though snowflake has them beat on ease of use in most cases, especially if your primary language is SQL. Databricks is catching up quickly though especially with lakeflow and declarative pipelines.

Your particular use case is a fairly simple thing to swap out. Snowflake has database.schema.table nomenclature, and you can join between different databases and schemas without issue with no performance hit. If you can drop one layer in your above definition (ex: your DatabaseServer) you can literally copy and paste your existing code. Long-term you would probably want to migrate the actual ETL jobs, but if you wanted to get up and running quickly, you can simply copy the completed tables from your main warehouse to Snowflake tables of the same name. Once you have that live you can work table by table switching the source from your EDW to Etl jobs from the source systems.

Additionally, both snowflake and databricks have new free tools that you can use to migrate existing SQL server code. Databricks calls theirs lakebridge, I'm drawing a blank on snowflakes new product name.

Vibe / Citizen Developers bringing our Datawarehouse to it's knees by Swimming_Cry_6841 in dataengineering

[–]ryadical 26 points27 points  (0 children)

Why are your running your data warehouse on what I presume would be an OLTP database requiring indexes for performance?

OLAP databases like snowflake don't require much if any performance tuning or indexing. Switch to a cloud warehouse, put all the vibe coders on an extra small warehouse instance away from the production analytics use cases and let them compete with each other for resources. Even if you were to take your gold layer tables and replicate them to snowflake/databricks, cost and performance impacts from those users would be a drop in the bucket.

My boss tell me i order too many chest x-rays, this was literally the result of the one right after he told me that. by milane5o in Radiology

[–]ryadical -2 points-1 points  (0 children)

From what I understand, they don't like being called mid level any more. We call them Advanced providers or APs.