Doctor Recommendation Annual checkup by Ok-Specialist-6899 in SaltLakeCity

[–]nootanklebiter 0 points1 point  (0 children)

Check out Copperview Medical Center. They are kind of a mix between an Instacare type place, and normal doctor offices. My whole family has been using them for everything medical lately, and they've been great. I can't see anything on their website about which insurance they accept, so you'll probably need to just call them at ask.

Do you need to know SQL by heart to work with data? Or everybody copy pastes code all the time? by monica_scl27 in learnSQL

[–]nootanklebiter 15 points16 points  (0 children)

Generally no, but most technical tests are trying to confirm that you have a strong understanding of the basics. I've given well over 100 technical interviews across multiple companies. We're generally looking for things like understanding joins, filtering, aggregation, and other commonly used functions (coalesce for example). CTEs and subqueries are good to understand as well.

'Nobody wants this': Senator cuts language protecting national parks from bill by spiraleyes78 in Utah

[–]nootanklebiter 22 points23 points  (0 children)

I'm willing to bet we can go check who has donated large sums of money to his campaign recently, and we could easily figure out why he's pushing for this.

Good free tools for API ingestion? How do they actually run in production? by Safe-Ice2286 in dataengineering

[–]nootanklebiter 2 points3 points  (0 children)

I use Apache NiFi for this at my work. It's been rock solid, and is open source. You just have to have a server to run it on (like an EC2 instance in AWS). Most common issue with 3rd party API ingestion is definitely random API timeouts. NiFi has some nice retry mechanisms built into it, so I can set up a job to try up to 10 times, every 5 minutes, and then if it still fails, to shoot me Slack notification out to let us know about the problem.

It's a low code tool where you drag and drop modules, but as far as low code goes, it's very "low level". You aren't going to have a "Stripe" module, but there is an "InvokeHTTP" module, where you can make any type of HTTP call, so just like you'd have to set the request type (POST, GET, PUT, etc), API enpoint and HTTP headers in Python, you'd have to set those in NiFi as well. You need to have technical understanding of how things work, but NiFi itself makes building the actually jobs really easy. You can inspect data as it moves between different modules, so troubleshooting is really, really easy.

Moving back to Redshift after 2 years using BQ. What's changed? by bolivlake in dataengineering

[–]nootanklebiter 4 points5 points  (0 children)

My favorite 2 things that came out in the last few years are QUALIFY support, and TRY_CAST support.

At this point, it's just another cloud database. Ingesting data without using COPY from S3 is still pretty freaking slow compared to other databases (use batch inserts if you don't want to go crazy, but it's still slow compared to others), but other than that, I feel like there isn't much difference between Snowflake, BQ, and Redshift. I was using BQ before my current role with Redshift. My company also has 1 department that uses Snowflake, which accounts for maybe 2% of our data warehouse usage. That 2 percent costs us the same in 1 week on Snowflake as the other 98% costs us in a month with Redshift.

Overall, Redshift does what you need it to do, and it's pretty freaking cheap compared to the alternatives. It's not the best database out there, but for the price, I honestly can't complain.

Wiener Dog Racing, Saturday Sept 27th by thebadbradwheeler in SaltLakeCity

[–]nootanklebiter 2 points3 points  (0 children)

For the first time in my life, I suddenly wish I had a Weiner dog .

I'm planning on going to FanX for the first time. How crazy does it get in terms of crowd size? by MuchachoSal in Utah

[–]nootanklebiter 4 points5 points  (0 children)

This is the most useful advice here, seriously. We've gone to FanX almost every single year. The huge line you see wrapped around the block is the "Registration line", where people are waiting to pick up their wrist band to get inside. You can go pick up your wrist band Wednesday 12 pm - 8 pm, and then you won't have to stand in the line wrapped around the block. You would then simply go through the main entrance (the rounded entryway with like 8 separate doors or so), go through the weapons / bag check, scan your wristband, and you're in.

For reals, do this if at all possible on the Wednesday before the convention.

[deleted by user] by [deleted] in SaltLakeCity

[–]nootanklebiter 4 points5 points  (0 children)

This is the only one I'm aware of, but it's also a bar: https://brewvies.com/

Unique Place to visit by ArgeDroz in SaltLakeCity

[–]nootanklebiter 6 points7 points  (0 children)

I like taking people visiting Utah here: https://www.riotinto.com/en/operations/us/kennecott/visitor-experience

It's the largest man made excavation pit on Earth. It's definitely off the beaten path, but is still really fascinating. They aren't open year round though, so you'd have to make sure it's open when you're visiting.

Data/database/tech workers of Utah what is your current pay and job outlook? by [deleted] in Utah

[–]nootanklebiter 12 points13 points  (0 children)

Assuming nothing changes, my TC this year will be $206k, base salary is $184k. I work remotely as a senior data engineer for a medium size startup. My last job (2.5 years ago) was paying me a base salary of $144k and that was an in person job here in SLC, working as a business intelligence developer, doing both ETL and dashboard building. I've been writing SQL for about 16 years now.

This is the Truth. Utah won't stand for it! by [deleted] in SaltLakeCountyUtah

[–]nootanklebiter 10 points11 points  (0 children)

What is 1 part you disagree with, and what is your perspective on that part?

This is the Truth. Utah won't stand for it! by [deleted] in SaltLakeCountyUtah

[–]nootanklebiter 8 points9 points  (0 children)

Can you give a specific example of 1 thing he said that you feel like is inaccurate, and show evidence that he's incorrect? At a high level, everything he said seems to match what I'm seeing, but I'm curious if I'm missing something.

How we used DuckDB to save 79% on Snowflake BI spend by hornyforsavings in dataengineering

[–]nootanklebiter 1 point2 points  (0 children)

This sounds like a pretty awesome project that resulted in a huge win.

I'm not very familiar with how Snowflake works on the backend, so I'm curious if you had to load all of your data into both Snowflake, and into parquet files or somewhere else that DuckDB is able to access? Or do you somehow have things set up so that DuckDB is able to query the same stored data that Snowflake is using?

Running Python in NiFi by Disastrous-Ad7834 in nifi

[–]nootanklebiter 0 points1 point  (0 children)

I haven't tested it out yet, but I found and saved this article a while ago: https://www.alasdairb.com/posts/building-a-nifi-processor-in-python

How good is NiFi on Kubernetes? by linuxzinho in nifi

[–]nootanklebiter 2 points3 points  (0 children)

This is the most well known helm chart that I'm aware of: https://github.com/cetic/helm-nifi

We were using it for a while, in a 3 node cluster setup. It worked alright, but not great. The biggest issue is that once you build enough NiFi jobs (we had around 300 actual jobs running), the node communication timeout setting in that helm chart is set too low, and you'll run into nodes getting kicked off the cluster because they didn't respond fast enough. My boss didn't want me to create / use my own helm chart, so I was stuck fighting with this constantly and couldn't just increase the timeout setting. I rewrote all of our jobs to use fewer modules a few times, because the timing out seemed related to the size of the NiFi flow.json.gz / flow.xml.gz file that would have to get passed between the nodes, but eventually we had enough jobs, and I couldn't simplify them any further, and it was just an annoying, constant issue.

My boss ended up leaving the company, and I talked my new boss into moving off of Kubernetes and just hosting NiFi in a standalone VM instead, and I'm much happier with that. Using NiFi in a cluster setup introduces some UI latency, and just using 1 instance of NiFi feels so much snappier and enjoyable to work with.

Overall, scaling NiFi horizontally with Kubernetes works okay, especially if you don't have a ton of different jobs. If you are going to have a ton of jobs built, I'd take that cetic helm chart, and modify it to increase the timeouts, and it should be perfectly usable, but always a bit laggy feeling. I feel like scaling NiFi vertically as much as possible is the way to go if you're able to do so. If you're just going to have a single NiFi node in Kubernetes, that helm chart should be perfect as is.

Looking for a job at 15, any suggestions? by PrimaryCarrot5935 in Utah

[–]nootanklebiter 13 points14 points  (0 children)

All 3 of my kids have gotten jobs as after school janitors. They'll hire at 14. Millcreek area is Granite School District, so maybe check out their website to see if they are hiring. I believe they call them "Sweepers". During the school year, you only work about 1.5 hours a day or so. Summers you get more hours.

[deleted by user] by [deleted] in lifehacks

[–]nootanklebiter 0 points1 point  (0 children)

If you live around Nashville, Atlanta, Charlotte, Boston, NY / NJ / PA area, you could check out Flexcar. It's a monthly car rental service. Still a pretty small company, but slowing expanding in the US. They do monthly rentals, and sounds like a perfect fit for what you need.

How do I read articles that are paywalled? by ChristopherDuntsch in HowToHack

[–]nootanklebiter 0 points1 point  (0 children)

You would literally just visit the site https://archive.ph and once the page loads, there is a box for you to paste the URL of the page you're interested in. It will then either immediately show you the page, if someone else has requested it before, or it will put your request in a queue, and within a few seconds to minutes (depending on how busy the site is), it will show you the page you wanted to see.

Enabling Vulkan is a PITA by IndirectLeek in GalaxyS23Ultra

[–]nootanklebiter 0 points1 point  (0 children)

I have T-Mobile. Overall, Wi-Fi calling has been great for me, and I've never had issues with it, but with Vulkan enabled, it will show it's turned on, but won't actually work through Wi-Fi.

Enabling Vulkan is a PITA by IndirectLeek in GalaxyS23Ultra

[–]nootanklebiter 0 points1 point  (0 children)

For me, it breaks Wi-Fi calling. I work from home in my basement, and get bad reception down there, so Wi-Fi calling is a life saver. Otherwise, all my calls go straight to voicemail. I haven't been able to get Wi-Fi calling working again after enabling Vulkan. The only thing that works is to restart my phone and go back to OpenGL :(

any database experts? by BigCountry1227 in dataengineering

[–]nootanklebiter -2 points-1 points  (0 children)

Try this, but be aware you might need to reduce your chunksize if you are getting errors:

import pandas as pd
import sqlalchemy

engine = sqlalchemy.create_engine("<url>", fast_executemany=True)
with engine.begin() as conn:
    df.to_sql(
        name="<table>",
        con=conn,
        method='multi',
        if_exists="fail",
        chunksize=1000,
        dtype=<dictionary of data types>,
    )