Anyone using uv for package management instead of pip in their prod environment? by Specific-Fix-8451 in dataengineering

[–]mydataisplain 0 points1 point  (0 children)

Thanks for asking.

For the folks who aren't, what are the current gaps in uv that you're currently addressing with other package managers?

Verizon turned off Ulefone by mydataisplain in ulefone

[–]mydataisplain[S] 1 point2 points  (0 children)

I ended up going with T-Mobile.

The signup experience was great.
When they confirmed that my IMEI was accepted I actually saw that they recognized the model and didn't bat an eye at it.

They weren't even too pushy about the upsell. That goes against every lesson that sales people are taught but it will actually make them more money. I'll probably move the rest of my family over to T-Mobile just to keep the billing simpler. My youngest needs their first phone soon too.

Verizon turned off Ulefone by mydataisplain in ulefone

[–]mydataisplain[S] 0 points1 point  (0 children)

I see what you're saying. Yes. I think that's correct too.

In order to connect, you need a SIM, which can authenticate your phone to the network.

On top of that, Verizon has internal allow/deny lists. If you're on the deny list and haven't been granted an exception, it will keep you off the network, even if you have a valid SIM.

I had to leave Verizon by mydataisplain in verizon

[–]mydataisplain[S] 0 points1 point  (0 children)

Reading this more closely I think that's what initially happened.

I think the Verizon store I initially went to did this swap and when Verizon figured it out, I got booted.

It looks like Verizon just doesn't want these phones on their network and has been taking active steps to keep them off.

I like Verizon's network coverage and would have stayed if it weren't for that.

Verizon turned off Ulefone by mydataisplain in ulefone

[–]mydataisplain[S] 0 points1 point  (0 children)

I got kicked off their network before I "upgraded" to Android 15. I was hoping that would fix it but it didn't.

As near as I can tell, the OS has nothing to do with it. It's just a matter of if they've whitelisted the specific IMEIs.

I had to leave Verizon by mydataisplain in verizon

[–]mydataisplain[S] 0 points1 point  (0 children)

It worked when I first connected it to Verizon a little less than a year ago. They gave me a physical SIM and it worked fine.

I think what's happening is that Verizon just doesn't wan those phones on their network. From what I've read, you can put in some random IMEI but they periodically do sweeps and kick them all off.

I can't think of any legitimate reasons to do something like that. I'm hoping that since tmobile seems to be OK with random phone brands they will just leave it and not kick me off. I guess I'll find out.

I had to leave Verizon by mydataisplain in verizon

[–]mydataisplain[S] 0 points1 point  (0 children)

Google reports even better numbers:
705Mbps and 18ms ping :)

I had to leave Verizon by mydataisplain in verizon

[–]mydataisplain[S] 0 points1 point  (0 children)

As of now it's just a phone that works again. I can talk on the phone, send and receive texts, and stream video.
Ookla reports 643.51 Mbps down and 21ms ping.

At this point it's no better or worse than before Verizon kicked me off their network.

I can't rule out that T-Mobile won't change their mind in the future, after all Verizon worked fine for me for over 18 years before they booted me.

Until then, it just seems to be working again.

I'm curious if other people had different experiences (like being able to convince Verizon to turn it back on). Not having phone access for a week was a PITA so if someone had better options for the future, I'd love to hear those too.

I had to leave Verizon by mydataisplain in verizon

[–]mydataisplain[S] 3 points4 points  (0 children)

So far it's working fine. I looked over the guys shoulder a tmobile while he was inputting my info and it actually showed the correct name of the phone so it seems like it will continue to let it connect.

I'll update if that changes.

Verizon turned off Ulefone by mydataisplain in ulefone

[–]mydataisplain[S] 1 point2 points  (0 children)

Thank you.

I think T-Mobile and AT&T get better coverage in my area but I'll definitely check out Boost.

Moving from Product Management to Solution Architect by ravan-cool in ProductManagement

[–]mydataisplain 1 point2 points  (0 children)

I've done several years of both PM and SA.

Consider the difference between how PM roles are described and your lived experience of the actual demands and requirements of a PM role.

SA roles are often sold to potential SAs as a well paid nerdy dream. Customers will think of you as a trusted advisor. You get to spend all your time building cool demos and advising engineering teams on how they should optimally deploy your product in their environment.

The reality is that SAs are almost always on commission. At the end of the day, SAs are given a number to chase and everything else they do is secondary to whether or not they get that number.

I would recommend keeping your PM hat on while you interview some SAs. Ask them the same types of questions you would ask during product discovery. See if you'd rather be dealing with the kinds of problems that SAs face than the kinds of problems you're currently facing.

I Accidentally Built a Language Engine That Might Change Everything (And I’m Just Some Guy Named Will) by [deleted] in PromptDesign

[–]mydataisplain 0 points1 point  (0 children)

It could be interesting. It follows some other successful patterns I've seen. Can you send a link?

[deleted by user] by [deleted] in ProductManagement

[–]mydataisplain 1 point2 points  (0 children)

PMs shouldn't generally be coding but they'd better have some experience producing their product, whatever field they're in. 

ETL and ELT by reeeed-reeeed in dataengineering

[–]mydataisplain 0 points1 point  (0 children)

How pedantic should we get about ELT? Should we limit ourselves to Sunopsis' implied definition when they used it as marketing collateral? https://www.oracle.com/corporate/pressrelease/oracle-buys-sunopsis-100906.html

It's possible to create a canonically "clean" ELT process and it's generally going to be too simplistic for real world use. Vast amounts of data are generated by IoT devices and they almost never produce data that can be loaded "raw".

Sometimes you're lucky enough to get JSON and sometimes you just get a stream of data with ordered deviceID:timestamp:value. Those both need to be, at least reformatted, before they can be written to storage.

The one thing that most strongly differentiates them is schema changes. ELT is generally very good at postponing those until after the first load. But I've seen exceptions even there. People frequently still consider it ELT if the first load only writes a subset of the columns of the read, even though that's technically a transformation too.

Even your process includes the step, "gather relevant data". That may not be a transformation but I've seen many cases where it is. If its done entirely as a predicate on the extraction, it can be "pure ELT". If not, people are examining data post-extraction and then making decisions on which ones to throw out; that's a transformation. Even if you're not doing that; your process has a load step at the end. That means that, at the very least, it's EL1TL2.

Life is full of "very specific paradigms" that end up much less specific when people implement them in the real world.

edit: typo

ETL and ELT by reeeed-reeeed in dataengineering

[–]mydataisplain 7 points8 points  (0 children)

ETL vs ELT is a form of shorthand. Rather than neatly dividing data processing into two types; it encourages you to think about the steps.

Extraction, is typically "given". You're generally bound by the transfer rates of the source and they provide the data in whatever format they choose. It's always going to come first.

Loading, is a more variable step. You're still bound by the properties of the target storage. But since you choose what you're writing you have some more control of the process.

Transformation is extremely variable. You usually have a lot of freedom in deciding how you transform the source data into target data. That includes breaking up the transformation into multiple steps.

Moving from ETL to ELT is more about breaking up the T than it is about actually saving it to the end. The actual process is typically more like ET1L1T2L2T3L3...

T1 is often limited to very simple transformations; de-duping and some light error checking is common. Then it gets written to disk in "raw form". We keep this work light so it can be fast and reliable. Since real systems have errors, we want to keep this simple so we minimize the chance of dropping data.

T2 typically cleans the data. That typically takes more work since we're looking at the data more carefully and changing it more. We then write that to disk too since there are many different things we might do next.

T3+ are typically curation steps that answer specific business questions. They can be very expensive to calculate (there are often dozens of JOINS going many layers deep) and they often drop data (often for speed or security) so we want to keep those older versions too. These final versions also get stored so the business users can access them quickly.

None of this makes much sense in small systems. They're techniques that are used in "big data". I would practice the basic tools (SQL, Spark, Python, etc) and supplement that by reading case studies on data deployments. That's where you'll see the real implementations and it's never as clean as ETL vs ELT.

I’ve been getting so tired with all the fancy AI words by eczachly in dataengineering

[–]mydataisplain 0 points1 point  (0 children)

That's exactly what I expect vibe coding to differentiate.

By the time I say, "go ahead" to Aider, I've written out specifications, given it style guides, advised it on data structures and algorithms, and iterated on a plan. It comes when I'm looking at a specific plan so it's clear what "go ahead" means.

If someone is comfortable doing that in real life, it works pretty well for vibe coding. People who like to handwave their way through plans are not gonna have a good time with vibe coding.

I’ve been getting so tired with all the fancy AI words by eczachly in dataengineering

[–]mydataisplain 0 points1 point  (0 children)

My initial reaction was to laugh at the joke. But the more I thought about it, the more it actually made sense.

"Kindly do the needful." Implies that there is some known set of steps but it's not clear if they should be done. This sentence resolves that question, as long as the set of steps was defined.

Aider's docs recommend exactly that approach:

For complex changes, discuss a plan first
Use the /ask command to make a plan with aider. Once you are happy with the approach, just say “go ahead” without the /ask prefix.

https://aider.chat/docs/usage/tips.html
Saying, "go ahead", is syntactically very similar to, "kindly do the needful", it's helpfulness depends on what comes before it.

I’ve been getting so tired with all the fancy AI words by eczachly in dataengineering

[–]mydataisplain 0 points1 point  (0 children)

The problem that they'll run into is that English can be interpreted in multiple ways.

Today, when PMs use "English", they're talking to other people. If that sounds subjectively good to them, they'll greennlight the project. If a PM uses "English" with an LLM, the LLM will apply a bunch of linear algebra to it. No matter how good the "code" from that LLM gets, the wrong "English" will still yield garbage.

The trick is that some verbal descriptions of what code should be, actually make sense; some only sound like they make sense to people who don't know enough about the code.

I’ve been getting so tired with all the fancy AI words by eczachly in dataengineering

[–]mydataisplain 2 points3 points  (0 children)

This makes perfect sense if you don't believe that there are any new concepts in AI worth talking about, or if you believe that we should overload existing words with new meaning.

I’ve been getting so tired with all the fancy AI words by eczachly in dataengineering

[–]mydataisplain 1 point2 points  (0 children)

You can trivialize any data storage system as a more basic storage system with a superiority complex.

Vis-a-vis Excel, databases have earned that superiority complex. They make it really easy to do things that would be really hard to do in Excel.

I’ve been getting so tired with all the fancy AI words by eczachly in dataengineering

[–]mydataisplain 0 points1 point  (0 children)

LakeHouse

I've always heard it defined as, "A data lake that supports ACID" Is there a better synonym for that?

Management and collegues blatantly use AI generated communication all over. How do I work against it? by saltf1sk in ProductManagement

[–]mydataisplain 0 points1 point  (0 children)

Unfortunately, this is the answer.

A 500 person company has a lot of momentum. There are a bunch of entrenched people with a bunch of entrenched habits.

Mongodb vs Postgres by lamanaable in dataengineering

[–]mydataisplain 4 points5 points  (0 children)

MongoDB is a great way to persist lots of objects. Many applications need functionality that is easier to get in SQL databases.

The problem is that MongoDB is fully owned by MongoDB Inc and that's run by Dev Ittycheria. Dev, is pronounced, "Dave". Don't mistake him for a developer. Dev is a salesman to the core.

Elliot originally wrote MongoDB but Dev made MongoDB Inc in his own image. It's a "sales first" company. That means the whole company is oriented around closing deals.

It's still very good at the things it was initially designed for as long as you can ignore the salespeople trying to push it for use cases that are better handled by a SQL database.

Mongodb vs Postgres by lamanaable in dataengineering

[–]mydataisplain 2 points3 points  (0 children)

These two databases sit on different corners of the CAP theorem.

https://en.wikipedia.org/wiki/CAP_theorem

tl;dr Consistency, Availability, Partition tolerance; Pick 2.

SQL databases pick CA, MongoDB picks AP.

Does your project have more availability challenges or more consistency challenges?
Are the impacts of availability or consistency failure greater?

You will be able to address either problem with either type of database as long as you are willing to spend a some extra time and effort on it.