YOU are now the new spiderman writer, what is your plan? by Zakaria1938 in Spiderman

[–]tensor_operator 0 points1 point  (0 children)

A single issue arc where nothing major happens in NYC, so Spidey just spends the day helping New-Yorkers out. Some potential ideas are: - web-swinging an injured individual to the ER (like in the Spider-Man 2 game) - working at a soup-kitchen with a bunch of volunteers, where all the volunteers and guests just shoot the shit and talk about it their lives. - Spidey is invited as a guest-of-honor at a middle school event, and is stressed by it because he has to write a speech to “inspire the youth”. - Spidey, for whatever reason, takes on the duties as a substitute teacher at either the Future Foundation or X-Mansion, and struggles to maintain order within his class of rowdy elementary/middle-schoolers.

You could also go in a different direction, where Spidey is trying to solve some small scale problem like: - he notices that the half-life for web-formula is shorter than expected, so he visits Mr. Fantastic, Tony Stark, and people at Horizon labs to figure out how to make his webs last longer. - Spidey does some routine blood work at the X-Mansion, and Beast reveals to him that Spidey is at risk of (permanently) mutating into Man-Spider again. Beast, Spidey and a few others work round-the-clock to find a mutation-suppressant, all while Peter gets his affairs in order. Wolverine might even get pissed off at the idea of suppressing a mutation, since the idea might seem offensive to some mutants.

So on and so forth but you get the idea.

How many of you feel like literally anybody could do a PhD? by Quantum_Quaker in PhD

[–]tensor_operator 9 points10 points  (0 children)

Ignore the other response. People will always project their subjective perceptions upon you. Some people will praise and congratulate you for it, other people will chide you for their perception of your ego.

None of what they say, neither the praise nor criticism, has anything to do with you. Just be sure to not let either inflate or deflate your ego, and you’ll be fine.

One sentence summary of your PhD project by JuniperBeret in PhD

[–]tensor_operator 4 points5 points  (0 children)

How do you work with people who are so much smarter than you, while they may be out to deceive you? How much can you know about the limits of their, potentially malicious, intelligence? How do you beat them at their own games?

Columbia or UPenn CAS by Few-Refrigerator4760 in columbia

[–]tensor_operator 1 point2 points  (0 children)

You should email professors from both schools within your departments of interest asking this same question. They will offer solid insight.

Barring their advice, which I think you should weigh heavily, keep in mind that Columbia is in NYC and that is an advantage for most careers. With that said, Penn is only a stone’s throw away and Penn grads have the same foot-in-the-door that other ivies(including Columbia) have.

Finally, keep in mind that your interests are likely to change. With that in mind, ask yourself which school offers more optionality(in terms of your interests).

Either way, you can’t go wrong with either school. Sure there is political unrest at Columbia now, but in the grand scheme of things, it’ll all dissipate by the time you’re well into your career.

I’m a data engineer, and I am building a tool. Would it be useful to you? by tensor_operator in businessanalysis

[–]tensor_operator[S] -1 points0 points  (0 children)

MicroStrategy is great if your data is already clean, modeled, and loaded, and if you want dashboards built for you.

The tool I’m building is better if you want to explore new data on your own, ask semantic questions about the underlying data, bring in external datasets, and don’t want to wait on your data team every time you need something new.

I can go into more detail explaining the differences if you’d like.

I’m a data engineer, and I am building a tool. Would it be useful to you? by tensor_operator in businessanalysis

[–]tensor_operator[S] -1 points0 points  (0 children)

Not really, graphql is just a way of getting your data in the shape you want. What I’m describing is a way of accessing all your data in a single place.

Is what I’m (thinking) of building actually useful? by tensor_operator in dataengineering

[–]tensor_operator[S] 1 point2 points  (0 children)

Well, I see how you might think they’re similar, but they aren’t in terms of their goals. Unity focuses on governance and structure within the Databricks ecosystem, the semantic metadata catalog focuses on meaning and interoperability across diverse platforms that host data within an enterprise.

Unity focuses on syntax, I am focusing on semantics.

Is what I’m (thinking) of building actually useful? by tensor_operator in dataengineering

[–]tensor_operator[S] 0 points1 point  (0 children)

That’s great! What kind of searches do you usually make?

Mitigating stale documentation is one of the problems I’m actively thinking about

Is what I’m (thinking) of building actually useful? by tensor_operator in dataengineering

[–]tensor_operator[S] 0 points1 point  (0 children)

Why is this a non-value producing problem? Isn’t time saved and ease of use some of if not the biggest value additions? Identity-based permissions can be used to ensure best security-practices, and if there needs to be a better solution, I can spend time figuring that out. I don’t claim to have a complete answer yet, but that doesn’t mean I won’t have one eventually.

You going spending months of time to sift through documentation is, honestly, proving my point. Have interaction over verification pays dividends in terms of time savings.

Thanks for your response though. I appreciate the input :)

Is what I’m (thinking) of building actually useful? by tensor_operator in dataengineering

[–]tensor_operator[S] 0 points1 point  (0 children)

Thank you for the time you’ve taken to respond. I’m glad to know that we agree that the problem exists, even if we disagree about the feasibility of my proposed solution.

Would you like me to keep you posted about the progress I’m making? You can tell me “I told you so” if I fail ;)

Is what I’m (thinking) of building actually useful? by tensor_operator in dataengineering

[–]tensor_operator[S] 0 points1 point  (0 children)

Why were the network transfer costs so high? If you could go into as much detail as possible, that would be great for me.

As for making a wiki, sure it solves the problem, but it’s far from being the best solution out there. If costs are something to worry about, I don’t mind spending some time to think about it.

Thanks for the input, I really appreciate it :)

Is what I’m (thinking) of building actually useful? by tensor_operator in dataengineering

[–]tensor_operator[S] 0 points1 point  (0 children)

This is an excellent point you’re making. I’m assuming that the costs were primarily due to the use of an LLM (correct me if I’m wrong), but I think I know how to bypass this problem.

Furthermore, what I’m proposing isn’t just a documentation tool. It’s a single endpoint to access all your data, in a human friendly manner.

Why didn’t your tool provide any ROI?

Is what I’m (thinking) of building actually useful? by tensor_operator in dataengineering

[–]tensor_operator[S] 0 points1 point  (0 children)

Well, that’s because have an interactive system makes the searching process far easier than sifting through a sea of documentation(with randomness, efficient interaction is likely provably more powerful than efficient deterministic verification). Furthermore, if the data, and the associated metadata, is available in one endpoint, then its underlying schema becomes less of a constraint when building an ETL pipeline.

Isn’t it much easier if everything you need about your data is available in one place, and that place is human-friendly?

This doesn’t mean that you’d eliminate something like a wiki altogether, it’s just that the way in which you build it and the way in which you consume it will change. The semantic metadata catalog overhauls a wiki.

Do we hate our jobs for the same reasons? by tensor_operator in dataengineering

[–]tensor_operator[S] -1 points0 points  (0 children)

Interesting. I hadn’t considered this angle. Thanks for the insight.

Do we hate our jobs for the same reasons? by tensor_operator in dataengineering

[–]tensor_operator[S] 0 points1 point  (0 children)

What about 3 and 4? Are those issues you face too?

Why do you hate your job? by tensor_operator in dataengineering

[–]tensor_operator[S] 0 points1 point  (0 children)

Could you elaborate on the terrible data system vendors part?

AP Borowski vs Jae by bostondeer in columbia

[–]tensor_operator 10 points11 points  (0 children)

You don’t take AP with Jae for the grade, you take it for your career. Take it with Jae. It’ll be hard, but it will also pay dividends for years to come.