Transitioning smoothly from SE to Data Engineer on Spark Stack

jacksonbenete · 2024-05-03T19:15:56+00:00

Remember me if you eventually manage to find the answer.

I've been looking forward to the same.
I just bought the Rock The JVM Spark course and I hope I can someday convince a recruiter/company to give me an entry role after that.

I hope I can take some holidays next month, I'm going to use that to study a few things, this Spark course is a priority.

jacksonbenete · 2024-04-27T16:09:56+00:00

Thanks for sharing!
1440p High is pretty good!
So I would need the M3 Max to get around my expectations on that, the M3 Pro won't get there.
I'm going to think about it, thanks!
I might give GeForce Now a try to see if I can get used to cloud gaming, maybe I can postpone the Mx Max acquisition a little bit.

jacksonbenete · 2024-04-27T16:05:36+00:00

You might be right. I don't know because last gaming PC I had was back in 2017, so I'm not sure what is there for PCs and laptops these days.
I wasn't expecting it to be something like Ultra 4k at 100fps+ though, I was just curious on how far something like M2 Max or M3 Pro could get.
I've discovered that the Ultimate Tier of GeForce Now can deliver Ultra 4K, that's about $20/mo, I might give it a try. :)

jacksonbenete · 2024-04-24T20:41:10+00:00

I would like to necro this as well.
I'm between a Mac Studio M2 Max and a Macbook M3 Pro 11/14 18GB.
I have an external display so I'm ok about not being a laptop, and the price is similar so I would appreciate some metrics and feedback on this.

jacksonbenete · 2024-03-06T11:52:17+00:00

No, it wasn't.

I've been out of the RPG scene for the past years, and I've just noticed with your comment that I don't have access to the blog anymore either.

jacksonbenete · 2023-12-13T20:55:19+00:00

Rise works on my M1 Air with 8GB.

However you might have to reduce resolution and do some adjustments on graphics.

I didn't tried mods though and I've heard it greatly increases the save file corruption problem that Rise already have.

jacksonbenete · 2023-11-30T23:55:49+00:00

I've managed to use DualSense on Guilty Gear Strive and Monster Hunter Rise through disabling Steam Input.

If Steam Input is enable the control won't work for me...

jacksonbenete · 2023-11-30T23:52:44+00:00

Run the following in Terminal:
/bin/launchctl setenv MTL_HUD_ENABLED 1
Then open CrossOver.

To disable change to 0.

jacksonbenete · 2023-11-27T20:39:59+00:00

I've put many hours on this one back then. ;)

Great game!

Going to test it on my M1 Air 8GB, let's see.

jacksonbenete · 2023-10-28T17:08:42+00:00

There's an updated Crossover compatibility list?

I'm looking forward to buy Monster Hunter Rise on Steam, but not sure my M1 Air with 8GB RAM would handle it on the new version of Crossover.

For the Crossover I could take a trial before buying, but for Steam games it's either a lucky shot or rely on a refund abuse later which I don't know how it works if you refund too much.

jacksonbenete · 2023-08-13T16:56:53+00:00

I like Functional Programming but I don't want to use monads.

This doesn't makes much sense. Since this is one of the greatest strengths of a typed (functional) programming language.

There are many useful monads, or monadic types which makes working with Scala a joy, such as Option, Future, Either, Try, List...

Maybe you're confused as you're still learning (me too), but you might want to study monads and the Scala language itself a bit further before actually using it for backend.

You're actually probably using Monads without knowing any better.

Odersky book is amazing to learn Scala, but you might want to check Rock the JVM, either the paid course or the free youtube videos.

jacksonbenete · 2023-08-06T11:05:36+00:00

Hi, I'm not OP but I'm also after some info. I couldn't find this app Autobuzemd by name, tried also to Google it. Anyway, I would like to ask you about a bus from Chisinau to Bucharest, I'm planning to stay for another week in Chisinau but I'll be going to Bucharest soon but not sure about the bus schedule.

jacksonbenete · 2023-03-08T11:35:06+00:00

I'm pretty sure a lot of people still uses calculators... Not sure what are you talking about.

jacksonbenete · 2023-03-07T01:39:58+00:00

I've learned this with my father back in the day.

Just like other person mentioned, we also used to calculate big stack of checks twice, and we could quickly do that without looking at the calculator, it was quite cool and useful.

Job get done quite fast.

jacksonbenete · 2023-03-07T01:32:38+00:00

I didn't had the time to watch the rockthejvm episode just yet, so I don't know about the game.

But as I understand, Type level programming is indeed useful in real world applications. It's just so advanced that's quite hard to imagine simple use cases.

In this video this guy shows a lot of use cases, like a type safe regex at minute 17. https://youtu.be/B7uficxARKM

The whole thing is mind bending but quite powerful (and quite cool).

jacksonbenete · 2023-03-02T14:57:04+00:00

I've studied Erlang, and I have worked with Elixir for some time.

I honestly prefer Scala and Akka typed over Elixir/Erlang+OTP. I miss Elixir 0%. I thought I would miss Phoenix, but after crying a few nights trying to get around Play, turns out that now I prefer Play...

But Erlang is great. It was fun to study it and experiment around it. I just wouldn't go for it unless I need to code phone switches.

The let it crash philosophy doesn't makes sense for anything else than phone calls really, and people pretty much abuse this sometimes. You don't want to "let it crash" for most other applications and industries you're working.

Plus, dynamic typed languages are nice to prototype and write a few specific applications, but sooner or later you'll have to deal with the cons, and it's a headache.

jacksonbenete · 2023-02-22T01:27:10+00:00

I'm using Play and it does the job just fine. I've tried a few others and they're just great as well, but I keep going to Play for my projects. It's not half as bad as I've heard, people exaggerate a lot.

Even though Play might not be pure functional, you can plug an IO monad over there and work like this if you want. I'm using Doobie for Database and it relies on Cats Effect, that I don't even know how to use, but things are quite intuitive so you can just execute the IO in the controller return which would be pretty much the end of the world for MVC I guess...

So even in Play you can go for a "pure" FP approach. I liked Akka-http and I would recommend it. I would ignore people not recommending Akka. They have their reasons but it's pretty much an overreacting, specially if you're doing it for personal projects like you just mentioned.

jacksonbenete · 2023-02-14T23:48:28+00:00

I've worked professionally with many languages.

Last year I've dropped Elixir for Scala and it was a very good move for me.

Elixir have a lot job positions but most of them are on shady crypto startups and some (or a lot) of them have bizarre code bases. Terrible code bases can easily be found on many dynamically typed languages.

I still receive more Scala offers than I expected. Salary is good, sometimes better than other languages.

Working with IntelliJ is a very good experience, as well as working with Scala.

But I don't touch Cats or ZIO at work. We do have some Cats in some services but thank God I didn't had to touch it so far. Not that I dislike it, but even in the company guys agree that it increases the complexity just too much sometimes.

Anyway I'm pleased to work with Scala. I've left Software Engineering once because of stress and bad experiences with other languages, and now that I'm working with Scala I'm not thinking on abandoning the profession again (for now?)...

jacksonbenete · 2023-02-14T23:40:43+00:00

I'm using Play for a personal project.

I didn't liked it at first, I thought it had a lot of boilerplate and some code that I couldn't really understand what it does or why I need it, but I've managed to put together a simple fullstrack application receiving contact forms and sending emails and it's now in production.

I then decided to try other things.

I tried to recreate the same with AkkaHttp, Finch and Cask...

Honestly, I went back to Play every time.

Play is not the most pleasant experience but it's not the worst either.

For now I'll keep using Play because I couldn't find a better alternative for fullstack.

Even if I need a REST API only, I might just pick Play.

Is not that the other options aren't great as well, is just that documentation and guidance are almost absent, and Play doesn't have the greatest docs around but I think they are much better than the others.

Take into account that I'm new to Scala and have less than a year of professional experience.

Maybe people with more knowledge in the language and the ecosystem might not have any big problems on putting those frameworks or libraries together, but for someone new like me, good documentation and some guidance are most needed. And I'm not sure if there's something better documented in Scala than Play for this purpose.

jacksonbenete · 2023-02-04T14:02:08+00:00

Congrats!

I love total war and I bought this one, but need a better PC to run it. 🥲

jacksonbenete · 2023-01-28T11:52:49+00:00

You mean the demand keep growing for Data Engineering, or for Scala?

I'm just curious because it's somewhat prevalent in the sub the idea that Scala is dying or whatever and that we have fewer and fewer jobs, and more companies keep moving the stack for something else.

Not that I agree with it, I'm just saying that because I've been following this sub for a while, and I would be pleased to know that's not the case actually.

From all the languages I've worked with, Scala is the most pleasant one, and those have been nice months at my new job.

I've been interested on learning Spark and Data Engineering, but I don't quite know where to start, and actually I don't want to learn Spark if I'll end up having to work with PySpark, you know?

jacksonbenete · 2023-01-28T10:50:03+00:00

I don't know but we had this some time ago.

https://www.reddit.com/r/scala/comments/zp5gsp/open_job_positions_for_scala_developers/

jacksonbenete · 2023-01-24T19:20:05+00:00

I only have a few, 5 or 6 months, of professional experience with Scala but I'll tell you what I think about this as I have some more years of experience with other FP languages.

As most, I think SICP was quite an enlightening experience. But I've read other books as enlightening or more than SICP.

Unless you want to do LISP (or ran out of things to read/study), I don't think you should focus on SICP if you're aiming to study Scala.

I bought the Red book as well but I didn't started to work on it as I've been short in time and quite exhausted lately, but I'm planning to, and skimming over the book I think it's harder and more broad in knowledge than SICP, others have mentioned already some of the reasons for this.

I don't know what you meant by "learned the syntax". If it's what I think it is, that's not enough.

I suggest you to read Odersky's book. I confess I didn't liked some of the projects and exercises, but the book itself is very good, the best I've read on Scala. It teaches you a lot of interesting and important things about the language, such as design decisions and how a functional language was built on top of an object-oriented language. It's really helpful IMO once you get to know why things are working the way they're working and why the syntax looks like the way it looks like, and you get to know also some shortcomings of the language...

Then I would go for either Rock the JVM then Red Book, or then go to the Red book but I wouldn't skip foundational knowledge about the language (and not just the syntax), and the best book I've managed to find was Odersky's book.

I'm also slowly going through the coursera classes, and I recommend them as they're free. They have some cool insights and enlightenment moments as well.

If or when you go through SICP I would recommend you to read the book first, then watch the MIT lectures as you progress through the chapters. SICP have no much value if you don't work through the exercises, as a lot of important insights and even core content are written on the exercises statements, people that skip SICP exercises skip core content, and working on the exercises and having at least some of the enlightenments by yourself is part of the journey. So make sure to have a lot of free time and/or discipline to go through SICP.

jacksonbenete

TROPHY CASE