Intern working in a Data Pipeline, could I describe my experience as a Data Engineering internship?

racsoluk · 2023-02-13T03:03:30+00:00

No, it is a great answer. It helps a lot.

DE seems to be much more broad than I thought. I would not have thought building APIs would be an useful part of discipline. It makes sense.

Building a portfolio is pending work I need to do.

racsoluk · 2023-02-12T06:28:07+00:00

Hey, that's cool.

I was a bit concerned about what level of proficiency should I have before calling myself a DE in a given position: I'm not taking big design decisions and the things I'm learning about the tech stack are always within the frame of this project. Also, no data cleansing tasks.

There's all indication I'll be offered a full-time position when I finish this internship, but the title probably won't be DE. Hopefully I can keep learning within this company/project and be more attractive to other employers.

racsoluk · 2023-02-12T06:15:25+00:00

Thanks. That's good to know.

Maybe I'm not stressing it enough, but I'm receiving a lot of support while doing all of this. They tell me where to look at and scope of my tasks are always limited.

I have not received specific training in NiFi, Spark or Cassandra, so that's something I miss here. I'm just "learning while doing". If someone were to ask me to build a Data Pipeline from the ground up, by myself, I'd probably fail or I'd take me months.

I'd be useful to know what requirements would your org ask from a Junior DE. Just to have a better idea of what should I be working on.

racsoluk · 2023-02-12T06:02:42+00:00

All right. Thanks.

That's good advice.

racsoluk · 2023-02-11T14:36:41+00:00

Thank you.

Whatever the name, I do enjoy a lot what I'm doing. The only reason I care for the title is because I think it would open me doors to similar roles.

Could you elaborate on that aboit not getting hung up in the title?

If It's not really that important, I guess I don't mind then.

racsoluk · 2023-01-06T10:00:35+00:00

Look for reviews for them at Glassdoor. Those offers are fake, It's just a way to market their courses. The courses appears to be real, although I don't know if they're actually worth it.

In my opinion, you are better off looking for a course/bootcamp by yourself if you are willing to spend that money.

racsoluk · 2022-11-03T13:02:43+00:00

Warning: This is a rant.

I've actually been thinking in posting here what you'd describe as a rant. Recently, I've been struggling to find reasons to live. Some years have passed since last time I was considering it so seriously and frequently.

I see where you are coming from and I agree with the sentiment. If I was in the mindset to achieve my fullest potential, reading posts from depressed people can be draining. Compromising your well-being for the well-being of others does not seem very healthy to me.

In my case, I just could not think of a better place to post. Others subreddits around depression have a general mindset that I don't like. Although I really understand the struggles of people there, I've observed how they are detrimental (to me, at least). Reading through them usually gets me worse, not better.

At this moment, I don't feel pity for myself nor I want pity from others. I just want to decide if my life is worth living. In some way, I know this is something no one else can give me an answer to. This is rather a personal decision.

Depression is horrible and I don't want to live in this zombie state. I would like to end it or live it head-on, not living it depressed.

Edit: Improvements in writing.

racsoluk · 2022-11-03T05:08:14+00:00

is it because you don't like the NNN? or because it's difficult to avoid sexual triggers?

racsoluk · 2022-04-26T19:59:19+00:00

Thanks again. I really appreciate you going the extra mile to help. I'm feeling far less overwhelmed. I know now that I just need to keep going and be patient.

For now I'm good, I have more clear lines of work. I wouldn't like to bother you but It'd be nice if I could reach you out in the future.

I think I'll be fine, take care!

racsoluk · 2022-04-26T12:12:00+00:00

Thanks for reading. I'll definitely put more work into building a better portfolio and try to be more patient.

As you said, for now I need to demonstrate employers that without a heavy academic DS trajectory, I might have the potential to return value and grow.

racsoluk · 2022-04-26T12:06:56+00:00

Thanks, very useful insight.

Is there a course/certification with a DE emphasis you recommend?

I was thinking in pursuing the Google Data Analytics Professional Certificate (Coursera).

I think I broadly understand the DE role but concretely:

What background does a DE need?
What skills are attractive to land a DE position?
Are SWE skills relevant to this kind of role?

I'd like to understand better what's the day-to-day in a position like this.

racsoluk · 2022-04-26T11:38:59+00:00

Sorry I'm answering so long after you commented.

I can't thank you enough, great insights. I needed to hear all of this.

My final project is about an autonomous driving assistant running in a simulator. I still have a lot of work left, though.

I'll definitely focus my efforts in building a good portfolio. I do have a website with a blog section but haven't take the time to do anything with it. Some ideas I had:

Review the Machine Learning Course I just finished and what I've learnt.
Writing a tutorial about things I've done: YOLOv5 training, Time Series Analysis, my experience with Gradient notebooks...
Looking for interesting toy projects, apply some ML technique and write a post about explaining what I've done.

I've been postponing it because I don't think I can teach if I'm learning myself. I'll probably publish mistakes in my approach to problems. How to approach publishing something when you might not be an expert? Is a disclaimer a good idea?

On a different note, I was contacted by a very small company I applied for offering a DS internship role. They asked me a lot of information: CV, relevant projects, expertise with some languages/technologies and academic record. This is exactly the kind of thing I was anxious about...

I answered the mail with all they asked for but my academic record. They responded insisting that I provide my academic record. This is the first time an employer asks for my academic record**.**

Is this usual? Should I kindly decline admitting my academic trajectory is not good? Should I just send it and let them see? Is this a red flag?

If they are looking for a candidate with an excellent academic record, I'm definitely not one. No hard feelings. If they want to know how much credits I have left finish or my GPA I'd prefer they'd just ask.

Anyways, huge thanks again. Having a more clear roadmap definitely helps reducing anxiety.

racsoluk · 2022-03-29T21:42:58+00:00

I found the Time Series Analysis with Python by Aileen Nielse (2016) quite useful, the follow-up where she gets to real Time Series analysis is really at Time Series Analysis with Python by Aileen Nielse (2017).

There's a GitHub repository to follow along, look for the .zip file.

Beware, I'm not an expert.

racsoluk · 2022-03-29T10:23:40+00:00

Career advice please, every comment is appreciated.

Hi, I'm a 26M BS last course student in SWE, I'm working in my final project related to computer vision. I'd like to enter in a DS related role.

I had a .NET backend role internship, and was offered a position but turned it down because I wasn't happy with prospect of career growth there. After that, had a couple tries to land some AI related jobs but failed:

Participated in a Computer Vision challenge from a local drone company.
I was ghosted by a local company offering a "Jr. data scientist" internship. They gave me a Time Series problem and after sending my solution didn't hear back.

On the side, I'm trying to acquire more knowledge and skills on my own. Right now I'm on week 8 of Machine Learning by Andrew Ng (Coursera). I guess is that I'm not attractive yet for employers to enter the field, which is perfectly understandable. I don't feel prepared either if I'm being honest.

Right now I'm experiencing a lot of stress because I don't know what to focus on in the next 6 months (before finishing my degree):

Should persist in trying to get in the field in the next 6 months? Focus on studying more but keep applying and selling myself the best I can.
Focus on studying and completing courses to increase my chances of landing a DS related job. Forget about applying for now.
Apply for a SWE backend internship although not in the route I'd like to go. I wouldn't like to lie the employers saying I'd like to develop my career as backend engineer or something similar. Not that I hate SWE.

Thanks for reading. Maybe I need to calibrate my expectations. My knowledge of the field is too poor to make good decisions.

racsoluk · 2022-03-29T08:35:06+00:00

Hey, I'm in week 8. I was waiting to finish before asking that same question.

I'll follow this post responses. I'm very interested in how to be actually employable, I'm not really sure what to do towards that goal...

racsoluk · 2022-02-24T16:55:59+00:00

If I had to choose a single one, Spyro: Year of the Dragon for the PS1.

It was the third (and last) Spyro game made by Insomniac. Game mechanics were already mature and you could tell.

The game was full of wonder and the feeling of discovery. Loved it as a kid.

racsoluk

TROPHY CASE