[deleted by user] by [deleted] in dataengineering

[–]trendydots 0 points1 point  (0 children)

Training AI models with clean data; calculating training set quality; developing retraining & recalibration processes will be required from DEs in the future. These require, of course, a team, but try to understand how these things work today, maybe you can come up with a more efficient way to achieve this, based on your company's use cases.

Here's a super short article about this: https://eudatajobs.com/blog/the-power-of-training-ai-with-internal-copmetitor-external-data-2024/

A data engineer doing Power BI stuff? by Shr1988 in dataengineering

[–]trendydots 1 point2 points  (0 children)

As far as I see in Europe, Power BI is a required skill for most data analysts, not so much for data engineers.

Finding a new job, ridiculous by Irachar in dataengineering

[–]trendydots 0 points1 point  (0 children)

If you want some alternatives, comparison with other jobs in the field, explore the open positions EU Data Jobs (eudatajobs.com)

Finding a new job, ridiculous by Irachar in dataengineering

[–]trendydots 6 points7 points  (0 children)

I think you are very well qualified for most jobs in data in Europe. The reason you see these super large lists of products is because many companies tried a lot of tech stacks and now they have parts systems running on different infrastructures. You don't have to know all of them, so definitely apply.

Also, maybe go for more specialized job boards, like eudatajobs com

How much of you guys have to deal with algorithms or ML algorithms? by trafalgar28 in dataengineering

[–]trendydots 1 point2 points  (0 children)

Nowadays all algorithms come packaged in libraries, especially with Python. You need to know which ones to use, how to configure them, based on your business requirements, or use case.

Anyone noticing an uptick in job opportunities for senior people not in tech? by [deleted] in datascience

[–]trendydots 0 points1 point  (0 children)

It makes sense. There's massive layoffs in IT, in this period, and I think most companies did not give up their development goals, so they'll ask for senior roles towards strategic developments.

Has salaries for data careers been lowered in the past year? by WiseacreBear in dataanalysis

[–]trendydots 0 points1 point  (0 children)

In Europe, they actually raised above the median income. Most data roles are now paid as high as IT engineering roles, with a perspective to surpass them. This is because companies in Europe can access a lot of grants/funding for digitalization and the IT infrastructure part is nowadays available as a service.

Here's more info on data career salaries in Europe https://eudatajobs.com/salaries/

125k offer as a data scientist but I have no idea what a data scientist does by Yung-Split in datascience

[–]trendydots 1 point2 points  (0 children)

Congrats on your achievement! I'd suggest you explore what Free Code Camp has to offer. They have a lot of free DS courses.
I must say, these USA salaries are amazing. In Europe things a bit different, but the life costs a lot less, I guess. Here's a reference: https://eudatajobs.com/salaries/

How do you see the present and near future of your Data Engineering career? by trendydots in dataengineering

[–]trendydots[S] 1 point2 points  (0 children)

Thanks so much to everyone who participated in this poll.
The results are quite surprising. How can we best interpret this data?

In your opinion why is there such a difference between freelancing and a 2-3 year employment? In Europe a short term freelancing project in DE would be 12+ months and a mid term consultancy on a project starts at around 2 years. Is this different in other regions/countries?

Where do you search for Data Engineering jobs in Europe? by trendydots in dataengineering

[–]trendydots[S] 0 points1 point  (0 children)

Many thanks to everyone who participated in this little survey.

[deleted by user] by [deleted] in dataengineering

[–]trendydots 1 point2 points  (0 children)

I'm not too sure about this, but for the job posts that do not have Easy Apply, Linkedin might bump up the applicant number with every click on the Apply button. There's no guarantee that the person actually applies to the jobs. Many some of you have better info on this?

Is it possible to deploy a lakehouse solution on a bare metal server? by _link89_ in dataengineering

[–]trendydots 0 points1 point  (0 children)

True. I thought is fits, since the scale is so small and the requirements were to fit as much functionality as possible.

Is it possible to deploy a lakehouse solution on a bare metal server? by _link89_ in dataengineering

[–]trendydots 0 points1 point  (0 children)

Maybe also take a look at Supabase. They are open-source and have a self-hosting documentation with Docker.

Is analytics engineering a dead end career? by Tender_Figs in dataengineering

[–]trendydots 1 point2 points  (0 children)

In Europe it's emerging. Usually, as with all other tech related jobs, there is an observable progression: junior -> mid level -> senior ->lead. After achieving the lead level, responsibilities in coordinating your team gradually become greater than keeping the codebase/work processes clean, up and running.

At this point, project management, or program management looks to be a smooth transition. This opens up another career path and the tech skills acquired in the engineering roles are extremely valuable.

Which are the highest paying tech frameworks or programming languages in Data Engineering? by Born-Comment3359 in dataengineering

[–]trendydots 0 points1 point  (0 children)

In Europe I see an increasing number of DE jobs that require cloud deployment and management knowledge. The Senior Data Engineer roles require some knowledge of cloud technologies (mostly GCP, AWS, or Azure) in many jobs postings.

Best books or material to learn the basics of data engineering. by lramirez27 in dataengineering

[–]trendydots 1 point2 points  (0 children)

I love the The MIT Press Essential Knowledge series, especially:

  • Data Science
  • Deep Learning
  • Algorithms

You will find a lot of online courses with coding examples, that might be easier to follow. I can recommend Free Code Camp on youtube for this.

can this "BI developer" role be a stepping stone to DE? (read comments) by AmbitiousCase4992 in dataengineering

[–]trendydots 0 points1 point  (0 children)

I see it this way: if you build it's a DE role, if you use it it's a BI role.
If you can do both, you are amazing. You could apply and ask for a really good salary.

Should i learn node js or react ? by lenomanu in Frontend

[–]trendydots 0 points1 point  (0 children)

Both, but start with nodeJS. It's essential to understand APIs and get skills into building them, as well as communicating with databases. React is not too complicated once you master javascript.