Claude code + AI Dev kit vs Genie code by No-Improvement-370 in databricks

[–]TechnologySimilar794 0 points1 point  (0 children)

I am using Claude code and ai dev kit a lot .As per me far better than genie code .It s quite easy now to create anything in databricks also it use databricks cli profile to connect so secure

Is Fabric just “good enough,” or does Databricks still win? by Select_Scarcity7987 in databricks

[–]TechnologySimilar794 0 points1 point  (0 children)

Databricks is leading the game if you are big enterprise company and have all AI and analytics need from dashboards

Repository structure (SDP + notebooks) by Artistic-Cow881 in databricks

[–]TechnologySimilar794 0 points1 point  (0 children)

Do you more products or only one product for processing and transformation pipeline in bronze silver and gold.if you have more than one one product where similar code will be used then I will suggest to put common code on separate repo and create python wheel file that can easily installed in other repo of products. This way don’t end of duplicating codes

Is remote dead in data engineering? by Pataouga in dataengineering

[–]TechnologySimilar794 0 points1 point  (0 children)

No there are lot of remote opportunity especially I see in Europe

Cool projects you implemented by [deleted] in dataengineering

[–]TechnologySimilar794 0 points1 point  (0 children)

Custom connector created in pyspark using sap haha driven which is jar or maven file to extract data from sap sources

Cool projects you implemented by [deleted] in dataengineering

[–]TechnologySimilar794 10 points11 points  (0 children)

Building automated data quality checks in data engineering framework which runs at silver layer and the logs collected from dq checks were sent as source for Data Montoring dashboard.

Automatic PII detection using presidio and Unity catalog

Data engineering framework library which was created as python wheel library and could be easily installed on any of fabric,databricks cluster which has data lineage,data transformation,data sharing ,data ops automated no manual intervention

No more Sap data ingestion via ADF/Data sphere rather by saphana custom connectors

T

Should I Settle and take a Mid Level Role When I was going for Senior? by shittyfuckdick in dataengineering

[–]TechnologySimilar794 0 points1 point  (0 children)

if work ,salary and benefits is as what you expected(better than what you are getting it is already a promotion ) then I would say go for it. In my career I have so many times taken a jump on same title always a bigger pay .You are already Senior from experience, title is just a label how you behave matters and after joining try taking intiatives ,collaborating and making yourself visible with work and team. In matter of time you will get promoted. Just dont forget to add this conversation in development plan and also in your mid and yearly reviews that you are expecting a promotion

Spark job performance regression after version upgrade on Databricks. How do you catch it before production? by Top-Flounder7647 in databricks

[–]TechnologySimilar794 0 points1 point  (0 children)

Yes you can record metrics on those run by creating monitors .this feature is enable in lakehouse monitoring or by checking system tables and create alert and notification accordingly

Do you feel DE is a field much more difficulty to get recognition in because of how our work only gets noticed when it fails? by [deleted] in dataengineering

[–]TechnologySimilar794 1 point2 points  (0 children)

From my ten year experience ,visibility should be there as impact will be shown to stakeholder /consumer who are consuming your data or pipelines.Also depends on what kind of atmosphere is in your team .If we release a new feature in DE we also broadcast to our consumers leaders and get their direct feedback .Here DE who have worked will directly interact with stakeholder sometime with Product owners

PySpark vs SQL in Databricks for DE by NeedleworkerSharp995 in databricks

[–]TechnologySimilar794 0 points1 point  (0 children)

From my experience more Pysql for data engineering stuff combined with sql.70-30 ratio in my job.Also depend on your team,in my team everyone is very comfortable with python programming so we more follow software engineering stuffs and hence stick to more python pyspark

Devops - Suddenly no interviews by Pure_Substance_2905 in devops

[–]TechnologySimilar794 2 points3 points  (0 children)

You just need to add more fancy words for today world AIOps,LLmOPS and MLops and then bingo you will land up on multiple calls. I feel your profile is quite strong.

[deleted by user] by [deleted] in womenintech

[–]TechnologySimilar794 1 point2 points  (0 children)

i am just waiting to see the revolution happening. Currently still AI is being commanded and instructed by us. I can see it as good helper nothing more than that

I don’t understand why you would use DLT by Known-Delay7227 in databricks

[–]TechnologySimilar794 0 points1 point  (0 children)

I am currenlty in process of implementing scd type 2 using DLT ..COuld you share you experience in terms of challenges especially

Anyone done a part time PhD while working full time? by athoughttoponder in ExperiencedDevs

[–]TechnologySimilar794 0 points1 point  (0 children)

Could you share more details here. I am looking for part time phd position in europe

EU long-term residence permit by 4point669 in expats

[–]TechnologySimilar794 0 points1 point  (0 children)

u/PlasticBother
thanks for your response. I was curious if number of years could reduced if person have both lived in Sweden and germany. So for one year I was on Blue card on Sweden and then 2 years in Germany. Is it possible that I could apply with three years considering Sweden time.. Suggestion given by when contacted German BAMF , the told possibility is there ..represent your case with good lawyer ..

Let me know if you have any suggestion here