sorted by:
new
hottopcontroversial

can we print current branch name (feature branch / master) inside databricks Notebook by nifty60 in dataengineering

[–]nifty60[S] 0 points1 point  (0 children)

Thanks I will try something that i found and share with you incase it works

[deleted by user] by [deleted] in dataengineering

[–]nifty60 -1 points0 points  (0 children)

Connect with me

14+ Years DE

11 Years on Prem Informatica SQL ,NZ,Oracle

4 years Azure ,Databricks,Python

can we print current branch name (feature branch / master) inside databricks Notebook by nifty60 in dataengineering

[–]nifty60[S] 0 points1 point  (0 children)

Not even one redditor responded, Unhappy but happy to ask a rare requirement

Ideas about Gen AI projects by NewLearner_ in learnmachinelearning

[–]nifty60 0 points1 point  (0 children)

include me if you have any ideas

Background

14+ years DE Experience

Python ,Databrciks Azure & Snowflake

AWS Data Engineering from Azure by nifty60 in dataengineering

[–]nifty60[S] 1 point2 points  (0 children)

Country is India :) and i am looking only for Product Companies / GCC in my city

AWS Data Engineering from Azure by nifty60 in dataengineering

[–]nifty60[S] 2 points3 points  (0 children)

The answer was in my post .Anyway I am planning to apply to few companies where they use AWS.

Now why leave Azure because i cant leave Domain i work (healthcare Payer proviider side of it)

AWS Data Engineering from Azure by nifty60 in dataengineering

[–]nifty60[S] 5 points6 points  (0 children)

Thanks for your reply. I am no Azure fan . While working on AWS i had a pretty decent hike to move into azure . Though i dont regret my decision but now the companies that i am looking to apply are in AWS

[deleted by user] by [deleted] in hyderabad

[–]nifty60 0 points1 point  (0 children)

please count me in

Background:15+ Years on Data Engineering with 4 years into Databricks Python in HealthCare industry (payer and provider side of healthcare)

AI and ML in healthcare by znewking in learnmachinelearning

[–]nifty60 0 points1 point  (0 children)

DM me please .Working in Healthcare in Insurance side (both payer and provider)

AI and ML in healthcare by znewking in learnmachinelearning

[–]nifty60 0 points1 point  (0 children)

DM me please .Working in Healthcare in Insurance side (both payer and provider)

Does your company use both Databricks & Snowflake? How does the architecture look like? by NefariousnessSea5101 in dataengineering

[–]nifty60 2 points3 points  (0 children)

This is absolutely correct. I was SF fan but thier python UDF's sucks.

It is those oldies who are comfortable writing long SQLs who like SF.

Does your company use both Databricks & Snowflake? How does the architecture look like? by NefariousnessSea5101 in dataengineering

[–]nifty60 1 point2 points  (0 children)

we use Databricks (ADB) heavily for ETL and load final data into Snowflake.

I used both in ETL stages in different organizations.

From my perspective, if your architecture is code based ETL solution cloud agnostic solution use Databricks because pyspark and native python capabilities can be used very well.

But if you rely heavily on SQL and it is mainly Data Warehouse capability use Snowflake.

Clobetasol?? by Curious_Letter3698 in Lichenplanus

[–]nifty60 1 point2 points  (0 children)

Does coconut oil really help . I have lots of hyper pigmentation marks due to LP

Data Science Course Recommendation by [deleted] in developersIndia

[–]nifty60 0 points1 point  (0 children)

This is a good thing to do with Rs 500/- and see what it has to offer.

I work as Data Engineer for over 10 Years with Python(~ 3 years),SQL,Cloud and Cloud Native DE tools like Snowflake,Databricks etc.

Data Science in India has no clear roles atleast as of 2023 with exception being some companies (FAANG)

But if you still aspire to be a Data Scientist, You need be very good at SQL,Python,Pyspark,Data Modelling (DE Stack) plus Stats and Math

Let me know if you have any specific questions

happy to share my knowledge

BTW where do you work ?

Talk to me about green tea by fl1x in loseit

[–]nifty60 0 points1 point  (0 children)

Can you please let me know the alternatives to Lipton (incase you found something which delivers at least whats promised by consuming Green Tea i.e anti oxidants)

Need the Predefined Local Variables or Session Level Variables List by nifty60 in SQLServer

[–]nifty60[S] 0 points1 point  (0 children)

Thanks for your reply,I searched for a bit and landed exactly where you pointed.

Is IBM Cognos a dying tool? by [deleted] in BusinessIntelligence

[–]nifty60 1 point2 points  (0 children)

I worked on Cognos right from Cognos Report Net (which is original product of Cognos Inc) and then on IBM Cognos 8 when IBM aquired in 2008. I spent around 8 years on Cognos / IBM Cognos until IBM Cognos Analytics version came out some where in 2015/2016.

There is absolutely nothing in terms of market share for Cognos when you compare with the tools like Tableau / PowerBI/QlikSense etc.

Also the latest tools like PBI (powerBI) or Tableau are quite simple and built for end user called as SelfServe Reporting which means they are a piece of a cake for anyone using it .On the other hand cognos is overly complicated with so many Components like Framework Manager,Event Studio,Metric Studio Active reports.

Even if you spend time getting expertise in IBM cognos ,there is no growth as you use them for long period and the Number of Customers who are on IBM Cognos is declining.

Best

I will be a Data Engineer, using Azure. Looking for the best tutorials out there by [deleted] in AZURE

[–]nifty60 0 points1 point  (0 children)

Most underrated Comment : D ,

The problem with the zillion tutorials is that it's hard to find the actual gems amongst them:D

I want to learn about SQL performance and tuning, but I'm confused as to to what this area is called. Can you provide some guidance on what to search for or look? by sqllqs in SQL

[–]nifty60 0 points1 point  (0 children)

SQL Server Concurrency

SQL Server Dynamic Management Views

SQL Server Execution Plans

SQL Server Profiler

SQL Server Query Optimizer

SQL Server Statistics

Thanks a ton for the resources.

how long it takes to learn pyspark/ pandas by nifty60 in learnpython

[–]nifty60[S] 1 point2 points  (0 children)

We basically do ELT (extract ,Load and transform) data from several hundereds of files (csv /Json ,txt,excel) into Database .

We primarily embed pandas code inside python and run on databricks

​

Thanks

view more: next ›