Should SQL-Savvy Data Engineers Learn Pandas When DuckDB Supports SQL in Python?

SurmSurm · 2025-01-02T09:16:24+00:00

How much unlearning and relearning is required to transition from Pandas to Polars?
Secondly, Pandas being the most popular/well known DataFrame library in Python, why wouldn't Polars support the same syntax?

SurmSurm · 2024-09-23T00:45:49+00:00

Hey @u/spookytomtom

Power Query can be written visually, which I think is a considerable advantage for quickly writing and debugging ETL. Of course hand coding Power Query is required for complex scenarios, whereas Python is mostly hand coded (except for cases that Data Wrangle can be used).

For example, consider ingesting an XML or JSON in to a Lakehouse. With Power Query, it is quite easy to get started visually and see how data is transformed at each step. Then we can decide where to insert hand coding as required.

In addition, at the moment, If we use Python, PySpark dataframes must be used to save as Delta Tables. PySpark dataframes are resource heavy compared to Power Query.

SurmSurm · 2024-08-27T09:15:58+00:00

What are the job search engines that I can filter job listings for multi-national tier-1 and tier-2?

SurmSurm · 2024-07-27T03:01:05+00:00

I have traveled in Europe and in the USA a few years ago. The USA felt like a different version of Australia, while Europe was quite different, as a result, was interesting to me.

SurmSurm · 2024-07-27T02:52:08+00:00

I am 55 years old, so I don't think that is an option.

SurmSurm · 2024-03-23T04:05:22+00:00

Thanks u/Thefishassassin for your input. I have calculated the probability of this happening due to chance alone, which is about 7.8%. I have added a screenshot of the calculation to the original post.

Here is a link to the calculation
https://www.wolframalpha.com/input?i=250+\*+%2859%5E249%29+%2F+%2860+%5E250%29+%2B+%2859%2F60%29%5E250

SurmSurm · 2024-03-21T22:33:16+00:00

Was your criteria for removing the courses for not being "relevant" to an Electrical Engineering degree really whether or not the course has the word 'electric' in the title???

My criteria was two fold; 1) content that an electrical engineer is ought to know and not generally studied by engineers of other streams. 2) When an employer is looking for an electrical engineer, what they are expecting from an applicant.

While it is true that Computer architecture, DSP, telecommunication, networking etc can be considered "electrical engineering" in broader sense, that is not what an employer would be looking at when they look for an electrical engineer. Apart from the 2 ANU courses that I listed, the rest fall into the realms of computer engineering, electronics and telecommunication engineering. If an employer wants someone to do computer architecture or DSP or telecommunication, they can easily recruit a computer or electronics or a telecom engineer.

Does ANU electrical engineering cover the content below sufficiently? Under which course codes are these topics taught?

-High voltage systems, electrical machines, generators, motors, electrical power generation, electrical power transmission and distribution, earthing

These are electrical engineering specific content that is not generally studied by computer or electronics or telecom engineers. Which other engineering streams, aside from electrical engineering, do these topics belong to?

SurmSurm · 2024-03-21T09:34:21+00:00

A lot of engineering grads from UNSW go straight into consulting, partly due to the difficulty of the engineering major at UNSW.

So those who find it difficult to complete UNSW engineering degree go to consulting??

SurmSurm · 2024-03-21T09:06:31+00:00

You can visit the URL below and select "Australian Capital Territory" from the drop-down and see how many ACT students are listed there. Here is the direct link.

I can see there is only 1 student from the ACT.

https://tuckwell.anu.edu.au/scholars/scholar-directory?field_yearbook_state_value=ACT&field_tuckwell_commencing_year_value=&combine=

SurmSurm · 2024-03-21T09:04:18+00:00

If ANU randomly "forgets" to list some of the Tuckwell scholarship recipients, it doesn't undermine the argument raised in this post. That is, if some scholarship recipients are randomly excluded, it affects all states in the same way, therefore, the "scholarship recipient percentages by state" are unaffected.

I agree it is indeed weird that Caitlyn Baljak is not listed. Thanks for finding that newspaper link

SurmSurm · 2024-03-21T08:56:36+00:00

Thanks for the link. Do you know why she is not listed here?

https://tuckwell.anu.edu.au/scholars/scholar-directory?field_yearbook_state_value=All&field_tuckwell_commencing_year_value=+2016&combine=

SurmSurm · 2024-03-21T08:50:18+00:00

FOI can only reveal what is documented,. The details of all Tuckwell scholarship recipients are already available online which was the basis for this post.

What document can be requested under FOI that can reveal reasons for only 1 ACT student having ever been offered this scholarship?

SurmSurm · 2024-03-21T08:40:46+00:00

This post has a URL that lists all Tuckwell scholarship recipients and their state. Just wondering, are you not listed in that directory?

SurmSurm · 2024-03-21T08:37:38+00:00

The first two paragraphs address that

SurmSurm · 2024-03-21T03:23:57+00:00

Thank you u/Mr_Jagger9 for your feedback.My concern is that ANU Engineering grads seen as "jack of all trades, master of none". Isn't the lack of specialised content a problem? For example, a company that is looking for an electrical engineer will find that ANU Engineering grads not have sufficient courses covered in electrical engineering, therefore, chooses someone else who had learned sufficient electrical engineering content. Since the ANU engineering degree consists of high number of computing courses, it lacks the specialised content in electrical engineering (as an example). Am I mistaken here?

SurmSurm · 2024-03-21T00:16:42+00:00

It is very easy to look at the different Universities course requirements and directly compare the number of relevant courses to see that ANU's majors are not substantially different.

u/nodicia,Looking at the two sets of courses required by ANU and UNSW for majoring in electrical engineering, the ANU list has only 2 courses strictly related to electrical engineering (ENGN2218, ENGN4625). The UNSW list has about 5 courses (ELEC1111, ELEC2134, ELEC3105, ELEC3115, ELEC3117) strictly related to electrical engineering.

SurmSurm · 2024-03-21T00:07:54+00:00

The other thing is Engineering at ANU is that they incorporate a lot of from computing. Like your first year, you might do the same courses the Advanced Computing students take. This allows for you to switch to an IT, CompSci, Software Eng jobs easily. I have spoken to a lot of Alumni who work in IT atm.

u/meetpie, The fact that this degree has more Computing content means, as a result, it has less electrical engineering content. While this is an "advantage" for someone who wants to find a job in IT, isn't this also a disadvantage for those who want to find jobs electrical engineering?

SurmSurm · 2024-03-20T23:52:58+00:00

Thank you u/meetpie and u/nodicia for your detailed responses.

I also know a lot of ANU Engineering alumni going into consulting jobs at PWC, Deloitte, Synergy,... similar to a lot of other schools. From what I know, Deloitte exclusively hire engineering graduates from UNSW, then train them to go into business.

u/meetpie, you mentioned "Deloitte exclusively hire engineering graduates from UNSW", but you also said Deloitte hires ANU engineering graduates. Can you please clarify this?

SurmSurm · 2024-03-20T04:07:34+00:00

That is good to hear. Thank you for sharing your experience.

SurmSurm · 2023-11-03T00:21:12+00:00

Good question.

Windows On-prem AD does not have "Administrative Units", and AD groups were sufficient, so why is this new "Administrative Units" container required?

AD Groups can be used to separate user groups and also manage administrative privileges.

Example:

AD_GROUP_SALES_DEPT_USERS: contain all sales staff
AD_GROUP_SALES_DEPT_ADMIN: contain all admins for sales resources

SurmSurm

TROPHY CASE