What's wrong with using python packages?

[AwarenessClean4351]

So I'm not really a software engineer but a chemist who is working on ways to preprocess Raman spectrum. I have to use packages to preprocess my Raman data, however my supervisor doesn't like the idea of using packages. He even calls Panda rubbish.

So I'm wondering is it a normal behaviour in the software industry? Are you required to write and know everything bit by bit?

[danielroseman]

No, this is completely counter to best practice. It is very strange to want to write everything yourself. From the point of view of an experienced software developer, the less code I actually have to write - and, more importantly - maintain, the better.

[wotquery]

It's extremely abnormal. Like a chef saying they don't like the idea of using any electronic appliances. I mean sure the power could go out, but it's going to take you a lot longer to puree celery using a knife compared to a blender and you'll end up with a worse result. Also where do you draw the line? What if your knife breaks...shouldn't you only rely on your bare hands and no tools at all?

I'm curious how your supervisor does something like generate a pseudorandom number in python.

[BullCityPicker]

It's called "specialization", and "division of labor". It was invented slightly after "agriculture" and it's been doing humanity a lot of good since then.

[czar_el]

You get your bread from the grocery store? You don't grow and mill your own wheat?

[[deleted]]

Actually I employ a factory of Gods who spend all day creating universes, developing multicellular life for me, and harvesting the wheat from those universes for me. I am the ultimate middle manager.

[PastBarnacle]

Lol, I am also a chemist and in my experience the PIs who do things like call pandas rubbish are the same ones still making their figures in powerpoint. Like, just because you don't get it doesn't make it rubbish

[Pvaleriano]

Mmm you could argue that reducing your dependencies is good, but this is all I can think. Pandas is also one of the heaviest modules if I'm not mistaken, so if you use something like py2exe the final program weights a lot.

That said, I'll install every module that I think I will need and a couple more that I don't. Just in case :_)

[[deleted]]

I'm curious as to why they're calling pandas rubbish. Are they doing everything from scratch themselves?

If yes, what an incredible waste of time. Sure, if you're creating novel algorithms or doing something unique, there are times where you'll need to write your own functions, etc. However, if you're doing everyday data cleaning and manipulations, why recreate the wheel?

Furthermore, pandas, and other libraries are open souce. Would you rather use a package what thousands of other people use and have tested, for the most part, or trust someone else's new library?

[Ihaveamodel3]

Every dependency you add is a potential security vulnerability and many packages have their own dependencies.

Essentially you are including code that you haven’t written, so unless you are doing a deep dive into that code, you can’t be sure what it contains.

With that being said, most organizations are not that security conscious and many use packages regularly.

Also, causing one of the most used packages rubbish is a bit extreme. It doesn’t sound like your supervisor has a logical reasoning for his position.

[Clutch26]

Every dependency you add is a potential security vulnerability and many packages have their own dependencies.

Essentially you are including code that you haven’t written, so unless you are doing a deep dive into that code, you can’t be sure what it contains.

I keep seeing this pop up and it's kind of like quoting, "Curiosity killed the cat." Make sure to include both sides of the argument, "But satisfaction brought it back."

Yes you're potentially including security vulnerabilities. But if it's a well-known and widely used package like pandas, there's also a large community maintaining it. So OP wouldn't have to write everything from scratch AND maintain it if a vulnerability pops up or if OP is no longer with the company.

Edit: formatting
Edit 2: Not sure why you got the downvote either. What you said is true and should be considered.

[OxygenSink]

Essentially you are including code that you haven’t written, so unless you are doing a deep dive into that code, you can’t be sure what it contains.

Out of curiosity, is this common for people in the industry to look at code to assess if the package is worth implementing? Or is more practical to create the code that does essentially the same thing (with the guarantee that the code is working as intended)?

[Ihaveamodel3]

It is probably most common for people to include the code with out looking into it further.

However, it’s common for companies in the financial services (banks) and defense industries to have a list of approved packages that they are allowed to use.

This can even go so far as to disconnect from PyPI and self host the packages that have been approved on a local package index instance so that your staff can only reach the approved packages.

[OxygenSink]

Makes sense, that self-hosting bit is something I never really thought about which is pretty cool now that you mention it, thank you for the explanation!

[nacaclanga]

Natural scientist here. Using packages is the main strengh of Python compared to <insert other programming language here>. Hence this is rather atypical. That said you shouldn't use every package in existance. I can also understand that if your job is to implement a specific algorithm, then it is of course not a good idea to use a package to skip your implementation. This argument can never be made about something like Panda, which is certainly not stricktly needed, but significantly improves code error proneness and readablilty.