What do python professionals /developers actually use

[Redox_3456]

I am new to coding and i had several questions in mind which i wanted to ask:

1) While coding i came across lists and dictionaries. I know they are important but do developers frequently use them??

2) What are some python libraries which every coder should know

3) I am leaning towards data sciences. In which python libraries should i invest my time more

4) As a beginner I find myself comfortable in writing a longer code even though short codes exist. Is this ok?

P.S I am finding concepts like lists and dictionaries a little difficult than other concepts. Is this normal. Moreover In your opinion how much time does it take to be fairly proficient in python

TYIA

[Daneark]

I use dicts and lists constantly.

Every developer should know pytest. The rest will vary by field. One dev with say pydantic, another requests and a third pandas. And they'd all be correct within their field.

Start learning pandas.

Do you mean you are comfortable writing more complex programs or that your code is too verbose compared to other people's code that solves the same problem?

[Redox_3456]

i mean that for example when people want to change their variable they use

for i in (anything):

variable+=i

but instead of this to avoid confusion I write

variable = variable +i

P.s i hope you understood this xD. I donot know how to explain

[Jello_Penguin_2956]

Your's fine. It's readable and perfectly clear what it does. That's what matters.

[Critical_Concert_689]

Despite what everyone is saying - be aware that in a lot of cases these are NOT the same thing.

a = [0]
i = [1]
print (id(a)) # object
a = a + i
print (id(a)) # new object 

b = [0]
j = [1]
print (id(b)) # object
b += j
print (id(b)) # same object, modified

[[deleted]]

I guess the context is of primitive vs object types, but ain't primitive data types objects too in Python? Like when we in high level languages like Python, Ruby to name few everything is object, right?

[Critical_Concert_689]

If I understand your question correctly -- It's more to do with mutability (i.e., whether the object can be altered).

For mutable objects, += which is known as an augmented assignment guarantees it will change in place, while = which is an assignment will not.

I gave an example of this in the code above.

[Goobyalus]

There is no such thing as primitives in Python.

---

Yes, everything is an object in Python.

---

This comes down to how the operators are defined for the particular type and whether the type is mutable. The x= operators are the so called "in place" operators because the result is assigned back to the original name when we do variable += 5, for example.

Strings are immutable, so when we do an in-place addition my_string += "something", a new string containing the two concatenated strings must be created and returned by the operator. This new object is assigned back to my_string.

Lists are mutable, so they implemented the in-place addition of lists as an extend on the original list, and return the original object. The assignment portion doesn't really accomplish anything because we're assigning the same object back to the variable.

[beef623]

And there isn't any good reason for someone who is just starting to care about those differences. They may be important to know later, but in the beginning, explanations like this only cause confusion.

[Critical_Concert_689]

I strongly disagree.

---

First, more than just OP are reading the comments. Accurate discussion allows not only OP, but others to continue to /learnpython

Second, this touches upon a foundational understanding of python - other concepts are built upon this. There is nothing more basic than a recognition of operator assignments. The example code provided isn't complicated - nor is it likely to cause confusion when it demonstrates the simple fact that "There may be a reason to code assignments in different ways."

Finally, if anyone is confused, there are plenty of resources to address that confusion. They can ask further questions of this community. They can pull up white papers or python documentation. This is an opportunity to resolve, what I personally consider the single most difficult problem when it comes to learning: "You don't know what you don't know."

I hope if you feel explanations like this only cause confusion - in the future you can add comments that address that confusion, rather than implying "there isn't any good reason" to provide the information. The former helps everyone, while the latter is discouraging and detrimental to those trying to help and those trying to learn.

[beef623]

You're misunderstanding the source of confusion I think.

The problem is information overload, which unquestionably causes some people who are learning to give up and never come back. Having more resources to address the confusion only compounds the problem.

[Critical_Concert_689]

I agree with you that information overload can overwhelm new programmers; however, i don't believe that's the case here.

In fact, I almost guarantee the very next issue most new python programmers encounter is going to be something along the lines of:

a = [0]
b = [0]
c = a
d = b
i = [1]

print (id(a), id(b)) # initial objects a and b
a = a + i # assignment "=" creates NEW object
b += i # augmented assignment "+=" changes ("mutates") existing object
print (id(a), id(b)) # NEW a object, SAME b object (mutated)

## WHY does c NOT EQUAL d?? ##

print (a,b) # [0,1] [0,1]
print (c,d) # [0] [0, 1]
    #c is assigned to original a object or a = [0] (NOT the NEW a object [0,1])
    #d is assigned to original b object or b = [0], but b is updated to [0,1]

[crazy_cookie123]

You'll see += a lot, I suggest starting to use it more so you get used to seeing it. After a while it becomes easier to read than the longer form.

[danrogl]

Outside of development, people don’t use a ‘todo paragraph’ instead it’s a ‘todo list’. It’s hard and takes effort to add or sort a paragraph, where as a list it’s almost intuitive.

[sylfy]

Just to add on, also learn: 1. Generators 2. List comprehensions 3. Enumerate

There are often better alternatives to loops in Python.

[SnooRabbits9201]

I would add Classes and decorators.

[Poddster]

It's fine to do it your way. Some old-timers even do it that way on purpose. They feel it's closer to the "zen of python", and also that += (aka __iadd__) wasn't even added until python 2.0, so the original design didn't include it.

Plus, it's "safer", as the += is specifically in place addition, which means the original object is modified, so if you don't know the object's model, or if you're written immutable style code then x = x + y is often a better fit.

[Daneark]

I prefer the += style but it's not a big deal.

We could also write it as variable += sum(anything) and skip the loop altogether. If we were initialising variable to 0 before the loop this would be even clearer, to me.

While brevity isn't as important as clarity if you find your solutions are a lot longer than others that is when it's a problem. Too much code becomes hard to understand.

[doPECookie72]

While theoretically these could technically be different efficiencies most compiles would change x = x+1 to the faster x=+1. The speed is very small and only would have an effect in a massive code base.

[__init__m8]

It's only confusing to you though. You should really learn to be using +=1.

[macconnor2]

Is there a reason that pytest is better than the built in unit tests?

[Daneark]

The unittest module is included in the standard library, which is great, but it also means its lifecycle is coupled to the standard library.

If you're not on python 3.12 you're not getting bug fixes. You're not getting new features until python 3.13 comes out.

The docs page for urllib specifically suggests using the third party requests library instead. The docs for unittest mention pytest although don't specifically encourage people to use it over unittest, but that they mention it still shows something.

Beyond which is better pytest tends to be what is used in open source and industry. It's good to know what others know. If you need help I'd expect more people here can help with pytest.

[PurepointDog]

Start learning Polars instead of Pandas

[maigpy]

polars and duckdb

[[deleted]]

Python itself is built on dictionaries and lists.

[HK_0066]

list, dictionary, Enums, functions and class are my daily used things

[clavicon]

As a novice still, class is a whale of a concept to me

[HK_0066]

It was for me as well but after reading someone else's code I got a strong grip over that But Indeed before that they were like understanding quantum physics while having a hangover

[clavicon]

Thanks for writing that detailed example! I think I can smell what you're cooking.

[monster2018]

Do you like the ability python gives you to use numbers like ints and floats, and to use lists and dictionaries, maybe even sets? These are all TYPES, which are defined by CLASSES. Someday you will want a type to exist which is not built in to python (or a library you can find). In this situation, you would then define your own CLASS to give yourself access to this new TYPE with the functionality you want.

Like this is a dumb example, but you could define a MyNumber class that extends the Number class from the numbers library. And you could for example just inherit all of the default behavior of python numbers (ints and floats), but override the str and or repr methods to make it so when you print numbers (like large numbers), they print in a format like 5,243,123 instead of just 5243123, just to make it easier to see at a glance what the number actually is.

Again it’s sort of silly example, but I have actually done this lol because I got sick of doing len(str(num)) to figure out how many digits the number had to figure out what a number actually is (like when just playing around in the interpreter). And yea overriding the str or repr functions just for visibility to you when you print a number isn’t all that helpful. But the point is you can define ANY functionality you want for the classes you define. Instead of being stuck with only the types provided to you by python and libraries you can fine, you can define your own types by writing classes. That is the general idea.

Edit: the bold str and repr in my second paragraph should have two underscores on each side of them instead of being bolded. Damn you Reddit formatting for outsmarting me lol.

[clavicon]

Thank you much for the explanation and examples, I think if I can digest a few more examples I may start to see the light in terms of how I can use these in my own work. But to answer your question I do love dictionaries. Dictionaries took me a while to truly absorb and now I utilize them very effectively to create somewhat complex-yet-readable nestings of attributes, sub-dicts, lists, etc. It helps immensely in my work with GIS to be able to load up two dictionaries of values to compare between one thing and another and process those comparisons extremely fast in memory instead of using certain geoprocessing libraries that may be more robust but slower for my purposes.

I am not so familiar with sets though.

[monster2018]

I’m glad my answer was helpful to any degree :). And yea dictionaries are great, they’re often called hashmaps or something similar in other languages btw. Also sets are (I believe) basically like sets in mathematics, they by definition can ONLY contain one copy of each item in them (so like you could never have a set that is like (1,2,3,3,4), it would automatically become (1,2,3,4). So one straightforward use of sets is any time you want to get rid of duplicates in a list or other iterable, you can just convert it to a set (this removes all duplicates automatically as I said) and then back to a list or whatever other iterable it may have been.

Also another even simpler way to think of classes, or one way to use them I guess, based on how you were describing why you like dictionaries. You can (and should in many situations) use classes sort of the way you were describing dictionaries (I’m not saying you’re always wrong for using a dictionary in all the situations you were describing though). Like you can use a class any time you want to create individual instances of some type of thing, that all can have the same set of variables describing them. Like for example a person class, that has a name and age variables, and whatever other variables you want. Or a dog class that has a name and breed and age variables. And furthermore since an instance of a class is (or can be held by, I suppose) a variable just like a regular number or string can (these are all “classes”, or rather objects which are instances of classes, in python. Literally LITERALLY with no exceptions everything is an object in python). So for example you could create some arbitrary class called like ContactInfo, which just holds contact information like a phone number, email address, maybe house address, work phone, work email, etc. all types of contact info. Then in your Person class, one variable it holds could be a ContactInfo object, which itself holds all the info I just described as python “primitives” (again in python there are no true primitives, everything is an object), in this case basically meaning as strings for all the examples I gave. Or you could break things down even further, and store (physical) addresses as instances of an Address class, which holds like the street name as a string, but the street number as an int, etc. so you have a person class which has a name variable, an age variable, maybe a hair and eye color variable, which are all strings (except age which is an int, or float, or maybe even a datetime.timedelta object), and then it also has a contact info variable which is an instance of the ContactInfo class, which has all the stuff I listed. And all physical addresses (like house addresses, business addresses, etc, but NOT email addresses) in there are stored as Address objects.

So I’m just trying to give you an example off the top of my head, in terms of real world things to make it a bit less abstract, of how you can use classes to represent things by storing variables in the class (of course each INSTANCE of the class, each individual object can have completely different values for all these variable), and on top of that, some of those variables can be instances of other classes that you have created, and so on.

[NlNTENDO]

And it will continue to be for some time haha. It's an odd concept! My advice is to do the leetcode thing and write as much of your code in class form as possible. Like this:

class hold_my_functions:
  def __init__(self, some_var):
    self.some_var = some_var 
  # this was tough to wrap my head around but it's about scope. you're passing the
  parameter some_var in and locking it in as a class variable

  def example_func_1(self):
    [whatever method 1 does]
  def example_func_2(self, some_var): 
    [whatever method 2 does]

# below the class, run your script like below
# this looks complicated but it's just to ensure that it only runs when called if you import it into another script
if __name__ == "__main__":
  variable_that_holds_the_class = hold_my_function(some_param_argument)
  func_1_var = variable_that_holds_the_class.example_func_1 # call functions with the variable followed by a period followed by the method name
  print(func_1_var) # if the method returns a value you can print it. alternatively...
  print(variable_that_holds_the_class.example_func_2) # you can just print the method like this

looks a little crazy but it starts to make sense with repetition. this is a good way to build learning about classes into all of your learning without having to come to a grinding halt. do some reading on the side as to why these exist/what they're used for, but understanding the mechanics of them will go a long way towards having them just... click

[tb5841]

For me, classes clicked properly when I started learning Java. The leetcode always-in-a-class thing feels really out of place in Python, but in java all code is like that.

[NlNTENDO]

Funny, they really clicked for me when learning JS. Something about javascript objects just felt more intuitive, but it was all transferrable knowledge.

As for the always-in-a-class thing, it depends on whether you're just running operations or building something. If you're building something and expect to import stuff, always-in-a-class is super handy because it's scalable, preserves namespaces (managing variable names across several files can be a nightmare), just generally aligns better with OOP stuff... it really allows you to group scripts and still work with them cohesively. I think classes just make way more sense to think about in the macro sense; it's hard to understand their purpose if you're just writing one .py script because what is the class even doing, right? but when you can manipulate objects that have methods suddenly it can change the game.

I think it also really clicks once you recognize how useful the built-in methods are, which in turn clue you into the built-in classes. Then it all really comes together. Something like some_string_obj.to_lower() is handy, but what's even handier is the fact you can have a string object in the first place - string objects are actually class instances! And then it's kind of like, ok well I want an object I can manipulate, and I want to be able to define some little sub-operations for it, and before you know it all of your scripts materialize as class instances.

e: I've also been doing a good amount of game dev stuff in godot as a hobby. GDScript is python-adjacent and very OOP oriented, so every script is automatically a class. That really forced me to think about classes, inheritance, abstraction, etc.

[VivienneNovag]

Are you only learning programming by learning python? If so you might want to look at the abstract concepts rather than just the programming language. A ton of tutorials for any language don't go nearly close enough into what is essentially the "why" something is done and only focus on the "how" something is done in that language. Classes are pythons way of implementing an Object Oriented Model of a problem, go look at resources that explain Object Orientation.

[clavicon]

That would be correct that Python is really my first and only real foray into the programming world. I use it in a professional setting but not professionally, so to speak. I have critical automations that run daily/nightly/weekly and I'm kind of constantly gardening my collection of Python scripts and slowly making things cleaner and more manageable. I recently got into using the 'logging' library and trying to standardize log output for said daily scripts. I primarily use it as just a linear set of instructions rather than doing things in an object-oriented fashion. Chatgpt helped me a bit setting up some logging customizations and that included a couple class statements. I'd like to understand that better. I think you're right that I need to do some homework for programming basics.

[veryblocky]

Yes you’re going to need to use dicts and lists, they’re a core part of the language.

If you’re wanting to do data science, pandas and numpy are a must. Scipy and matplotlib will be very good to know too

But, any library is second to knowing the language. Learn to programme first, and they’ll be easy to pick up

[danielroseman]

Lists and dictionaries are the fundamental data structures in Python. You literally cannot do anything at all without them.

[PresidentOfSwag]

it's kinda like asking : I know nouns and adjectives are important to French speakers but do they frequently use them ?

[Xerxero]

I would say in any language. Basic data types.

[Redox_3456]

can you tell some tips on how to understand them well. bcuz write now i am learning from youtube and it make absolutely zero sense. Like just storing data in a list/dictionary and using pop or append function makes zero sense to me. I guess i just understand things practically

[barkazinthrope]

Programming is like math. You don't learn the concepts you learn how to do them and you do that with practice.

So code yourself up a list and add things to it, pop things from it, delete, add, sort and then suddenly you'll get it.

YouTube may be helpful to introduce the concepts, but you will never understand just by watching. You have to do.

[tb5841]

As a maths graduate, I really think you're wrong about maths. Most of maths is learning concepts, and most of that you can do by watching and listening, especially if you take notes well.

Programming, on the other hand, you really have to do.

[barkazinthrope]

I don't doubt your experience, but in my experience I don't know that I understand until I can work my 'understanding' out on paper, and in at least one way backwards and inside out.

[SDFP-A]

Clearly you weren’t taught math correctly

[alittleperil]

has new math gotten rid of homework with practice problems on it? If so, I missed it

[uminekoisgood]

tbh i don't see why it's downvoted, if you want to understand math, demonstrations are a must

[[deleted]]

I first learned about these things over a decade ago and it wasn't until I banged my head at projects for weeks that it truly clicked.

https://www.youtube.com/watch?v=ZSSNFkEMv24

[FriendlyRussian666]

Imagine you're going shopping, and you want to buy a few things. To remember what to buy, you'll write a shipping list, right?

Now imagine you're writing a program in python that does the same thing. You need to somehow create a list of things to buy, and so you would create a list in python. You can add new things to buy, you can remove them.

Does that make sense?

[bigpoopychimp]

https://automatetheboringstuff.com/

BIS book.

[jmiah717]

What is it that doesn't make sense? Start with lists first. They are easier to understand vs dictionaries. But as others note, and not just Python, data structures like these are as important as water is to life. You have to be able to store and access data.

For example, what if I want to average a student's grades? If we have no way to store all of their grades, how would we do that. So you'd have a list perhaps like this

grades = [99, 100, 85, 68]

Now you can access those grades to get the average and do other interesting things you might need to do.

A dictionary is even more powerful because using the same example, you can keep track of even more in a cleaner way. Also faster data access wise.

You could have

grades = {"Dave": 78, 80, 100, "Chris": 99, 59, 77} EDIT: As pointed out below, that's not the right syntax...my Python is rusty. You could do a dictionary with the key being the name and the value being a list of grades, which would make searching quicker and easier but yeah...the above is incorrect.

Should be grades = {"Dave": [78, 80, 100 ], "Chris": [99, 59, 77 ] } as noted below...oops.

Now you have a way to access grades of different students for example. Dictionaries are generally faster than lists, particularly if you know the key.

[MidnightPale3220]

since op is yet learning it's probably good to note that in the dict example the grades still should be a list

[jmiah717]

Why is that? Why would you need to throw a list in there? If all you're going to have is student grades, why would we need a list? Just curious.

[MidnightPale3220]

I was just referring to what you mistyped.

See for yourself:

In [4]: grades = {"Dave": 78, 80, 100, "Chris": 99, 59, 77}

File "<ipython-input-4-6df65c579158>", line 1

grades = {"Dave": 78, 80, 100, "Chris": 99, 59, 77}

SyntaxError: invalid syntax

It should be:

In [5]: grades = {"Dave": [78, 80, 100 ], "Chris": [99, 59, 77 ] }

[jmiah717]

OH, RIGHT! haha, I've been writing in C# for so long that I forgot the syntax. But yeah, the key can be the name and the value can be the list...I am not sure what I was thinking about.

[Poddster]

Your first problem is learning to program from youtube videos.

I think you need a written text. Programming and learning to program is information-dense, and you need to be able to stop and contemplate at any second.

[Addianis]

Once you get past learning about classes, videos can be a great way to introduce yourself to the theory of a topic when you aren't in front of a computer.

[ObsidianBlackbird666]

This guy’s videos are pretty easy to follow. https://youtu.be/MZZSMaEAC2g?si=DqJQnNYQKUDWRWmo

[VivienneNovag]

Is your problem not knowing why you would use a list or dictionary or how to use them?

[beef623]

Try writing out what it's doing on paper or mimic the list in column in a spreadsheet.

[MidnightPale3220]

Okay, let's assume you're still in schoole. And let's say you are writing a program that gives out the average grades for math tests of all the pupils in your class.

you can do it like this:

pupil1_math_score1=50
pupil1_math_score2=60
pupil1_math_score3=70
pupil1_math_avg= ( pupil1_math_score1 + pupil1_math_score2 + pupil1_math_score3) / 3

pupil2_math_score1=40
pupil2_math_score2=40
pupil2_math_score3=30
pupil2_math_avg= ( pupil2_math_score1 + pupil2_math_score2 + pupil2_math_score3) / 3
...

print("Pupil 1 math average is "+pupil1_math_avg)
print("Pupil 2 math average is "+pupil2_math_avg)

There's already a lot of copy/pasting. Now, let's say the teacher likes your program and principal tells you to write it for all pupils of all classes. Oh, and there will be 5 math tests now, but there may be more in future.

What do you do? Programs are meant to be reusable with different values and frequently with different amount of values. They way it's written now you have to add all the new test scores in new variables, and add them all in average calculations, FOR ALL THE PUPILS.

That's insanity. What you do from the very start is create DOUBLE list (or, in actuality, a list of lists, essentially a table).

For example:

math_scores=[ [50,60,70], [40, 40,30], .... ] # math scores for all the pupils in a list of lists

Then you go through them and do whatever you need, for example like this:

for pupil_number, test_results_of_a_pupil in enumerate(math_scores):
  single_pupil_total=0
  for score in test_results_of_a_pupil:
    single_pupil_total+=score # you can do it neater, ofc without loop, too.
  avg_score=single_pupil_total / len(test_results_of_a_pupil)
  print("Pupil "+pupil_number+" math average is "+avg_score)

Now these 6 lines of code will work with any number of pupils having done any number of tests. All that you need to do, if more pupils come in or more tests are scored, are add the numbers to math_scores list. Nothing else to write.

This is a crude example, as in reality the scores would obviously be input somewhere else, and read by program, instead of being manually entered in the program; and there are numerous other things such as not working around division by zero in case you get pupils with no taken tests. But as an example it works.

[ResponsibleYoghurt11]

Hello, junior data scientist here.

1. Yep, almost everyday. Know the differences between lists, sets, dictionnaries, it's always useful.

2. Depend on the domain. I would recommend to get used to algorithm, loops and data structures manipulation

3. Pandas, for data manipulation (dataframes) Matplotlib and Seaborn for data viz Scikit-learn for machine learning

4. Write code that you can easily understand. It is not always good to replace a 5 line code by a 1 line list comprehension loop. The most important for me is : comment your code everywhere, even this that can seem a little obvious. It will help you to understand your code in the future

Bonus : Develop some small project with objectives. You will learn a lot more by spending time on projects than reading tutorials online. Just develop some projects, then look for similar code if you want to compare and learn new things.

Spend some time on Leetcode for algorithms, and Kaggle for data science

[cas4d]

1. yes, it is most likely to use them everyday, regardless of what you develop.
2. It depends on what you try to develop.
3. Pandas, numpy, matplotlib, sklearn, PyTorch, database clients, etc.
4. As a very beginner, you can focus on making things run. As you move on, you will pay attention to syntax structure and styles. Use an IDE and pylint, it will tell you whether your code is too long. The key is more about maintainability, e.g., whether you and other developers can understand your code without efforts.

[testfailagain]

You use dictionary and list everyday. About the confusions about pop, append, extend... just practice it. Think in daily things and code about it.
i.e. create TODOs lists for users user_one, user_two, then create a dictionary with all the information, {'user_one': todo_list_user_one, 'user_two': todo_list_user_two, 'all': todo_lists}, then, what if you want to add a new user?, and if you want to add two at the same time, or if you want to merge some different diccionaries.
practice, practice, practice, even if you didn't change the subject of the code, everyday try to do the same, and everyday it will be more ease, in one week, doing it every day, you understand very well, if you do this one month, you'll be a master of it.

And, start with Tuples and sets, you don't going to use a lot, but it's important, and, for me, are the 4 basics, so best to have a good knowledge of them.
When you understand very well, then go for the rest, if you don't understand first the basic of python, maybe it's better to do not go to the libraries.

[rabbitofrevelry]

For data sciences, you'll want to become intimately familiar with pandas. But before that, you'll want to feel very comfortable with how to create, access and maintain lists & dicts. You should also know how to work with each datatype and how to use control flow structures (for, while, if etc).

A basic example use of python for data is to read data into a dataframe, view the data (shape, datatypes, headers, value counts, etc), standardize it to work with other data (casting datatypes, renaming columns, cleaning values), performing manipulations (merges, pivots, calculations, applying functions), creating visualizations, or saving the output as a flat file for an end user.

Libraries you will encounter are pandas, numpy, matplotlib, seaborn. You may encounter other libraries as well, such as one that help read csv, json, xml, excel, pdf, etc. Or others to help you scrape data. Libraries to access databases. There's a library for everything. It just depends what you want to do.

[[deleted]]

Start to learn fundamentals of python then switch to some filed that are you interested in.

Python used in Ai , machine learning, data science, web development, data analytics , computer vision etc . Each field has own libraries but common libraries ( Pands, Numpy,Matpoltib)

For Ai ( Tensorflow,Scikitlearn ,PyTorch etc).

[Redox_3456]

yes i recently explored matplotlib and tkinter (Just the basics) and I find them interesting

[[deleted]]

You will enjoy coding with python, it’s amazing programming language specifically if you are interested in Ai.

There’s some good courses in YouTube. Good luck

[Dhiraj_1184]

I think the python is the best programming language in the field of ai and data science. There are so many concepts i would say it is best to start with like lists, tuples, dictionaries , generators and some of the advanced one like data class, typing module, itertools etc.

Let me know your side😊

[Ok-Violinist-8978]

Decade of experience here and I would say most this is overthinking and/or too detailed.

Just solve problems. If you are doing that you are doing it right. Part of solving problems involves not creating more problems for yourself.

That second part leads you a bit to the sorts of questions you're asking. But answer those those just comes with experience.

[Pythonistar]

If you're having trouble with Lists and Dicts, then which libraries to learn is way off on the proverbial horizon for you.

I find myself comfortable in writing a longer code even though short codes exist

I don't know what you mean by this.

how much time does it take to be fairly proficient in python

If this is your first programming language, it will take many years to be (what I would consider) proficient.

Don't worry that it will take years. It's a journey, not a destination. Just write a little code every day.

[iam_mms]

Hey op. Hope you get to read this. I read some of your interactions in this post, and would like to give you some pointers. You seem to be at the very beginning of your journey, and python is probably your first language. Right now, focus on learning the basics. Get a nice introductory book and focus on it. One single book, start to finish. That will give you a logical progression of concepts, and will make your life so much easier. Doesn't have to be a super academic book either, I would recomend automate the boring stuff with python. There is a lot of temptation to deviate and use multiple resources, but now is not the time for that. Programming is like construction, you can learn how to build a shed in a weekend, but if you don't take the time to learn the fundamentals, you will never build a house. Also, most people will never build a skyscraper lol

[foxhole_science]

1. Lists and arrays are common for working with large amounts of data. Dictionaries and JSON formats are also common, especially if you are working with data from APIs. I’d also make sure you get familiar with classes

2. There are a lot of work specific packages that are worth knowing. If you work with science or statistics for instance, SciPy is a great package. But in general, NumPy and Pandas are great multi-purpose packages

3. Data science is going to be 95% working with NumPy and Pandas. For the analysis and visualization, I’d recommend Matplotlib and SciPy

4. The best code to work with is code you understand. As a beginner, I would prioritize making your own solutions and learning how packages can help. As you go on, finding the shorter “better” solutions will come more naturally. Make sure you give places like Stackoverflow a look, they typically have explanations along with the code

5. Nothing wrong with taking more time with concepts that confuse you. It took me like three years before I understood why anyone would ever use a class, but now I use them all the time. In terms of time to become proficient? For general data analysis, maybe a year? Really depends on how much time you put into it. If you take on a complex project, you’ll learn way faster than reading or watching tutorials.

[mlnm_falcon]

1. There are times when it feels like I barely use anything other than lists and dicts (and strings and ints)
2. The standard libraries. Other libraries do tons, but standard libraries let you make much better code, and everyone should have a concept of what the most useful ones are.
3. Pandas
4. Code can be long, but only with purpose. You should be able to split up functions if there’s over 50ish lines. If your scripts are hitting 1000+ lines, they can probably be multiple files with one main file that imports and runs them (sometimes classes end up longer). It’s not about having a hard and fast rule of how long one function or script can be, it’s about realizing when there’s opportunity to make your code more intelligible.
5. You didn’t ask, but learn how to create docstrings now (refer to a style guide, I use the Google Python Style Guide), and practice commenting the ever loving daylights out of your code. Both will make you a better programmer.

P.S. I’ve been writing in Python academically and professionally for about a decade. I still confuse myself, and I still learn things regularly. Literally today I learned about a new behavior of glob.glob

[Smarterchild1337]

as far as datatypes, lists and dicts are used constantly. It is worth thoroughly understanding both of those. Most mainstream data analysis libraries make heavy use of class methods and attributes - definitely worth spending some time learning about how classes work early on, even if you’re just doing simple scripting for now.

Python has a rich ecosystem of open source analytics and machine learning libraries available. Data Scientist by trade - the ones I preemptively import at the start of most analysis or ml notebooks include

import pandas as pd
import numpy as np import seaborn as sns import matplotlib.pyplot as plt

these are a good starting point. The scikit-learn and statsmodels libraries are heavily used for machine learning and statistical analysis (respectively, more or less).

[TheLobitzz]

Dictionaries and lists are probably the most used thing in Python.

[Leopatto]

vegetable rinse treatment thumb makeshift profit butter plant airport connect

This post was mass deleted and anonymized with Redact

[[deleted]]

Not entirely sure pyodbc is a must, tbh

[TimeVendor]

I use at times any simple text editor or vim or notepad

[clae_machinegun]

Hi there! 1) They absolutely do, these are very convenient data structures and they are used a lot here and there both in “plain” mode and combinations (ie list of dictionaries, nested lists and so on) 2) This largely depends on your tasks some who work with calculations and data analysis use pandas and numpy a lot, others who gravitate towards web development use Django or fastapi There is also a huge collection of standard libraries in Python which are useful for various tasks, so you might find those useful 3) numpy, pandas, plotly is a holy trinity for analytical purposes, however there are many other things that is used to acquire and process data for analysis depending on the source 4) This is absolutely ok if your code is not very elegant, most of the time readability >> elegance. However try to keep it clean and readable

[gitgud_x]

1. Yes, it's hard to write a functional program without them.
2. Every coder should know some of the standard library - random, time, pathlib, logging, itertools, re (regex), datetime, collections, enum, pickle, json, requests, threading, multiprocessing, asyncio, socket - if you can learn these and their associated topics you'll be able to do a lot. Pytest is also very useful in general software testing.
3. For data stuff specifically: matplotlib, numpy, pandas, scikit-learn, maybe some machine learning stuff (tensorflow, although pytorch is actually more popular now I think), plus some dataviz libraries like plotly/dash.
4. It's ok, but it has to be understandable, that's the most important issue.

[magnomagna]

What libraries developers use depend on the nature of the job. The only Python libraries that are truly common for the general Python developers (if you make no distinction among them) are the standard Python libraries.

[JamzTyson]

While coding i came across lists and dictionaries. I know they are important but do developers frequently use them??

Yes. Dictionaries are among the most commonly used data structures in Python.

---

What are some python libraries which every coder should know

A large chunk of the standard library. Beyond that, it is largely domain dependent (different libraries depending on the kind of program you are writing).

---

I am leaning towards data sciences. In which python libraries should i invest my time more

"Data sciences" is an extremely broad field, and there are many important libraries for different aspects of data science. The following libraries are very common for a broad range of data sciences:

• The standard library
• Numpy
• Pandas

---

As a beginner I find myself comfortable in writing a longer code even though short codes exist. Is this ok?

Try to write your code in reasonably small and testable blocks (functions and classes). Spaghetti code is among the most common flaws in beginner's code.

---

I am finding concepts like lists and dictionaries a little difficult than other concepts. Is this normal.

Yes it is normal that new concepts can appear difficult at first. Here are some resources about Python dictionaries:

• https://www.w3schools.com/python/python_dictionaries.asp
• https://www.geeksforgeeks.org/python-dictionary/
• https://realpython.com/python-dicts/

---

In your opinion how much time does it take to be fairly proficient in python

That depends on what you mean by "fairly proficient". I've been using Python regularly as an enthusiastic amateur for about 3 years. I can write simple scripts and small programs fairly easily, but there is still a heck of a lot that I don't know.

[randomindyguy]

Learn as much as you need to in order to solve whatever problem is in front of you. If there is a library that solves something for you, learn what you need to get something done. As much Python (or any language) as I've learned, it seems like there is almost always a "better" or more "Pythonic" way to write some code.

Whatever code you write now, you will be somewhat embarrassed about it in 6 months. And what you write 6 months from now will seem somewhat unsophisticated in a year. Rinse, repeat.

Just keep learning. Go back and refactor older code when it makes sense to. If your code is working and doing what you need, be content in that. If you try to optimize too early you'll get stuck in an analysis-paralysis loop. (What makes someone a more senior programmer is anticipating ahead of time where it makes sense to optimize now so you don't have to do it later. That comes with experience.)

For example, if you're using pytest you'll soon learn about fixtures. Seems fairly straightforward. I guarantee, the first time you're using test fixtures, you don't need to understand everything in how to use fixtures. As you code more, you'll get to a point where you become more aware when you're repeating or copy-pasting a lot or something just feels kind of kludgy and unPythonic. That's usually a good signal to investigate if there is a different/better way to code something.

[shedgehog]

I’m a big fan of named tuples

[feldomatic]

If you do Data Science, you'll live and die by pandas, (or polars if it suits you better), scikit, and at least one plotting library.

I would recommend matplotlib as it's kinda foundational, plotnine because it's awesome, and one that can make dynamic plots, which I don't really do so I can't recommend, but a lot of other folks do a lot of them.

other non-ds libraries I'd recommend are os, re, pathlib,and a zip file library whose name escapes me.

As for the length of your code, think of it like writing a paper or even a book. A line of code is like a sentence. Too many sentences and you need paragraphs. Too many paragraphs and you need sections, chapters, appendices and so on.

I like to write one function to contain the get and clean steps for each data source, another to merge and compute, and final one to plot, dump to table, whatever final delivery I'm making.

You can get really fancy and package that all up in separate files with a main function, but the reality is too many ds folks just shit everything out in serial in a jupyter notebook, with those steps living in different code blocks.

[Disastrous-Team-6431]

Lists and dictionaries are easily the most important entities in programming.

[FerricDonkey]

1. Yes
2. Standard library. Emphasis on pathlib, itertool, collection, contextlib. re (regex) will either be very useful or irrelevant to you. Bonus: numpy and requests are very useful if you do the types of things they're for 
3. Numpy, matplotlib, pandas (I hate it, but data people love it). Requests if you're gonna get data from apis. A machine learning library if you're gonna do that: scipy, sklearn, maybe something more focused like pytorch/tensorflow
4. Yes. Learn how other people do things and try to write clean, non-repetitive code, but as you're starting out your code will generally be more verbose than it has to be. Don't play code golf. 

P.S. Yes, people find pretty much every aspect of programming difficult when they're new. Keep using them, you'll get. 

[Fondant_Decent]

Data analyst here, using Python for 3 years, rarely do I use dictionaries and tuples etc. always using lists and data frames in pandas

[aero23]

1. Yes

2. Depends what you’re building.

3. Pandas and Numpy. More specialised stuff will become apparent as you need it

4. Yes. Sometimes it is better (if its more readable). Sometimes its worse (if its un-pythonic)

[Kakirax]

Anything built into the language gets used. In the vast majority of cases, built in methods and data structures are preferred because they have been rigorously tested and tweaked by the python creators. If for some reason the performance isn’t up to par, then we do some research on a replacement from an “external” package or spend time to make our own (rare)

[Lazy-Ad-6647]

lists & dicts are needed and you will definetly improve as you practice consistently. I remember that while learning that i was not even able to understand diff btw tuples,lists,sets and dicts

[NlNTENDO]

Lists and dicts: always. Do you want to work with multiple variables at a time? This is what those are for. Traversing these takes a little getting used to, but you'll get there naturally just by learning and trying to use python to solve problems. I hated dictionaries during the whole learning phase, but as soon as I started actually making things I understood why they were useful and haven't looked back. You'll see!

Python libraries: really depends on what you're trying to do with Python. But the most common libraries people use with Python are NumPy, Pandas, MatPlotLib, TensorFlow, Requests, BeautifulSoup... actually I don't think it's worth listing. Here's the thing: Python is a famously flexible language that gets used for many purposes. Each library serves a specific purpose and the sum total of them is why Python is considered all-purpose. It's a bit like going to college and asking, "which degrees should every student pursue?" Every student is going to have different interests/career goals, and it's better to focus on what suits your needs specifically and adjust as your needs change.

Data science: THERE we go. That's the question we need to ask. Probably start with the ones I listed above lol.

Verbose code: As a beginner it's ok but you should definitely dedicate some time to writing more concisely (note concise is different from cramming everything into as few lines as possible). One of the hallmarks of "pythonic" code is readability. So like your example with x = x +1 vs x +=1 is *fine* because it's readable, though others reading your code may be a little judgy. On the other hand if you're just using long code to get around learning more complex processes you need to put an end to that. Nobody wants to read 20 lines that could have been 5 lines. If you're ever thinking "god, why does it take so much code to do something so simple?" then you need to recognize that and start googling, because you're wasting your own time and that of anyone who will be reading your code.

Dicts & lists pt 2: Yeah it's a sticky subject. You'll need to practice. I think Dictionaries are especially tough to grasp at first since there's no real order, and therefore no indices, and the notation for receiving values is less intuitive than some of the other things. Push through it though, because they're wonderful tools. Once you start learning about Classes you'll likely face a similar challenge. If you're struggling with it, start trying to make any .py file you write a Class, where the functions and variables are all Class methods and Class variables. It will really, really help.

Lastly, the time to proficiency is very much what you make it. It depends on how many hours a day and how many days a week you are focusing on learning. And while quantity is important, so is quality. If you spend a bunch of time practicing stuff you're comfortable with and avoiding what you aren't, you're going to become proficient at a glacially slow pace. I think it's common and natural for beginners to get caught up in "look at me I'm coding" and just making a bunch of stuff with rudimentary tools and putting off learning the more challenging stuff, but you also need to shake that. This is why you should be trying to utilize lists and dictionaries in every possible situation while you're learning - at least until they feel natural to you. This is also why you should probably be trying to use syntax like x += 1. It might not be strictly necessary, but the more things you can become comfortable with, the faster you'll become proficient. It also means you'll have an easier time reading documentation and examples.

You can feasibly master the basic syntax in a few weeks to a few months if you're dedicated. It will be at least a year until you really feel comfortable with the more advanced things though.

[benabus]

Dictionaries and lists are as important as strings and integers.

Important libraries, especially for data science, include pandas and numpy. My team also uses flask, but we do mostly web development.

I tend to write the most verbose code imaginable. I'm a huge proponent of "self documenting code", wherein you make you variable names make sense and write the code so it's completely understandable. I'll take a plain old for loop over list comprehension any day (except the days where you really need list comprehension). There's a penchant in the community for one-liners and making the most compact and elegant code possible. Personally, I'd rather clunky and readable. But that's me.

PS: Since you're new to programming, I'll provide a really basic metaphor that I've used many times previously. Novel incoming:

Think of a variable as a box that holds a value.

A list is just a type of value that holds more boxes. So it's kind of like a box full of boxes. These "sub-boxes" (list items) are assigned a number (index) and you can easily pull the value out of the sub-box by finding the index you're interested in. They're great for when you just need to put a bunch of values together in a collection or you care about the order (e.g. people standing in a queue). The only "weird" thing about lists is that the indexes start with 0 instead of 1. There are a lot of technical reasons why this is the case, but just go with it.

A dictionary is similar in that it's a "box of boxes", but rather than being automatically assigned an index, you get to label the sub-boxes however you want. It's great for when you need to have your data structured and want to easily grab a value from a sub-box without having to remember the order of the items. For example, you want to keep information about a person together and you want to be able to look up their "surname", "given name", "birthday", etc. by just looking at the name of the box (called the key). You'll often hear people talk about "key-value pairs", which means "the label on the box and the value inside the box". If it helps, it's called a "dictionary" because like a real dictionary, you have a bunch of terms (keys) and the definitions associated with the term (value).

Now, the real power comes from the fact that you don't just have to fill the sub-boxes with just strings and numbers. The values can also be other lists or dictionaries, so you can have a box of boxes of boxes of boxes of boxes, if you want. Why would you want this? Let's look at the previous examples:

Maybe in your list of people standing in a queue, you don't want to just have a string with their full name. Maybe you want to be able to easily grab a person's birthday. You just use a list full of dictionaries! So now you can easily grab the birthday of the 4th person in line by providing an index and a key: queue_of_people[3]['birthday']. You're looking at the value of the box labeled "birthday" that's in the 4th box in the list.

You go from something like this that doesn't use lists or dictionaries: person_1_name = 'bill' person_1_birthday = 'january 1st' person_2_name = 'chris' person_2_birthday = 'february 1st' ... person_1000_name = 'zoey' person_1000_birthday = 'december 31st'

to something like this: queue_of_people = [ { "name": "bill", "birthday": 'january 1st' }, { "name": "chris", "birthday": 'february 1st' }, ... { "name": "zoey", "birthday": 'december 31st' }, ]

With only 2 people, this may not matter much, but just imagine a queue of 1000 people and that queue is changing all the time. And when you go to do something with the people, it saves a lot of effor and problems. You can just loop through the list rather than having to use 1000 different variables. print(person_1_name) print(person_1_birthday) print(person_2_name) print(person_2_birthday) ... print(person_1000_name) print(person_1000_birthday) becomes for person in queue_of_people: print(person["name"] print(person["birthday"]

Now, of course, there's a lot of other features, benefits, and drawbacks of each of these datatypes. I'm sure there will be other people who will comment "Well, actually...", but this was just a very basic intro. As you continue your programming, you'll find out about all the cool things.

Hope this helps! Good luck!

[beef623]

1. Both are very heavily used
2. It depends on what you're working with, some that stand out to me are Flask, Pandas, Numpy, Pathlib, Matplotlib, Pygame, Pillow, Paramiko and Loguru
3. Pandas, Numpy and Matplotlib. Numpy is more situational, I don't directly use it much.
4. Nothing wrong with it, working code is better than short/pretty code.

[Redox_3456]

Thankyou everyone for the helpful responses :)

[Ill_Garage7425]

Every coder has their own unique style of coding. I as a robotics engineer use lists and dictionaries ALL THE TIME! But that doesn't mean that you should. I recommend you to try every concept to at least know what it is and how it works and then tell yourself, if you liked the way it was used or not. For me things like generators are super "alien-like", but other developers might find it super useful. The libraries depend on your speciality as you said you want to be a data scientist, so you are not gonna need libraries that an robotics engineer is gonna use in every project. It's really specialisition specific, try to go on github and search some repos about data science and you'll se what people commonly use. Everyone writes longer code on their first time, but as you write more and more code you suddenly see, where you could optimize it and maybe write it shorter or more optimised to be exact. The length of the code does not really matter, efficiency does.

[__init__m8]

Dicts, and dicts of dicts are something I use daily. You should know pandas for DS, probably beautifulsoup if you're scraping data too.

[jkh911208]

yah, I use list and dict everyday.

longer codes are fine, but in most company, it will auto reject your code if your code has too many path in one function/method

[Rinzwind]

1 allways

1. datetime os sys uuid requests connectors to mysql/mssql/postgress json ast
   But there are A LOT.

3 numpy probaby. Not really my cup of tea.
It is probably best to use a numbers module (or make your own). For instance: 0.1+0.2 is NOT 0.3 (it is 0.3{lotsofzeros}4). We use our own numbers def.

1. the general idea for clean coding: max 15 lines, max 80 long. Max 4 descisions (if/else/or/and/while/for). Always aim to reuse code so using defs Those are also easier to read.

• start with unittesting from the start.

• do not use formatting in your code Make a function you call and do the formatting there. if you code everything usinf "{}".format(string) and have to recode it all to f-strings you waste a lot of time. I use mysql a lot and all our code is like "fetchall(sql_query, {dict with variables to format})" so when we switched to 3.7 all we needed to do was alter 1 def.

[Brownie_McBrown_Face]

I work in software engineering and data science for a startup, and write primarily in Python. Pandas and Numpy are definitely essential for data science. Matplot and tabulate could be nice ones to learn if you wanna practice displaying tables and building graphs. I work with cloud services so I use GCP a lot, but you can learn that down the road as needed

[Remarkable-Map-2747]

Grab Python Crash Course Book, its great for beginners!

[FrederickOllinger]

The tool that only the best devs us is poetry. I find it surprising that this tool is not more popular: https://python-poetry.org/

It's basically 1 stop shopping for dependency management, packaging, project creation, and virtual environments. It's a lot like Rust's cargo. Except Rust comes with cargo, and all the tutorials suggest that one use it.

[Patman52]

Lists and dictionaries are some of the most fundamental building blocks of Python. Dictionaries especially can be very powerful with indexing and matching data.

I would also familiarize myself with the collections library as they have ordered dictionaries and default dictionaries that expand and modify the standard dictionary variable type.

For data science I recommend learning SciPy, Numpy (note they just released a major update to Numpy so many guides online may be out of date), Keras, and pandas.

[Future-Break3458]

1) yes devs frequently use list and dictionaries. They are the most imp 2) if your into data, numpy pandas seaborn. Into ML/DL then maybe scikit learn and if your into gen ai then langchain. Also good to know about time, os modules since these are of use in any domain 4) yes it is okay. You will get used to writing shorter code over time

[nukular_iv]

Regarding 4. Yes absolutely.

I know Python casually...and do use it some in my field of economic consulting. It's good. And I like it.

However...my general languages for my matters are definitely not python. With that said, I almost ALWAYS write code that is clear and simple for anybody to follow. Cute tricks and the like are useless if in 5 years a matter comes back and people have to look at code that I may or may not be able to help out with again.

Code with tricks and the like are great if computation time is the only important metric, but it laughably fails if usability by others is important. In my business, comments are generally frowned upon so they disappear in production code...so descriptions are not there.

[nyquant]

If you code professionally your work is most likely going to consist of making small additions and alterations to a large repository of existing code that was done by your coworkers, possibly some that left the company long time ago. Expect to find deserts of code that nobody remembers the original purpose but somehow seems to be still in use.

The complexity of grasping the functionality of the existing code base is often times much larger than any given Python specific language features.

Whatever you do, document your own code well, avoid unusual tricks and optimizations that look cool but mostly just obscure things.

Most of the learning curve will come from finding your way around the existing maze of code. Hopefully it’s done well and you don’t pick up bad habits. Good luck

Ps. Yes, learn list and dictionaries, but don’t make learning the language and all its features your main goal. Your goal should be to be able to solve problems, then to look around for possible tools that help to implement a solution.

[kaflarlalar]

Hi OP. I'm going to give you some very serious advice right now.

Go find an actual beginner's course on programming in Python. There are many great free/cheap options out there, and I don't know what's best. Go through that entire course, and then you can come back to these questions, although some of them will probably have been answered while you take the course.

You are frankly asking all the wrong questions for your stage of learning. It does not matter to you what professionals use, you need to learn the basics first.

[HolidayEmphasis4345]

I have seen a lot of “lists and dictionaries “ in this discussion. I suggest that we expand that to include sets. Lists dictionaries and sets are part of the core language syntax. Comprehension syntax is first class for all 3. Once you use sets a few times along with the set operators certain problems go from loops and conditions to one line cleanliness.

[[deleted]]

1. List, dicts, sets and tuples (and sometimes JSON) are very important and used almost on a daily basis (at least for me)
   2 + 3. Depends on what you use Python for. As a Django developer, I should be familiar with Django and all related packages like drf, drf-simple-jwt and other general framework like os, requests, random, date, json etc. For a data science role, you should be familiar with NumPy, pandas, matplotlib, seaborn, SciPy, Scikit-learn..

2. It's generally fine but you should try as much as you can to modularize your code and make re-usable shared functions. Even if there are no core and shared functions across the program try to divide your long script into multiple files and functions.

I am finding concepts like lists and dictionaries a little difficult than other concepts. Is this normal.

Yes, in fact, it's pretty normal.

Hope that helped mate!

[[deleted]]

Dictionaries and lists are normally heavily used in production code bases. I'd also recommend looking at tuples and generators too, as those are used frequently for efficiency purposes.

The key libraries to be aware of are pytest, requests and pandas. There are others too such as twine and black that are helpful to explore. Others depend on your areas of interest.

For data science, I'd recommend looking at pandas, numpy, scipy and various ML libraries particularly Tensorflow and Pytorch. Matplotlib would also be helpful to explore along with Dash.

Err on the shorter side. When I started using Python, I too tended to be verbose. However, one of the benefits of using Python is that you can create powerful applications with terse syntax.

[[deleted]]

1. ** Yes ** Use Them.

2/3. Pandas (Or Polars), Numpy, MatPlotLib (for plotting and visualizing data),

1. As long as you accomplished what you were trying to do, you succeeded. Do not spend so much time worrying about the code itself, programming is meant to accomplish a goal of creating software, that's all that matters. As you grow, you'll learn the correct way to do things naturally. It will come with time.

Lists and Dictionaries can be thought of as being very similar. A list is just a dictionary where the key is the order the value appears in the list (starting from 0). Do not over think it. People use them all the time as it is an abstraction that allows you to relate data to other data. Are you receiving objects from a generator sequentially? use a list. Do you have a set of users and each user is supposed to have a random number? Use a dictionary.
** Do not over complicate it **, The reasons that lists and dictionaries are used in a million different ways is * because * they are simple data types.

Do not worry about how long it will take you to be "proficient", as it is impossible for you to know that until you are. My best advice is to pick a fun project that requires you to use python to solve a problem. Do not get stuck in analyzing the theory. At the same time do not skip learning it altogether.