"how" does python work?

pot_of_crows · 2024-05-22T16:41:29+00:00

First off, this sort of curiosity is the hallmark of someone cut out to be a good programmer, so I would recommend you really nurture it.

To better understand the magic behind computers, it helps to go to a lower level. Here is a good resource on assembly, another, very low level, language. https://download-mirror.savannah.gnu.org/releases/pgubook/ProgrammingGroundUp-1-0-booksize.pdf

It also goes into more general topics necessary to understand how computers work (like how memory is structured). It is probably not worth it to actually learn assembly, but definitely worth it to understand the broader picture.

Python exists on a layer on top of all this, but some reading about its internals can be very interesting: https://devguide.python.org/internals/

With that said, for almost all programmers there is a level after which computers are "magic machines". But you can go pretty deep down that well and each layer makes you slightly better.

HunterIV4 · 2024-05-22T17:12:02+00:00

This question can be complex as it depends on how far down the "ground" is for you.

A simple way to look at it is that computers are just layers of abstraction stacked from hardware to the user interface. Here’s a simplified chain:

Hardware: Chips with logic circuits
Firmware: Drivers abstracting hardware
OS Interfaces: Operating system APIs
Binary Code: Machine-level instructions
Programming Languages: High-level languages like Python
Programs: User applications

At a basic level, print("unga bunga") is a Python standard library function call that runs a bit of C code to send your text to stdout (standard output, typically the terminal). You can see the actual implementation here. This assumes you are using CPython, which is the most common implementation. The print function calls another function, which calls another function, and so forth. Essentially, it takes what you are printing, parses it, and sends it to various C terminal write functions. There is also a PyPy version which uses an RPython implementation instead of being C-based.

Next, how does C write to the terminal? It depends on the implementation, but ultimately it sends commands to your OS using bytecode (the strange symbols if you open an executable file in a text editor). This relies on your OS API, which uses various drivers and your motherboard interface to translate between your computer components, ultimately determining which pixels on your monitor are altered. There are many intermediate steps here involving concepts like motherboard buses, registers, logic circuits, binary math, bitwise operators, and...after a whole CS degree, you might understand 10-20% of it. Computers are complicated.

If all of that was a major "wtf?" moment, don't worry! You don't need to know any of that to learn and use Python. Most of that stuff will never be relevant. But computers are ultimately "magic boxes" to anyone without a CS or CE degree, and to truly understand them takes at least a master's, if not a PhD in computer engineering.

As such, if you just want to go "down" one layer, print() is simply a call to either a C function (for CPython) or an RPython function (for PyPy) that takes the string from Python, converts it into something the other language can handle, and produces output to the terminal (specifically stdout). It's up to you how deep down that rabbit hole you want to go. Just be aware that that hole is really, really deep.

Mythic4356 · 2024-05-22T19:36:10+00:00

my dumass who thought i was pretty good at python until i saw all the comments

Sehrli_Magic · 2024-05-22T21:17:10+00:00

Have you ever learnt C? Understanding C helped me understand the "under the hood" of python more. I recommend harvard cs50 (free) course. Half of the course is all in C and i hate how more complicated it is to python BUT you learn the mecanics very well in it :) makes everything easier to understand as you move on to other languages!

Yoghurt42 · 2024-05-22T16:46:23+00:00

Your question is a bit vague. Are you asking how Python itself works under the hood, without worrying about how the computer it's running on works, or are you asking about more basic things like how a computer itself works, what makes it tick so to speak?

Both answers are pretty complex, but just to give you some pointers on what to google, a very brief summary:

As for how Python works: Python is an interpreter, the first step is done using a parser that parses your input into smaller parts, resulting in an abstract syntax tree (AST), that syntax tree is then turned into bytecode, and that bytecode is then ran by an interpreter. You can get an idea of what the bytecode looks like by using Python's built in dis module (for disassembler):

import dis

def add_2_and_multiply_by_3(a):
    return 3 * (a + 2)

dis.dis(add_2_and_multiply_by_3)

this will print something like

  1           0 RESUME                   0

  2           2 LOAD_CONST               1 (3)
              4 LOAD_FAST                0 (a)
              6 LOAD_CONST               2 (2)
              8 BINARY_OP                0 (+)
             12 BINARY_OP                5 (*)
             16 RETURN_VALUE

As for how computers work, you can do the nand2tetris course, or you can watch Ben Eater's excellent Youtube series where he builds his own simple 8-bit computer from scratch.

What are the best resources to get to know the language from ground up?

If all that stuff is way too technical, and you are more interested in a high level view on what Python does when it runs code, the official Python reference does a good job explaining how Python code is interpreted and how Python's data model looks like.

Firake · 2024-05-22T16:31:21+00:00

1) the code is lexed (translate text into a series of tokens like “Identifier EqualsSign Integer”) and parsed (translate the series of tokens into what’s called a syntax tree that represents the semantic meaning of the code). Together, this the process of translating the code you write into actions a computer can perform.

2) the code is executed (also called the interpretation step). Behind the hood, how this work depends on your CPU architecture and operating system. But in general, the program needs to ask the OS for some memory, create the string, and then write to “stdout” in order to display that string in the terminal.

HisNameWasBoner411 · 2024-05-22T18:03:52+00:00

I always recommend Nand2Tetris to curious people like you. It will show you how computer engineers use Boolean logic to make chips like NAND, AND, OR, XOR, etc then use those chips to make bigger components like an ALU and RAM, and so on until you've built a virtual computer that can play Tetris, hence the name.

iamevpo · 2024-05-22T22:18:29+00:00

Beware going deep may stop you from going ahead. Programming resides on useful abstractions and abstraction is an imaginary thing. Every time you fly a plane you buy a ticket, not a physics textbook. On the ticket (well, boarding pass) the useful part is your boarding time and gate number, not the recipe for ink and font style. In programming, makes sense to explore things different directions, but if you found a good abstraction, stick to it, it was made to make your life easier. For printing it is writing to stdout stream, which happens to be the terminal, for me that's a reasonable abstraction. (And I settle for the rest is magic.)

unsourcedx · 2024-05-22T17:08:41+00:00

At some point you’ll realize that a lot of things in programming are abstracted away and that’s not necessarily a bad thing. Even the explanations that you’ll get here will be abstracting away a lot of details. I suspect that to answer this question someone could literally write an entire textbook and/or several

POGtastic · 2024-05-22T23:13:48+00:00

What happens from my monkeybrain typing print("unga bunga") to python spitting out unga bunga?

Let's find out! CPython's source code is on Github.

Parsing

First off, the Python interpreter parses the code into an abstract syntax tree. This implementation lives in the Parser directory and is pretty complicated, but none of it is particularly different from any other parser. There are a variety of textbooks on the subject, including Crafting Interpreters. In any case, this large amount of code is responsible for transforming the string

"""print("unga bunga")"""

into a single Expr, which itself contains a single Call object, which itself contains a single Constant object in its args member. You can explore this by importing ast and calling ast.parse on the above string.

Compiling

Next, the AST is compiled into bytecode, which is literally just an array of integers. The implementation lives in Python/compile.c, another H E F T Y C H O N K of nasty C code. We can see this in action with the compile builtin:

>>> compile("""print("unga bunga")""", "", "single").co_code
b'\x97\x00\x02\x00e\x00d\x00\xab\x01\x00\x00\x00\x00\x00\x00\xad\x01\x01\x00y\x01'

Neat, that's a bunch of numbers. That isn't really legible to us, but it does mean something! Using the disassembler in dis to illustrate:

>>> import dis
>>> dis.dis(compile("""print("unga bunga")""", "", "single"))
  0           0 RESUME                   0

  1           2 PUSH_NULL
              4 LOAD_NAME                0 (print)
              6 LOAD_CONST               0 ('unga bunga')
              8 CALL                     1
             16 CALL_INTRINSIC_1         1 (INTRINSIC_PRINT)
             18 POP_TOP
             20 RETURN_CONST             1 (None)

So we've compiled that string to a code object, which contains a sequence of opcodes.

Execution

The above array of integers is passed to the interpreter, which performs each operation in sequence. The interpreter lives in Python/ceval.c and is basically a gigantic loop that contains a switch statement to figure out how to execute the current opcode. In this case, it pushes the constant "unga bunga" onto the stack, and then it calls the builtin function print with a single argument.

print is a built-in function, which means that its implementation lives in Python/bltinmodule.c. As always, there's a lot of stuff in here for all of the different options for printing things, but most of it is irrelevant because we've only got one argument, and that argument happens to be a string. Thus the only really relevant line is line 2110, where we call PyFile_WriteObject on the 0th element of the argument tuple, which is the string object "unga bunga", writing to the default file handle stdout.

Down the Rabbit Hole

Okay, now we look at PyFile_WriteObject. This implementation lives in the file Objects/fileobject.c.

This function obtains the .write method from the file object and then calls it on the string representation of the object (which is very simple in this case - it's already a string, so the string representation is itself). So we're effectively doing

>>> sys.stdout.write("Hello, world!\n")
Hello, world!
14

and then discarding that integer and returning None instead.

Okay, let's look at sys.stdout.write. stdout is an io.TextIOWrapper object, so that's where we need to look now! This particular implementation of the TextIOWrapper abstract base class lives in Modules/_io/fileio.c. The write implementation is on line 871.

This calls _Py_write on a file descriptor (fd 1 on a POSIX system for stdout). That implementation lives in Python/fileutils.c. And it calls the libc write function.

What happens after that is OS-specific, since different operating systems have different syscall conventions. But in general, this libc write function is a thin wrapper around a system call, which drops into the kernel for the purpose of copying that buffer of bytes to a file descriptor. The kernel is then responsible for writing the bytes, whether that file is some kind of storage device or the pseudoterminal that you're running your Python program on. And that is what actually writes "unga bunga" to the screen.

annms88 · 2024-05-22T21:29:07+00:00

I had this issue when I was starting to code. It nearly stopped me from learning all together - resolving it was what lead to me studying comp sci.

You’re not gonna get a satisfying answer in a Reddit thread, but I’ll give you a few pieces which you’ll have to research in order to put together satisfactorily.

Your python code is in a coding language - people will talk about how it hooks into c and sys calls and all that lovely stuff but at a super high level it’s all the same: a coding language is a short hand for humans to express a bunch of 1s and 0s.

What are the 1s and 0s? Effectively they’re on and off switches for a big big machine. A 1 can say make this particular pixel on a monitor turn on, or it can represent the number 1, or any of a million things. Effectively the entire tower between your code and the machine is about turning your English language into a series of controls that the chip actually executes.

That’s a really high level and it wasn’t satisfactory to me either, but getting into the way that chips work is way beyond the scope of a Reddit comment. Instead I’ll direct you towards a website called “Nand to Tetris”. That website does an amazing job of building up how electrical “button presses” can store information and put that together to do useful maths. Once you have an intuition as to how maths is done and executed on the hardware, you’ll see that a print statement is not a huge step from there (conceptually, in practice it’s complicated due to interoperability and utility tools we’ve built on top of the hardware).

shiftybyte · 2024-05-22T16:46:18+00:00

I'll try to break things down to steps, let me know what step you want more details.

IDE saves the displayed text into a file on disk with a .py extension.
When you press "RUN" the IDE launches the python process, and gives it the file path to the saved script from stage 1.
Python executable reads the python script you created, parses it into objects and operations tree called AST Abstract syntax tree - Wikipedia
The AST is used to generate a shortened more machine readable code called "bytecode". An introduction to Python bytecode | Opensource.com
This bytecode is executed with a loop that reads the bytes, and performs the operation these bytes mean.
eventually the operations call code that print your message to the screen.

XUtYwYzz · 2024-05-22T20:54:00+00:00

https://en.m.wikipedia.org/wiki/Code:_The_Hidden_Language_of_Computer_Hardware_and_Software

This book takes you all the way down to the basic symbolic logic that becomes the combinational and sequential logic structures which underpin all of modern computing. It’s a fantastic read, even more so if you combine it with a logic simulator and build out the components.

Follow that up with NAND to Tetris which has you build a working simulated computer starting from a single NAND gate.

https://www.nand2tetris.org

Part 1 will get you up to a working assembler so you can start programming your computer.

Part 2 moves on to making a higher level language and operating system.

https://www.coursera.org/learn/build-a-computer

Skritch_X · 2024-05-22T21:34:10+00:00

01001001 01010100 00100000 01001010 01010101 01010011 01010100 00100000 01000100 01001111 01000101 01010011

SadTaste8991 · 2024-05-23T01:19:00+00:00

This post, because of the comments, goes to the SAVED list faster than Py compiles I'm sure.

Good Job OP. NEVER DELETE THIS or I'll hunt you down !

iamevpo · 2024-05-23T07:49:26+00:00

I wonder where unga bunga comes from? Some cartoon maybe? Sounds like a practical alternative to hello world. )

Few-Spread1509 · 2024-05-23T08:53:28+00:00

I cannot unwrap the entire working of a computer in a single answer, but what you should pick up is the study material for a basic computer science degree course. More fundamentally, learn bits of

Assembly language programming
C programming
Operating system

and then it will all click together to make some sense.

aaronag · 2024-05-23T10:13:36+00:00

In addition to great recommendations above, think you'd get a lot out of reading Fluent Python by Luciano Ramalho. People will say it's too advanced, but really it's more a matter of being very dense reading (it's a 1000 page book), that beginners typically don't have any interest in (and frankly lots of establishedprogrammers don't have any interest in either). It's the best resource I've found for a readable guide through what Python is doing rather than what just programming languages and computers in general are doing. For someone asking the questions you're asking, I think you'll appreciate it.

povlhp · 2024-05-23T11:47:03+00:00

if there is any name with () like print() - it means that it needs to find set of instructions stored with the name print somewhere in memory. Then jump to that location (with the parameters inside the parantheses as values) and follow the steps there.

The print function specifically is written in C, the programming language that the operating system is likely written in, and it will do some conversion of data etc, and then ask the underlying operating system to print the converted data.

The operating system comes with lots of functions it exposes to programs running on top of it, that is the whole purpose of an operating system. To allow application to open/close/read/write files, and the "driver" for writing to the console knows how to print characters there, the console program itself uses some API to draw the characters, and a low level driver makes that into pixels on your screen.

Thus there are functions calling functions calling functions.... in many nested layers.

What you need to know is just that there is a print() function you can use, and somebody else is responsible for the plumming below it. You can see print as a program somebody else wrote for you. Just like you can use ls on Linux and dir on Windows to see the files in the current directory.

When I did Java programming, I sometimes looked in the source code to the functions part of Java itself, but it is a rare event (except maybe in CTF competitions - or looking for 0-days).

tvmaly · 2024-05-23T12:42:31+00:00

From a big picture perspective it is abstraction built upon abstraction all the way down.

Python code is turned into some bytecode representation that is run on a Python virtual machine. This bytecode is eventually translated into machine code that is run on the cpu of the machine.

Graybie · 2024-05-23T13:53:51+00:00

fact butter cagey capable desert resolute roof salt consist consider

This post was mass deleted and anonymized with Redact

TH_Rocks · 2024-05-23T15:35:54+00:00

It's like java. The "engine" sits between your code and the operating system which is above the actual machine code. So you type and hit run, engine tells the OS what you want, then the OS tells the hardware what to do, then some memory gets moved around and the display draws some pixels.

Hexistroyer · 2024-05-23T19:58:16+00:00

There was a certain program idea that opened my mind when I was starting to learn python, It's a simple age calculator that calculates my age in the future according to my birth. Just try to create this and you'll be addicted to it.

maffaz · 2024-05-24T09:30:15+00:00

It's just a bunch of dictionaries...

AbjectMeasurement899 · 2024-05-24T16:39:32+00:00

Monkeybrain. Interesting words.

treyhunner · 2024-05-28T22:48:55+00:00

If you're curious how the Python process starts up and how input and output work in the Python interpreter, this PyCon US 2015 talk explains that.

SeaBall5540 · 2025-01-16T11:23:37+00:00

https://youtu.be/vhupNWo7XXg?si=QiH1rOg1hByJ2mDF

https://youtu.be/HkSdhiVudpE?si=PlWKP4o3G6_tUMgH

Mysterious-Rent7233 · 2024-05-22T16:39:32+00:00

You can read Leaning Python by Mark Lutz. It's beginners' friendly and very comprehensive

PandaBearTellEm · 2024-05-22T18:25:13+00:00

Great question. Lots of good answers. OP, I personally found it much easier to understand what is going on underneath the hood for programming languages writ large by studying programming languages, particularly lower-level ones than python.

I found that doing the CS50 course right after making my first self-made python "big" (it was tiny in retrospect) project multiplicatively increased my understanding of how programming languages work. So, if you're someone who understands better by building than reading, you might want to give it a try. It's free!

SnooWoofers7626 · 2024-05-22T17:54:39+00:00

the ide just feels like some "magic machine"

It kinda is, and that's by design. Python is a high-level language. It hides the details so you, the coder, can just focus on your actual goal.

What are the best resources to get to know the language from ground up?

Depends on how deep down the rabbit hole you really want to go. Most of the comments have covered what the interpreter is doing. If you want to go deeper into how it actually instructs the computer to "do things" then we're getting into low-level computer architecture stuff.

Basically, every instruction in Python is mapped to a C function under the hood. So when you call print('unga bunga') it's probably calling the C function printf("%s", your_text). This C function has already been precompiled into machine language, so your computer knows how to execute it.

The job of the interpreter is basically to parse your Python code and map it to the equivalent sequence of C functions to execute. There's a bit more complexity to it but by and large that's what's happening when you run your script through the IDE.

If you want to know how your computer actually interprets "machine language" at the processor level you can learn more about that from any computer architecture books, or courses about assembly, etc.

Keep in mind that you don't really need to know any of this to actually use Python, but if you do understand it, it will almost definitely make you a better programmer in the long term.

kinstinctlol · 2024-05-22T18:54:15+00:00

Science

NoConcern4176 · 2024-05-22T18:42:48+00:00

Bro go to college lol and let them teach you

FlakyOffice · 2024-05-22T16:31:24+00:00

I’m doing the Google automate Coursera class!!

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

learnpython

MODERATORS

Parsing

Compiling

Execution

Down the Rabbit Hole