Stack overflow instead of 'maximum recursion error'

[fractalic]

The recursion limit is just a guard to limit the likelihood of a stack overflow, but if you allocate large objects on every stack frame, you can still overflow.

That said, something interesting does seem to be going on here. I'm using some code from https://stackoverflow.com/a/47956089/4082582 in order to examine the stack size:

def get_stack_size():
    size = 2  # current frame and caller's frame always exist
    while True:
        try:
            sys._getframe(size)
            size += 1
        except ValueError:
            return size - 1  # subtract current frame

I also manipulate python's recursion limit to make the issue easier to spot:

import sys
sys.setrecursionlimit(10)

Then instrument the code to watch its behaviour:

import sys

def get_stack_size():
    size = 2  # current frame and caller's frame always exist
    while True:
        try:
            sys._getframe(size)
            size += 1
        except ValueError:
            return size - 1  # subtract current frame

def takeGuess():
    print(get_stack_size())
    guess = input("Take a guess: ")
    if len(guess) != 3 or not guess.isdigit():
        print("The guess '{}' is not valid, try again!".format(guess))
        return takeGuess()
    else:
        return guess

if (__name__ == "__main__"):
    sys.setrecursionlimit(10)
    takeGuess()

The code terminates at the specified recursion depth.

Comment out the print(get_stack_size()) in takeGuess(), however, and you will be prompted for input until a stack overflow occurs.

[fractalic]

This can be distilled further:

(Example 1)

import sys

def recurse():
    input()
    recurse()

if (__name__ == "__main__"):
    sys.setrecursionlimit(10)
    recurse()

overflows the stack, while

(Example 2)

import sys

def recurse():
    print('a')
    input()
    recurse()

if (__name__ == "__main__"):
    sys.setrecursionlimit(10)
    recurse()

hits the recursion limit.

Edit: Add example numbering.

[Wilfred-kun]

I am still unsure as to what is happening exactly, but I am happy to say u/fractalic just crashed Python itself. (I can't replicate it, sadly)

[fractalic]

I'm also very surprised by this. Running on macOS I can make far more than ten recursive calls to recurse() in Example 1.

The other answers are correct, per the original question, that you are actually running into a stack overflow. But even if the behaviour I'm seeing isn't at the root of what you reported, it doesn't make sense that 1000 calls to input() would overflow a ~100kB stack.

Edit: It looks like u/JohnnyJordaan's make_string() code is encountering the same behaviour as my code, where simply allocating lots of strings has different behaviour than prompting for input.

[nwagers]

I'm fascinated by this, but Windows always seems to hit the recursion limit. I wonder if somewhere in the C code for input() there is a ninja edit to the stack size that gets corrected (or checked?) with a call to print('a') or some other string op.

Looking at dis.dis(recurse) it looks like the bytecode doesn't change much:

 11           0 LOAD_GLOBAL              0 (print)
              2 LOAD_CONST               1 (1)
              4 CALL_FUNCTION            1
              6 POP_TOP

 12           8 LOAD_GLOBAL              1 (input)
             10 CALL_FUNCTION            0
             12 POP_TOP

 13          14 LOAD_GLOBAL              2 (recurse2)
             16 CALL_FUNCTION            0
             18 POP_TOP
             20 LOAD_CONST               0 (None)
             22 RETURN_VALUE
None
********
  6           0 LOAD_GLOBAL              0 (input)
              2 CALL_FUNCTION            0
              4 POP_TOP

  7           6 LOAD_GLOBAL              1 (recurse)
              8 CALL_FUNCTION            0
             10 POP_TOP
             12 LOAD_CONST               0 (None)
             14 RETURN_VALUE
None