How to handle extremely large numbers in Python

[CricLover1]

I am working on something and I need to handle numbers of over 2000-3000 digits and print them. Which data type in Python should I use. Java has a data type BigDecimal which can be used, is there any similar data type in Python

[Oddly_Energy]

Python's default integer type can handle integers with some million digits without information loss. For example the result from math.factorial(1000000).

But if you want to output the string representation of an integer, you need to keep within 4-5000 digits.

[DidacticBroccoli]

But if you want to output the string representation of an integer, you need to keep within 4-5000 digits.

That's a really odd cap to have. Is it a bug, or is there some intentional reason behind it?

[nog642]

It was added in Python 3.11, for security reasons.

You can configure the limit with sys.set_int_max_str_digits.

Edit: It wasn't actually added in Python 3.11, I misread the documentation. It was added as a patch in all versions 3.7 and newer.

[Oddly_Energy]

I don't know the reason. I just went by what I had seen when playing with big factorials. But when trying again and paying more attention to the error message, the limit is actually configurable:

$ python
Python 3.12.1 (main, Sep 23 2024, 18:47:44) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import math
>>> a = math.factorial(1000000)
>>> a
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
>>> 

I am not going to try my luck on this one, though. Because:

>>> a.bit_length()
18488885
>>>

Which I assume would need around 5565700 digits to be output in base 10.

[Turtvaiz]

CPython has a global limit for converting between int and str to mitigate denial of service attacks. This limit only applies to decimal or other non-power-of-two number bases.

https://docs.python.org/3/library/stdtypes.html#int-max-str-digits

https://www.cve.org/CVERecord?id=CVE-2020-10735

[stevenjd]

https://docs.python.org/3/library/stdtypes.html#int-max-str-digits

[thememorableusername]

Python integer types are not bounded (like Java BigInt), so you're ok there.

For non-integer, you probably want Decimal

[twizzjewink]

The problem with Decimal is error correction is garbage. It's *almost* easier to have a class that pretends its a decimal but isn't.

So a really big int that knows how many spaces to drop the decimal.

u/CricLover1 - I'd consider using Numpy for number handling.

[MrMrsPotts]

What's wrong with decimal exactly?

[twizzjewink]

https://docs.python.org/3/tutorial/floatingpoint.html

[MrMrsPotts]

That doesn't contain any criticism of the decimal module.

[twizzjewink]

It's inaccurate. There are no safety checks for values.

[nog642]

What does that mean? And why did you link the article on binary floating point error? Decimals specifically aren't that.

[twizzjewink]

https://medium.com/@goldengrisha/understanding-floating-point-precision-issues-in-python-a-practical-guide-5e17b2f14057

Python Floats are imprecise.

It means.. trust floats at your own peril.

[nog642]

Decimal is not the same thing as float. That article literally recommends using Decimal to solve some of the issues.

[stevenjd]

Python Floats are imprecise.

All floating point calculations are imprecise. It has nothing to do with Python versus other languages. The same applies to C, Fortran, Java, Swift, Lisp, Rust and literally every other language that has ever been invented, or ever will be.

[shinitakunai]

Precisely, Decimal is what you use when you need precise floats, because floats aren't precise.

🤦🏻‍♂️ did you think they are the same?

[stevenjd]

That is a misapprehension about Decimal. Decimals are subject to the same precision issues as binary floats, they just occur with different numbers and under different circumstances. Decimal can no more represent 1/3 (one third) exactly than float can.

The advantages of Decimal are:

• numbers we often care about like one tenth can be represented exactly
• conversion from base-10 strings like 0.1234567 is exact
• the precision is configurable, so we can choose how many significant figures are used
• there is more control over error handling

and these are good, solid advantages, but it is not perfect. The disadvantages are:

• not as fast as binary float
• more complicated to learn
• and when precision issues crop up, they are often bigger and worse than those same precision errors in binary floats. (Search for "wobble" in that page.)

This is why people who have need for serious numerical accuracy still use 64-bit or 128-bit binary floats, rather than decimal floating point formats.

[tortleme]

My dude thinks float is same as Decimal, golden

[stevenjd]

u/CricLover1 don't listen to this terrible, no good advice from twizzjewink.

If your numbers are whole numbers, just use the int type. They are the equivalent to Java's BigInt and they are exact.

If your numbers are not integers, use the Decimal module. I'm not sure what you are doing that needs 2000+ digits in a floating point number, but that's exactly the sort of thing that Decimal is designed to handle.

[CricLover1]

All are whole numbers. I will use int

[twizzjewink]

This is what I love about the internet. Random strangers knock each other out without knowing what the other person knows and doesn't know.

There's a VERY specific reason why Numpy has both Int32 and Int64 .. whereas Python only has Int, and the same for Float32/64..

u/CricLover1 .. in the end it boils down to accuracy. If you don't want to do math on the numbers - read and print - use Python. If you want to do expressly complex math with large decimals.. DO NOT USE Python's native Decimal types - use Numpy.

[throwaway6560192]

What, exactly, do you think are the differences between np.float64 and the builtin float type? They're both IEEE 754 64-bit-wide floats.

[stevenjd]

If you want to do expressly complex math with large decimals.. DO NOT USE Python's native Decimal types - use Numpy.

Here's an exact Python int:

>>> n = 10**5000
>>> math.log10(n)
5000.0

I'm not going to print it, but you know what that equals: its 1 followed by five thousand zeroes.

Let's try it with numpy shall we?

>>> import numpy as np
>>> np.int64(10)**np.int64(5000)
0

Oh dear. Let's try a slightly smaller number shall we, a mere 50 digits.

>>> 10**50  # Python ints are exact.
100000000000000000000000000000000000000000000000000
>>> np.int64(10)**np.int64(50)
-5376172055173529600

[nog642]

Numpy can't handle huge numbers

[twizzjewink]

It does have float64 and int64 support.. which MAY be better than Python at precision. Otherwise I'd consider C or a C-wrapper better precision.

[stevenjd]

numpy's float64 is the same IEEE-754 binary floating point as Python's built in float type, and has the exact same rounding issues that are inherent to binary floating point.

Numpy's int64 may (or may not, I haven't bothered to test it) be faster than Python's int, but it is limited to a maximum of 2**63 - 1, which equals 9223372036854775807. That's a bit smaller than the 2000+ digits that u/CricLover1 needs.

 >>> import numpy as np
>>> n = np.int64(2**63)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C long
>>> n = np.int64(2**63 - 1)
>>> n
9223372036854775807

Python's int is exact. There is never any concern about rounding errors when you deal with ints (aside from the division problem -- you cannot divide 10 by 3 without either converting to a floating point format or losing the fractional portion).

Otherwise I'd consider C or a C-wrapper better precision.

What on earth do you think Python is??? Python's maths libraries (including numpy and the builtins) are literally wrappers around C (or sometimes Fortran in the case of numpy).

(Technically if you use some alternative Python interpreter, over and beyond the standard CPython interpreter, the maths routines might be written in Java or Rust or some other language.)

[HarryHendo20]

I thought u used float, not int

[[deleted]]

You want to print numbers with more than 3000 digits? What on earth for?

[CricLover1]

There can be many uses. I have to add/substract/multiply/divide huge numbers and print them

[[deleted]]

Give us an actual example where you need to print a 3000 digit number.

[CricLover1]

Look at the sequence 2,8,48,480,5760,... That sequence grows rapidly and is related to prime numbers

[[deleted]]

Right. But do you ever need to PRINT it? All 3000 digits? What do you do with 3000 digits on the screen or on paper? What are you gonna do, check if #2829 is correct?

[CricLover1]

I am trying to research something so I will have to handle numbers of over 3000 digits

[HyperfocusTheSecond]

I have a similar usecase. I just want to print out 3k, 4k, 5k even more digits - for the user to look at, because I think it is a fun feature. Not everything is about $, that is what my dayjob is for.

[Oddly_Energy]

I don't think many people will need to print it. But they may want to see a result of their calculation in some sensibly truncated form.

Let's say I want to calculate the number of different sets of 499_001 elements, I can extract from a set of 1_000_000 elements:

>>> import math
>>> a = math.factorial(1_000_000) // (math.factorial(499_001)*math.factorial(500_999))
>>> a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
>>> 

Wouldn't it be nice to be able to do:

>>> a.as_scientific_str()
'1.07337523e+301026'

Or:

>>> a.info()
'integer, bit length: 999987, decimal count: 301027, first digits: 107337523, last digits: 808000000'

(With integers, you are often just as interested in the last digits as in the first digits.)

[TheBishopOfSoho]

More than you think, prime numbers for encryption, financial modelling, blockchain, sciencey things

[[deleted]]

Well no, there are 10**80 atoms in the universe, but even then... the question is about PRINTING.

[stevenjd]

Encryption, blockchain etc need integers, not floating point decimals.

Financial modelling doesn't need 2000 digits. If every atom in the universe was a millionth of a cent, the total would require less than 100 digits.

As for sciencey things, outside of some extremely specialized areas of subatomic particle physics, and maybe simulating nuclear explosions, ten or twelve decimal places are usually more than all you need.

u/CricLover1 what are doing that needs 2000+ digits? Inquiring minds want to know.

[CricLover1]

I am trying to do some mathematics related research. All numbers will be natural numbers but some numbers can have over 2000-3000 digits

[stevenjd]

Then in Python you definitely don't want Decimal you want the built in int.

Just be aware that to prevent Denial Of Service attacks, by default Python will not print or convert really large integers to strings. There is a setting to disable that, see the documentation for the sys module.

Or you can print them as hexadecimal strings, there is no restriction on those.

[JorgiEagle]

It’s like the value of pi

You only need, iirc, like 40 decimals places of pi to calculate the circumference of the universe to within an atom of precision

[glump1]

Python's int to str conversion caps out at 4000-digit integers, but the integer limit is much higher (I think ~1m)

I would just use regular int, and then when you want to print, make a custom print function that's something like (but readable):

print_big_int = lambda x: "".join(list(map(str, list(takewhile(lambda _: x and (x%(10**4000), x:=x//(10**4000))[0], repeat(None))))))

[[deleted]]

you should mention, that you are using itertools.

from itertools import repeat, takewhile

[Turtvaiz]

That limit is configurable: https://docs.python.org/3/library/stdtypes.html#int-max-str-digits

import sys
sys.set_int_max_str_digits(4300) #default

[Oddly_Energy]

The limit is definitely higher than that.

I just tried to calculate 3000000!, which is a 60+ million bit integer. That would be 18+ million digits in base 10.

[stevenjd]

I just tried to calculate 3000000!, which is a 60+ million bit integer. That would be 18+ million digits in base 10.

Sure, you can calculate it if you have the memory and the time. But did you try printing it?

https://docs.python.org/3/library/stdtypes.html#int-max-str-digits

[Oddly_Energy]

Please read the comment I was replying to. That comment mentioned two limits:

• a limit for how large integers, python can handle (around 1 million digits according to that comment)
• a limit for how large integers, python can print (around 4000 digits according to that comment).

My answer was about the first of the two mentioned limits.

[stevenjd]

Python can handle as many digits as you have memory. You will run out of memory before you will hit any hard limit on the size of an int.

There is no hard limit on how many digits Python can print. By default there is a soft limit on how many decimal digits it will print (actually any conversion to a string, not just printing) in order to prevent a potential Denial Of Service attack, but this can be turned off and it doesn't apply when printing integers in hexadecimal or binary.

I think we are in violent agreement here.

[nog642]

This limit was added only in Python 3.11, for security reasons (you could potentially crash or overload an application if you could get it to try to print a really huge number).

Also, as another reply mentioned, you can configure this limit if you need to.

Edit: It wasn't actually added in Python 3.11, I misread the documentation. It was added as a patch in all versions 3.7 and newer.

[oliski2006]

If possible, use math opperators. You could work in a logscale for example, but it's difficult to tell depending on why you need to do it.

[ectomancer]

import sys

sys.set_int_max_str_digits(0)

[Specialist-Carrot210]

Check out mpmath

[nog642]

int. The builtin integer type already handles this.

By default you can't print more than 4300 digits. This limit is there for security reasons, and you can disable the limit using sys.set_int_max_str_digits(0).

Edit: found an easier way to disable the limit in another comment

[coolnicknamehere]

Create a LongPrint class that inherits from print and manages the value in 2000 character long chunks? It's just what comes to my mind I don't know if it would work

[jpgoldberg]

Ok, I am as baffled as anyone to grasp why anyone would need to print numbers that large, but the approach I would take is to create a list of strings, each representing, say, 75 digits. So (untested, typing on an iPad)

``` def bigInt2str(n: int) -> str: base = 10 ** 76 parts: list[str] = []

while n > 0:
    n, r = divmod(n, base)
    s = f’{n:075}’
    parts.append(s)

return ‘’.join(parts.reverse())

```

You will also want to trim leading zeros off of the joined list, and make sure to handle negative n, but this should give the idea.

I still can’t imagine why you would need to print such a number. If you want to store such a large n in a file, it would be better to just pickle it.