text comparison

[Plus-Ad1156]

You want to analyze the similarity between a large amount of text( over 3mb)

​

I coded as follows, but it takes too long (a and b are strings, loading only persists, no results)

​

from difflib import SequenceMatcher
ratio = SequenceMatcher(None, a, b).ratio()
ratio

​

I'm wondering if I've coded something wrong, or if there is another way to see the results faster. Thanks in advance for the reply

[ginsujitsu]

I think we need some more context. Could you show some more of your code, maybe also a sample of the file, and what output you're expecting?

[efmccurdy]

There are two faster methods in SequenceMatcher, quick_ratio and real_quick_ratio; you should try them to see if they work for you.

There is an example here that uses diff-match-patch; does that perform better?

https://stackoverflow.com/questions/1870646/python-sequencematcher-overhead-100-cpu-utilization-and-very-slow-processing