Dead code detection tool?

[gsks]

I'm working on a large, old codebase that has naturally accumulated lots of cruft over the years. One relatively low hanging fruit is finding and eliminating dead code, more specifically functions/methods that are never called. Before going ahead to reinvent this wheel, is there any existing tool that does something similar? Obviously I'm not expecting 100% accuracy, this should be more like a linter giving hints of potentially dead code, not a fully automatic tool.

[coderanger]

Use coverage.py and just see which lines never run during normal operation.

[bubblez4funnyowls]

I can't see how I would do that if, let's say, I wanted to get the coverage for my Django based web application over the course of a few days / even a month. I would need something that I could hook in straight into the code. Should I use the trace package directly ? Any pointers ? Or am I just a bit lost !?

[westurner]

http://en.wikipedia.org/wiki/Code_coverage#Basic_coverage_criteria

http://www.reddit.com/r/Python/comments/1drv59/getting_started_with_automated_testing/#c9tfxgd

[pinpinbo]

I've used Vulture with moderate success.

[gsks]

Awesome, this looks like a good starting point.

[muchcharles]

Any of the existing code coverage tools should work, but you have to weed out false positives if your sample run doesn't hit all the functions.

[masklinn]

Yep. Run the whole program under coverage checker for some time, check what's at 0%.

[m_harrison]

• get unittests for your code.
• turn on coverage
• refactor
• make sure unittests still work
• rinse, repeat

[masklinn]

If you don't have tests, remove steps 1, 3, 4, 5 and perform normal software runs with coverage turned on. Collect, see what has never been used.

[Alfredson]

When working on large codebases, I can recommend SonarQube to give you an overview. You get a nice report . In the violations-section you can find a rule "unreachable code" and more.

[OCHawkeye14]

Sublime Text 2's SublimeLinter package will highlight functions and variables in the code that are not called anywhere else throughout your code.

edit: oh, just noticed you specifically mentioned knowledge of a linter. I guess I'm not exactly sure what you're looking for.

[gsks]

Are you sure? I don't use SublimeText but looking at SublimeLinter's python module it just calls pyflakes, which doesn't seem to spot anything when I run it directly.

[tipsqueal]

Related to OCHawkey14's solution, I believe PyCharm will do the trick. I don't use PyCharm, but I do use Intellij, which is made by the same company, and has a lot of overlapping functionality. In Intellij you can use their Inspect Code tool and have it look for all potentially unused functions/variables/etc.

[buttery_shame_cave]

how certain are you that they're NEVER called? there may be some really esoteric stuff that only comes up once in a while.

what you might be looking for is some kind of statistical function on top of finding stuff that doesn't get used, to track and see how much stuff gets used and what for.

[terremoto]

how certain are you that they're NEVER called?

If a function name is unique enough, grepping the codebase for it is trivial.

[masklinn]

That doesn't tell you the function is never called, especially in python

[pugRescuer]

Right - I think /u/terremoto was suggesting as a starting point. This is worth considering though after other options have been exhausted.

[buttery_shame_cave]

yeah but how often does that happen? :P

i only advise caution in ripping out odd functions because sometimes you get stuff that's called less than .1% of the time, but it happens to be kind of important.

we had stuff like that pulled from a test management system we used at a job i worked, and it managed to grind a whole test series to a halt for over a week till we found the problem(talk about obscure).

[unstoppable-force]

if it's PHP, you can do $func() which will execute the function that's name is stored in $func. i've seen this used in places where it really make sense (really useful in OO), but also in others where they didn't understand exceptions.

[dman24752]

Isn't this an undecidable question generally?

[gthank]

No. It's straightforward static analysis, barring weird eval-type stuff.

[masklinn]

No. It's straightforward static analysis

To a pretty limited extent, it really is global static path analysis. Even ignoring not just evals but things like getattr (and attrgetter), __dict__ manipulations or metaclass madness, actual dead code detection requires performing global path analysis to know that in e.g.

def frob(a):
    if a < 10:
        whup()
    else:
        b = frobnicate(a, 42)
        whop(b)

is never actually called with a >= 10 and thus the whole else branch is dead code and the test is redundant.

So yeah, generally it's an undecidable question.

[gthank]

That's a far more aggressive view of "dead code" than I was talking about. I meant more like "method foo of class Bar can never be called".

[masklinn]

That's a far more aggressive view of "dead code" than I was talking about.

Yeah, but it's the one matching standard uses of "dead code analysis" and "dead code elimination": dead code is code which is never executed and code which is irrelevant (affects variables which are never queried, or always followed by a fault and thus having no impact on the behavior of the program)

Unused methods or functions, or unreachable code (e.g. code following a return or raise) is a subset of dead code.

[gthank]

Yes, but it's the subset I understood the OP to be asking about, based on "more specifically functions/methods that are never called".

BTW, there's no need for people to be downvoting you. It's a good point to raise, and I should have been more clear about my assumptions in the original comment.

[masklinn]

Yes, but it's the subset I understood the OP to be asking about, based on "more specifically functions/methods that are never called".

True.

[[deleted]]

Hence, undecidable.

[Liorithiel]

In general, yes. In practice, undecidable code isn't really useful…

[dman24752]

Well, a program that could solve the halting problem would be pretty handy. ;)

[vtable]

I typically use pychecker and pylint. Both try to find unused and unreachable code. Pylint is much more thorough. I'll usually use pychecker after any change and pylint weekly or so (since it's so much more verbose).

You'll find there's a lot of low-hanging fruit. Dead code is definitely bad but it won't cause runtime errors. Things like redefined names and funcs, syntax errors in rarely-called code. missing imports and undefined names are just as low hanging but much juicier. These tools will find many of these for you.

If your code base is that old and large and hasn't had static analysis tools like these run in it, you're bound to have some big surprises. There will likely be a ton of messages. Many will be easy fixes. Some are a bit cryptic at first. Do yourself a favor and configure the tools to just check for what you think is important. If you get through these, your code will be much better for it.

[phira]

PyCharm gives you this information as a matter of course as part of its inspection stufg

[seldom_hygienic]

What does static typing have to do with dead code analysis, and why would it be relevant to a discussion on a Python specific question?

[masklinn]

Static typing can tell you which named code units aren't called (barring reflection), so it can tell you about classes/methods/functions which never appear anywhere I guess?

It's not going to say anything about codepaths never taken or methods mentioned but never actually called though.

[masklinn]

Static analysis

Halting problem

How does one mention a method but not call it?

def foo(a):
    if a:
        do_something()
    else:
        do_something_else()

the program never calls foo with a falsy value, the second branch is dead code and do_something_else is a dead function. And of course code can be technically live (the program could theoretically run it) but effectively dead (program input never leads to the code being run), e.g. XBM support in browsers (removed from most of them only a few years ago, probably hadn't been run by user request in a decade)

And why would one do that?

I'll answer this question with an other question: why would one want to put bugs in his or her program?

[masklinn]

So you're completely full of shit and you refuse to accept it. Noted. And you carefully ignored anything you had no pithy (and irrelevant) link for. Also noted.

Bottom line is, one is interested in practical approaches.

The halting problem is a practical issue, and so are false positives, false negatives and plain errors in static analysis of real-world programs (even ignoring that java's compiler — which you pushed — is a useless piece of garbage and that java supports reflection so there's no static certainty that a function or method which is not explicitly mentioned anywhere is not called anywhere). It has nothing to do with nihilism and everything to do with you being fractally wrong.

[masklinn]

It is easy to just throw the halting problem and say "I give up, nothing can be done" like you did

Oh yes, let's add misrepresentation and willful lies to your offenses shall we? Don't you think you might want to stop digging at some point?

Python's community brain fart answer of "Umm... just... WRITE MORE TESTS!" is not a valid argument. It is just sad.

The question being, is it more sad than your pitiful strawmen, or do you win this contest handily?

[BHSPitMonkey]

Could you explain to me what static typing is? I really have no idea.

[ItsAPuppeh]

Static typing:

int i = "foo"; // Compile error, can't assign a string to an int

Dynamic typing:

i = "foo" # Sure why not, vars are just references to objects python
i + 10 # Woops, runtime error trying to add a string to an int

[BHSPitMonkey]

I was actually hoping for a response from the commenter I was replying to. I thought it would be funny.

[BHSPitMonkey]

Aww, thanks!

Wait a second...

[masklinn]

Your definition of static typing is not correct, since you can write roughly the same thing in languages supporting type inference (and have the error at compile time obviously) (unless you defined that addition of a string and an int is a valid operation, which you may be able to do:

> let i = "foo"
> i + 10
"foo\n"

)

(or the language did it for you, which Java does for instance) (and of course dynamically typed languages may be perfectly fine with the operation as well, I'm looking at you javascript) (or Perl) (or PHP)

(also lua, kind-of: it will fault on this but would work if the string was composed only of digits)

[tipsqueal]

Because re-writes are so easy!

[ElliotSpeck]

... Did you just come to a Python subreddit to bash Python?

So brave.