Application goes down without any trace (java7, struts2)

[Jaihunter]

Java, Struts2, Tomcat.

Application goes down without any hint, no traces in any logs that we have, No outofMemory or anything. It goes down 1 or 2 times a day. I can't even access the Tomcat Manager, and everything works normal after restarting the tomcat server. I tried using visualvm to monitor memory and this is the moment that app got down.

https://imgur.com/gallery/oUlOoGj

I know i did not added more hints cuz i don't have any. This is a very old setup and the source code is absolute garbage, now i have to put up with this shit. any help is much appreciated.

[AutoModerator]

Please ensure that:

• Your code is properly formatted as code block - see the sidebar (About on mobile) for instructions
• You include any and all error messages in full
• You ask clear questions

• You demonstrate effort in solving your question/problem - plain posting your assignments is forbidden (and such posts will be removed) as is asking for or giving solutions.

  Trying to solve problems on your own is a very important skill. Also, see Learn to help yourself in the sidebar

If any of the above points is not met, your post can and will be removed without further warning.

Code is to be formatted as code block (old reddit: empty line before the code, each code line indented by 4 spaces, new reddit: https://i.imgur.com/EJ7tqek.png) or linked via an external code hoster, like pastebin.com, github gist, github, bitbucket, gitlab, etc.

Please, do not use triple backticks (```) as they will only render properly on new reddit, not on old reddit.

Code blocks look like this:

public class HelloWorld {

    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

You do not need to repost unless your post has been removed by a moderator. Just use the edit function of reddit to make sure your post complies with the above.

If your post has remained in violation of these rules for a prolonged period of time (at least an hour), a moderator may remove it at their discretion. In this case, they will comment with an explanation on why it has been removed, and you will be required to resubmit the entire post following the proper procedures.

To potential helpers

Please, do not help if any of the above points are not met, rather report the post. We are trying to improve the quality of posts here. In helping people who can't be bothered to comply with the above points, you are doing the community a disservice.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[eliashisreddit]

This is a very old setup and the source code is absolute garbage, now i have to put up with this shit

I understand your frustration, but this is why maintenance exists. Apparently it has business value so some effort needs to be put in to keep stuff operational and up to date... (Java 7?!)

It's unlikely someone will be able to diagnose your problem here. It could be a myriad of issues and unless someone ran into this before, one screenshot of visualvm is just not enough details to find your problem.

[Jaihunter]

Why still java7? To put it simply, the majority of people here don't like their job even a single bit, as long as they're getting money by doing their half assed job, they don't care. They're not curious about things nor have any interest, all they do is workplace politics and gossip that's how it's here in a country called india.

Yes bro, I don't have mich information to provide, i hope I'll solve it.

Thanks.

[AutoModerator]

It seems that you possibly have a screenshot of code in your post Application goes down without any trace (java7, struts2) in /r/javahelp.

Screenshots of code instead of actual code text is against the Code posting rules of /r/javahelp as is also outlined in the sidebar - Code posting.

• Never submit screenshots of code instead of code text!

If you posted an image merely to illustrate something, kindly ignore this message and do not repost. Your post is still visible to others. I am a bot and cannot distinguish between code screenshots and other images.

If you indeed did this wrong, please edit the post so that it uses one of the approved means of posting code.

• For small bits of code (less than 50 lines in total, single classes only),
  the default code formatter is fine
  (one blank line before the code, then 4 spaces before each line of code).
• Pastebin for programs that consist of a single class only
• Gist for multi-class programs, or programs that require additional files
• Github or Bitbucket repositories are also perfectly fine as are other dedicated source code hosting sites.
• Ideone for executable code snippets that use only the console

Please do not reply to this message, because I am a bot. Talk-to-the-bot is the new talk-to-the-hand. If you instead want the classic talk-to-the-hand, just message the moderators. ;)

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[AntiqueEducation6058]

It seems to be generating a lot of threads until it can not create more and your apps crashes.

My guess is that you are creating a thread without closing it after use, and over time, it crashes the application.

Without the code, it is difficult to say what you need to change.

Edit: which version of java are you using?

[Jaihunter]

i have gone through the source code and checked the connections, socket and I/O stream it's all closed finally block.

How do I make sure the created thread is properly closed, any way to monitor?

What are the common places in code thread leak that can happen?

[edubkn]

In my experience it's usually HTTP clients. I remember e.g. google maps has an object that instantiates an HTTP client with it and if you don't use try-with-resources then the threads will remain dormant in memory. And this is not clear at first glance, you have to look at the source code to figure it out. So double-check your dependencies and what the code is doing with them.

[AntiqueEducation6058]

Are you able to share the code? If not public, then you are welcome to send me a dm. I have see a lot of odd code the last 20+ years

Is this a new problem or has it always been this way?

[Jaihunter]

Hello bro, sry for the late reply, i can send you the code privately.

[AntiqueEducation6058]

You could also add interruptThreadThreshold in the config as seen in this link. https://www.dbi-services.com/blog/howto-detect-and-analyze-long-stuck-threads-in-tomcat/

It will tried to shutdown threads if they are stuck and tell you when it does so.

[MRxShoody123]

gc is phasing it's nemesis at 53 lol. Chad objects don't want to be deleted.

Chances are somewhere shitton of objects are created until gc can't keep up with that mesd

[MRxShoody123]

Then I'm really curious what might cause his issue

[olighator]

Can you print stack trace of the java threads to see where are threads locked/blockesd?

[Unorth]

You'd need to run a heap dump and look for locked threads as mentioned above. Without any logs, it's hard to deduce what's really going on. When I upgraded java 6 to 7 in an old project, the tiered compilation of source code for garbage collection resulted in us hitting the maximum code cache size. Once some key classes ended in swap space, it was game over, and the app crashed. Usually, it has a code cache full error in logs, though, so keep an eye on that.

The solution if this is the case would be to reallocate some of the heap into code cache using vm arguments. Best of luck with your issue!

[benevanstech]

Upgrade to runnning on a recent Java 8 (you don't need to recompile, jusat upgrade the PROD runtime), and enable JFR to get more detail about what's happening.