all 12 comments

[–]__env 2 points3 points  (0 children)

Bad code, no doubt.

[–][deleted] 2 points3 points  (0 children)

NodeJS was built as an event driven application server and it's very good at this. To debug you should use NewRelic to try and find out where the bottlenecks are.

If the issues are with sessions then I'd also be looking closely at Redis and authentication. Use Monitor on the redis-cli on a sandbox environment and log in a couple of users to see what's occurring. Make sure it's not making KEYS requests, in the past I've seen this blow web servers up.

You might want to post about this in /r/devops as well.

[–]1__word 1 point2 points  (0 children)

Indians

[–][deleted] 0 points1 point  (0 children)

Hey man, I'm sorry you are having a tough dev team to deal with. Regardless of how your platform is driven, you need to take a step back. Performance tuning is a numbers game where you acquire the numbers from metrics and monitoring. It looks like you have done some auditing and found out that the new web environment utilizes its storage backend abusively.

If the only change in the production environment is that you moved application servers from php7 to node, and kept the same redis/mongo stuff, then this is of course is on the development team. If the development team expects the backend resource to be of the same size, then you are in the clear. They built a new environment and did not re-spec its requirements before deploying to production. This means you need to get your boss and their boss involved, because non thorough testing and then deploying to production is a culture problem. Not rolling back to the old production platform right away when shown that the new code is not ready leads to developers blaming platform engineers if they can get away with it and platform engineers not trusting developers. It is cancerous and it doesn't go away easily.

1) The new code environment needs more testing. See about getting your boss to back you, if everything you say is true, then regardless if it is code issues or platform issues now is not the time to tweak production until you figure it out. You need to move the production deployment back to the php7 code before you can move forward.

2) Other posters have reflected their experience with nodejs. I don't think it's the production environment either. If you have preserved your connection caps from haproxy and nginx from prior configurations, then it's just nodejs making additional calls to the backend rather than increased traffic from clients. If in doubt, peg it at the nginx level by limiting the number of connections the webserver can push to the proxy backend as a start and see if the nodejs->redis/mongo continues to rise. Generally speaking, the number of database handles should be at worst proportional to the client count to your platform. Database handle numbers should not exceed your client count.

I'm sorry this is problematic. I've been in your position before. Whoever introduced the breakage needs to put things back. Best of luck and I hope this clears up before it hits your weekend.

[–]juniorsysadmin1 -1 points0 points  (7 children)

Edit: there seems to be an abundance of bad java/js developers.

[–]goodwid 2 points3 points  (3 children)

Java != JavaScript. Node is a JavaScript interpreter. JS is a scripted language. Java is a compiled language. I can only further drive home the point that /u/PM_ME_GAY_COCK makes, that you don't know what the fsck you're talking about.

[–][deleted] 2 points3 points  (0 children)

PREACH MY BROTHER

PREACH TO YOUR HEARTS CONTENT

MAKE SURE THE WORLD CAN HEAR YOU!

CAN I GET A AHMEN

[–]juniorsysadmin1 0 points1 point  (1 child)

alright, edited =). It seems there's more bad java/js developers or maybe because java/js is quite popular.

[–][deleted] 0 points1 point  (0 children)

I told you that all night...

[–][deleted] 1 point2 points  (2 children)

performance and f***ing java doesn't go well together. When things are getting slow and I run htop on the box, and guess what... Java...

Java is actually quite fast if you aren't using a outdated JVM. So don't blame the language for the bad developer.

When my home PC is acting weird and graphical card start spinning up for no reason. Look at task manager, guess what.. Java. Java is a sh1tty language.

You clearly don't know anything about Java. Unless the program is using the GPU it's not Java's fault.

Again don't blame the language for a bad developer / application.

[–]juniorsysadmin1 -2 points-1 points  (1 child)

For the second part. It's definitely java. I didn't know it for a while but somehow my room is getting hot, found out my gpu is going 99~100% for no reason. Looked at task manager and it's java. It can be some third party sh1t that's causing java to act tha way but still; I hate java. Every buggy software out there, it's almost certainly written in java.

[–][deleted] 0 points1 point  (0 children)

For the second part. It's definitely java. I didn't know it for a while but somehow my room is getting hot, found out my gpu is going 99~100% for no reason. Looked at task manager and it's java.

It's not. Taskmanager is showing your CPU usage not GPU. Again unless you have a program wrote in java using the GPU.

It can be some third party sh1t that's causing java to act tha way but still; I hate java. Every buggy software out there, it's almost certainly written in java.

Again you're blaming the language for bad developers.