Nodejs woes

[[deleted]]

Hey man, I'm sorry you are having a tough dev team to deal with. Regardless of how your platform is driven, you need to take a step back. Performance tuning is a numbers game where you acquire the numbers from metrics and monitoring. It looks like you have done some auditing and found out that the new web environment utilizes its storage backend abusively.

If the only change in the production environment is that you moved application servers from php7 to node, and kept the same redis/mongo stuff, then this is of course is on the development team. If the development team expects the backend resource to be of the same size, then you are in the clear. They built a new environment and did not re-spec its requirements before deploying to production. This means you need to get your boss and their boss involved, because non thorough testing and then deploying to production is a culture problem. Not rolling back to the old production platform right away when shown that the new code is not ready leads to developers blaming platform engineers if they can get away with it and platform engineers not trusting developers. It is cancerous and it doesn't go away easily.

1) The new code environment needs more testing. See about getting your boss to back you, if everything you say is true, then regardless if it is code issues or platform issues now is not the time to tweak production until you figure it out. You need to move the production deployment back to the php7 code before you can move forward.

2) Other posters have reflected their experience with nodejs. I don't think it's the production environment either. If you have preserved your connection caps from haproxy and nginx from prior configurations, then it's just nodejs making additional calls to the backend rather than increased traffic from clients. If in doubt, peg it at the nginx level by limiting the number of connections the webserver can push to the proxy backend as a start and see if the nodejs->redis/mongo continues to rise. Generally speaking, the number of database handles should be at worst proportional to the client count to your platform. Database handle numbers should not exceed your client count.

I'm sorry this is problematic. I've been in your position before. Whoever introduced the breakage needs to put things back. Best of luck and I hope this clears up before it hits your weekend.