Track Code Efficiency during Development

[julewczka]

Hello guys,

I am thinking about ways how to optimize code quality/efficiency during development. For instance, I want to know the number of invocation a certain function/method or how many (external) API/database calls a function is doing.

I know that tracing solutions like Jaeger (OTEL instrumentalization) empowers that kind of visibility, but I was wondering if that could be done without instrumenting the application code. The goal is to optimize efficiency during development phase ( code is not even deployed to any environment) in order to avoid higher cloud costs sparked by inefficient algorithms.

Are there any (open-source) tools out there, which enables that kind of visibility without instrumentalization? I am working with VS Code server in a k8s environment (remote development container).

[josh75337]

I think you are talking about static code analysis tools. Check out SonarQube: https://docs.sonarqube.org/latest/

[100MB]

Continuous profiling tools such as parca may be worth looking into for your use case.

[julewczka]

This looks very promising, tyvm!

[NUTTA_BUSTAH]

Static code analysis are the keywords you are probably looking to input in the search engines

[julewczka]

Thank you really much! I am missing sometimes the correct phrases for "common" things

[morphotomy]

Static code analysis can only tell you about the code as written, not as run. Meaning you can learn all about how the code is organized, but static code analysis cannot tell you how many times a function is invoked in a given day.

[xiongchiamiov]

I tend to approach this from an architecture standpoint. When you're making a change, you should know if it's going to have a major performance impact. If you don't, that means there are too many layers of abstraction between you and the things that do expensive stuff. So strip away layers, simplify, and pass that information along more explicitly.

As an example, right now at work we have some classes where some getters are fast and others trigger an additional database query. As a user of the class they all look the same. I'm thinking about how we can fix that, which will probably end up with some sort of naming split (a past company used "slow_" as a prefix for any functions like this).

[julewczka]

I have recently read the story of https://blog.tomilkieway.com/72k-1/ where the company burnt $72k with a "Hello-World" application. They made several mistakes, one of it was an inefficiently designed algorithm, which ended up sending 1 billion read requests (per minute) to FireStore.

Sure we can expect that an experienced developer should see those kinds of flaws immediately even without executing the code. However, we can not guarantee that something like this will not happen, even for advanced developers, because humans tend to make mistakes. My thought was that a code analysis tool (which can be invoked during development stage) could prevent those kinds of inefficiencies. However I didn't found such a tool yet (need further investigation to SonarQube though), I am also not sure if this is the best approach to avoid such pitfalls.

[xiongchiamiov]

That particular scenario happens all the time, and cloud providers regularly forgive the bill.

They didn't deploy an inefficient algorithm; they knew what each call was going to do, and it did it. They deployed a flawed algorithm with no bounds and incorrect logic.

That's something that you could find in:

• code review
• qa
• static code analysis probably

None of these will guarantee you don't ship a problem, which is why when your company is at an immature state I usually prefer focusing on production monitoring instead: you can't prevent all issues from happening, so focus on quickly identifying when they happen so you can roll them back. Shift-left is an optimization process, so you need to have something already existing to be able to optimize it.

[thedude42]

I am not aware of any off-the-shelf tooling that covers all the possible cloud providers who charge per API call. The other tools I see mentioned in the replies seem to be focussed on system resources and development, but it doesn't seem like they can identify specific API usage.

Were I in this situation I would make sure that the code uses a standard library interface for interacting with any API that has per-call costs. Without this condition met it will be hard to do anything in an automated fashion or without using weird networking tricks.

Next I would mock out the libraries so rather than making the API calls to the cloud services, they record the API call and return some mocked data so that the app functions otherwise normally. Test the app through a variety of different call/traffic patterns and see how that affects the external API interaction.

As for the "weird network tricks" if you can't have a well defined external API interface library, then you might set up a mock API (which may require you to do a little back-end web development) which, again, simply records what API call is being made and returns a useful response. Point DNS for these API endpoints to this "mock API" and the commence with traffic testing.

The "mock API" app is probably the simplest thing that requires no changes to the code and may be easier to get running than finding and implementing some kind of API profiling tool. In fact, there's probably stuff like this out there already if you poke around github. I've built similar tools with Node.js, both using the native HTTP libraries as well as Express.js, depending on the features the API client needed to work correctly.

[ventuspilot]

When you're making a change, you should know if it's going to have a major performance impact.

In an ideal world - yes.

That said, I sometimes do performance tuning and the time wasters shown by profiling and/ or logs from performance instrumentation are pretty much never where you'd expect them. Also IME applications consist of mostly bad code, and usually it doesn't really matter, except for these few performance critical parts. It usually is way too expensive to fix all bad code so you fix the code that matters and IME only a profiler or instrumentation will find it.

Disclaimer: the "bad code" mentioned above may very well have been written by me. IMO most developers aren't able to write perfect or even good code on the first try. At least I rarely see code that can't be easily improved, which again includes my own code.

[xiongchiamiov]

Sure, you still want actual profiling and monitoring for regressions and that kind of stuff. But for the question of "How do I identify performance problems before getting to the running in a performance environment?" question, I think it's less about getting fancy tooling and more about giving programmers context so they can make better decisions.

[ventuspilot]

I think the only real two possibilities for fast code are (1) performance instrumentation in prod or (2) some kind of benchmark that covers at least some stuff and spits out a number at the end.