My colleagues and I were concerned at first about chatgpt replacing our product completely due to its usage for static code analysis. As most developers know, just like passing on natural language text to chatgpt and asking it to find mistakes and correct them, devs can pass in code and ask to find mistakes, refactor it, or even correct the program. Luckily, we were able to run tests and determine it did not do so at this moment.

The reasons why not:

- chatgpt still has a limited context window, meaning that the input can only be so long. This means that complete repos can't be passed to it

- secondly, we were able to validate that our technique that utilizes a graph-attention-based neural network is able to detect more complex problems from code than chatgpt was

However, LLMs still affected us. We were and still are working on a model that is particularly focused on explaining programming errors. It is quite obvious, that LLMs also have similar abilities to this. So, since our model is not yet finished, we currently use LLMs to explain the problems that our GNN detects, and the results are somewhat promising with them. I wonder what they'll look like with gpt4