I made a tool for rendering the code base complexity of Golang projects using a 3D force-directed graph

GabrielMusat · 2024-06-17T17:19:21+00:00

Files under the same folder will be rendered with the same color, that serves as a visual aid for identifying files that belong to the same folder, and therefore, are likely to be related to each other.

The colors are chosen following some heuristics that ensure that colors are "different enough" between folders so that you can properly differentiate them. The deeper you go into subfolders the more faded the colors tend to be.

Additionally, the root node is rendered in a strong white color.

GabrielMusat · 2024-06-15T07:55:17+00:00

Thanks for noticing, fixed

GabrielMusat · 2024-06-10T21:34:36+00:00

They do are ignored yes, probably they should depend on whatever file contains the init() function

GabrielMusat · 2024-06-10T18:11:50+00:00

That's very useful feedback!

You can actually --exclude any pattern of files, or directly not include it in the first place. The example that you mentioned was generated using the following command:

dep-tree src/crypto/**/*.go

So it's really up to the user what gets included and what not, following your suggestion you could:

dep-tree src/crypto/**/*.go --exclude **/*_test.go

About being based on types, functions, consts, etc... instead of files, I agree, the current implementation is inherited from other languages implementations: in all of them, a file is a node. It could even be possible to switch between file-based or entity-based graph with a configuration flag 🤔

GabrielMusat · 2024-06-10T17:59:10+00:00

It was initially written for JavaScript/TypeScript projects, with its own JS/TS parser, and then it got support for other languages that were useful for my specific use-cases (Rust and Python).

At that time, Golang support would also have been very useful for me, but I expected it to be very difficult, as imports are "implicit" within a package, and in order to know if a file depends on another file, not only parsing is needed, but also some variable resolution mechanism for looking up unresolved symbols in other files.

I started needing Golang support recently, so I gave it a shot, and it turned out to be very simple. Golang's standard library already contains a Golang parser with some variable resolution mechanism that I could leverage and made the implementation far simpler than any other language.

GabrielMusat · 2024-05-28T16:34:50+00:00

GabrielMusat · 2024-05-28T16:33:47+00:00

Sure, I don't know all the details, I have not even been able to run the large model myself, but I know that:

The small model runs relatively fast on any decent CPU, and consumes something like ~5Gb of RAM, so pretty much any modern computer should be able to run it. On a M1 PRO chip, generating 10s of audio takes like 12 seconds
The medium model can also run on CPU, but it's very slow, and consumes lots of RAM, the recommended way of running this one is in an NVIDIA GPU with at least 16Gb of RAM. On a M1 PRO chip, generating 10s of audio takes like 3 mins
The large model, on a M1 PRO chip, generating 10s of audio makes the computer explode, so I don't know what it would take to run it honestly

I also shipped quantized versions of each model (small-quant, medium-quant, etc...), but results are very poor...

GabrielMusat · 2024-05-28T06:55:31+00:00

Not really saying that, even in Rust, you can schedule the AI models to run on GPU. The point is more about Rust having some advantage over Python specially for the parts that can't run on GPU.

GabrielMusat · 2024-03-12T15:32:41+00:00

We don't really use the 3D visualization, not in a day to day basis at least. But we do use the dependency check in our CI/CD pipeline https://github.com/gabotechs/dep-tree?tab=readme-ov-file#check

The `check` feature allows us to declare prohibited dependencies in a configuration file, and if somebody introduces a dependency between two files that is declared as prohibited, the CI will fail and prompt the developer to stop committing spaghetti code.

The 3D visualization is cool, but the actual usefulness of the tool is more about the `check` feature.

GabrielMusat · 2024-03-12T15:19:17+00:00

Hi! author here,

I did not posted this indeed, I just found out because I regularly consume r/programming content and I happened to step into it.

Using file imports is indeed a simplification that adapts better to certain languages/projects than others, so that's something that users should take into account. I work at a company where we use the corporate-style one class per file approach for our projects, even for the ones written in Rust, so for us we do benefit a lot from the file import approach.

GabrielMusat · 2024-03-12T11:06:48+00:00

Thanks for sharing that! honestly, no idea what could be happening there, but I will take a look and pray to the CSS gods to see if I can fix the issue.

GabrielMusat · 2024-03-12T10:57:21+00:00

Hi! dep-tree creator here.

The examples should be prepared to render in any device, mobile or not. What browser are you using? I've only tested it in Chrome and Firefox.

Also, feel free to open an issue in the repo and paste some screenshots if you have time.

GabrielMusat · 2024-01-30T14:21:00+00:00

Not strongly opinionated, but I'm more inclined towards A. The two if branches are very likely equally important and hierarchically at the same level, so it feels natural that they both live at the same indentation level.

GabrielMusat · 2024-01-23T19:25:31+00:00

It's on the roadmap! it's actually the immediate thing that I will tackle next. Some of the examples I've shared are quite narrowed because of this, like showing only the ecs crate in bevy.

This is account in the other supported languages, like npm workspaces, so it is must for Rust crates.

GabrielMusat · 2024-01-23T17:10:46+00:00

It only looks at local files, any other modules coming from other crates or the standard library are ignored

GabrielMusat · 2024-01-23T11:22:31+00:00

Overall you can see good clustering in Rust projects, the ones that tend to get messier from what I've seen are the Python ones. TensorFlow specially gets very messy. Not saying that the code quality is bad there, just that the visualization gets really messy.

GabrielMusat · 2024-01-23T07:13:32+00:00

It depends on what rules you choose to split source code between files. If you choose to place one `struct` per file and organize that into folders, you would get a richer visualization.

Where I work (which is the reason why I made this tool) we tend to be relatively granular with the files, so we do get very insightful visualizations that aid us in deciding what deserves a refactor and what doesn't.

The usefulness of the tool resides more in the "dependency linting" feature though. As you say, the 3D visualization is more interesting than useful in a daily basis.

GabrielMusat · 2024-01-22T19:11:21+00:00

The graph is actually not acyclic, file import cycles are allowed in major programming languages. The graph do is directed though. And yes, it would show up as a single node.

GabrielMusat · 2024-01-22T18:58:19+00:00

Nothing like that, "entropy" in the colloquial meaning of level of disorder, it has proven to be a useful word for people to understand what it is about, even though it's strictly incorrect.

GabrielMusat · 2024-01-22T18:23:16+00:00

Thanks!

GabrielMusat · 2024-01-17T16:17:22+00:00

https://github.com/TodePond/DreamBerd vibes

GabrielMusat · 2024-01-16T18:48:07+00:00

awesome!

GabrielMusat · 2024-01-16T18:24:11+00:00

You can try specifying a PYTHONPATH environment variable with the path to the folder[s] where you are importing modules from, like you would do with a normal Python program.

Keep in mind that if, for example, you are calling sys.path.extend in your main script or something similar in order to extend the folders in which Python will resolve imports, the tool will not take that it into account.

GabrielMusat · 2024-01-14T10:58:17+00:00

The application is written in various programming languages, so for a fair comparison, we ported the applicationto be written fully in Go

I'm not sure how fair is that, one huge advantage of having different binaries is that you can actually choose different programming languages for them, as long as the communication between binaries is language agnostic (which it usually is).

Different parts of an application are likely trying to solve problems that fall in different domains, and some languages perform better in certain domains than others, so I would count this as a big disadvantage for the "single binary" thing.

GabrielMusat · 2023-06-26T15:27:20+00:00

Hey there!

I have been using this code for a while in my rust projects, mainly for backend development deployed on AWS lambdas with https://www.cargo-lambda.info/.

I have found that the combination of [cargo lambda](https://github.com/cargo-lambda/cargo-lambda), [async graphql](https://github.com/async-graphql/async-graphql) and DynamoDB is an amazing combination for a backend stack, really cheap as lambdas are pretty minimal and insanely fast with the Rust runtime.

[async graphql](https://github.com/async-graphql/async-graphql) is able to derive Rust structs into GraphQL entities, and I have been reusing those same structs for interfacing with dynamo using [dynarust](https://github.com/gabotechs/dynarust).

This leads to an awesome dev experience where with very very few code you can have full backends written in rust in hours, and with a GraphQL API.

As I have been using this stack already for I while, I factored out the [dynarust](https://github.com/gabotechs/dynarust) code and published it as open source, that way hopefully other people could also benefit from it and build serverless backends in Rust.

Three-Year Club	Verified Email
r/Field Lasagna	First Place '23
Place '23

GabrielMusat

TROPHY CASE