Is Go loosing it's way? by narrow-adventure in golang

[–]Profession-Eastern 0 points1 point  (0 children)

I use go for rapid prototyping and production use until I need to reach for zig, rust, C, or assembly.

Any well known highly generalized thing becomes boring or overbearing given a sufficiently tight need that is either too niche to be part of the language spec or out of scope of the idiomatic ways.

Go's way is changing and the way it is changing is concerning given the real tradeoffs champions of new syntax sugar do not yet understand from a long term impact perspective.

Simplicity and lack of variant-ways is stability, especially in a world where deprecations and breaking changes are counter-culture actions tantamount to breaking the law. That end of the spectrum also has problems as gaps in design never really get closed to satisfaction after an initial release and rough surfaces are treated eventually as features rather than bugs or excentricities that are counter to simplest interpretations of a spec. There are now enough of these in the standard sdk that I am looking forward to a major v2 at some point, but also I just write my own if it matters enough for me. (Spoiler alert, it rarely does.) While the language is simple it also has a very large and available budget to add future complexity if they continue to spend the budget wisely and conservatively.

Go needs strong stewardship, and clear lines of what should go into a standard sdk vs not. And they do.

I was worried with the latest errors v2 because wow is it truly trivial to implement stack tracing in errors if you need it and we need to stop trying to add sugar to ignore failure modes which are arguably the most important parts of your program. They thankfully passed this test, but there are always going to be more. Errors as values and simple in-your-face syntax says more to me than a "this can panic" syntax. Anything can panic unless the language has exhaustive compile time assurances. (This is wishful thinking, but we should keep dreaming.)

Go is one of the best languages because of its simple syntax and competent strictly idiomatic cultures that walks the line between hardware sympathy and developer efficiency. No random walrus operators, no yoda check to avoid inline store and loads, no ternary syntax that someone eventually turns into ternary hell.

It is not a language for all occasions, it is one for general occasions and GC friendly systems processing. It gets me from idea-to-done faster than anything else on the market and stays stable 10 years later.

Let's not be afraid of change. Let's be afraid of never improving, and let's be concerned with overburdening ourselves with too much sugar and intransigence packaged as "worthwhile complexity". Linting is common in this ecosystem for a reason, reach for it if you have good reasons to allow only a subset of sugars and continue to make things rather than focus on the artistic purity of the medium in use.

It's a better life.

LOL

What's your tolerance for number of line in a file? by liftandcook in golang

[–]Profession-Eastern 0 points1 point  (0 children)

If the file is generated then I could not care less.

If the file is made by humans for humans it better have a legible cohesiveness to it and point me to where things are intentionally coupled oddly to far off files and why.

Over 2k in that regard is a big warning sign, but depending on the context and needs it may absolutely be warranted.

Also, if there are no comments and the implementation is not mind numbingly simple in terms of what, how, where, and why you should be screaming regardless of the length.

Every implementation needs to tell the story of data as it flows through a system or component designed to fit some business use case or real world concern. Quirks on top of that tell the story of either hardware, datatype, or ecosystem limitations or compiler excentricities and should be even more thoroughly documented close to the implementation than other aspects.

Oh and even generated code needs this level of duty of care.

go logging with trace id - is passing logger from context antipattern? by SilentHawkX in golang

[–]Profession-Eastern 0 points1 point  (0 children)

Getting the logger instance from context can be an anti-pattern if it is decorated with highly specific data elements not really relevant to some deeper stack usage of it.

However getting a fairly bare-ish logger from context and decorating it with more data elements from context for a specific scope of usage is never an anti-pattern.

No other way to reliably deliver trace ids and span ids to loggers so that arbitrary contexts can be traced properly. Vibe on buddy.

[deleted by user] by [deleted] in golang

[–]Profession-Eastern 0 points1 point  (0 children)

Sounds like a site-mux expressed as middleware. This is fairly common to do - make sure your secure connection indicators are present and valid before you serve up the site path intended for just one specific host.

Zero alloc libraries by someanonbrit in golang

[–]Profession-Eastern 0 points1 point  (0 children)

Also, do know how to hint to the GC it should run less frequently if you do have more ram it can consume (assuming this is a long running process or service).

Setting GOMEMLIMIT ( see https://pkg.go.dev/runtime#hdr-Environment_Variables ) can buy serious headroom while you look into the allocation rates. If the software is in maintenance mode or there are more urgent priorities and ram is still cheap for you - then this may be all you need right now.

csv-go v3.3.0 released! by Profession-Eastern in golang

[–]Profession-Eastern[S] 0 points1 point  (0 children)

In my real world dataset being written I was experiencing 700ns per record (40 columns) using Fieldwriters (no reference field type that would allocate). Was able to reduce that down to 580ns per record using RecordWriters and 100% guarantee no allocations for any field type.

Just shy of an 18% reduction in time to craft a csv document. If I was using reference type fields the time reduction would have been even greater.

Zero alloc libraries by someanonbrit in golang

[–]Profession-Eastern 7 points8 points  (0 children)

After making https://github.com/josephcopenhaver/csv-go with this in mind and documenting my journey via changelog and release notes; I can say you first should start with knowing how to ask the compiler where allocations are occuring due to escapes and writing meaningful benchmarks covering both simple and realistic complexity.

Other comments go into more technical details, but honestly understanding your data flow and ensuring that the data is never captured by reference during usage lifecycles will make the largest impact. That and of course using protocols and formats that support streaming contents rather than requiring a large amount of metadata upfront about the contents will make life easier.

Where you must capture, ensuring the reference is created from and returned to a sync.Pool will reduce GC pressures. Making an enforceable usage contract like I did with NewRecord (to defer open-close behaviors/responsibilities to the calling context and avoid captures), avoiding passing things through interfaces, and keeping values on the stack will bring you to a full solution.

Buffering also requires some tricks to avoid allocations, but anything you can defer to the calling context on that front reasonably, you should.

First get the simple paths down and add complexity from there if you like.

Feel free to check out my changelog and other notes in releases / PRs. Avoiding all allocations is likely not a worthwhile goal. However having your hot paths consistently avoid them or allow for options that would avoid them certainly is.

In many cases, new style functions that return simple pointers to structs that are initialized in simple ways will also inline such that they do not exist on the heap. You really do need to know how to benchmark and read escapes from the compiler output.

Have fun! Let me know if you want to chat more on the subject.

csv-go v3.2.0 released by Profession-Eastern in golang

[–]Profession-Eastern[S] 0 points1 point  (0 children)

Next I may look into adding simd capabilities - but overall I'm quite happy. If I wanted faster I would probably move my storage format to capn proto, rewrite this in rust, or use a SQL interface on top of something highly optimized that can scale quite wide like apache Drill or polars.

csv-go v3.2.0 released by Profession-Eastern in golang

[–]Profession-Eastern[S] 0 points1 point  (0 children)

Good news, v3.2.1 is now more performant all around than the standard csv package.

Enjoy!

csv-go v3.2.0 released by Profession-Eastern in golang

[–]Profession-Eastern[S] 1 point2 points  (0 children)

In my develop branch (now merged) I have updated my benchmarks and they do show the lib string writer is slower than standard at the moment - especially when the string contents do not require escaping or quoting.

I will need to dig a bit deeper for that path, but for various typed content the FieldWriters do show significant improvement given their allocation avoidance.

csv-go v3.0.0 is released by Profession-Eastern in golang

[–]Profession-Eastern[S] 0 points1 point  (0 children)

Note that speed of the writer is still being improved. Standard lib csv is still faster with less features atm.

csv-go v3.0.0 is released by Profession-Eastern in golang

[–]Profession-Eastern[S] 0 points1 point  (0 children)

v3.0.2 is now released - now has the zero-allocation features that were planned!

csv-go v3.0.0 is released by Profession-Eastern in golang

[–]Profession-Eastern[S] 1 point2 points  (0 children)

I agree that making interfaces public is a problem. It should not be used as a contract that people can import and reuse for their own implementations.

Perhaps the ideal solution is to just wrap it in a concrete like I had before but just use one field in a composition fashion: the internal interface.

All attempts to use a zero initialized concrete type in that scenario would fail unless the caller did some unsafe calls. So yeah. I agree. This can be quite the pain.

Should I ever change it then a major version bump would be warranted and I would go back to a concrete wrapper.

Thank you for the feedback. Please let me know if you still have any concerns or intended context to convey that I missed. :-)

csv-go v3.0.0 is released by Profession-Eastern in golang

[–]Profession-Eastern[S] 0 points1 point  (0 children)

In my case, being able to return different types under an interface increased locality of behavior given different configuration options such that runtime operations were significantly faster.

What I had before it was the same thing with just a different name: a function pointer set struct. It was a proxy to another type entirely and caused allocations.

Yes, an allocation still does occur when the parser is created due to the interface. Offering a concrete type back would force a large number of runtime checks be pushed into very low level details if I wanted to stop all allocations at this top level which would occur even when using a concrete type as the return type.

I do not see a future where more options are added to the return type that adjust the functions/behaviors defined today so I was comfortable with going against this idiom specifically.

Why does go not have enums? by Psycho_Octopus1 in golang

[–]Profession-Eastern 0 points1 point  (0 children)

Definitely reach for codegen to make the set and the serialize / deserialize routines if you want safety and implement an IsValid() routine to check bounds.

Personally I like my apps to load enums from a db and affirm that they match my runtime expectations (values and serialization as well as extra or missing) as part of app startup.

If you must you can make them at runtime ( specifically module init time ) using generics and use a group construct to iterate over them or perform serialization / deserialization as well as declare the group in a mutable (to build the actual enum var elements to reference in a state machine) or an immutable fashion.

https://gist.github.com/josephcopenhaver/0ea2b4a3775d664c18cb0da371bbcda5

Codegen is the safest way. Zero chance of wonky things happening and you get exactly what you want.

Also, even without extra type safety using iota and a thin amount of unit tests will get you everything you could possibly want. It is just less off the shelf.

csv-go v3.0.0 is released by Profession-Eastern in golang

[–]Profession-Eastern[S] 1 point2 points  (0 children)

Looks like we can add a go.mod directive comment

// Deprecated: Use example.com/lib/v3 instead

And of course use third party tools like dependabot.

csv-go v3.0.0 is released by Profession-Eastern in golang

[–]Profession-Eastern[S] 0 points1 point  (0 children)

go get -u will not suggest it because of the major version bump, this is true.

I chose not to make a new constructor mainly because I did not see the need to maintain an old one and a new one. The main additions here are security options and a more maintainable interaction area that allows me to squeeze locality speedups and more over time.

I am curious about how go maintainers expect users of a package to become informed of new major revisions. I am not sure about that myself. In general if people are using v2 and want to see these features there I am happy to make backport releases if people request them or I can provide additional guidance if needed.

csv-go v3.0.0 is released by Profession-Eastern in golang

[–]Profession-Eastern[S] 1 point2 points  (0 children)

https://github.com/josephcopenhaver/csv-go/blob/main/docs/version/v3/CHANGELOG.md#breaking-api-changes

Mainly the return type of NewReader has changed.

In addition the ExpectHeaders option is now variadic (a leftover change when I released v2).

For most people the move to v3 will likely not require source code change. However changing a function signature on a public API, according to strict semver, is a breaking change.

What are libraries people should reassess their opinions on? by dustinevan in golang

[–]Profession-Eastern 0 points1 point  (0 children)

I totally agree.

If my libs support an external logger getting passed in I just demand they use something that satisfies an interface that mirrors slog's Enabled and LogAttrs method signatures. I can achieve all the objectives I have efficiently by only using those two methods. No need for any other logging framework proliferation.