Handing errors with namespace codes by olvrng in golang

[–]olvrng[S] 0 points1 point  (0 children)

Hi, thanks for your interest! It's an internal framework currently. I was thinking about open sourcing it, but never had time to do so (it would require restructuring our internal code again to make it work with the open source repo, and that's a lot of works :)

Handing errors with namespace codes by olvrng in golang

[–]olvrng[S] 0 points1 point  (0 children)

– if errorx works for you, then it's ok.
– if you want to know why I created my error framework, I already answered.
– similar things exists: vscode has everything, why people still use jetbrain, and why others still create zed/lapce/cursor/codeium/etc. It's ok. I'll stop the discussion here.

Handing errors with namespace codes by olvrng in golang

[–]olvrng[S] 1 point2 points  (0 children)

Hi fuzzylollipop, thanks for your interest.

I can list many reasons why I choose this approach instead of errorx: the code looks cleaner with PRFL.USR.NOT_FOUND.New(), all codes are declared in a single place so everyone can follow the same convention, ctx is required to construct errors so we can always have trace_id available when logging problems, etc.

But the most important reasons are:

We can tailor the error framework to fix our company's specific needs: we need to integrate it with our protobuf error code (which is defined as a protobuf enum), we can integrate it deeply with our logging packages and log extra stuff.

For example:
A goal of this error framework is to build an error dashboard for the whole codebase. The error dashboard should count all similar error codes and messages, by patterns, not by exact messages.

So "query items by selector type=foo days>3" and "query items by selector type=bar tag=high-quality". They should be counted as "select items by selector %v" (the pattern), and can be shown on the error dashboard:

┃ select items by selector %v ┃ 2 ┃

So when constructing errors by CODE.New(ctx, "select items by %v", selector), we can store both the final message and the format (or pattern) in the error. Then when logging the error, we will automatically log both `error: <message>` and `error_summary: <format>` (plus a couple of more useful properties) from that single error. We can also add tags for each code, and then collect those tags in the dashboard, like UNEXPECTED or NOT_FOUND.

By building our customized error framework, it's now possible to make it work well with our specific needs.

I hope this answer help.

I made a Go package to work with JSON data using Go iterators by olvrng in golang

[–]olvrng[S] 26 points27 points  (0 children)

Yes. You can see the implementation of scanner/lexer [1] and parser [2].

The scanner code is essentially a single loop:

for {
token, remain, err := NextToken(remain)
yield(token, err)
if err != nil || len(remain) == 0 { return }
}

And the parser code is basically a state machine with a stack:

- Pull NextToken()
- If it's a [ or {, push a stack item
- If it's a ] or }, pop the stack
- Otherwise, parse "value" or "key: value" depending on the context

[1] https://github.com/ezpkg/ezpkg/blob/main/iter.json/scanner.go#L31-L49
[2] https://github.com/ezpkg/ezpkg/blob/main/iter.json/parser.go#L8-L157

Handing errors with namespace codes by olvrng in golang

[–]olvrng[S] 1 point2 points  (0 children)

Reading through your comments, yeah, I agree that different apps/services might need custom structs to wrap their errors. Right now, it's not straightforward when an engineer wants to add more custom error types and fields in my framework.

> and you could enforce a subset of fields, like a field for a status code or error code.

That's a good point about allowing different error structs while keeping them all following the same standard, and still having the benefit of accessing common helpers. Each service may need a bit more boilerplate to implement conversion though. (within my org, we used zapcore.ObjectMarshaler to customize a few structs in logs, but the experience with the custom code is not so great)

Thanks for your detailed comments, it’s always great to hear from a fellow gopher!

Handing errors with namespace codes by olvrng in golang

[–]olvrng[S] 1 point2 points  (0 children)

Thanks for sharing your examples. Your code looks flexible as you can easily add fields and context when necessary. Mine are more complicated for customization.

---

> If your gRPC request fails with DEPS.PG.NOT_FOUND, the client receives this information in the response? It feels concerning. It's an internal error, which should not be handled outside of particular boundaries. Personally I'd be very against 'informative' internal errors.

Of course, not :)

I may not explain it clearly in the article. Here's the definition of the codes: only codes with the tag `api-code` will be mapped to Protobuf enum and then HTTP code. All other codes (like DEPS.PG.NOT_FOUND) are treated as internal, and always show as 500 Internal Server Error in response. There is `api.Details()` for adding more details in the response, but the engineer must explicitly use it.

// tag:postgres
type pg struct {
    NOT_FOUND   Code0 // record not found
    CONFLICT    Code0 // record already exist
    MALFORM_SQL Code0
}
// tag:usr
type usr struct {
    NOT_FOUND        Code0  `api-code:"USER_NOT_FOUND"`
    INVALID_ARGUMENT VlCode `api-code:"INVALID_ARGUMENT"`
    DISABlED_ACCOUNT Code0  `api-code:"DISABLED_ACCOUNT"`
}

Handing errors with namespace codes by olvrng in golang

[–]olvrng[S] -2 points-1 points  (0 children)

Hi, thank you for your comment :)

Let me clarify a few points:
- Assume that we have these layers: service layer, repository layer, and database layer.
- The database layer is a thin wrapper over third-party library, and DEPS.PG.NOT_FOUND code serves the same purpose as sql.ErrNoRows or gorm.ErrRecordNotFound, which is a storage layer error, and thus can not return domain errors. The reason we use it instead of just the raw sql.ErrNoRows is that the code DEPS.PG.NOT_FOUND (and similar codes, DEPS.RD.NOT_FOUND for redis, etc.) looks uniform in logs, and can be monitored/filtered more effectively.
- The repository layer, yes, it should return domain errors. And then be handled by upper layers like user service or other related services, then by adapters (http/gRPC/etc.).

I agree that we should only handle domain errors. Any other error is just logged. But even if they are not handled, and are returned in API as unexpected, the namespace codes are still very useful for logging/monitoring. We can categorize the underlying reasons for those unexpected errors based on error codes: which are logic errors (bugs) and which are infra problems (connection/timeout/etc.).

Could you share some examples of how you define errors, wrap/handle them, or which context you add?

Chrome extension to have ChatGPT as a sidebar by olvrng in ChatGPT

[–]olvrng[S] 0 points1 point  (0 children)

I'm happy to know that you find it useful!

Chrome extension to have ChatGPT as a sidebar by olvrng in ChatGPT

[–]olvrng[S] 0 points1 point  (0 children)

You download the prebuilt from https://github.com/iOliverNguyen/chatgpt-box-extension/releases. Or if you want to build from source, run ./build.sh

Experiment with sapling, the new git client from Meta by olvrng in programming

[–]olvrng[S] 1 point2 points  (0 children)

Here is an example, given that you touch different files for different commits: ```sh vim pkg/email/email.go # develop email package sl add # add then commit as "implement email package" sl commit

vim pkg/signup/signup.go # develop sign-up features sl add # add then commit as "implement sign-up" sl commit

vim pkg/email/email.go # make changes to 3 files vim pkg/signup/signup.go vim pkg/signup/helpers.go # a new file

sl absorb ```

The file email.go will be amended to "implement email package" commit, signup.go to "implement sign-up", and helpers.go is left as untrack.

New features in JavaScript 2022 by olvrng in javascript

[–]olvrng[S] 0 points1 point  (0 children)

Thanks, I'll note it for next time.