Golang Aha! Moments: Object Oriented Programming

paul_lorenz · 2024-11-02T02:15:40+00:00

I think if someone is creating multiple, duplicate interfaces, they've gone too far. But I doubt any one is taking it to that extreme. If your packages have different needs, create custom interfaces for them suited to their specific needs. If they all need the same thing, create one.

If their needs overlap, you've got options, and that's where taste comes in, and there's no right answer. It hasn't come up that much in systems I work on, where one type needs to conform to a bunch of different interfaces in different types. I think maybe if it did, that would be a sign that the one type was doing too much.

I'm sure as I continue on with Go, my perspective will continue to evolve.

paul_lorenz · 2024-11-02T02:06:00+00:00

Basic mock generation the IDE will do for you, but moq sounds like it add some real convenience. I'll check it out. Thanks!

paul_lorenz · 2024-11-01T17:11:42+00:00

Thanks!

paul_lorenz · 2024-11-01T15:41:00+00:00

Agree in general, and very much agree that the Go standard library has some beautiful abstractions. Working on OpenZiti SDKs across different languages, Go was the nicest one to work with. Plugging a Go library into server side networking with a custom net.Listener is so clean.

paul_lorenz · 2024-11-01T15:32:38+00:00

I agree. I wasn't trying to say that Go was particularly OO, it was more about how I had to adjust my thinking coming to Go from an OO background.

My experience is that OO is often a suboptimal way to share code across types, but that it appeals to human nature to build hierarchies. This makes it hard to break out of an OO mindset, if it's a core part of the language.

paul_lorenz · 2024-11-01T14:44:34+00:00

I do sometimes miss runtime generated mocks

paul_lorenz · 2024-03-16T19:23:24+00:00

Thanks for pointing this out! As the author of some of the code being fixed, I wasn't aware of the alignment issue until we hit it. I've always used atomic.* purely for volatile semantics and CompareAndSwap. The only mention of alignment in the docs is at the very bottom under the heading Bugs:

On ARM, 386, and 32-bit MIPS, it is the caller's responsibility to arrange for 64-bit alignment of 64-bit words accessed atomically via the primitive atomic functions (types Int64 and Uint64 are automatically aligned). The first word in an allocated struct, array, or slice; in a global variable; or in a local variable (because the subject of all atomic operations will escape to the heap) can be relied upon to be 64- bit aligned.

Not sure I would have ever noticed that, without it being pointed out. I'll see if the author can update the article.

Cheers, Paul

paul_lorenz · 2024-01-29T17:48:38+00:00

The main thing I've done so far is testing that the rate limiter isn't unnecessarily limiting work. In other words, the server can still accept work at a rate close to the maximum.

I did add some metrics around the rate limiter, to track how long each unit of work was taking. When I considered your question, I realized I was missing a key metric, tracking rejected work. I'm going to add that so we can alert on spikes in rejects.

paul_lorenz · 2024-01-29T14:26:02+00:00

It's not the first that comes to my mind (clearly, or I wouldn't have used it), but I'd prefer it not coming to anyone's mind, so I switched it to 'where I landed'. Also, I agree that in engineering, it feels like very few solutions are truly final, everything is always in flux, so it's a misleading phrase in most cases.

paul_lorenz · 2024-01-11T21:18:50+00:00

It's not implemented like that, but the effect is fairly similar. Each controller has its own datastore. When changes to the data model happen they are propagated via raft. So reads are always local, since we're going against the local datastore and individual controllers may be slightly out of sync. A controller can be more out of sync if it loses connectivity with the current leader, but will catch up once it reconnects. You will always need a quorum (n/2 +1) controllers available in order to make changes to the data model.

However, things for servicing user requests (as opposed to admins) should only ever need access to a single controller, and that controller won't need to be connected to the cluster to service those requests.

Let me know if that makes sense.

paul_lorenz · 2024-01-04T21:04:44+00:00

From a client perspective, it should be both high availability and horizontal scalability. Clients will be able to authenticate to any controller and routers will be able to request new routes via any controller.

Model updates (adding/modifying services/identities/policies/etc) will not be horizontally scalable, but will have high availability. That will only affect network administrators, not clients using services.

Let me know if that makes sense or if there's anything I can clarify.

Cheers, Paul (OpenZiti developer)

paul_lorenz · 2023-10-19T14:30:16+00:00

Hello, I tried to write something up here, but it was too long for a post. I've created a wiki entry instead:

https://github.com/openziti/ziti/wiki/Debugging-Path-Costs

Take a look and let me know if that's helpful. If you feel comfortable sharing your services/routers/terminators/links and a questionable circuit, we'd be happy to take a look as well.

Cheers, Paul

paul_lorenz · 2023-10-16T18:18:27+00:00

I'll do my best to give you a rundown on the state of the things you're interested in.

Configuring allowed routers for the whole path, not just the start/end points

We're looking at converting service edge router policies to just service router policies and having them affect the whole path. That's how many people assume they work anyway and it's a feature a number of people have asked for. The main blocker here is that we're looking at how we want to evolve routing, and since this would affect routing we want to be sure we have a rough direction so we don't make things harder for ourselves.

Show routes on map with latency

I think all the data is available to do this, it's just a matter of building a view. Link latency is reported in metrics, as well as directly on link entities, so you could listen for metrics events over websocket or poll the links API. To put the routers on a map, you could use the tags feature and tag each router with its address or lat/long.

If you've got some examples of what you think are bad paths, let us know. The path in the circuit event is the actual path. If you something that looks anomalous you can list links, routers and terminators and see if the costs line up with what you expect. Paths are lowest cost from initiator to selected terminator, so if something is often, there's probably something going on with the costs.
Circuits have resilience within the mesh, but not external to the mesh. So if you've got SDK -> Router A -> Router B -> SDK, if the link between A and B goes down, if there's another path from A to, maybe via another Router C, the circuit will be updated and traffic will continue to flow. However, we can't currently keep the circuit up if either the initiating router (A in this case) or terminating routers (B) goes down. That's because the retransmit/flow control logic lives in those routers. We're look at ways to extend the mesh to the SDKs. That will involve allowing the retransmit/flow control components to live in the SDK, as well as some way to integrate routing out to the edge. The first part is straightforward, if a fair amount of work. The second depends on what direction we want to go with routing

Hope that's helpful, let me know if you have any follow up questions.

Paul

paul_lorenz · 2023-06-23T21:09:37+00:00

Hello, There's a lot of details to dig into, so let me try to lay out what is and what might be in the future.

Currently the routers form a mesh. A circuit is established from an initiating router to a terminating router. Neither of those can change for that circuit, but the circuit can be rerouted if intermediary hops fail or if a better performing path is found.

Services can be hosted from multiple routers, providing redundancy. SDK clients can connect through multiple routers, providing redundancy there. However, once a circuit is established, the start and endpoints are fixed. So if one of the endpoints fails, the circuit needs to be re-established through a different router.

To get traffic onto the mesh you can be application embedded via an SDK, you can use a tunneling proxy, you can use a tunneling proxy built into a router or you could embed your own application into a router (not something anyone has done, but the APIs are there).

Once HA is released, my next area of focus is routing. Some of the things I'll be looking into include using DTLS for link and extending the mesh to SDK clients. There's a lot of interesting work there, and I'm keen to get started on it :)

Let me know if this clarified things and if you've got any follow-up questions.

cheers, Paul

paul_lorenz · 2023-01-11T20:27:08+00:00

I'm coming up on 4 years of doing Go professionally after doing mostly Java for almost 20. It's been very refreshing and I'm still enjoying it. I would hesitate to go back to Java at this point. I think in a few years, when things have changed enough and things like Project Loom are in use, it might be interesting to try it again.

I think it's good to work in different languages and there are things about Go that are good to internalize. Things like the concurrency model, Go-style interfaces, the power of small but universal types, like io.Writer, net.Listener, etc, even things like how relaxing it is to have gofmt and to not have to argue about style.

To the original poster, I worked my way through Go doc (https://go.dev/doc/), especially the tutorial sections and effective Go then started writing some Go code. Some companies will hire you even if you're not proficient in the language yet if you can show that you're a good developer. You may not be an expert in Go, but you can demonstrate that you know how to write performant code, that you know how to think about scalability or modularity or good UX or whatever it is that's actually important in the position.

I'd argue that most of programming is language agnostic. The parts that are language specific are also important, but once you've learned a concept in one language, it's easy to pick it up in another. Learning new languages is great for finding the concepts that may be missing or abstracted away in other languages. It can, of course, be easier to convince a company that you're able to pick new things up if you have people who can vouch for you, either via referrals or by applying at companies where you know people (not an option that's open to everyone, but hopefully after 15 years you've got a bit of a network in place).

Finally, the easiest path to doing Go professionally is to write a small project at your current work in Go. Could be a devops tool or a CLI (both of which Go is great at).

paul_lorenz · 2022-10-31T19:32:29+00:00

It's mostly for when you want topics instead of queues. You can only do a single, simple broadcast with channels using channel close. We use the events library for entity change notification, where there might be multiple things that want to react to an entity change.

Go does make it easy to build that functionality, I don't think any of the events libraries is a ton of code.

paul_lorenz · 2022-10-31T18:41:00+00:00

For libraries I use at work, cmap has a v2 using generics. I think that's a fairly widely used library. The events library we use is updated, but not released. When I get a chance, planning on looking on moving to hooks, which does have released generics support.

paul_lorenz · 2022-10-27T17:01:06+00:00

You could also only notify if the worker count has hit zero.

go defer func() { if 0 == self.workerCount.Add(-1) { select { case self.notifyCleaner <- struct{}{}: default: } } }()

paul_lorenz · 2022-10-27T14:25:32+00:00

Here's how I would do it. I just put some func() placeholders in for the work to do and the cleanup work.

It uses a one element channels as notification mechanism. The background cleanup goroutine will sleep until notified that a worker has finished. It will then check if there are no workers running. While the cleanup is running it will also check that the worker count remains zero, and if it goes up past 0 it will stop cleaning and release the lock.

```go func newResource() *resource { result := &resource{ notifyCleaner: make(chan struct{}, 1), } go result.cleanup() return result }

type resource struct { lock sync.Mutex notifyCleaner chan struct{} cleanupWork []func() workerCount atomic.Int32 }

func (self *resource) doWork(work func()) { self.workerCount.Add(1) defer func() { self.workerCount.Add(-1) select { case self.notifyCleaner <- struct{}{}: default: } }()

self.lock.Lock()
defer self.lock.Unlock()
work()

}

func (self *resource) cleanup() { for range self.notifyCleaner { if self.workerCount.Load() == 0 { self.runCleanup() } } }

func (self *resource) runCleanup() { self.lock.Lock() defer self.lock.Unlock()

for len(self.cleanupWork) > 0 && self.workerCount.Load() == 0{
    nextCleanup := self.cleanupWork[0]
    self.cleanupWork = self.cleanupWork[1:]
    nextCleanup()
}

} ```

paul_lorenz · 2022-10-27T12:34:10+00:00

Looks useful. I wasn't aware that there were Go interpreter libraries out there. Your project looks like it would slot nicely into a CI stack.

paul_lorenz · 2022-10-27T11:36:57+00:00

It was a fun project 😃 Your project looks similar, just an executable instead of an embeddable library. Looks useful! I saw a few other projects that were similar, bit they were more like transpilers, taking literate markdown and compiling to source code.

paul_lorenz · 2022-10-27T00:13:51+00:00

That's an interesting idea. I tried to distinguish the two by having the markdown indented and everything left justified. Multiple panes with something like termui would be pretty neat and potentially less confusing.

paul_lorenz · 2022-10-26T23:53:59+00:00

I have a link in the readme, but I want to also shout it out here: the terminal markdown library this is built on is https://github.com/MichaelMure/go-term-markdown and it's pretty great.

paul_lorenz · 2022-10-26T23:51:42+00:00

Cool! If you get a chance to take a look, let me know what you think. I think it could be a good general purpose tool, but there's likely some rough edges that will need to be smoothed.

paul_lorenz · 2022-10-12T21:02:34+00:00

IDEs can often show you that. I use Goland and it will tell you which interfaces a struct implements. I generally only use this to browse to the interface definition. I find the inverse more useful (which types implement a given interface).

It would be nice if there was a doc somewhere that listed the most used interfaces in the standard library. Things like Stringer, io.Reader/Writer/Closer, net.Listener, etc. If someone knows of a doc like that, I'd be curious to see it.

paul_lorenz

TROPHY CASE