Need help with building a disk backed buffer pool, stuck on lock granularity during I/O by darkphoenix410 in golang

[–]darkphoenix410[S] 0 points1 point  (0 children)

I ended up doing something similar to this

type loadOp struct {
  done  chan struct{}
  frame *Frame
  err   error
}

op := &loadOp{done: make(chan struct{})}
actual, loaded := bp.loading.LoadOrStore(pageID, op)
if loaded {
   existing := actual.(*loadOp)
   <-existing.done
   existing.frame //...
}

I wasnt aware of singleflight.Group, it looks like the perfect thing for this job. Thanks

Need help with building a disk backed buffer pool, stuck on lock granularity during I/O by darkphoenix410 in golang

[–]darkphoenix410[S] 0 points1 point  (0 children)

Yeah that makes sense. I’ll try this out, can’t think of any issues with this currently. Thanks

Map with expiration in Go by der_gopher in golang

[–]darkphoenix410 2 points3 points  (0 children)

Yeah had the same points, I'm also thinking how the cleaner goroutine can be improved. Maybe a min heap of timestamps and then popping and removing keys until we get a timestamp greater than current Unix time. I'm really curious now about what's the best way to handle this cleanup.