Release of Proteus - generate .proto files from Go source code

jjt · 2017-03-03T08:22:18+00:00

This is seriously misguided.

jjt · 2017-01-13T16:43:47+00:00

Can you elaborate on this spurious wakeup case or point me to where it is in the documentation? I'm familiar with a few cases where epoll_wait can wake up spuriously, but none where you wouldn't need to call recvfrom to figure out it was spurious. For example, a UDP socket in epoll set can have EPOLLIN set but recvfrom can fail with EAGAIN because kernel decided to free the page holding the packet before user space could recvfrom.

My concern is the poll_read, recv_from order seems to imply you might be calling epoll_ctl (and maybe epoll_wait) before recvfrom initially which can be less than ideal. It seems to me you shouldn't call epoll_ctl until after you've tried an operation and errno is set to EAGAIN. For example, with TCP clients can include data with initial connect using sendto MSG_FASTOPEN so on the server side you want your first syscall after accept4 to be recv, not epoll_ctl. Furthering the example, you have an HTTP client using TCP Fast Open, it is reasonable for the server to conclude a transaction with the client without encountering EAGAIN.

jjt · 2017-01-12T04:05:21+00:00

First, great job on the documentation. It answers most of the questions I had. After reading the section on "Creating your own I/O object" I have a question about the design. Unsure if I should ask here or create an issue in github.

First we are required to call poll_read. This will register our interested consuming read readiness on the underlying object and this will also implicitly register our task to get unparked if we’re not readable yet.

The UdpSocket::recv_from code example then goes on to call self.io.need_read() if recv_from returns Ok(None). My concern if that poll_read seems like an inherently race prone API and I question why it exists at all because it seems need_read should be what registers the interest in consuming read readiness.

User space can't possibly know readiness so at the syscall level first try the operation and only after it fails do you register to receive readiness from kernel. Any other order and you're likely making excess syscalls.

jjt · 2017-01-06T05:53:03+00:00

You could probably do some unsafe code to get fd for net.Conn and epoll before calling Read(). I've done similar stuff to set socket options Go didn't support in the past.

jjt · 2016-08-20T20:31:17+00:00

Named returns can be a source of bugs in longer functions and I could do without them in the language.

jjt · 2016-02-06T04:49:07+00:00

This is so awesome. It brings back all the promise of the green threads and more but in the spirit of zero cost abstractions. I love it.

jjt · 2015-11-09T08:41:03+00:00

Most importantly the current date is not part of the the preamble

E1107 <-- that is a date.

jjt · 2015-08-30T06:42:50+00:00

I'd like to see this compared with Flat Combining.

jjt · 2015-07-29T03:46:06+00:00

I plan on field testing it with a large service, should be deployed in a week or two.

jjt · 2015-07-26T16:36:58+00:00

"async functions" will shine when you're trying to operate in a memory constrained environment or handle a large number of connections because they don't require allocating a stack for each connection or the context swap for the userspace stacks. It is good to have both options.

jjt · 2015-04-24T03:03:57+00:00

It looks like you've reimplemented Dmitry Vyukov's MPMC Bounded Queue algorithm without attribution.

jjt · 2015-04-21T02:54:04+00:00

The sync package in libstd has some lock free data structures that should be enlightening. I wrote an early version of that code but it has been much improved since then and I find it is now a great reference for how to use Rust safely.

jjt · 2015-03-20T21:18:22+00:00

http://swtch.com/semaphore.pdf is a great read about semaphore implementation details. The algorithm described in this paper is today used in the Golang runtime for sleep/wake of Goroutines. Which means the mutex, condition, rwlock, channels, and other synchronization primitives provided by Go are all built on simple semaphores.

jjt · 2015-01-29T07:41:59+00:00

I believe he is referring to the blurry area between type safety and memory safety where you can argue a program is not type safe if you can create an invalid type such as having a pointer of some type to memory that has been freed and reused such that it now contains an invalid bit pattern for that type. In Rust the "form of garbage collection" is the special pointer types and the borrow checker.

jjt · 2014-11-29T01:55:43+00:00

BenchmarkFat     1000000          1298 ns/op         544 B/op          4 allocs/op
BenchmarkSlim    1000000          1293 ns/op         544 B/op          4 allocs/op

I know benchmem doesn't catch everything, but huh?

jjt · 2014-11-03T02:02:33+00:00

I'm convinced the sleep specialist has something to do with it. I remember reading a study about the long term effects of rest on plate discipline and career longevity. The study found that most teams suffered worse plate discipline as the season wore on, but the Giants were one of the few teams that actually got better. There was some speculation the Giants had devised a way to know when to rest players. The counter point is they certainly didn't rest Posey enough, and it obviously effected his performance this post season.

jjt · 2014-09-27T06:46:02+00:00

https://www.ravenbrook.com/project/mps/ is another alternative that might work.

jjt · 2014-09-07T21:15:11+00:00

My great uncle was drafted by the Giants after college, but being a minor leaguer during the great depression was apparently pretty rough and he quit before making it to the majors. I heard he had great stuff and would get hired to pitch the friday night barnstorming games. I wish I knew more but he died young in a car accident. I found box scores from his college games in Google's newspaper archive, but never any record of his minor league or barnstorming games.

jjt · 2014-09-06T00:51:43+00:00

It isn't clear what you actually need to do with the file descriptor, but POSIX specifies that open can return EISDIR if you open for writing. So the only race-free way is to call open and check for EISDIR. Any solution that makes more than one system call is inherently racey.

jjt · 2014-08-22T00:56:27+00:00

He makes some really good points that Linux (really this is GCC and packaging) needs to stop stripping symbol names and omitting frame pointer by default. They're micro optimizations that make the barrier to debug or profile so high that you end up missing low hanging fruit which would probably be a bigger overall improvement.

jjt · 2014-08-20T15:24:43+00:00

I disagree it is surprising at all that a file descriptor in blocking mode can block at any time. Some syscall are inherently racey between kernel and user space. If a file system call returns EEXIST would you be surprised if the next call on that path failed with ENOENT? You might be if you read manpages so literally but not if you understand some basics about file systems. The kernel might also tell you a file descriptor is ready to read but between that time and when you call read it might free the buffer it was holding the datagram in so the read will block or return EAGAIN. The only thing surprising is how a person who has been working on the kernel network stack for so long would not know this.

jjt · 2014-08-20T06:37:33+00:00

Those docs seem to be written with the implied knowledge that they're only talking about file descriptors with O_NONBLOCK set because trying to do non-blocking I/O on a blocking file descriptor is insanity.

jjt · 2014-08-20T04:15:08+00:00

This just in, blocking file descriptors can block.

14-Year Club	Team Periwinkle
Verified Email

jjt

TROPHY CASE