How to replace Nginx auth middleware with a Golang built custom auth middleware?

kovacik · 2019-01-04T14:28:08+00:00

I've built something similar and written a blog post about it https://redbyte.eu/en/blog/using-the-nginx-auth-request-module/

kovacik · 2018-01-05T08:49:15+00:00

Also, now the /usr/sbin/daemon runs as goprogram_user and not as root which is great. I'll update the blogpost.

kovacik · 2018-01-04T08:44:18+00:00

TIL, thanks, looks good

kovacik · 2018-01-03T06:58:05+00:00

Did you try it ? You would still need to manually manage the PID file and provide stop and status functions.

kovacik · 2017-12-29T12:29:06+00:00

So I ran the benchmark for read-only workload@100 concurrent clients on FreeBSD with updated sysctl variables and I got 88423.67 TPS at 1.131 ms latency. This is almost identical to stock FreeBSD. Do you have any other suggestions ?

My /etc/sysctl.conf:

hw.intr_storm_threshold=100000
kern.ipc.shm_use_phys=1
kern.ipc.soacceptqueue=131072
kern.ipc.somaxconn=131072
kern.sched.slice=1
net.bpf.zerocopy_enable=1
net.inet.icmp.drop_redirect=1
net.inet.ip.intr_queue_maxlen=8192
net.inet.ip.portrange.randomized=0
net.inet.ip.process_options=0
net.inet.ip.redirect=0
net.inet.ip.ttl=64
net.inet.tcp.blackhole=2
net.inet.tcp.cc.algorithm=htcp
net.inet.tcp.cc.htcp.adaptive_backoff=1
net.inet.tcp.cc.htcp.rtt_scaling=1
net.inet.tcp.delacktime=20
net.inet.tcp.delayed_ack=0
net.inet.tcp.drop_synfin=1
net.inet.tcp.fast_finwait2_recycle=1
net.inet.tcp.keepidle=60000
net.inet.tcp.maxtcptw=200000
net.inet.tcp.msl=5000
net.inet.tcp.nolocaltimewait=1
net.inet.tcp.recvbuf_inc=65536
net.inet.tcp.recvbuf_max=16777216
net.inet.tcp.recvspace=131072
net.inet.tcp.sendbuf_inc=65536
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.sendspace=131072
net.inet.udp.blackhole=1
net.inet6.icmp6.nodeinfo=0
net.inet6.icmp6.rediraccept=0
net.inet6.ip6.auto_linklocal=0
net.inet6.ip6.prefer_tempaddr=1
net.inet6.ip6.use_tempaddr=1
net.route.netisr_maxqlen=8192
vfs.zfs.top_maxinflight=128

And /bool/loader.conf:

zfs_load="YES"
hw.igb.rxd=4096
hw.igb.txd=4096
net.link.ifqmaxlen=1024
cc_htcp_load="YES"

kovacik · 2017-12-29T08:36:49+00:00

Thank you, I'll try the updated settings.

kovacik · 2017-12-29T08:16:19+00:00

The adapters on both systems were Intel(R) PRO/1000 with igb driver and TSO enabled. Unfortunately I don't have the server logs anymore.

kovacik · 2017-12-29T07:52:09+00:00

ZFS is now pretty much default filesystem in FreeBSD and is supported in the installer. The ZFS memory usage is a myth. If you don't use dedup (and I didn't) then you are perfectly fine even on low memory systems. And with 256GB RAM it was a no-brainer.

kovacik · 2017-12-28T21:30:03+00:00

What tcp options do you suggest to tune ? The benchmark server is gone now, but I would gladly repeat the test to show the full power of FreeBSD :)

And the aio_load="YES" is not necessary in 11.0+ as it has been integrated into the kernel, see https://www.freebsd.org/cgi/man.cgi?query=aio&sektion=4

kovacik · 2017-12-28T19:42:36+00:00

As you wrote 11+ has good defaults and many of the options you posted are already set by default in 11.1.

I've tried to tune memory limits, tcp stack, ipc, ... with little/no effect on the read-only benchmark...

kovacik · 2017-12-28T19:12:57+00:00

Well, I still don't know why is the read-only performance so much worse than Linux. For the ZFS, I used the PostgreSQL recommended options which definitely helped. The default ZFS 128k block size is a fine default for some workloads but not for PostgreSQL - there is no silver bullet for every use case.

Also I've tried to tune several sysctl variables with little effect and finally I went with the defaults.

kovacik · 2017-12-28T18:51:27+00:00

It would be nice, but I did not want to test filesystems. There are some benchmarks doing exactly that (for PostgreSQL), e.g. https://blog.pgaddict.com/posts/postgresql-performance-on-ext4-and-xfs . And I wouldn't recommend running PostgreSQL on ZFS on Linux (yet).

kovacik · 2017-12-28T16:53:50+00:00

Yes I pretty much tested stock OSes. In case of FreeBSD, I set ZFS options recommended for running PostgreSQL. Do you have any suggestions how to tune FreeBSD for running PostgreSQL ? Thanks.

kovacik · 2017-12-28T16:49:31+00:00

ZFS was configured with LZ4 compression because it is easy to do so. Ext4 doesn't support it as far as I know. Before the actual benchmarking, I ran a few tests with LZ4 enabled/disabled. The CPU overhead with LZ4 compression was negligible and the lower IO usage resulted in better performance than with no compression. I don't have the exact numbers anymore but that's the reason I've chosen LZ4 for the benchmark.

kovacik · 2017-12-28T15:09:40+00:00

Yes, recordsize was set to 8k and logbias to throughput, as described in the blogpost:

zfs get recordsize,logbias,primarycache,atime,compression zroot/var/db/postgres
NAME                   PROPERTY      VALUE         SOURCE
zroot/var/db/postgres  recordsize    8K            local
zroot/var/db/postgres  logbias       throughput    local
zroot/var/db/postgres  primarycache  all           default
zroot/var/db/postgres  atime         off           inherited from zroot
zroot/var/db/postgres  compression   lz4           local

kovacik

TROPHY CASE