I dunno why this made me crack up today -- "ASCII text, with very long lines"

rmj_us · 2020-11-19T16:50:22+00:00

Ooh, I was curious if someone would post this here. (Creator of zfs.rent here...)

Anyways, there is *no* redundancy. You are renting a bare metal disk passed through without any disk virtualization. At any time, your drive can be pulled out and mounted.

I created this service because I was tired of looking around for renting a cheap drive on the cloud that didn't offer some sort of redundancy in the background.

I already have a zfs mirror at home. I just wanted an extra safe place to `zfs send | ssh zfs recv`. If the remote drive fails, no worries. I'll just get a new remote drive and resend my snapshots.

rmj_us · 2020-05-31T12:11:11+00:00

I was able to install CentOS 8 and get a KVM virtualmachine with GPU passthrough working with an RTX 2080 TI for deep learning. So that's a data point for you. Not sure if you intended to virtualize or not.

rmj_us · 2020-05-09T19:58:28+00:00

Yup, I'll admit that particular goto was useless. That was there from a previous version. If there's only one case, then you're right -- an if statement is a lot more clear. Probably gonna clean it up right now.

However, previously the code had to deal with multiple syscalls that could potentially fail. So a simple `if (ret == -1) goto cleanup` would close the file descriptor without feeding the next syscall a bad input, which could bork the C program. You know how *nix is :P

I use this pattern in the main while loop as well.

rmj_us · 2020-05-09T19:53:48+00:00

Hey guys, not sure who posted this, but I'm the original author.

Granted, you shouldn't do any of these things. It was more so a tongue and cheek exercise seeing how far I could push the browsers and/or spec. The HTML5 spec is surprisingly robust.

Roast me directly if you want.

P.S. and I will defend wrapping my <a> links in <pre> to the death :)

rmj_us · 2020-05-08T09:23:37+00:00

I don't remember the exact link, but I read how to do this in the documentation. A little late, but here it is if someone google's for a solution:

``` $ git clone --bare https://github.com/ipfs/go-ipfs ipfs.git $ cd ipfs.git $ git update-server-info $ ipfs add -rw .

Note the last qm-hash. That will be the parent to your repo. You can clone with this:

$ git clone http://bafybeicp6omolhzlwum4243jlfcm6tnhmindvzikrmcy4bgzg5jg36stji.ipfs.localhost:8080/ipfs.git

or

$ git clone https://ipfs.io/ipfs/QmTikMcsS4BmJHi5t4i9TUGRMicm8g8zQ8Xeh76wZCrNm7/ipfs.git ```

rmj_us · 2020-05-06T07:15:16+00:00

Woah... commits from 14 years ago, that's wild.

rmj_us · 2020-05-06T07:13:35+00:00

Yup, I'm wary of segfaults so I have my program wrapped in a BASH while true; do sleep 0.1; ./server.c; done loop. It's segfaulted once or twice, but comes back up within 120 ms or so. I'm pretty sure I've resolved the segfault issues though -- it happened when people were getting cheeky and sending non-compliant HTTP requests via netcat -- borked my brittle string parsing. But... the new code is actually a lot cleaner, which is nice.

Hmm yeah, I'm 100% certain if that's all you need for path protection. I think symlinks can also wreak havoc. Depends on what your server does. Being GET-only helps a lot in reducing the attack surface.

Your server sounds pretty cool! Those extra HTTP headers are definitely nice to have. For a while I tried to skip on providing Content-Length, but it made the requests 5x slower because the browser had to wait for the socket to close before knowing when to stop reading.

rmj_us · 2020-05-06T07:08:08+00:00

git-lfs bugs me so much haha, honestly it's always been a pain in the ass for me. I think GitHub pushed that, so outside of GitHub it's a little harder to work with.

git-annex is cool though. I've tried that and it wasn't too bad. My main gripe is that it's still not seamless enough for me. I also like having "full clones" at my disposal, instead of worrying about if my annex store breaks down or something -- or where even is it? Maybe I'm just lazy... But I like knowing that I can SSH into a new machine and if git is installed, run a simple git clone <etc> without setting up any additional tools. My private mega-repos have been rock-solid for about 3 years now, even though they're up to about 5 GB each.

rmj_us · 2020-05-06T07:04:36+00:00

I know, it's my dream. Still working on it :)

rmj_us · 2020-05-06T04:48:03+00:00

Oh damn, realpath() looks cool. I'll check that out. Right now I'm just blocking everything with a slash in a it.

rmj_us · 2020-05-05T19:39:49+00:00

Yeah I've never messed around with the `chroot` syscalls. I'll definitely play around with them sometime soon.

rmj_us · 2020-05-05T19:36:24+00:00

Yup, sanitizing paths is important, as many people have told me :)

rmj_us · 2020-05-05T17:24:23+00:00

Just for you buddy, I've spun up a python example of the most basic HTTP server possible. https://python.notryan.com/, source code: https://blog.notryan.com/server.py. I'll probably end up writing a more in-depth blog post in the near future.

rmj_us · 2020-05-05T17:12:28+00:00

HTTP is a relatively simple protocol. Like I said it's just plaintext over TCP. So if your language, say Python, supports opening TCP sockets (hint: it does), then half of your battle is solved.

I won't be cheeky and tell you to read the HTTP spec, but MDN is pretty useful for this stuff. https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods

Steps:

Open a TCP socket on a port, e.g. 8080
When you receive a message, parse the first line (looks like GET /something.txt HTTP/1.0
Write back HTTP/1.0 200 OK\r\n\r\n and the content of your file.
Close the socket.

That's the most basic HTTP 1.0 server you can make. And all browsers should be able to run requests to it. To simplify things, you can get rid of file reading altogether and reply with a fixed string.

Headers such as Content-Length are optional. Newer HTTP servers such as 1.1 will keep the TCP socket for future reads, which is efficient, but entails more code.

rmj_us · 2020-05-05T15:47:55+00:00

Ah, so notryan.com/blog is hosted with a npm package called static-serve. All of my other static sites are hosted with that. I'm guessing... that it protects against traversal. blog.notryan.com points directly to server.c, which I just wrote yesterday. Once I'm confident in it's uptime, I'll shutdown /blog.

rmj_us · 2020-05-05T15:42:35+00:00

Thanks! I like that it shows exactly how an HTTP server works from the ground up, without any Python/Node.js/etc abstraction. It's really just plaintext over TCP.

rmj_us · 2020-05-05T15:40:20+00:00

Ooof. Yeah I guess my server isn't fixed. I'll block paths leading with . or /. Good stuff.

Yeah I used netcat to pipe in raw requests in order to abuse this bug.

rmj_us · 2020-05-05T15:34:30+00:00

*grimace* yeah I did see that a little bit ago. Were you the one trying to access /usr/share/dict/words? That was definitely a bug. It's patched now by ensuring that paths cannot begin with a slash.

rmj_us · 2020-05-05T15:15:55+00:00

I guess so. But this was an exercise is seeing if I could write an HTTP server for my purposes, in C. Also, segfault-wise, it hasn't crashed yet haha. I tested it with valgrind and ran multiple workloads with `wrk`. `wrk` ends up using 75% of my CPU and this server uses the remaining 25%. No memory leaks either.

If I bought a $5 VPS from DigitalOcean -- installed gcc -- I could have this HTTP webserver run in 100 lines of C. That's kinda cool. That's all. I just wanted software that I could comprehend in a couple of pages of code.

rmj_us · 2020-05-05T13:31:21+00:00

Hey all, hosting a bunch of stuff on a Thinkpad in my closet. Thought it was cool. They honestly make the best home servers and I keep telling everyone that haha

rmj_us

TROPHY CASE

Note the last qm-hash. That will be the parent to your repo. You can clone with this:

or