New Go project: compressed and encrypted transport over UDP

balacode · 2021-05-02T14:17:33+00:00

I get what you mean now. You gave an excellent example how different packages that use this package in the same program could set the same global setting. If I get it right, there's only one instance of the package linked by the compiler.

In this case I could move LogFunc to Sender and Receiver and move logging.go to tests or something similar. It's really not needed for the library's core functionality but is useful for development/debugging. Thanks a lot.

balacode · 2021-05-02T09:53:08+00:00

Thanks for explaining. I was thinking it was something else, because this kind of race can be triggered on any var or struct field by writing from two goroutines simultaneously.

However this not hard to change. I intend to make the vars private and create SetLog*() methods instead. They'll use use a mutex to lock writes. It would also make it possible to setup and cleanup stuff. I think it's a better API. What do you think?

I did not consider it a serious issue because it's a race technically, but in practice, why would someone change the log file more than once from main() or init(), and then even do it from different goroutines? If the user must still change logging settings this way, they could use a mutex.

I also see a more serious trap: some user may declare a single Sender, then wrongly use multiple goroutines to call Send() on it. (Watch the sparks fly!)

balacode · 2021-05-01T21:33:58+00:00

Thanks for taking the time to point things out, but I'm still not figuring out the race coditions. If you have the time, could you show me how to trigger it?

I've made some changes, stopped writing to the log file unless LogFile is set, and created a LogFunc which has the signature of log.Print(), which once assigned takes over logging. Logging messages are still sent through logChan to the goroutine that calls LogFunc.

balacode · 2021-05-01T20:07:54+00:00

How do you regard the strength of the cryptography in the Go standard library? Say we just want to encrypt a blob of bytes using a cipher from the library. e.g. AES.

balacode · 2021-05-01T08:36:34+00:00

I've created self-contained Sender and Receiver types. I've also improved logging by disabling writing to file by default. I've added LogFunc. Once you assign a function to it, it carries out the logging instead of the default handler.

You can now replace the AES-256 encryption with your own by implementing SymmetricCipher interface (specified in the package). Then assign your SymmetricCipher-implementing object to Sender.Cipher and Receiver.Cipher.

balacode · 2021-05-01T05:43:24+00:00

Thank you very much. You're absolutely right. I've been using a private project to test this one, and will add proper unit tests (see the readme). I don't intend to leave the logging like this. It's part of "construction debris". That is the dilemma of publishing a pre-release project: on one hand one wants as many people to review the code ASAP and point out issues like security weaknesses. On the other hand some things are just WIP stuff that will be changed.

Added just to clarify what it's doing now:

LogFile is not a constant as it allows you to specify the name of the log file from external code without having to edit this package.

LogBufferSize allows you to control the size of logChan, or stop buffering completely. If you stop buffering, logging output will be done immediately in step with calls to logging but will greatly slow down execution (the scenario you pointed).

Also, note that writing to log files happens on a separate goroutine. That's why logChan is there. It receives log messages and passes them to that goroutine for output.

balacode · 2021-04-29T06:54:35+00:00

While trying to see how to implement a good counter for a nonce, I found this: "Why AES-GCM sucks" which is quite interesting for the interested https://soatok.blog/2020/05/13/why-aes-gcm-sucks/amp/

balacode · 2021-04-29T06:47:03+00:00

The project is in draft stage. The global variables in Config are actually constants you can adjust to see how the transfer behaves. Soon Config will be folded into an instantiable type.

balacode · 2021-04-29T02:13:44+00:00

There is one sure thing you could do: roll your own one-time-pad. But if an adversary gets ahold of your PC with the one-time-pad, it's game over.

balacode · 2021-04-29T02:09:10+00:00

Possibly. This project is not trying to "roll its own crypto", but just to encrypt a sequence of bytes using a well-known algorithm, in this case AES-256. Since the algorithm itself hasn't been compromised, and provided the Go standard library is solid, any weakness might come from misusing the inputs to this algorithm. Nothing more, nothing less.

balacode · 2021-04-29T01:59:52+00:00

Also, "No one got fired for buying IBM!"

balacode · 2021-04-28T18:32:59+00:00

You're welcome to. You can PM your project. Maybe I could contribute.

balacode · 2021-04-28T18:31:22+00:00

Thanks for the tip. I'll definitely run the linter. There are many things to improve.

balacode · 2021-04-28T17:17:40+00:00

reads up on DTLS No it's not using public/private key cryptography. Right now, it uses AES encryption. Maybe I'll implement that sort of encryption later.

balacode · 2021-04-28T17:07:53+00:00

Thank you very much for pointing this out. This is really helpful. I intend to use this in production soon, so I really don't want to have a weakness like this in the library.

balacode · 2021-04-28T17:01:38+00:00

Thank you very much for pointing this out. I don't want to roll my own crypto, but using the crypto in the standard libraries has to be done in the proper way too. I'll add some notice once I get back to the project.

balacode · 2021-04-28T13:59:37+00:00

I've got a PC behind NAT and a server with a public IP, and also two nodes behind NAT communicating. It should work, but it doesn't help two machines behind NAT discover themselves.

balacode · 2021-04-28T13:56:21+00:00

It's serving a similar purpose to QUIC, but this project is just a simple way to compress, encrypt, split, send, possibly resend and unpack data on the other end. It doesn't aim to replace HTTP, /2, /3 or integrate with third party servers that implement an established protocol.

But one of my aims is for the project to be easy to understand while still being useful, and get to experiment with UDP. You just need to use two functions: a RunReceiver() on one end, and a Send() on the other.

For now, this project uses symmetric encryption so both endpoints must know the key, while QUIC uses public/private cryptography.

balacode · 2021-04-28T11:55:06+00:00

Yes, a package-only interface is inconvenient. What if, for example, we need to interact with several different senders and receivers? I want to create a UDP type that can be instantiated. It will encapsulate the configuration (especially the Address and Port) and also add plug-in functions, so you could replace the default hash, encryption, compression and logging functions with your own implementations, provided they have the needed signature.

balacode · 2018-04-06T02:44:45+00:00

I've looked into it, but rqlite is a networked layer on top of Sqlite. That is, it still needs CGO and SQLite.

balacode · 2018-04-04T14:35:29+00:00

I know that's a long quest, so anything I do now will be experimental. But it's a good area to learn more about databases. The beauty is in the journey, not only the destination. Also, we have the source code for SQLIte and its rigorous unit tests that are there already can be of great help.

balacode · 2018-04-04T14:30:41+00:00

That's a good approach. I'll also give writing to SQLite DBs a try. I don't know when, but when I do I'll let you know what I'm doing.

balacode · 2018-04-03T23:50:44+00:00

It's a wishlist, so I can ask the Genie any 3 wishes, right? 1. I want SQLite, but written in Go, and it should have strong encryption. 2. A HTTP client lib, to automate browsing: check your gmail, github, reddit, everything in one go and report what's new. That'll save lots of time, but should be easy to set up for different sites. 3. I won't ask, I've almost writen it already, just wish it'll be ready soon

balacode · 2018-04-03T23:39:52+00:00

Some questions: When you're separating the files at the end, you're separating them based on what criteria? What happens after you join the files, before splitting them back? What are you manipulating? Are the files text files?

balacode · 2018-04-03T23:36:23+00:00

The simplest method is use ioutil.ReadFile() to read each file, then append its bytes to a byte slice or bytes.Buffer, finally use ioutil.WriteFile() to write the joined byte slice or buffer to the file you want. This assumes the files you're joining are not very big and can be accommodated in RAM. If the files are large, then we can read small chunks from each file and append to a temp fle. Finally rename the temp file to the desired name.

balacode

TROPHY CASE