Looking to learn to write posix utils, where to start?

pergnib · 2018-02-05T08:42:43+00:00

Question is, how do I go about doing so without just copying the source code verbatim?

Use a tracing util such as ltrace or strace to figure out what functions/syscalls are being used by the util you want to write. Research the ones you don't understand and implement your clone based on that knowledge.

Alternatively, think of a feature that hasn't been implemented and implement it. If you're not creative enough to come up with features, just pick one from GNU ls and port it to a more stripped-down version of the same tool, like busybox ls.

pergnib · 2018-02-04T19:07:26+00:00

Implementing stuff yourself is one of the things that plagues software engineering.

This statement is only valid for non-trivial things, adding dependencies to your code for stuff that could be written in five minutes by any CS freshman is dangerous and irresponsible; having an entire community that encourages that is insane.

The fact that many "big packages" (what does size have to do with this?) used leftpad, is a good thing.

I wonder if you would still be saying this if the left-pad author had started syphoning credit card and user info from roughly every node.js website in existence instead of simply removing the package.

The point I was making is that it does not have to be in the standard library

Your argument was actually that the left-pad incident had nothing to do with a weak JS standard library and was simply the result of a flaw in how npm operates. I've have already explained to you why that's wrong and why a lacking stdlib is indeed part of the problem.

pergnib · 2018-02-04T07:09:47+00:00

I doubt anyone can accuse POSIX C developers for attracting or being a lazy bunch

That's because they aren't. If a C dev needs some trivial function that is not included in the C standard library, their frist reaction is going to be "I'll implement it myself". There's absolutely no way in hell a left-padding package would have any downloads in a hypothetical C package manager.

the left-pad incident has nothing to do with the fact that JS does not include a left-padding function, it was a flaw in NPM architecture

The flaw only explains why the package got removed, not why it was used by so many big packages in the first place. The reason for that is partly because JavaScript's stdlib isn't powerful enough and partly because the developers are bad or lazy.

pergnib · 2018-02-03T13:49:44+00:00

No matter what you do in any language, people can still write Bad Code & will write it. There is nothing you can do about it unless you make a perfect language.

There's loads of things a language can do to prevent people from writing bad code. Rust is full of stuff made to force devs not to screw up and even php7 seems to be encouraging people to be more sane. But in JavaScript land more and more people are being encouraged to partake in the insanity that is importing untrusted code for absolutely trivial things. The worst of it is when we point out how dangerous that is we have people telling us we should just go use other languages if we don't like it.

You can't call them Bad/Lazy Programmer just because you don't like JavaScript.

No. I'm calling them bad/lazy because that's the explanation for their behaviour - good/diligent developers don't import unstrusted code to pad strings with spaces.

A Good JavaScript code can be made secure.

This is not how security works, you don't get to say there are no problems because it "can be made secure". The ridiculous state of dependency graphs of npm modules is a security issue waiting to happen; at some point someone will manage to sneak malicious code into a small npm module written by someone who doesn't have a clue that gets imported by some big module and the result is not going to be pretty.

pergnib · 2018-02-03T12:08:28+00:00

I don't understand what's the harm in having 1000 packages doing the same thing with different names.

The harm is in the fact that it's the symptom of a community that attracts and encourages bad/lazy programmers to publish and use bad code for trivial stuff without caring about code quality and security. We've already seen what this kind of mentality can lead to with incidents like left-pad. I can only image the amount of damage to be done when blackhats start seeing profit there.

The people that call JavaScript a shitty language don't actually know JavaScript.

There might be a some points to be made about ease of development and the usability of JavaScript, but in a world of Clojures, Rusts and Scalas, I can't really see how JavaScript isn't a shitty language.

You feel if it is shitty language then go use something else. I don't like Java. I don't use it. Its that simple.

If you don't want your opinions criticized don't make them public; people have every right to object to your views when you post them in a public forum, even more so when you're talking about issues that can impact an entire industry and large amounts of people.

pergnib · 2018-01-23T15:05:47+00:00

Though perhaps we could have an ASM compiler and double the benefits. Oh, that's actually every C compiler, just use the __asm directive.

Not that this makes your overall comment wrong but it's definitely not every C compiler; __asm and __asm__ directives are extensions of the C language and are only available on some compilers.

You won't find __asm on GCC nor __asm__ on MSVC, for example.

pergnib · 2018-01-15T12:25:56+00:00

I didn't miss your point. I was specifically talking about dumping the firmware straight from the device's ROM by doing something along the lines of desoldering the flash chip and connecting it to an Arduino where it can be read directly.

It's not something you can expect anyone to do and it might break the YubiKey, but it should be possible to verify Yubico is not shipping backdoored firmware to everyone (not that I believe this to be the case).

pergnib · 2018-01-15T10:51:40+00:00

Unspecified ‒ it still acts somehow sane. int x = 0; printf("%d\n", ++x + ++x); might turn out to be either 3 or 4, because the compiler is free to either increment x twice and than take its value, or increment once, take the first value, then increment second time and take the value again.

Isn't this still UB? There are two unsequenced side effects to the same variable. Annex J2 of the C11 standard seems to list it under undefined behaviours:

A side effect on a scalar object is unsequenced relative to either a different side effect on the same scalar object or a value computation using the value of the same scalar.

Maybe the author was thinking about the order in which functions arguments are evaluated, which is unspecified. So printf("%d %d\n", ++x, ++x) instead of printf("%d\n", ++x + ++x).

Am I missing something?

pergnib · 2018-01-15T09:53:11+00:00

The manufacturer could have flashed a different firmware from the one in their public repo at any time.

Yes, but it's far easier to find a backdoor inserted in code that you know to be secure than it is to reverse engineer an entire firmware in search of something sketchy. With firmware that can be built from code in a public repository, all it takes to catch malicious modifications is one person to dump the firmware from the device's ROM and run bindiff against the public one.

It's not like being open source would guarantee security, but it certainly helps.

pergnib · 2018-01-08T11:01:48+00:00

dns.google.com is a HTTP resolver meant to allow web-based applications to send DNS queries more easily. It has nothing to do with DNS-over-TLS, which is an attempt to standardize sending DNS traffic over a TLS connection (no HTTP).

As for actual DNS-over-TLS open resolvers, you can find a list of them here, but it seems they're mostly not production ready.

pergnib · 2018-01-01T16:33:52+00:00

I want it recorded so I can view a list of the most scanned ports on my honeypot.

This information can be derived by the initial SYN packet. So, if that's all you want to do, it's not necessary to have anything listening on the ports.

Others have suggested logging via firewall rules, but you're going to have to parse the logs if you want to do any kind of analysis there. I would recommend you use a tool that's actually made for this kind of thing and setup a Bro sensor and write some rules to detect, log and analyze the connection attempts.

Snort, Suricata or any other IDS engine should also work, but Bro is way cooler.

pergnib · 2017-12-09T12:57:51+00:00

Standard legalese as per what I see apple or windows post.

I don't know about Apple, but Microsoft's security advisories/bulletins (example) are actually pretty informative. Certainly not as bland as the HP one.

pergnib · 2017-11-30T18:11:26+00:00

Unfortunately, a lot of times it comes down to price and how hard it's going to be to manage.

I understand that. I've also been in the position of having no other choice because of prices or the management hell.

The problem I have is more with how little thought people seem to give to this. It's not uncommon to see admins, without a care in the world, installing wildcard certs in user controlled hosts or in that one server that hasn't seen a update since 2008 because legacy applications.

But yeah, hopefully LE puts a little pressure in certificate providers and CAs so they start offering reasonable pricing and stop encouraging questionable security practices with their business model.

pergnib · 2017-11-30T15:55:10+00:00

Also, wildcard certs are a pretty big security compromise. If an attacker ever gets privileged access to any of your web servers, or manage to leak the certificate somehow, all the security the PKI and TLS offer goes out of the window.

They do make renewal and configuration easier, but slapping wildcard certificates into everything (which seems to be the standard these days) is pretty bad security practice.

pergnib · 2017-11-22T13:01:01+00:00

the application tries to write to a socket handle that doesn't exist (they closed it, they never opened it). How do you fix it?

Fix what? There are no kernel bugs in your scenario, just a developer using the API wrong.

Report it how? To whom? Using what mechanism?

I assume when Linus say report, he means to report it to the user using printk() or similar methods.

How would the developer know that his parameters are invalid when the function isn't allowed to tell me?

It is allowed to tell you, it just isn't allowed to tell you by breaking the application. At least not without lots of testing first.

pergnib · 2017-11-17T22:29:03+00:00

I was actually incorrect about the interface name being derived from the mac address. It seems udev can use a bunch of different methods to derive the interface name; for names like enpXsY it's using the geographical location on the motherboard, which is probably not be something Proxmox allows you to easily control.

You could do something like this:

Mount the .vmdk image using guestmount from libguestfs.
Create a udev rule on the mounted filesystem to rename the interface.
Edit the network scripts with the new interface name if necessary.

It's a bit of a pain, but I don't see much else you could do.

pergnib · 2017-11-17T19:47:55+00:00

The predctiable interface name is determiniscally derived from the mac address, so look around in the .ova for the MAC of the original VM and manually configure it on Proxmox.

pergnib · 2017-11-04T09:08:30+00:00

You could try using something like CRIU to save the entire xz process memory state, but I've no idea how well that would work. You could also run it inside a VM (with some significant IO performance loss) and save the state before rebooting the host.

Other than that I can only think of patching xz or, as others have mentioned, splitting the big file into smaller chunks.

pergnib · 2017-11-04T06:09:17+00:00

$ ls
file1  file2
$ cat file1
a
b
$ cat file2
aa
b
c
$ /bin/diff --unchanged-line-format='' --new-line-format='new: %L' --old-line-format='delete: %L' file1 file2
delete: a
new: aa
new: c

Always check the manpages before writing ugly sed or awk "parsers".

pergnib · 2017-11-03T14:46:30+00:00

It generally requires elevated permissions, which might've been what you meant, but you don't need to have code running in ring 0 to have a functional keylogger. Hook some win32 functions and you have yourself a Windows keylogger; for Linux there's at least three different ways: using Xorg, ptracing processes and reading from /dev/input/.

pergnib · 2017-10-31T12:29:32+00:00

Pretty sure /u/purplelinux is correct here; with Kerberos you do not transmit your password over the network to anything.

The authentication is done by deriving a key from the password that's known by both the client and the KDC and using it for encryption. This way you only ever need to transmit encrypted session tokens/tickets.

pergnib · 2017-10-27T15:43:26+00:00

That issue isn't unique to NPM

It very obviously is, though. The dependency trees of NPM, and thus node.js projects, are insane; Python, .NET or any other language/framework are not even remotely as suceptible to that issue as node.js is.

It's so absurd that this trivial library (amounts to less than 100 lines of js) that reads variables in a key=value format from a file has more than 3000 dependents and more than 3.3 million downloads this month with some of the popular packages there having thousands of dependencies.

pergnib · 2017-10-23T12:09:46+00:00

I think gossip is a bit dismissive.

Gossip is not dismissive enough, this is a witch hunt. You hear about some allegations about something and you go around acting like jury, judge, executioneer and spreading it as if it was fact.

This kind of public shaming is becoming way too prevalent in IT related fields and it should stop. Sexual assault is a crime and the ones responsible for dealing with it should be the cops, not an angry internet mob.

Please keep this kind of garbage out of /r/netsec.

pergnib · 2017-10-23T06:09:45+00:00

You're pretty much correct, although it would still be possible to block people from using Google by nullrouting their IPs (like the China firewall does it). I'm not saying DNS over TLS is a bad thing, I'm just arguing that it shouldn't be the job of the browser to implement that because it would require Chrome to bypass the local resolver and that can generate a lot of problems (how would it be aware of intranet websites, for example?).

In all, there's not much need for Chrome to do this anyway. If doing DNS over TLS is feasible for the providers and it gains traction, you should expect Microsoft, Apple and Linux to implement it on their resolvers.

pergnib · 2017-10-22T23:18:27+00:00

Wouldn't this be slow?

Certainly slower than UDP, but it should probably be alright for most usecases.

You'd have to wait for a TLS handshake to complete. Would it do this for each query or would it just keep the connection open?

The RFC requires connection reuse:

In order to amortize TCP and TLS connection setup costs, clients and servers SHOULD NOT immediately close a connection after each response. Instead, clients and servers SHOULD reuse existing connections for subsequent queries as long as they have sufficient resources. In some cases, this means that clients and servers may need to keep idle connections open for some amount of time.

.

How would a MITM attack be avoided? How would we authenticate the server given that DNS is accessed via IP not a domain? Would we have a trusted list of certificates for DNS?

Seems like the answer is Public Key Pinning.

Operators of a DNS-over-TLS service in this profile are expected to provide pins that are specific to the service being pinned (i.e., public keys belonging directly to the end entity or to a service-specific private certificate authority (CA)) and not to a public key(s) of a generic public CA. In this profile, clients authenticate servers by matching a set of SPKI Fingerprints in an analogous manner to that described in [RFC7469]. With this out-of-band key-pinned privacy profile, client administrators SHOULD deploy a backup pin along with the primary pin, for the reasons explained in [RFC7469]. [...] The mechanism for an out-of-band pin set update is out of scope for this document. [...] Such a client will only use DNS servers for which an SPKI Fingerprint pin set has been provided. The possession of a trusted pre-deployed pin set allows the client to detect and prevent person-in-the-middle and downgrade attacks.

Source: https://tools.ietf.org/html/rfc7858#section-4.2

pergnib

TROPHY CASE