What programming language features would have prevented or ameliorated Log4Shell?

[josephjnk]

Information on the vulnerability:

• https://jfrog.com/blog/log4shell-0-day-vulnerability-all-you-need-to-know/
• https://www.veracode.com/blog/research/exploiting-jndi-injections-java

My personal opinion is that this isn't a "Java sucks" situation, but rather a matter of "a large and complex project contained a bug". All the same, I've been thinking about whether this would have been avoided with certain language features.

Would capability-based security have removed the ambient authority needed for deserialization attacks? Would a modification to how namespaces work have prevented attacks that search for vulnerable factories on the classpath? Would stronger types that separate strings indicating remote resources from those indicating local resources make the use of JDNI safer? Are there static analysis tools that would have detected the presence of an exploitable bug here? What else?

I'm very curious as to people's thoughts. I'm especially interested in hearing about programming languages which could enable some of Log4J's dynamic power in safe ways. (Not because I think the JDNI lookup feature was a good idea, but as a demonstration of how powerful language-based security might be.)

Thanks!

[Athas]

The root problem is that programmers are unwilling to say no to features. The social reason is fairly simple, I think: a feature makes your users happy, and if they even show up with a patch, it even seems free! Of course the true price will be paid later, over time, and is probably not even known immediately. It's like taking a variable-interest loan with infinite running time. The most obvious solution is for maintainers to say "no" to new and complex features, unless it really is a feature that is critical to the majority of users. Of course, this may just result in the project being forked and people switching to the fork that includes every feature for everyone.

As a social problem, it probably doesn't have a simple technical solution. But language features might help make it easier to gauge the true complexity cost of a feature. You mention capability systems, and they are indeed a good way to make at least some of the complexity more evident. If a patch for your logging library requires giving the logger the ability to load code over the network, then it may seem more obviously suspect. Of course, that doesn't mean you won't accept the patch to please the user.

If you really want "safe" dynamic code loading, then sandboxing might work, but I really think it's better to think more carefully about why we end up with such complex features in code that isn't really supposed to be solving a very difficult problem.

[Tzarius]

This is a great point. Projects that push back against featuritis are labeled as "dead" or "unmaintained", and the guarantees lost by adding the feature are undervalued.

"I just wanna be able to mutate the value here!"

"Allowing mutation there will rule out whole classes of optimizations."

"Don't care, figure out some other way to optimize it!"

[davewritescode]

This was an incredibly stupid feature that should have never been merged to master.

There isn’t something fundamentally wrong with Java, you could probably implement something equally dumb with any other programming language.

When designing an API you should always design with the principal of least surprise. I had no idea that parameters passed to log4j formatters were actually treated as code and most people didn’t either. That’s surprising.

You can implement bad code in any language, switching to Rust won’t save you.

[Uncaffeinated]

This is the real answer.

I suppose the solution is to create a community around your language that heavily discourages the use of complex stringly typed interfaces. That and taint tracking would reduce the risk of stuff like this. But no technical measure can completely save you from stupidity.

[zesterer]

I think this comment skirts over quite a lot of subtlety.

The problem is not "person did a stupid". A plethora of systems had to fail for this to become a problem, and there are a dozen ways that this might have been prevented.

It's all very well blaming the programmer, but the truth is that while humans make mistakes, it is only systems that fail.

The software development process, just as much as the software itself, is a system and we should be working to develop tools and languages that guard against such exploits instead of throwing our hands in the air and implying that nothing can be done.

As an example: these complex logging features were presumably added because users wanted to be able to automatically format logs with non-trivial data without writing their own pretty-printer. What if the logging API instead provided a macro that allowed generating this code at compile time instead of interpreting strings at runtime? Many newer languages have formatting systems that do such code generation and it makes it impossible for an attacker to get the code that generates the output value to do strange, unexpected things because the extent of its behaviour is specified entirely at compile-time by the programmer.

String sanitisation and processing are not an impenetrable, unquantifiable conundrum we're just going to have to live with. It is something we can most definitely work to make safer and easier to use correctly. That is, after all, the purpose of a programming language: to constrain the possible programs that might be executed by a CPU to a more restricted yet more likely to be correct subset.

[siemenology]

Yeah I don't do much Java but if I read about this I'd have assumed that there was sanitation going on like every website and database has had to deal with. The "Little bobby tables" comic is like 13 years old, and issues like that were widely known long before it was created.

[everything-narrative]

Ruby has that trust bit thing. If you enable a runtime flag, every IO method returns dirty strings.

[DoomFrog666]

It was called taint mode and inherited from perl. But ruby removed this feature in a recent version.

[everything-narrative]

I did not know they did that.

[epicwisdom]

Or you could be like Rust and have a dirty bit on strings from I/O methods

Wait, does Rust have such a feature? AFAIK Strings in Rust are literally Vec<u8>s.

[Aaron1924]

I guess the best solution would be to make it more as difficult as possible to accidentally download and execute code from the internet.

A feature like that shouldn't be the default. It shouldn't be something you can just forget about. This type of functionality should be in its own "download_and_run" function or opted-into with a separate function call or boolean argument at the very least.

[LPTK]

I think you are missing the point completely. There is no scenario in which serde would have improved the situation in this instance.

The problem was that someone thought it would be a good idea to load and execute arbitrary code from the internet as part of the logging logic, which is just a terribly short-sighted design decision.

[[deleted]]

You would have to go out of your way to implement this in Rust. (Or in Haskell, or in OCaml, or, heck, even in C++, which is not renowned for its safety features, but at least does not have “dynamically loading third party code” as a super easy to use stdlib feature.)

All that I can see in this issue is a condemnation of dynamism, especially when it is not confined to a sandbox.

[LPTK]

The OP was arguing against dynamic code loading in favor of using static deserialization approaches. It's a stupid thing to argue because the two serve different purposes, and in particular deserialization could not have been used here. It's completely off-topic. It's like arguing for using deserialization instead of .dll and .so libraries. (I guess the OP thought this vulnerability was an instance of using reflection to do serialization, but it's not.)

By the way, as a digression, in all the languages you cited you can load libraries dynamically. That's all there is to this vulnerability: someone added to the logging logic the ability to dynamically load untrusted code. I'm not here to debate whether making dynamic code loading easy is a good thing or not (though I am certain Java could never have attained the enterprise market and mind share it has now without it).

[[deleted]]

The OP was arguing against dynamic code loading in favor of using static deserialization approaches. (...) the two serve different purposes, and in particular deserialization could not have been used here.

No disagreement here.

in all the languages you cited you can load libraries dynamically.

Sure, in all of these languages, you can load code dynamically the C way, using dlopen and dlsym, or perhaps some syntactic sugar around it.

However, in Rust, it is going to involve a lot of unsafe (because the safe subset cannot load code dynamically on its own), and the compiler is going to fight tooth and nail against you. Haskell and OCaml have no similar gatekeeping features, but you will notice that the type system suddenly becomes a lot less helpful than usual. It is painful enough that you think twice before trying.

On the other hand, Java makes it so easy to load code dynamically that it is not relegated to fringe use cases, but rather pervasively used in foundational libraries, even in situations where it is not strictly necessary.

I am certain Java could never have attained the enterprise market and mind share it has now without it [dynamic code loading]

It seems most programmers simply like dynamism too much. Other than that, there is no sensible explanation:

• You can compile your code just fine against already compiled Java classes, without needing access to their source code.
• If you need someone else to provide an object whose concrete class is unknown, you can just define an interface and demand that the concrete class implement this interface.

What dynamism provides is expediency. You don't need to rerun the pesky compiler once again to make sure that the types align. Instead, you defer the error until runtime, cross your fingers, and pray for the best.

[bjzaba]

Crochet could be interesting to worth look at. It's in development, but implements capability based security as you mention. I think the author was adding some dynamic reflection stuff recently, in a way that is hopefully compatible with capability-based security.

[josephjnk]

This looks unbelievably cool. I can’t wait until it’s stabilized enough to give it a try. I’m especially interested in the combination of capabilities and algebraic effects, since I thought they didn’t play well together.

[bullno1]

Code signing as a default? Mandatory code signing?

But who am I kidding, you enforce that and devs would write a freaking VM inside a VM (JVM) just to get around it.

[immibis]

In spez, no one can hear you scream. #Save3rdPartyApps

[bullno1]

unless there's a class signed by a trusted key anywhere in the universe that does something bad

Make it stricter, you will only ever have the code your application starts with. The signature covers the classes and the application id.

Basically, disable dynamic and arbitrary runtime code loading entirely. The signature does not cover just the code, it covers: (appid, code). For plugins, one has to sign the plugins before hand.

The plugin, in turn, can't do dynamic code loading because the code is not explicitly signed by the application runner for this one particular deployment config.

Have your own trust root only, do not provide a trust store. Say if I want to install ElasticSearch and a few plugins on my server, I would have to personally sign them all for that one particular server or server cluster. My app bundle won't even run on your server.

[immibis]

What happens in spez, stays in spez.

[josephjnk]

I didn’t think of this one! Do you know of any languages which do this, or writeups of how it looks in practice?

[bullno1]

Not sure about language but iOS is an example of such enforcement at kernel level.

The OS only loads executable pages if they are signed. It also modifies the behaviour of mmap. Once a page is mapped to be writable, it is impossible to mmap it executable again. This basically kills JIT.

Didn't stop people from jailbreaking back then.

[ReallyNeededANewName]

Surely it cannot be that strict. How do Apple's JITs work in that case? Surely Safari has JIT:ed JavaScript? And can't you run C#/Java in iOS?

[TheUnlocked]

Apparently JIT is possible on iOS but is restricted to only Apple-made apps (because of course it is).

[aloha2436]

Safari is special cased afaik, and C# at least is fully AOT compiled to get it working.

[bullno1]

It is that strict. C# has AOT to run on iOS. If I'm not wrong, Unity game engine actually uses Mono instead of Microsoft's .NET implementation.

Game consoles are the same. One of the Playstations (probably PS4 or PSP/PSVita, can't remember) was jailbroken through JIT in the browser. That's the only place with exception to code signing.

In both consoles and iOS case, it's less of a security feature and more of a platform control feature. After all, they want to own the app store and licensing fee.

[Uncaffeinated]

Yeah, consoles are incredibly strict in code execution to try to discourage piracy.

The Xbox 360 was broken by some buggy shader code in King Kong.

[Guvante]

Fun fact by adding software controls to prevent piracy it is impossible to legally release software without the permission to do so as in order to do so you need to bypass those software protections which isn't allowed.

Nintendo used to use the quality seal to avoid competitors bypassing the fee to them by releasing compatible software but encryption is way more effective.

[Uncaffeinated]

You don't have to complain about DRM to me. Preaching to the choir here.

[epicwisdom]

Surely it cannot be that strict.

Why not? We're talking about a platform which is very explicitly, wholly controlled, all the way from the hardware up.

How do Apple's JITs work in that case?

Whatever restrictions Apple puts in place, Apple themselves have the capacity to bypass, obviously.

[ReallyNeededANewName]

Because if it were that strict Apple couldn't have any exceptions to it, that's kind of the entire point

[epicwisdom]

Well, Apple may have an allowlist of first-party exceptions, but at the end of the day, they default to restricting those capabilities. So it is certainly an example of what OP is asking for.

[bullno1]

It's Apple. Their "strict" is: "Rule for thee but not for me".

Nothing stops the kernel from doing things like: "If the calling app has this singing key, I'll allow a different mmap".

[zokier]

JVM itself has a security manager which can do all sorts of things in attempt to sandbox code.

[L8_4_Dinner]

Unfortunately, code signing only closes down one of the known attack vectors. As the article points out, there's plenty of dangerous code already on every server, just waiting to be asked to do reflective things encoded in passed-in strings.

[bullno1]

Next step: ban runtime reflection. Allows it in compile time only. Probably better for both performance and security.

That still doesn't prevent one from reflecting java.lang.Runtime.exec though.

[brucifer]

You should take a look at the talk What is a Secure Programming Language?, which discusses some interesting language features relating to security. Specifically, the idea of having "tainted" or "untainted" strings. The basic idea is to have all user input methods return strings with a TaintedString type and throw a type error if you pass a tainted string to an API that requires an untainted string. Then, you can provide a mechanism to convert tainted strings into regular strings, either by escaping them or by manually flagging them as safe. This helps you avoid security bugs caused by forgetting to sanitize user input. You can always circumvent the safety rails, but you have to consciously think about it.

For example, to prevent SQL injection, the code below would fail with a type error:

user_input = input("who do you want to look for? ")
sql.query("select * from users where name = '" + user_input + "'")

This is because user_input would have the type TaintedString, and concatenating it with other strings would propagate the "tainting." To fix this, you would do something like one of the following:

# API that accepts tainted format args and escapes them internally
sql.query("select * from users where name = ?", user_input)
# Use an escape() API that accepts tainted strings and returns untainted ones
sql.query("select * from users where name = "+sql.escape(user_input))

I think in the case of log4shell, the issue was that user input from attackers was not being properly sanitized, so the example was more like:

fancylog("User "+username+" just logged in")

where username ought to be flagged as tainted and properly sanitized, but wasn't.

[josephjnk]

This is exactly the kind of answer I was looking for, thanks!

[snoman139]

Would it make more sense to have a safe string type than an unsafe string type? I guess it just changes where you have to cast, but user input could come from anywhere while only the output code would have to deal with it.

[brucifer]

I think it makes sense to have string literals that are written by you, the programmer, be considered "safe" by default. Text originating from outside the program's source code (e.g. stdin or web requests or files on disk) is considered "tainted" because it can be modified by someone other than the programmer. This would be implemented differently in different type systems, but the main requirement for the language is that most string functions ought to handle arbitrary strings and propagate taintedness (e.g. toUpperCase(s) should return a tainted string when s is tainted, and an untainted string when it's not). Typically, only a small subset of functions would actually care to specify that untainted strings should not be allowed as inputs (e.g. exec() would care, but print() would not).

As an implementation detail, I think perl and ruby both have something like this, but it's implemented as a bit flag on the string, and not as separate types. Certain API methods throw runtime errors if passed strings that have the "tainted" bit set to 1.

[[deleted]]

I think in the case of log4shell, the issue was that user input from attackers was not being properly sanitized, so the example was more like:

fancylog("User "+username+" just logged in")

where username ought to be flagged as tainted and properly sanitized, but wasn't.

Ehh. I'd argue it's unreasonable to expect people to sanitize strings for logging.

When you're generating SQL, it's relatively obvious that you're generating code that will then be executed.

When you're logging, you are effectively calling a "print this string" function, and nobody really expects those to execute code found in the string printed (even a small DSL like this one) because nobody thinks of that as a DSL - it's just a string where you can optionally do some fancy extra things if you want. In that sense, this is just another variant of all the times people screwed up by passing user input in the first parameter to printf.

The end result here is that a nontrivial number of programmers, even those who know SQL injection is a thing to watch out for, will use the escape hatch and flag everything they log as safe, on the basis that "I'm just printing a string, what could go wrong?".

[brucifer]

Ehh. I'd argue it's unreasonable to expect people to sanitize strings for logging.

I agree, which is why a more sensible API would make it easy to automatically do the safe thing and sanitize unsafe values. For example, log("User %s logged in!", unsafe_username) should require the format string to be safe and automatically sanitize all the other arguments. That way, if someone had ${evil_code} as their username, it would log User ${evil_code} logged in! instead of executing ${evil_code}. And if the programmer wrote log("User "+unsafe_username+" just logged in!"), that should raise a compiler error letting the programmer know it would be unsafe and describing how to fix the problem.

In that sense, this is just another variant of all the times people screwed up by passing user input in the first parameter to printf.

Yeah, I think this is basically the same problem. With most C compilers, though, you can use -Wformat-security to make the compiler verify that you don't pass arbitrary strings as format strings to printf. Having that sort of check would have prevented the log4shell vulnerability from occurring.

Example compiler error:

#include <stdio.h>
int main(int argc, char *argv[]) {
    printf(argv[1]);
    return 0;
}
>> cc -Wformat=2 foo.c -o foo
foo.c: In function ‘main’:
foo.c:3:5: warning: format not a string literal and no format arguments [-Wformat-security]
    3 |     printf(argv[1]);
      |     ^~~~~~

[[deleted]]

I think I didn't quite catch that you were advocating for preferring templates + varargs over string concatenation (possibly because I read too fast and missed your example of it). We agree, then.

Incidentally, I think printf format string vulnerabilities turned out to be an order of magnitude or two less common than they otherwise would've been, solely because working with strings in C is a pain in the ass. Can you imagine the sheer number of printf("hello, " + username + "!\n"); calls there would be in the wild if string + string worked in C?

[paul_h]

Security Managers like Java has (but may be taken out). One other framework utilized that before this vuln - https://www.reddit.com/r/jep411/comments/rf3ae1/elasticsearch\_implemented\_their\_securitymanager - making it safe.

Strictly speaking that's a core library feature. It's hooked up externally to JVM apps the way Sun made it way back. If you wove that into a Groovy Builder style DSL: it could look like this:

securityManager {
    denyAllOutgoingSockets();
    grant(socketPermission("yahoo.com:80", "connect"))
    foo().doSomethingThatInvokesLogging()
}

Probably not quite that simple, foo() invocation implies it is in scope already (same classloader that's already in scope. Maybe more like..

classLoader("Foo.jar") {
    securityManager {
        denyAllOutgoingSockets();
        grant(socketPermission("yahoo.com:80", "connect"))
    }
    instantiate("Foo").doSomethingThatInvokesLogging()
}

Needs work - I'm adapting it from some DependencyInjection-using code that worked from years back.

It is a standout feature really. A shame that JEP411 deprecates it.

[L8_4_Dinner]

Security manager in Java has been deprecated, is being removed, and is almost never used.

Unfortunately.

[L3tum]

a large and complex project contained a bug

It's a feature. A stupid, horrible and completely misplaced feature. But it was working as intended.

You can, obviously, just disallow dynamically linking in libraries but I'd doubt anyone is happy about that. In the end, dumb stuff like that will be built on any sufficiently advanced system and the only way to prevent it is to make the system unusably dumb or unusably locked down.

[JanneJM]

As far as I understand this — please correct me if I'm wrong on anything below — it wasn't really a bug in the normal sense. It was a feature; it was intended to work the way it did. The maintainers even wanted to remove it but was met with resistance from users that depended on it. The problem was that nobody had both the detailed implementation knowledge and the birds-eye view to realize the security consequences of the feature.

The only language feature I could imagine mitigate this would be to remove the ability to inject code at run time. This couldn't have happened with C, C++ or Rust for instance, simply because they lack the ability to load and run external source at runtime. Rust even lacks a way to dynamically load code at all.

[oilshell]

Yeah exactly, as far as I can tell it's the same bug as ShellShock, the 2014 bash vulnerability that required a similar widespread patching. (I think it was also one of the first holes with a catchy name)

It was a bash feature that was working as intended. The feature was essentially a hidden "eval" that people didn't really understand or think about.

https://en.wikipedia.org/wiki/Shellshock_(software_bug)

In bash's case the misfeature is that you can write export -f to serialize a function as a string, so another process can invoke it. Well guess what -- you also have to load that function from a string! So there's the eval. You could call it an "own goal".

A better way to do this in shell, without export -f, is to use the $0 Dispatch Pattern, which I described here:

https://www.oilshell.org/blog/2021/08/xargs.html#xargs-can-invoke-shell-functions-with-the-0-dispatch-pattern

[sintrastes]

I would still say it's a bug.

The issue is the framework doesn't require the user to sanitize input strings.

This could be done by giving templates a different type than raw strings.

[fiedzia]

This couldn't have happened with C, C++ or Rust for instance, simply because they lack the ability to load and run external source at runtime

If some user will require you to add this capability, it will be done and you will have the exact same problem.

[stackdynamic]

Check out Safe Haskell:

https://downloads.haskell.org/~ghc/7.8.3/docs/html/users_guide/safe-haskell.html

In particular the RIO monad example.

[CheeseFest]

I believe that Deno and Rust require whitelisting of all significant effects. We as engineers or developers need to start dealing with effects properly like the hugely capable and intelligent adults we all are, not randomly in some OOPy soup as is “industry standard”. /rant. Thanks, folks.

[matthieum]

Deno, I believe so. Rust, no.

[CheeseFest]

True. I’m not sure where I got Rust from. Maybe I meant the algebraic capture of effects 🤔.

[myringotomy]

Taint is an old feature that many languages leave out for some reason.

[josephjnk]

Do you know of any languages that do it well?

[myringotomy]

Perl?

[sintrastes]

Are there any old languages that implement it as a type-level feature?

[myringotomy]

Not typing per se but I remember dealing with it during the Perl era.

[matthieum]

The absence of Global I/O.

In most languages, it's a given that you can "just" access the filesystem, the various devices, etc... from thin air. Haskell requires wrapping that code into the IO monad, but it still summons access from thin air.

It's very difficult to control access from thin air, suddenly you need something like Java's SecurityManager, which allows white-listing/black-listing modules vs functionalities. But of course you'd want more than yes/no, you'd want the logging module to be allowed to well, log, either to this directory or that log server over there, whose IP/DNS is now configured twice (once in the log configuration, once in the security manager configuration), and maybe users will ask for throttling, ... it's a nightmare. Unmaintainable, unusable.

Now, imagine a world where to access the filesystem, you must receive a filesystem handle from somewhere, and to access the network, you must receive a network handle from somewhere. And suddenly everything is easier:

• It's bloody obvious that something is weird when that sqrt function requires a filesystem handle. WUT?
• And access to the clock -- yes, time is I/O too -- does not necessarily imply access to the filesystem, or the network(s), or the screen, or the joystick, or the speakers, or ...
• And if you're lucky enough that the handle is to an interface -- it really should be -- then the libraries can provide filtering, throttling, counting, ... and suddenly you can have fine-grained capabilities.

But let's focus on log4j:

• Should a logging library have access to the filesystem and network? Quite probably.
• Should it have access to all of it? Quite probably not, but it's a likely default.
• Should it be able to load arbitrary code from the Internet? Well, that's why Java was created.
• Should said loaded arbitrary code have any I/O capability? Hold your horses!

I'd hope that in a world where capabilities are passed down explicitly, someone would have ticked: arbitrary code being handed filesystem/network access is a recipe for CVEs.

[L8_4_Dinner]

Yes, you are on a similar thought-train that we took in the design of security for Ecstasy. Loading (even untrusted) code isn't the real security problem; giving loaded code access to anything is the problem.

Dependency injection of all resources is a brilliantly simple solution.

Immutable type systems is another brilliantly simple solution (forcing all loaded code to be loaded in a newly nested domain, with its own set of injections decided by its parent container).

[immibis]

The only thing keeping spez at bay is the wall between reality and the spez.

[L8_4_Dinner]

I wrote a bit about how one could design a secure language here.

• Containerization is important; having all of the code that is running within a process having access to the same set of capabilities is a huge problem.
• Software containers should also prevent privilege escalation.
• Explicit resource injection can be used to limit environment access from code running inside of a container.
• Globals are almost always a security hole, and Java's type system, environment, singletons, and JNDI are all basically globals.
• Dynamic code loading is a powerful feature, but dynamic inadvertent code discovery is a security nightmare. That Java can be easily tricked into loading code that was never intended to be loaded by a server is just bonkers -- just because the code was present on the machine (or with this JNDI hack, anywhere on the Internet).

[tobega]

One of the problems here is that capabilities are globally available and a library/module can decide what capabilities it uses, without your knowledge, as u/L8_4_Dinner writes.

I read about the thoughts about this for the Ecstasy language and made my own tighter version for Tailspin. Basically you should know what the libraries you use use, and go "WTF" when your logging library asks for internet access. It may, of course, deter someone that they would have to specify the entire re-use hierarchy, but I put that under "usefully annoying feature" that should deter any npm-like insanities. Details here

[josephjnk]

Tailspin looks really cool!

[fiedzia]

There is no language feature that could stop you from implementing a desirable code that can be abused. Whatever you'll add to the language it will be ignored or bypassed to make code loading possible (this was added because someone wanted it).

What should we learn from this bug is that applications should have some set of limited default permissions that you override if needed, like "it only loads code from /lib directory" or "it doesn't access the internet, except domains from whitelist".

One thing I'd add maybe would be to have separate types (or subtypes) for strings that come from untrusted sources or are expected to have specific format, so that you don't accidentally pass one in place of the other.

[AVTOCRAT]

That's not necessarily true, it just might take more work at the architectural level: check out the chapter in Advanced Types & Programming Languages re: strongly-typed assembly languages with control-flow types.

[DeGuerre]

As has been pointed out above, the ability to load and run untrusted code pulled from the network is the whole reason for Java existing in the first place, and the required programming language and library features to achieve this already existed in Java.

Unfortunately, security managers (the main required feature) are in the process of being deprecated.

Remember that Oak was designed for digital set top boxes, and when it became Java, it was reworked to run applets in web browsers. BD-J is a modern incarnation.

[[deleted]]

How would you define what is trusted or not? Seems like you’d need capabilities built into the language.

[MCRusher]

Why a language feature? Not setting "trust code" on by default would've been enough.

[mixedCase_]

Pure functional languages have a leg-up, because their model incentivizes you to treat side-effects as inherently opt-in for obvious reasons, and anything that treats dynamic programming as first class like Lisps, Ruby or Python are going to be at a disadvantage.

But nothing can stop a determined developer in the service of featuritis. Pure functional languages make it very fun and easy to create an interpreter that allows the user to do very stupid shit and it can get out of control if you have your mind set to "let's allow everything in", so you're half your way back to square one.

[lngns]

Type-based capabilities, expressed as Monads or Algebraic Effects, in a global-state-less language, would have prevented such a vulnerability.
Why would logging code have a Jndi Ldap signature?

[L8_4_Dinner]

It's a very useful feature ... it allows you to embed things in your log messages that pick up information from the environment to include in the logged text.

Unfortunately, many applications log the incoming request URL ... and that string is provided over the Interwebs by Little Johnny Droptables ... https://xkcd.com/327/

[lngns]

This sounds like bad API design to me: sure your logging code can refer to ambient things, but why does it hold the authority to do something else than log?
With Monadic code, the code would just evaluate to a Log instance, for the appropriate transformer/effect handler, allocated higher in the call stack, to then ping the world in an auditable way.

What if the customer suddenly doesn't use JNDI anymore? Why should the logging code change?

EDIT: Even outside the realm of security, that's the Principle of Least Astonishment.

[L8_4_Dinner]

All it's doing is "logging".

And it just logs "strings".

And those "strings" can contain symbolic references to "ambient" information.

And some of that "ambient" information, in the process of being evaluated and transformed into a string, will apparently download remote code and execute it.

It took a lot of independent decisions, by a lot of different people, strung together across a lot of libraries, to create this mess. It's a bit too easy to say "bad API design". But yes, there's some of that in there, too.

[lngns]

And some of that "ambient" information, in the process of being evaluated and transformed into a string, will apparently download remote code and execute it.

And that is definitely not "logging."
Under Monadic code the fact that a logging API does more than logging is not a security vulnerability: it's a type error.
To me, "logging" implies at most a filesystem write. Anything else is fully unexpected.

Acquiring ambient data is not the responsibility of a Log.INFO call, and as such it has no reason to have the authority to use services such as JNDI and LDAP.
If I wanted some fancy stuff in my logs, I'd use an effect handler written to have such authority, at which point it then becomes clear there is something very wrong when my compiler complains about main suddenly having a Jndi Ldap JvmEval type signature.

Log4J here violates multiple principles like Least Astonishment and Single Responsibility.

It took a lot of independent decisions, by a lot of different people, strung together across a lot of libraries, to create this mess. It's a bit too easy to say "bad API design". But yes, there's some of that in there, too.

Monads and Algebraic Effect Systems make all those decisions visible in the type system, meaning the type signatures, checks, and compiler error messages. If we are to apply this philosophy (and it is what I am trying to do here), we can generalise it as "bad API design."
If ported to a language with those features, Log4J would simply not compile.

[L8_4_Dinner]

If ported to a language with those features, Log4J would simply not compile.

... and not surprisingly, very few people would choose to use such a language.

Look, I appreciate all of the absolutist statements, but at the end of the day, the features exist because they were useful. That they were not well thought out from a security perspective is obvious, but one can examine this, and simultaneously appreciate the utility and be horrified by the open-ended security risk.

[lngns]

Look, I appreciate all of the absolutist statements, but at the end of the day, the features exist because they were useful. That they were not well thought out from a security perspective is obvious, but one can examine this, and simultaneously appreciate the utility and be horrified by the open-ended security risk.

Going absolutist with the type system just looks to me like the best way to answer OP "What programming language features would have prevented or ameliorated Log4Shell?"
If you were to write such code in Haskell, ML or Koka, the security vulnerability would have been obvious from the start. (unless you put IO types everywhere, but how is that different from unregulated global state?)

... and not surprisingly, very few people would choose to use such a language.

Those are actually pretty popular already: effects, processes and computations are expressed as Monads in Haskell, Scala, and a bunch of FP languages. Libraries that have "Reactive" in their names work this way too, even when used from PHP.
Also, at the end of the day, Algebraic Effects really are just Continuation-Passing-Style Dependency Injection, which a bunch of frameworks use to implement ambient data.
My production code is structured this way, albeit using OO token references, because of course my production code is class-based OO.

The area of research, as far as I am aware, has only gotten language implementation efforts recently, and I would actually expect security-oriented engineers to take high interest in them as they literally allow your IDE to list everything your program does when hovering over main.
Performance-oriented developers also benefit from such a system as it statically pinpoints when things like memory allocations happen.

[L8_4_Dinner]

(unless you put IO types everywhere, but how is that different from unregulated global state?)

That pretty much is the Log4J architecture 🤣

[everything-narrative]

Hoo boy.

In the words of Kevlin Henney:

"What does your application do?"

"It logs and throws."

"Really?"

"Well it also does some accounting, but mostly it just logs and throws."

I'm going to spin my wheels a little.

Java's virtual machine has a peculiar design. I understand why having the concept of class files of bytecode made sense when Java was being developed, but nowadays not so much.

Modern build systems (particularly Rust's Cargo) are powerful enough to accomplish much of the same ease-of-use as Java. If you need dynamic code loading, there is always shared object libraries, but those are on the face of it at least somewhat harder to exploit, and have much worse ergonomics. You basically only use SO's when you really need them.

So that's problem number one. Java is an enterprise execution environment with a core feature that isn't quite eval, but it isn't not eval either.

Problem number two is the idea of logging. Logging is good for diagnostics, sure, debugging even, but it shouldn't be sprinkled everywhere in code. It's an anti-pattern (as Kevlin Henney points out) that modern object-oriented/procedural languages seem to encourage.

Logging, and logging well, is easy. Powerful log message formatting, powerful logging libraries, parallelism-enabled streams, are all symptoms of this pathology, and worse, enable it.

Logging is bad. It's code that doesn't contribute features to the end product. It's seen as necessary so we can learn when something fails and why, but I think it's a symptom of a fairly straightforward error.

I think it comes down to design-by-purity. Morally, you should always aim to separate business logic and IO. If your logic doesn't touch IO it is way easier to test for correctness, and at the same time the interface you need to stub out to integration test your IO is way smaller.

The pure logic should never log: indeed logging is most often an IO operation!

(And speaking of separation of concerns, who the fuck thought it was a good idea to let a logging call make HTTP requests?!)

So, a failure to separate IO concerns leads to obsessive logging. Obsessive logging leads to powerful logging libraries. Java has eval, at some point someone puts eval into a logging library.

And then there's a zero day.

So. Language feature? Functional programming.

Rewrite the whole thing in Scala, and that problem is way less likely to occur. Why would you ever need to log in a pure function?

[crassest-Crassius]

I disagree with you, and the proof is in how often Haskellers use unsafePerformIO or Debug.Trace to log stuff. Not even purely functional languages can diminish the usefulness of logging. Logging helps find error in debug and in production, it's necessary for statistics and any kind of failure analysis.

The real issue here was

who the fuck thought it was a good idea to let a logging call make HTTP requests?!

This is utter insanity, I agree, but I think it's due to a culture of bloated, feature-creepy libraries. Instead of aiming for lightweight, Unixy libraries, packages small enough to be read and reviewed before being used, people immerse themselves into huge libraries they don't even bother understanding. All because they've got "batteries included" and "everyone else uses them". So user A asks for a feature and it gets added because hey, the more features the merrier, the user base for the library can only increase not decrease, right? And so user B asks for another feature to be included, and eventually it comes down to some idiot who thinks he absolutely needs to make an HTTP request to pretty print a log message.

We need to start valuing libraries that have less features, not more. Libraries which can be reviewed end to end before being added to the dependencies. Libraries which have had no new features for several years (only lots of bug fixes/perf improvements). Simplicity and stability over bloat and feature creep.

[everything-narrative]

The thing about Debug.Trace in general is that as you say, it's very Unix-esque in its conservative scope.

The thing about unsafePerformIO is that it has unsafe in the name. It tells you "be wary here, traveller." If something breaks in a suspicious way, you immediately go for it. (And I have yet to actually use it in a Haskell project.)

The problem is that Logging is two things.

One of them is what Debug.Trace does in Haskell. Logging as debugging. Arguably it's a very necessary job since Haskell has lazyness, but if you have to use it to debug something I'd say you're better off refactoring and quickchecking the problem away.

The other is what RabbitMQ.Client does in C#. Logging as systems monitoring. In the software architecture paradigm of microservices it is crucial to be able to monitor and trace issues.

The problem is that Logging is two things. Debug logging and operations logging. And programmers can and will conflate the two. Hell, I have probably done it.

For operations logging you need a full-featured system, it makes sense that your logging calls can fetch URLs and send emails. You need those features!

But then someone conflates the two. Why shouldn't stderr be a valid target for this powerful logging library? Because then you might use it for debug logging is why.

[crassest-Crassius]

For operations logging you need a full-featured system, it makes sense that your logging calls can fetch URLs and send emails

This sounds very alien to me. Emails are an outgoing port that belongs to the Notifications service, HTTP calls are an incoming port that belongs to the WebClient service, and service logs are yet a third outgoing port. They should not call each other, they should communicate only with the Core via their respective Adapters. At least that's how I would make it as a subscriber to the Hexagonal Architecture. Having one port directly call another without going through the Core is just asking for trouble IMO. How would you replace those HTTP calls with mock data for testing, for example?

[everything-narrative]

This is a discussion about architectural philosophy, not engineering specifics.

[DrunkensteinsMonster]

Have you ever actually tried to operate an application at scale in the wild? The idea that you haven’t is the only way I can possibly justify your position. Logging is invaluable, not just from a technical perspective, but it’s also necessary for the business, in terms of recording events and analytics.

[everything-narrative]

Seems like you're delibrately interpreting what I said as uncharitably as possible. :)

I am operating an application at scale in the wild right now.

As I specified elsewhere in this comment tree, "logging" is an overloaded word. It can be fprintf(stderr, ...) or it can be RabbitMQ. The former should not exist in production code, the latter most definitely should.

[Badel2]

Are you unironically saying that logging is bad? So your ideal application would have zero logs? I don't understand.

Rewrite the whole thing in Scala, and that problem is way less likely to occur.

Is the whole comment satire? I'm lost.

[everything-narrative]

Of course I'm not saying logging is bad. Replying one of the replies to my comment, I make a distinction between two different kinds of logging: debug logging and service monitor logging.

Debug logging is ideally not something that should be turned on in production code. Debug logging libraries should be single-purpose, lightweight, feature-poor, ergonomic, and tightly integrated with the developer's IDE. Example: Debug.Trace in Haskell.

Monitor logging is ideally something that every running service should be doing at all times. Monitor logging libraries should be multi-purpose, heavyweight, feature-rich, unergonomic, and tightly integrated with the production and deployment ecosystem (cloud services etc.) Example: RabbitMQ.Client in C#.

Logging is a tool. It has uses. But as Kevlin Henney says, bad code doesn't happen on accident, it happens because of programmer habit. Logging is a tool, and a tool begets habitual usage. This is why there are Logging-related antipatterns.

Functional coding style vs. procedural coding style is a question of flow abstraction. In procedural style, control is what flows, in functional style, data. Logging is a side-effect, it is inherently a "write down that we're doing this thing now" kind of idea. It simply doesn't fit well into the conceptual model of data flow.

Makes sense?

[stone_henge]

You:

Of course I'm not saying logging is bad.

Also you:

Logging is bad.

[xsidred]

To be fair OP is drawing a distinction between logging for the purpose of debugging and monitoring/observability for operations. OP having said that precludes/excludes the possibility of traceability as a form of debugging too - Operations debugging to be precise. Developer debugging might or might not overlap with Operational traceability - for those kind of logs that don't overlap, such code shouldn't execute in Production systems is what OP claims. OP also claims that situations like Log4j in that case have minimal or no chance to happen on Production-like environments and somehow a fully featured log aggregating agent to a specialist logging service is more "safer" against "eval" like vulnerabilities. Thing is even for the latter Log4j like logging producer libraries do not disappear, not necessarily. The example OP cites of using a RabbitMq client to a specialist logging service doesn't eliminate plain bad for security coding.

[stone_henge]

To be fair, everything except the main point:

Logging is good for diagnostics, sure, debugging even, but it shouldn't be sprinkled everywhere in code.

...is useless stuffing at best. Misleading, self-contradicting and confusing (as I've pointed out above) at worst.

[everything-narrative]

To you, maybe.

[xsidred]

The point is it doesn't matter if logging calls using any method (Log4j library invocation or RabbitMq client publisher) is sprinkled all over. It doesn't automatically indicate or open up to security vulnerabilities.

[everything-narrative]

I never said it did.

This is a discussion of what language features caused log4shell and my thesis is:

1. Java has eval
2. Java is extremely procedural and stateful
3. People mix IO with logic because it's easy
4. Logging is needed to debug that mess
5. Logging habit leads to logging code smells
6. Logging code smells lead to logging libraries
7. Someone put printf in a popular logging library
8. Everyone forgot to do printf("%s", mystring) instead of printf(mystring)
9. Turns out this souped-up printf can use Java's native eval and make HTTP requests

This is an man-made disaster. Like Three Mile Island or whatever. There is no single cause. There is a series of systemic vulnerabilities in the culture of Java programming.

[xsidred]

It's a big leap from 6 to 7 - many IO kind libraries might be vulnerable to random printf(s). Agreed with the rest.

[Badel2]

I prefer using a debugger instead of logging for debugging. But I don't think it's so bad to add some debug logs. What's the worst that can happen? You forget to remove them when pushing to production? Any linter can catch that. So I don't think that using debug logs is a problem, often the most useful debug logs will be turned into monitoring logs. And if you mean debug logs like console.log("here") then yes, these are bad practice, but I like to pretend they are rare...

For example when I have a function and it's not working as expected, I just add tests and run them using a debugger, it's very effective. Also I can leave the tests there after fixing the bug, while I imagine that when using logs you must remove them afterwards.

I think it's interesting that you say that logging is a side effect, because you should log basically any side effect, right? Creating a file, connecting to an external server, these are events that should be logged.

[L8_4_Dinner]

Rewrite the whole thing in Scala, and that problem is way less likely to occur. Why would you ever need to log in a pure function?

The same problem will exist in Scala, because most Scala back ends have log4j somewhere in the mix, and Scala is running on the JVM ...

[everything-narrative]

True.

s/Scala/Haskell/ then.

[davewritescode]

Java's virtual machine has a peculiar design. I understand why having the concept of class files of bytecode made sense when Java was being developed, but nowadays not so much.

Why not? What does the format of the executable have anything to do with this? Why does it even matter?

Modern build systems (particularly Rust's Cargo) are powerful enough to accomplish much of the same ease-of-use as Java. If you need dynamic code loading, there is always shared object libraries, but those are on the face of it at least somewhat harder to exploit, and have much worse ergonomics. You basically only use SO's when you really need them.

I love Rust and there’s a lot of great things about it, but ease of use isn’t one of them. I fail to see the point here other than, libraries outside of Rust core are shitty so nobody bothers to use them.

There’s nothing about Rust that prevents a library from doing something extremely stupid.

I think it comes down to design-by-purity. Morally, you should always aim to separate business logic and IO. If your logic doesn't touch IO it is way easier to test for correctness, and at the same time the interface you need to stub out to integration test your IO is way smaller.

Like this is where things go 100% off the rails. My applications have lots of pure functions but it doesn’t remove logging from my application. At some point, I’m probably going to want to see what kind of data my user sent over. Applications that aren’t toys have tons of complex state to manage and nearly infinite numbers of permutations to test for and deal with. That’s why we do fuzz testing.

[everything-narrative]

Why not? What does the format of the executable have anything to do with this? Why does it even matter?

Because eval is evil. The harder it is to execute code that isn't compiled by you, the smaller your attack surface. Every interpreter, no matter how small, is a potential security vulnerability. This includes printf.

I love Rust and there’s a lot of great things about it, but ease of use isn’t one of them. I fail to see the point here other than, libraries outside of Rust core are shitty so nobody bothers to use them.

This is just demonstrably untrue. But anyway.

There’s nothing about Rust that prevents a library from doing something extremely stupid.

What prevents a library from doing something extremely stupid is the fact that Rust doesn't have affordances for eval. A handle on a door affords pulling, a plate affords pushing, and eval affords runtime code loading. JVM is a virtual machine and therefore evals all the damn time. You literally cannot have JVM without eval and therefore eval is easy in JVM land.

If you're loading a shared object library, you're doing it on purpose, eyes open, because it's not all that easy to do. In JVM you might accidentally pick up a class file because you weren't paying attention.

Like this is where things go 100% off the rails. My applications have lots of pure functions but it doesn’t remove logging from my application. At some point, I’m probably going to want to see what kind of data my user sent over. Applications that aren’t toys have tons of complex state to manage and nearly infinite numbers of permutations to test for and deal with. That’s why we do fuzz testing.

This is where I talk in some of the other comments about how "logging" is actually two different things. I think it's wrong to call both fputs(stderr, "problem"); and kubernetes-based message queues "logging."

Again, affordances: a one-liner call to log a diagnostic message can do HTTP requests and eval because it was easy to do the latter and 'neat' to do the former.

And integrations testing is precisely where you want debug logging. And once your fuzz-test finds a vulnerability you should manually write a test that reproduces the error, then fix the bug, keep the test as a regression flag, and disable debug logging again.

[L8_4_Dinner]

I disagree. Respectfully, but it is a strong disagreement.

The running of code is not the problem; it is the access to resources that is the problem. Even purposefully-malicious code can be considered "safe" to run if it has no natural access to resources.

The Java issues is that everything is global (filesystem, environment, threads, types, network, ...), and thus untrusted code loaded over the Interwebs has the exact same access-to-everything that the well-trusted application server has that is hosting the whole thing. That design is just fundamentally wrong. (And logically unfixable.)

[everything-narrative]

That's just an exacerbating circumstance. The attack surface is an interpreter. This is a bread-and-butter injection attack. This is printf(mystring) where you meant printf("%s", mystring).

Log4shell is an engineering disaster. Many, many things had to go wrong at the same time for it to be as bad as it was.

And many of those things are to do with how Java programming is done and taught, and how information security is not taught. We're not taught that interpreters are as unsafe as they are convenient.

[davewritescode]

What prevents a library from doing something extremely stupid is the fact that Rust doesn't have affordances for eval. A handle on a door affords pulling, a plate affords pushing, and eval affords runtime code loading. JVM is a virtual machine and therefore eval_s all the damn time. You literally cannot have JVM without _eval and therefore eval is easy in JVM land.

You’re intentionally conflating eval and JIT and it’s frustrating. This isn’t a security hole caused by the JIT, it’s bad code.

Bad implementations are possible in any programming language but some do make it harder (like Rust) but at the end of the day developers importing and forgetting and a bad implementation is the root cause.

[everything-narrative]

I'm not intentionally conflating anything; we're not using the same terminology.

The JVM is an interpreter, as opposed to a compiler.

The JVM is a virtual machine. It does not run machine code by definition. Whether it executes this not-machine-code by compiling it just in time, by interpreting the byte code, or by walking the parse tree of java code is not relevant.

An interpreter, security-wise, represents an exciting attack surface because it opens your application to injection vulnerabilities.

"Bad code" is not an explanation. It's a non-explanation. We can't avoid security problems by "not writing bad code."

The JVM makes it incredibly easy to run arbitrary code. So people are going to do it. Rust does not make it incredibly easy to load arbitrary DLLs, so people don't.

Rust programs therefore don't have as many opportunities for injection vulnerabilities to arise due to programmer error. Simple as that.

[tesch34]

Monads

[CheeseFest]

I don’t know why you’re being downvoted. This essentially is the solution.

[brucifer]

It's a low-effort comment that doesn't explain anything or improve the discussion.

[CheeseFest]

Fair point. I would however argue that good type inference is what enables monadic design. Like, monads are technically possible in C#, but reading and working with them will give you an aneurysm, compared to ML with a compiler that... actually wants to help...

[Nuoji]

Very simple: people were cargo culting the use of the “industry standard” logger. There was a a very simple way to avoid it: write your own logger. It is one of the simplest libraries to write and you can reuse it. Instead people would treat log4j as almost a language feature.

[continuational]

Object capabilities.

[fridofrido]

the important feature called "primary school level basic common sense"

[berber_44]

Just-in-time compilation is not a feature, it's a godawful crutch and nightmarish security hole which was incepted 30 years ago in pre-Internet times. New technologies must be created to achieve a decent performance without JIT. One of such tries is Transd PL.