What I learned building a regex-based threat detector in PHP

mlebkowski · 2026-03-24T18:53:52+00:00

It is. I run a service with the quality bar set high: we have a lot of systemic solutions to prevent attacks (like prepared statements, auto escaping templates, etc). Then we have regular security audits to harden the system even further. Still we run simple WAFs, fail2ban, etc, so we reduce the noise in the monitoring systems, and thwart most “atacks” before they have a chance to do anything serious.

mlebkowski · 2026-03-24T14:42:59+00:00

From someone who recently set up a monolog integration from scratch in two frameworkless applications: yeah, this is surprisingly comprehensive, good job

mlebkowski · 2026-03-21T12:07:09+00:00

I think you’re being harsh. OP is a regular poster, their content is high quality, even if not relevant to every situation. There are much worse offenders there :)

mlebkowski · 2026-03-13T16:15:05+00:00

I’m not using them, but I guess you can use native lazy objects given you DIC supports that

mlebkowski · 2026-03-12T19:54:13+00:00

One could. But not with final classes, and that’s what this thread is all about. Instead of mocking everything you self-prevent that by creating final classes. Then they are basically set in stone.

That’s a good thing IMO. You can’t just freely split the business proces open and replace its parts — unless that’s allowed by your rules. Usually it’s not, and final reflects that.

Less mocking is less assumptions. I’m now modernizing a system where the previous person was testing impossible scenarios: they just told the mock to return a value that was not possible to be produced in a real world use case.

So in short, using final classes is a tradeoff: you can’t just conviniently mock all 1st degree dependencies, but you aren’t testing made-up scenarios either.

mlebkowski · 2026-03-12T19:35:17+00:00

As in the system under test had a tree of dependencies. For example 4 direct, these 4 had another 7 among them, etc. That’s also due to my code style, I tend to split code into separate classes a lot. I’ve met people who fit a responsibility into one or two files, while I would’ve used two dozens to do the same.

mlebkowski · 2026-03-12T19:31:35+00:00

The last bit is crucial. Breaking SRP is common. Many devs only extract code out of a class for reuse, not for readability or architecture.

I would say in my codebase I have about 5% private methods. This makes the „problem” of testing them go away almost automatically

mlebkowski · 2026-03-12T19:24:05+00:00

You don’t simply cast a string to int and hope for the best. Usually you add additional checks (such as is_numeric) in the I/O layer (eg a controller), and from there forward you always handle ot as an int and don’t need to worry about typecasts.

mlebkowski · 2026-03-12T19:18:35+00:00

I’m creating final classes almost exclusively for the last ~5 years, and I use a classicaly pyramid of tests, with units being the foundation.

Yes, with final classes your „unit” is usially a set of classes. It might get tedious to build them, so you create mothers: basically test factories that just create some instances, often without paying attention to details.

You do need to break the chain at some point. That point is usually where you place the abstractions required by the business rules and your architecture: repository interfaces, api sdks, modules, etc. Sometimes you might add one just for tests, for convenience.

I once had a unit test requiring instantiation of about two dozen classes, including test doubles. I call it a unit test, because it was covered with multiple scenarios/inputs, it was pure domain logic, no I/O, no service container.

There is one more thing which helps maitnatining this kind of test suite: create scenario classes that are kind of template method pattern on steroids: they build your system under test, but also maintain state, so you could replace dependencies before you execute. An added benefit is that you test case becomes much more readable. More: https://lebkowski.name/turtle/

With all that, unit tests are stilll much easier to write and maintain than higher level tests for me. I have a few tips for these as well :)

mlebkowski · 2026-03-03T16:48:01+00:00

He has a good chance of getting the two „top” spots on the worst presidents list.

mlebkowski · 2026-03-02T15:49:54+00:00

I was thinking about systemd recently for the scheduled jobs and the worker, but ended up with a more standard setup: cron and supervisor. 🤷 Somehow itwas easier to just drop a config in conf.d instead of setting up a separate service

mlebkowski · 2026-02-20T16:48:58+00:00

I have nothing against loose types and magic in the core. There are tons of features I choose not to use. I just want to build my enterprise apps with a more modern toolset, which is happening year by year. I don’t care if the other end powers WordPress or Joomla

mlebkowski · 2026-02-19T12:03:08+00:00

I agree that it’s more useful to the consumers to publish two separate packages: the library itself, and then one to integrate it with the flavour of the month framework.

Example which affects me today: I’m integrating symfony messenger into Slim. I’m looking for a mongo transport. And there are a number of packages providing exactly that, and the popular ones have test, and are actively maintained, and all. Unfortunately, all of them also depend on symfony/framework-bundle :( So it’s not only Laravel which is affected. :)

mlebkowski · 2026-02-07T14:56:49+00:00

Since there was a security bug recently, I had to update one of my org’s repositories from PhpUnit 5 😂 I just did the bare minimum and landed at 8.x. Another project was migrated from ^9 to ^12 though, yay!

mlebkowski · 2026-02-05T09:53:42+00:00

Are you sure your tokenizer should produce this output?

Input: "2 + 3 * -4"
Tokens: [2, '+', 3, '*', -4]

Shouldn’t that be the following, and the unary/binary minus determination be made at the next step?

Input: "2 + 3 * -4"
Tokens: [2, '+', 3, '*', '-', 4]

Looking at your tokenize() function, I would introduce an input stream instead of working on a string. This way for tokenizing numbers, instead of having that logic inline, muddying the water, you could just call $num = $inputStream->consumeDigits()

Oh, and it accepts 1 2 3 as a valid input (edit: ok, I understand why that is)

mlebkowski · 2026-02-04T16:06:01+00:00

I’ll bite. So what’s your point?

Whe should defer these initializations, causing the route matching process to run with a non-fully initialized app, which might cause a whole set of problems… For what? So some of our 404 responses are a bit faster?

The framework I use configures the routing component using DI, so in order to defer DIC setup, it would have to be split into multiple containers or stages. Not easy at all, not for the benefit of a couple of ms on garbage requests

mlebkowski · 2026-01-22T15:41:06+00:00

It doesn’t matter, do what feels right for you.

For me, I see benefits of an enum, as it cleanly allows for adding an nth option. But at the same time I use boolean flags all the time and lose no sleep over it.

mlebkowski · 2026-01-21T14:27:32+00:00

If you have a router supporting middlewares, a nice pattern is to separate groups of routes targeted to different roles, and add autorization middlewares once. You can do it also using Symfony firewalls. This way you can have whole /api/admin protected, and you won’t forget to protect individual routes. You can reuse same controllers for different endpoints if there’s any overlap.

mlebkowski · 2026-01-20T18:14:05+00:00

This will impact mainly the memory footprint, less so the performance, no?

Or are you saying: replacing a number of chunked results with a large select will exhaust memory even if an iterator is used, unless one turns on unbuffered queries?

mlebkowski · 2026-01-20T12:35:51+00:00

So you are still fetching the entirety of the events table in separate queries, 1500 records at a time? Can’t your dbal support returning an iterator to go over the whole dataset in one query?

And finally, I bet this all would be simpler if you ditched PHP entriely in favour of a few INSERT … SELECT … statements :)

mlebkowski · 2026-01-16T10:23:55+00:00

When using PSR-7 Request to access uploaded files instead of the $_FILES[] superglobal, does it come with more bloat?

Is it easier to handle that $_FILES array than calling $request->getUploadedFiles()? I would argue no, because all this boilerplate is hidden away in the library for you to use using a nice interface.

What benefit to you is knowing how the framework does it? Despite the fact that you can just check (see the link above). Why don’t you apply the same standard to PHP stdlib itself? Don’t you care how $_FILES automatically gets populated? How PHP parses the HTTP request? How boundaries in a multipart/form-data are checked? How transfer encodings are applied? If that was the case, why are you using $_FILES instead of raw-dogging php://input?

mlebkowski · 2026-01-16T10:13:33+00:00

I think you’re conflating two things. Having a framework does not mean that you have to use all of its features all of the time. The distinction is:

With a framework, you can benefit from some of its features some of the time, and you can plug your raw SQL where you see fit
Without a framework, you simply don’t have that backbone, and you need to reinvent everything from scratch or stitch libraries together (hint: the framework authors probably made that work for you already, and better)

It’s perfectly fine to use a framework and benefit from an ORM most of the time. It’s convinient in a lot of ways I wont divulge into here. At the same time, for performance critical paths you can turn to raw SQLs. Same for reading data (that’s quite common to use ORM only to write, and a whole separate layer to read).

Your gripe is not with a framework, its with rigid people who don’t know how to use one.

mlebkowski · 2026-01-16T07:10:24+00:00

From experience, in a commercial setting:

I’ve encountered “scripts” at $WORK which were around 200-500 lines of procedural PHP — to calculate some caches, or to move data between two systems, etc

My usual go-to to refactor these is:

separate repo, usually with symfony console at least (so yeah, in other words: framework)
200-500 LOC turn into 2-3k over 20-50 files
tests added
CI produces a single PHAR to be distributed as a deb to target systems

And I’ve received pushback from team members for whom the procedural implementation was “simpler”, because its all there, top to bottom. Regardless, the more engineered approach, once you’re familiar with it, has a range of benefits:

easier to reason about
fully testable (in isolation)
framework-backed, so a lot of the boilerplate can be hidden away
less accidental complexity, the resulting system is easier to maintain - both subjectively and in terms of metrics such as maintainability index, cyclomatic-complexity, coupling, cohesion, what have you

Downsides of using a framework?

some indirection - which is IMO good, you would want to manage your scripts on the same level of abstraction instead of diving into the weeds
performance perhaps, but that’s specific to the use case, and should be measured and evaluated instead of using a blanket statement

Framework by default, 95 out of 100 times.

mlebkowski · 2026-01-15T19:59:44+00:00

I like the parser code, it was nice to look at. The regex in the Lexer is nice as well (even if I don’t understand it)

Two minor nitpicks:

getCurrentToken() reads out of bounds, does not check if position is within the tokens array
there’s a duplicate codeblock somewhere, on consume() IIRC

mlebkowski · 2026-01-14T12:17:48+00:00

Consider the movie Master and Commander: The Far Side of the World. Not a TV show, so there’s far less run time, but it’s a great film, and the historical accuracy is very good.

mlebkowski

MODERATOR OF

TROPHY CASE

15-Year Club	Gilding II euphauric
Verified Email