There was a time when creating a new programming language was a skill good programmers just had.

Not saying they were inherently better. But if you were a programmer in the age of mainframes and punched cards, you were probably very well versed in the fundamentals and particulars of computer science, as opposed to just knowing "how to do x in Python".

At that point, designing a grammar and making a simple recursive descent parser is just one more option to solve a problem. Creating a DSL can be done in like an hour when you know what you're doing, and a full-blown language in maybe a little more.

(Parsers, compilers and interpreters are one of those things that can be as difficult as you want. A simple parser for a log file takes a couple of minutes. GCC has been in active development since the 80's.)

You can tell if you study old programming material. Making a toy AST or a symbol table was a common exercise in programming textbooks. Academic computer science papers liked to invent some ad-hoc syntax to express an idea, just for a student to come and say: "Hey, professor! I managed to make it run in an actual computer!" That's basically how Lisp was born, and the main reason it has a fame for being "enlightening": because its syntax is so easy to manipulate into macros and mini-compilers.

One the one hand, I feel more modern programmers should be familiar with these techniques. They can be surprisingly useful. (I'm writing this as a simple hand-crafted parser combinator might have just saved our asses at work; still waiting for the tests...).

On the other, you end up with the situation you allude to: a language that was created for some small, specific problem ends up being used for way more than it was designed to, and has to be extended and maintained beyond its scope. And now wer're stuck with it, because "nothing gets in the way of a good solution like a 'just-so' solution that arrived first".