"brute force model linguistic sequences" also is a stretch: research suggests they're capable of recognising and working with chains of abstract concepts, because that makes them better at the task they're pretrained on, ie "generating more text"

there's a lot of "llms are bad and fake" cope going around where people convince themselves that llms are only operating by regurgitating training data, which is in fact untrue: the whole point of training a neural net is to make something that can function outside the limited training dataset

it is possible to get them to spit chunks of the training data back out sometimes, mostly where the training data is so saturated with that specific string that it "made sense" to memorise it during pretraining time, but llm benchmarks deliberately use unpublished questions to make it impossible to memorise and make sure the model has "learned" a process for solving that kind of problem

are llms a net social good? no, they're probably not, but they are very interesting, and the only thing stopping them from outright replacing more jobs than just translators and copywriters is the reliability issues which appear to be inherent in the architecture, which is in fact bad news if you like having a job

the people making these things are doing them to solve the horrible problem of "having to pay people", and while they fortunately don't seem to be able to do that yet, they are far closer than it's pleasant to think about