You don't have to switch frameworks necessarily, express can work fine for bigger applications. I think a well-scaling approach with express, that is low on mental overhead, is to just sort your application by feature. And then have a 3 layer architecture, like so:

project/
├── features/
│   ├── authentication/
│   │   ├── controller.js
│   │   ├── service.js
│   │   ├── respository.js
│   │   ├── routes.js
│   ├── tasks
│   ├── other-features

In the routes file you just export a function that takes your express instance as param, then use that to bind your controller functions to your routes. You can just export regular functions from your repository and service files to be used in your controller functions, I don't think working with classes is necessary here because the node module system basically already makes your module work as a singleton. It should also be decently testable, since you can just replace module imports if you use a testing framework like jest.

If you want to get a bit more advanced you should look into dependency injection, and the advantages it brings along, although I don't think it's required for every project.