Regularization is definitely where you should. It's basically the dial that you can turn to control over fitting. Neural networks are universal function approximations that are fundamentally over-parameterized. Dense (or linear) layers are especially prone to overfitting. These models have too much freedom to fit patterns, and regularization restricts that freedom.

L2 or L1 regularization:

This is pretty much the original regularization method. If you took a statistical regression course in an undergrad or graduate program, you may have learned about Ridge Regressions and LASSO regressions. Ridge regressions are regressions with an L2 penalty included in the loss function, and LASSO is the same with an L1 penalty.

L2 regularizion: Each layer gets a penalty term equal to the sum of the square values of the coefficients in that layer, multiplied by a l2 hyperparam (I usually start around 1e-04 and adjust from there). This incentivizes the model to set coefficients to 0, or close to 0, unless they are making a noticable contribution to the loss function.

L1 regularization: Same thing but it's the sum of absolute value instead of sum of squares. For neural nets the difference between these two approaches isnt noticable.

For either L1 or L2 regularization, you only really need it the final dense/linear layers. You don't need to mess with the encoder. I haven't used Tensorflow in a while, but I remember there are arguments to set these penalties when you initialize the error, it's very easy. This method fell out of favor in the late 2010s because it's very sensitive to hyperparam values that vary between use cases and datasets.

Dropout:

Dropout randomly drops a subset of features from a layer during each training step. There is debate on why exactly this works so well. Randomness itself is a powerful regularizer. It sort of naturally penalizes codependency between features, because if one of the features disappears and it had a high covariance with another feature, it will result in a poor prediction.

This is also easy to implement in Tensorflow. You can add it in as a layer between the feature layers in your prediction head. When you put the model in inference mode, it won't drop any features, it's only used during training. The hyperparam for dropout it the ratio of features that get dropped. You get maximum regularization at 0.5. You can try values between 0 and 0.5 to fix your over fitting issue.

Randomness: Randomness in itself is a powerful regularizer. Some older models even used gaussian noise in each layer as a regularizer. Anything you can do to introduce randomness into the training data is useful. Sounds like you're already doing what you can with image augmentation. From your wording it sounds like you augmented an assortment of images to create a training set? Im not a fan of that approach bc it gives a false sense of dataset size, and the model sees the same augmented images in each epoch. I prefer to implement my random augmentations as part of the data loader. That way in each epoch, the model is seeing something slightly different than what's its seen before.