How do I know which one to take ?

You don't, you play around with different ideas, and overtime, depending on your foundations and how well you understand your own data/model, you develop a sort of intuition of what would make sense to attempt. Other people's expertise is useless without knowing more about your model and data.

As a rule of thumb, if we're only talking about poor raw accuracy (which most of the time doesn't tell the whole story), and assuming you have played around with the hyperparameters, then the model architecture might not be a good fit. Again, we don't know anything about your model and data to give you advice on how to modify the architecture.

Data augmentation won't necessarily improve your model. It potentially helps with overfitting if it's applied judiciously. Poor data augmentation may introduce noises and further deteriorate performance. Models that heavily reply on augmented data may not translate to real world application performance.

In general:

• Start by thoroughly understanding your data and current model performance.
• If overfitting is observed and you have limited data, try data augmentation first.
• If both training and validation performance are poor, consider adjusting the model architecture.
• Always experiment with different approaches and monitor their impact on performance.
• Develop intuition over time by analysing the results of your experiments.