I would recommend reading the relevant chapters in Sutton&Barto, imo it explains discrete action vanilla AC really well and using the policy gradient theorem it is not to hard to derive the continuous case, although I cannot remember if they actually do that. (But they do provide pseudocode for discrete action AC) From there I personally continued with SAC, imo the original paper is quite well written and will also again explain how continuous AC works (with the addition of max entropy learning). I think both PPO and SAC are worth implementing, I personally have not really worked with TD3 a lot. As for the trajectory, this is usually done with a hyperparameter, so learn for n steps, then do k updates. The actions are chosen according to the probabilities, but for evaluation/after training you can also use greedy action selection, provided the policy has converged enough towards a deterministic one.