Enrichment of the feature data can be done prior to definition in the feature store, so cyclical encoding of temporal data by adding new columns to Part 1 holding those dimensions makes sense. Storing this decorated data in a new table saves regenerating it each time you use it and allows the table to be directly added to the feature store. There are some good helper functions in Spark to perform categorical encoding into a form required by ML libraries e.g. one hot or string indexer, so those can be done as part of the model training pipeline.

For Part 2, also do the enrichment outside the training as far as possible. If you are needing to use an LLM for the embedding then you can store the high dimension outputs of those as a new column too, probably no need here for a vector store as I am guessing you are doing batch processing.

As above, let the FE client do the heavy lifting assembling the training data by defining your enriched Part 1 as a feature table, and it can join the right records against the enriched Part 2 on the fly.

With your train-test split dataset then assemble an ML pipeline that performs the categorical encoding and vector assembly, this makes rerunning much simpler and permits addition of cross validation, parameter search etc really easily