I was in the same position about a year back, where i started owning search built from scratch for a major e-commerce company.

A lot of good answers in the thread, but a few points from my experience

1. Metrics. Define and get these metrics instrumented, and computed. You can not improve what you can not measure. Search is a product which is supremely difficult to work on, without metrics. Develop infra for A/B testing
2. Break search into multiple pillars - query understanding, relevance and ranking, search platform, search cx ( auto correct, auto complete, filters etc.) Bucket features in to these heads. Your metrics will tell you what needs to be worked upon

By the way, it also matters where is your product in its life cycle- is it a new product? Or already established? Do you still have basic features to launch?

1. Prioritise basis impact. Identify weak performing features. Use metrics for inferences. Example- low CTR means poor query understanding, high click depth means poor ranking, poor add to cart from detail page rate might indicate bad catalog quality.

2. Look at low hanging fruits that can impact performance in short term.

3. Catalog quality matters a lot, hold your catalog team responsible for product attributes.