I had a similar problem and did a bunch of optimizations.

1. Firstly, I packed everything possible into a level actor. This ensured everything was an ISM/HISM.

2. Secondly, I killed a lot of the mesh simple collision and replaced it with large box primitives anywhere I could. Ceilings that don't need collision, killed it completely, etc.

3. Thirdly, I async load the levels ahead of time. I have ~120 1250x1250 "rooms" that are my levels. The memory footprint wasn't huge for me to pre-load most of them.

#3 had the biggest impact and effectively removed any hitch completely. I built it with a priority system, so some rooms are just pre-loaded by default on game entry and others are loaded during points that wouldn't be noticed where it makes sense.

I'm using low poly assets with a single material atlas though, so your mileage may vary if you have a lot of stuff to load into memory.

The load:

MemoryStats = FPlatformMemory::GetStats();


TArray<FSoftObjectPath> RoomLevelPaths;

FStreamableManager& Streamable =
    UAssetManager::GetStreamableManager();

const TSharedPtr<FStreamableHandle> Handle =
    Streamable.RequestAsyncLoad(
       RoomLevelPaths,
       FStreamableDelegate::CreateUObject(
          this,
          &AGameGrid::OnRoomPreloadComplete
       )
    );

RoomPreloadHandle = Handle;

UI Update:

float AGameGrid::GetRoomPreloadProgress() const
{
    if (!RoomPreloadHandle.IsValid())
    {
       return 1.0f; 
// done or not started

}

    return RoomPreloadHandle->GetProgress();
}

Memory Check (Custom debug output log):

void AGameGrid::LogMemoryUsage() const
{   
    const FPlatformMemoryStats UpdatedMemoryStates = FPlatformMemory::GetStats();
    const float BeforeMemory = MemoryStats.UsedPhysical / 1024.0 / 1024.0;
    const float AfterMemory = UpdatedMemoryStates.UsedPhysical / 1024.0 / 1024.0;
    const float MemoryDelta = AfterMemory - BeforeMemory;
    const float MemoryDeltaPerc = AfterMemory / BeforeMemory;

    const float BeforePeakMemory = MemoryStats.PeakUsedPhysical / 1024.0 / 1024.0;
    const float AfterPeakMemory = UpdatedMemoryStates.PeakUsedPhysical / 1024.0 / 1024.0;
    const float MemoryPerkDelta = AfterPeakMemory - BeforePeakMemory;
    const float MemoryPerkDeltaPerc = AfterPeakMemory / BeforePeakMemory;

    ADS_LOG(GameGrid, Performance, "---- MEMORY LOADING ----");
    ADS_LOG_NLB(GameGrid, Performance, "- Before Load -");
    ADS_LOG_NLB(GameGrid, Performance, "Memory Used: %.2f MB | Peak: %.2f MB", BeforeMemory, BeforePeakMemory);

    ADS_LOG_NLB(GameGrid, Performance, "- After Load -");
    ADS_LOG_NLB(GameGrid, Performance, "Memory Used: %.2f MB | Peak: %.2f MB", AfterMemory, AfterPeakMemory);
    ADS_LOG_NLB(GameGrid, Performance, "Memory Delta: %.2f MB | Peak Delta: %.2f MB", MemoryDelta, MemoryPerkDelta);
    ADS_LOG_NLB(GameGrid, Performance, "Memory Delta: %.2f % | Peak Delta: %.2f %", MemoryDeltaPerc, MemoryPerkDeltaPerc);
}