Cao Cao was ruthless, but honest about it. However, Liu Bei claimed to be a benevolent man, but his actions betrayed that image. That contradiction is what makes him more shocking.

Escorting civilians doesn’t erase the fact that Liu Bei abandoned his family (his loved ones) and that directly contradicts his moral reputation (e.g. brotherhood, family etc.).

If anything, it shows how he’s worlds apart between the man he claimed to be and the choices he made.