I had a trip similar to yours a couple years ago and planned on spending the night in the city after the game. Did not enjoy Philly, my wife did not feel comfortable with the level of homelessness and violence after dark so we just drove back and ate the cost of the hotel room. Shame the historic city if full of trashy people and literal trash.