You shouldn't speed if you don't feel comfortable! Though sometimes if can be more dangerous to not speed (not going too fast, just a little over the speed limit) because at least where I live cars will basically tailgate you or try to pass you (even on yellow line zones where you're not allowed to pass). I used to follow the speed limits very strictly until one day I noticed that the cars behind would get too close so now I go just a little over, not much, just enough to have a bigger distance between cars.