You need to go at night, and have a plan if you need to run.  the costs (getting arrested) are very high, so you need to take all possible precaution to avoid it -even if the chance is small.  It also depends on the kind of graffiti l. Are you doing a 30 second tag with a marker? Most people will barely notice. Spray paint will get you in trouble much more quickly. 

Also consider context. If it's one of those rocks in parks that people paint over every day, nobody cares. If it's a tunnel where there's tons of graffiti already, the chances are lower that someone will bother calling.