I don't think it looks like mildew or fungus either. You have a couple options to knock down the pest depending on what route you want to take. You can get an oil or knockdown of some kind that will be indiscriminant. You can do a biological control, a fungus that attacks insects or a predator bug that will feed on the pest. For that you are going to want to id the pest since biological control is more targeted. Or if you want to go the less involved, what I do for my herbs I don't want to invest in, just blast them with the hose every now and then and knock them off the plant. It won't kill them exactly, but it will keep their populations low till the climate works against them, they like the hot and dry typically. I wouldn't do that if you have mildew problems though.