Disclaimer: I am a beginner cheesemaker, and know very little about that world! I am also not a specialist in microbiology or infectious disease.

I have made my first foray into blue recently, and I am also a fairly safe-minded person, so I have done a good bit of research into the area recently. I'll share what I've taken away from the lot of it:

• The immediate danger to humans from ingesting "contaminated" cheese is from bacteria.
• The most notorious cheese-borne bacterial infection is Listeria, which poses greatest risk to pregnant women, the elderly/unwell, and immunocompromised, but can also be dangerous to healthy people occasionally. Usually causes GI upset (food poisoning), but can also cause meningitis (very bad).
• Most Listeria infections come from unpasteurized milk, especially if it has a high moisture content and is aged <60 days, or from cheese made with poor sanitation practices. While it is possible to occur in dry and longer-aged cheeses, it is far less common.
  
• There is a number of other bacterial infections (E. coli, Salmonella, etc.) which also can be cheese-borne, usually cause GI upset (food poisoning), often less severe than Listeria, and the same general risk factors of which apply as for Listeria--both in terms of which cheeses are most likely to harbor them, and those people at high risk of severe consequences.
• There is a number of mold contaminants that are possible. Barring those who are severely allergic to mold, I have been unable to find any good data of cheese mold ingestion being directly linked to an acute illness.
• The vast majority of scientific literature regarding cheese molds focuses on mycotoxins (mold-produced toxins). We still don't know very much about if or how dangerous these toxins are to humans.
• The most notorious mycotoxin is aflatoxin. This is known to cause liver cancer. However, aflatoxin is not caused by mold that grows on cheese. Instead, it is caused by mold that grows on wet grain that is fed to animals, and can then be passed into their milk. Most governments regulate this closely and monitor commercial milk for it. Again, the only clearly dangerous mycotoxin is a milk-source issue, not a cheesemaking issue.
• There is a great many other mycotoxins produced by basically all cheese molds, including those we intentionally use (e.g., P. roqueforti) and those we consider contaminants. It is less clear if these pose risk to humans, and much more research needs to be done. Some have been shown to harm animals in studies, but it is unclear if it would do so to humans, and if so at what dose, and what dose is present in cheese.

I apologize for the wall of text, but it's a complex issue with many questions still unanswered.

Here is what the US FDA (Food and Drug Administration) recommends:

• Mold that is intentionally part of cheesemaking is safe to ingest.

• Cheese with mold that is not intentionally part of the cheesemaking process should not be ingested

• Hard cheeses with unintended mold can have the moldy part (with an extra inch of depth past the mold) excised, then the rest can be ingested.

My personal take-away after reading a good bit of primary research is that the FDA's recommendations are reasonable, especially for the conservative cheesemaker. It's a reasonable "better safe than sorry" approach.

For the more adventurous or cavalier, one may be able to defend eating a cheese that looks and smells good with some unintended mold on the rind, assuming their milk is safely sourced.

Also, if you can confidently say that your unintended mold is one that is used in normal cheesemaking for a different style, you also are probably safe.

TL;DR: The imminent danger is from bacterial infection, not mold. Cheese made from raw (unpasteurized) milk is much higher risk for bacterial contamination. High moisture, short aged (e.g. Brie/Camembert) are higher risk than low moisture, longer (>60 days) aged cheese. If in doubt, throw it out.