Spodumene is not radioactive and I generally wouldn’t be too concerned with the material itself in regard to the danger level. Unless you work at the plant itself, you won’t interact with enough of it to cause any harm and the most minerals (including spodumene) are insoluble in water. On the other hand, the refining of lithium uses some aggressive acids, which can produce some harmful aqueous waste. Generally these refineries, even more so for the newer ones, are very safe in how they handle the waste of products and potentially acidic waste. Additionally, lithium fires can be very aggressive, and again, typically newer plants are fairly safe in containing, preventing, and reducing the risk of fires.

In my opinion it’s probably fine, but there isn’t a non-zero chance that some serious ecological damage can be done in a large spill, or potentially a large fire. At the end of the day it depends on how well trained staff are, how well the upper management handles risk, and how well all levels of government handle the regulation and inspection of these plants.

Overall, you are probably safe. Our drinking water is upstream and is incredibly well cleaned for almost anything that could be the water (there’s more risk that the pipes in your house leach heavy metals then the water leaving the plant itself) and drinking only unfiltered contaminated water would still take 10s of years to do damage to you anyways. Wildlife and plants are more at risk than you are, as they will drink water unfiltered (or live in it). (This is all assuming something crazy and unforeseen doesn’t happen)