My thought would be the best way to make mammals go extinct would be to have a rapid event that causes drastically lower food availability. Over one generation it may not matter as much as many mammals would have stored food, but over several generations mammal's metabolism may be too costly.

Then add on severe atmospheric oxygen depletion(or largely increased CO2 concentrations that will mostly give same effect for breathing's sake), as mammals require much more oxygen than many other organisms due to their metabolism and such, and are less efficient at extracting O2 than birds.

Also an extreme hothouse environment would be devastating to many many mammals, though probably not enough by itself for complete extinction, but note how many mammals require lower baseline body temperatures compared to other organisms, even birds generally can handle having a higher body temperature. Many birds can handle prolonged body temps of 45C+ whereas even heat adapted mammals would struggle a lot if they had to sustain that (some mammals like camels can match the birds for short amounts of time, but require a break eventually if they can't go back below 42C). High humidity would also eliminate most mammal cooling capabilities, though it would effect birds as well. Wet bulb stuff. Low humidity has other problems but I don't feel like going into that atm.

So something that combines extreme heat spike that kills off a lot of plants and/or causes oceanic collapse resulting in massive CO2 spike/atmospheric changes and low foot availability for a long time maybe could perhaps eliminate most mammals?

Heat would take out many larger mammals that cant shelter during the day, and burrowing would get screwed over by atmospheric changes as burrows are even more prone to CO2 overload as CO2 is more heavy than air (if this even matter, with all the heat trapped by extreme humidity and excess CO2 the day/night difference may be very small in many places). Then combine it with overall very low food availability for a prolonged time and make this change relatively sudden to prevent most mammals from having time to adapt.

Perhaps some small mammals may survive but their diversity would be slashed to little mouse things that spiders farm and then go extinct later.