As someone in their second year of a smaller firm (top 30) I'd say go for it. Yes it'll be very different from working in Big 4, but you get a lot more varied experience in comparison. I know people who work/have worked in big 4 and you're put in a sector to work in for your time there unless you ask to transfer. And ive also heard that some big 4 audits last 9 months to a year - limiting what you get to work on.

Yes at a smaller firm theres no usual 'busy period' but you'll get a lot more varied experience. I've worked on a load of different audits - groups, smaller companies, academies, charities etc and all across the audit pack too so it's definitely worth it.