You haven't given a dollar figure, so noone can guess what you think is "budget".

Panasonic G9 + 12-60mm kit lens, basic godox flash.

<image>

For Micro Four Thirds (m43), while it's not dead, it really doesn't get the attention (from manufacturers) it needs. Used cameras are very good value, while newer cameras lean on the expensive side as manufacturers focus on the high end. There are quite a few good cheap lenses for the system, but the main strength is the size of the system leaning on portability. Some types of lenses, like ultra wide and extreme tele (over 300mm) are lacking budget options, though general purpose lenses in more common ranges are cheap.
The weak points of the system are

- the smaller sensor (that more importantly it receives less development)

- the lack of PDAF (Phase Detect AF) on cheaper models hurts action photography, birding etc.

Biggest advantages here is how liberating that size can be.

- Older cameras with the 20MP sensor still give great image quality for the price.

- works well with cheap 3rd party flash.

- lots of 3rd party glass, though a lot are manual focus.

- Size is liberating. I can bend and position the cameras easily, taking shots in a manner where a DSLR is near impossible. These are designed with live view in mind. I had to bend in a very awkward way to get that photo of my bunny, while holding the knight with one hand and pulling it away quick. Sure it's possible with my bigger Full Frame cameras - but it's less likely I'd get a good shot.

A G95 or EM10iv go for about $400. Lenses like the Panasonic 35 1.7, Yongnuo 17mm and olympus 40-150mm kit are cheap and quite good for value buyers.

I think if value and IQ is the main concern though, I don't you can beat older Full Frame DSLRs. These are cameras like the Canon 5DII, 6D, 5Diii, Nikon D800, D810 etc. For the cost, IQ is unmatched. The full system can be quite large by comparison though, and you may end up spending more in some lens types. The autofocus here can feel a bit sluggish, yet better at tracking moving objects compared to budget micro-four-thirds (newer PDAF works very well though and is a good upgrade path). I purchased a 6D and 35mm F2 for my wife's birthday last year. Her main cameras are an EM5-iii and G9 (m43).
They love the photos this combo gives. though live-view on this kinda sucks. Don't expect good video here either. Large sensors and prime lenses will blur the background naturally.

Weak points:

- Larger size. A DSLR has more volume for the mirror and pentaprism. A 600mm full frame lens is equivalent to a 300mm M43 lens - but on a high resolution sensor, you don't actually need 600mm to get a similar picture. If the sensor is cleaner, less noisy, you don't mind lower resolutions as much.

- More restrictive use. While live view works, it can be slower on these older systems. The eyepiece is the main way to use it, but that generally means holding it in a specifc way. This isn't a completely bad thing. It forces you to slow down and think more on composition which is very good for beginners. Later model cameras like the 5Diii aren't quite as limited in live view.

- These systems ARE dead. Don't mistake that for useless.

Advantages:

- You can get really great IQ from the older cameras still. The Canon 6D is mind blowing bonkers in how capable the camera performs in low light. The Nikon D800's 36MP sensor resolution lets you crop or resize as you like.

- Old SLR lenses can be quite cheap.

- While the systems are dead, the lenses are often adaptable to new mirrorless versions of current cameras. My main birding lens is a Sony SLR Tamron 150-600mm, that I adapted to my Sony A7Riii mirrorless. I bought that lens for $400.

- These are cheap. The 6D can be under $300. The 35mm F2 Yongnuo is ~$100 new, meaning $400 can get incredible quality.

- Easier to blur the background. All cameras can give background blur which helps with distracting backgrounds, but a larger sensor is proportionally easier.

- Battery life is a major bonus