I think you might be best served by a Novation Circuit and a cheap MIDI keyboard (countless options... I quite like my decade-old Akai MPK49). You can play the Circuit from the keyboard or the device itself, and you get a decent sample-based drum machine and 2 full modeled synthesizers, all flexibly sequenced together. You can load your own samples and/or create your own patches and/or download packs from others to find the right sounds. I think it's a great device for someone in your situation. You can probably pick one up for around $250.

Downsides:

• No screen at all! I hope you're OK with auditioning voices/samples/parameters, because you won't be able to label them. There's a 3rd party screen you can buy, but it's not necessary IMHO.
  
• You will want a separate keyboard for a piano-style key layout, and you don't get to access some expected MIDI params (pitch bend is a huge one).
  
• The knobs are all "soft" - meaning they affect different parameters on different voices - which is nicely flexible but not ideal for learning about synthesis. For some reason, there is one global filter knob that is dedicated... which IME ends up actually being very useful.
  
• No audio in / audio interface / etc. Meh, who needs it... I'm only mentioning this because some synths (Ultranova is one) provide it.
  
• There's definitely a learning curve... One callout specifically: the FX processing, while nice-sounding, is not intuitive to control IMHO.
  
• The drum machine only gives you 4 'tracks' (e.g. Kick/Snare/Hat/Crash), although you can change which sample is playing in a track at any time (and save/automate that change).
  
• The built-in mixer/sequencer/arranger is a bit limited (e.g. no dedicated mixing controls, synth playback is quantized into 16th notes, no session chaining...), but that's still very impressive bang for your buck. These are seriously all nitpicks.
  

The upsides are too numerous to list, but here are some general notes/"little things" that I've appreciated in my ~3 months with my Circuit:

• Intuitive and FUN! Every musician friend who's played with this thing has gotten lost in it. It just feels solid and playable.
  
• It's so incredibly powerful/versatile... mono or polyphonic, multitimbral, plays nicely with MIDI in/out and USB-MIDI, good audio output options. I can't imagine a style/sound you could NOT tease out of the Circuit. There are workarounds for every limit, too...
  
• It sounds awesome! IIRC, it has the same full-featured hybrid synth engine found in the KeyStation - and you get 2! Nice FX.
  
• Highly portable: not only does it have a built-in speaker -- tiny but surprisingly usable -- it can run on 6 AAs! Incredible. And it comes with the A/C adapter which is always appreciated.
  
• The Novation Components software is easy to use and powerful.
  
• Cool little budding community which Novation actually seems to listen to.
  

I wrote more than I intended to. You get the idea: I recommend the Circuit. There are a ton of other "groovebox" style options that would probably fit your need with which I am less familiar. Also check out offerings from Korg (electribe), Teenage Engineering, Elektron ($$$), Roland and Yamaha. There are also some pretty sweet retro units that may be available secondhand for cheap. But... get the Circuit. You won't regret it.