I'm pretty happy with my modded Xbox that has MAME and other emulators on it and thousands of games to play, but it's not the perfect solution. There's no way to play physical games and you can't save anywhere like most of the new retro machines can.
Of what's out there right now, the Retro Freak sounds good. It supports physical games from many consoles but also allows you to install Roms right on the system. It's pretty expensive though and not really plug and play as far as ease of use.
I'm not a huge fan of the Atari, Nintendo, or Sega throwback consoles just because they seem so limited and I don't think the value is there.
If I could buy a Sega box for example that had a digital game library and I could download every first party game from the Master System to the Dreamcast along with some third party support, that would be perfect!