I tried the bash completion stuff, and found a single thing I liked, ssh completion. Most of the other stuff just annoyed me… a lot (and was mostly useless, at least for me). I first looked at the code in the bash completion for ssh completion, but it was imho too large to stuff into my .bashrc, so I wrote a small perl one-liner that did the trick.
complete -C "/usr/bin/perl -e 'my \$match = \$ARGV[1] ? \$ARGV[1] : \".*\"; open(my \$INPUT,\"< \",\"$HOME/.ssh/config\"); foreach(> \$INPUT<) {next unless s/^\s*Host\s+//;chomp;foreach(split(/\s+/)) { print \"\$_\\n\" if(/^\$match/ and not /^\d/ and not /\*/);}}'" ssh
Stuff that into ~/.bashrc and off you go. It completes based upon hosts in ~/.ssh/config