Having a large number of followers may look impressive to some people but in the end these teeming hordes won't give you any real benefit. Better to have a hundred engaged followers than ten thousand who are either bots, or don't give a tinker's about what you do.
All this is true. However I don't think you should discount quantity completely. It's great to have a small number of consistent engagers in your network. But it's even better if you have a large number of people like this. Having many more followers is a real bonus, as long as they are of a good quality too.
If you get to this point you'll really gather momentum. You'll get many more retweets and mentions, as well as clicks on the blog posts you're sharing. It can lift your profile a lot. And you end up with still more quality tweeps finding and following you as a result.
Now, some people believe that quality and quantity in this context are ultimately mutually exclusive -- that the meaningfulness of the interactions is diluted by their number. Certainly this does occur when you get into really high numbers. It's impossible to even remember, let alone engage sincerely, with tens of thousands of followers, after all.
But remember that Twitter, being such a super fast, laser targeted and distilled medium, trains you to really get to the point and process information very quickly. After a few months of using the site you'll be amazed at just how much you can put in a tweet, as well as what you can actually recall about your followers and what's been said between you. Those interactions may have been brief, but it doesn't mean they were insincere or meaningless. For this reason I think it's a good idea to keep finding, following and interacting with new tweeps so that you build a following that's large as well as engaged.