Welcome to PSG, Colin
The white pixels around your button are called "jaggies". To get rid of these you must pick the correct background color in the Save for the Web dialogue box.
Obviously you had it set to white in your example. The best thing to do is figure out what color your background is that you're going to be placing the button on.
In the Save for Web dialogue box pick the same color (Matte) as that of the background you'll be placing your button on.
This way you will always have a nice blend between your image and the background you place it on.