Without looking at it I'd suspect the tensioner is weakening. Many are marked with a gauge that shows if the tension has weakened enough to warrant replacement, but such gauges differ a little between brands. There are postings elsewhere on this site, and postings out on the web, that show you how to spot a weak tensioner.
You can simply look at the HB in motion and at the security of the alternator mount to tel if those are problems.
Also, make sure that what you think is side-to-side movement is not merely an illusion due to whatever printing or wear is apparent on the back of the belt.