Why is my iPhone 4S making buzzing/interference noise internally?

I replaced my iPhone 4S battery with an aftermarket one off ebay.

Now when the cellular signal is on the iPhone is making buzzing/interference sounds. These sounds come from the internal of the phone (not the speakers).

The sound is similar to putting a cell phone next to speakers... blip blip buzz buzz. (If you know what I mean).

I was wondering if it could be related to the antenna ground piece (small black piece the sits under the top battery connector screw). It popped off during the replacement, I put it back according to the images here on ifixit.

Is this noise bad for my iPhone? How can I make it stop? Did I install something wrong?