First, check that you no longer have a "stutter dial tone." When you pick up the phone, is the dial tone steady? Or intermittent?
If it's not steady, that means the service thinks you still have some VM messages that you haven't yet heard. If that's the case, note it here and we can work on that.
Assuming you have a steady dial tone, you may want to see if you can clear the light from your phone. Some phones have a "reset voicemail indicator" function buried somewhere in their settings. Perhaps the phone hasn't recognized the messages have all been heard and it needs to be informed that is the case.
If that doesn't help, one other consideration: is it possible the indicator is also used to tell you something else, such as a low battery or or missing extension device?