I recently changed my avatar on this forum due to my last picture being so 2013. I noticed that when I swapped, my old one is still there as seen here:
http://cloud.stackideas.com/images/discuss_avatar/5709_1656f9976fefe7266689b7cbc92973f7.jpg

For a number of reasons I recommend some day having the old ones auto purge as the replacement one comes in.