This icon only appears if you own the blog post or if you are a site admin on the site. If you do not own the blog posts or you are not a site admin, these icons wouldn't show up.

I believe they are showing up because of either:

1. Check your ACL for the guest group

or

2. You have set your guest user group to a different user group in Joomla's user configuration

or

3. You have an extensive caching system which caches the html output of a logged in user.