No matter what backend image setting you choose for the popup setting (popup or not), the photo dropdown (frontend) offers to view in popup. You can see this live here: http://www.test.alpineascent.com/photos/item/6-torment-forbidden-traverse/user/josh
You can see the photo already opens in popup mode, so to have a drop down option for the frontend user seems rather silly. What it should do instead is offer the opposite effect. For example if I disable popup, the popup option should display so the user can choose if they want to use it. If popup is enabled, the option should say "View Photo Page". This way the choice is in the users hands, but the admin just sets the default.
You can see the photo already opens in popup mode, so to have a drop down option for the frontend user seems rather silly.