This could be because you've sent multiple invitations and multiple accounts have been created as a result. By checking the nicknames and login times displayed in the Settings tab, you can identify which accounts are no longer in use. Just tap "Remove from Album" to delete them.
Rest assured that even after removing people from an album, the photos and videos they uploaded, as well as their comments, will remain visible.