You can only add contacts that exist in the same account in which that Public Group exists. For e.g. if you sync contacts from iCloud and MS Exchange, if you add or edit an iCloud group, you will only be able to select contacts from the same iCloud account, and not the contacts that belong to the MS Exchange account.
Please note: you cannot add "Private Contacts" to a "Public Group". You can only add "Private Contacts" to a "Private Group".