Regarding assigned moderators to see replies.

I'm trying to show replies to "assigned" moderators in the categories.

I did not add the user group here: "who can view replies in this category" because if i add here, every other users in the moderator group will be able to see the replies. I want to make an exception and show replies to the "assigned moderators".

Also, "who can moderate this category", i added the moderator group to be able to assigned moderators by the admin. However, i want to hide replies from non-assigned moderators.

How should I do this? Please help.