Would be nice to have a Group Category Privacy/ACL Setting

Currently Group Categories have Access setting where you can only set which Profile Types can create groups but what I'm asking is to add a which Profile Types can see the Group Category as well. Here's my scenario, I created an Administration Category which no members are allowed to create Groups since that Category only allows Staff Profile Type and all of the Groups in it will be Invite Only Groups, therefore, it is illogical to even show a Category with no visible groups for regular users.

Thanks!