Perf: Optimize member handling in collectives with a large number of members

Perf: Optimize member handling in collectives with a large number of members #4085