I have created an attribute on the group level (SMSNumbers) and a stored procedure to provide the value.
The stored procedure loops all of my customer groups and adds the mobile phone number to the group attribute.
The result looks like this: 32486xxxxxx,3248675xxxx,324795236xx
Since my customer groups only contain 1-10 users, this is hardly a strain on the database and takes seconds to complete.
If anyone is interested, you can contact me and I will gladly give you the stored procedure