Problem with GPO settings for RDS-farm

Hello, im having troubles with my RDS-farm when using GPO.
I have 5x physical TS and the Connection Broker Role on one of my DC's.

The problem is when Group policy is applied on the Terminal Servers some of the servers randomly leaves the farm without any reason. 
When i remove the GPO and configure all settings manually, everything works like a charm.

My GPO contains:
Configure RD Connection Broker farm name: rds-pool1
Configure RD Connection Broker server name: DC01
Join RD Connection Broker Enabled 
Use RD Connection Broker load balancing Enabled

And has security filtering enabled on member of Session Broker Computers.