Powershell "related configuration data for the page is invalid"

I have installed the CAS role on my Exchange 2013 Hub/Mailbox servers. What I'm doing is moving from 2x CAS and 3x HUB/MBX to just 3x CAS/HUB/MBX.

Now when I open the Exchange Management Shell it cannot connect to either of the 3 servers and gives an IIS error (I excluded the bunch of HTML data):

IIS 7.5 Detailed Error - 500.19 - Internal Server Error
The requested page cannot be accessed because the related configuration data for the page is invalid.
C:\Program Files\Microsoft\Exchange Server\V15\ClientAccess\PowerShell-Proxy\web.config

So Exchange messed something up when I added the CAS role to the servers. Any idea what it could be? I have a web.config and web.config.bak file and I tried swapping the two with no luck.