Hi,
I hope someone can help. I am continually getting SCOM alterts with the above appearing for all 3 of our CAS servers. Also, intermittently, I am unable to logon to the ECP (either using the DNS round robin name given to all the servers, or the server URLs themselves).
The full email is pasted below. Really not got much of a clue on this one and, it would appear, neither does google. Any ideas anyone?
Alert: Health Set unhealthy
Source: CAS01 - ECP
Path: CAS01.domain;CAS01.domain
Last modified by: System
Last modified time: 6/9/2015 3:15:51 AM
Alert description: Exchange Admin Center logon is failing on Client Access Server CAS01.
Availability has dropped to 50REPLACE_PERCENT_SIGN. You can find protocol level traces for the failures on E:\Program Files\Microsoft\Exchange Server\V15\Logging\Monitoring\ECP\EacCafeLogonProbe.
Incident start time: 06/06/2015 20:15:30
Last failed result:
Failing Component - Ecp
Failure Reason - CafeFailure
Exception:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Microsoft.Exchange.Net.MonitoringWebClient.ScenarioException: Microsoft.Exchange.Net.MonitoringWebClient.ScenarioException:
Failure source: Ecp
Failure reason: CafeFailure
Failing component:Ecp
Exception hint: CafeErrorPage: CafeFailure 500 Inner exception: Microsoft.Exchange.Net.MonitoringWebClient.CafeErrorPageException
ErrorPageFailureReason: CafeFailure, RequestFailureContext: FailurePoint=BackEnd, HttpStatusCode=500, Error=500, Details=System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at Microsoft.Exchange.HttpProxy.ProxyRequestHandler.<>c__DisplayClass2c.<OnResponseReady>b__2b(), HttpProxySubErrorCode=, WebExceptionStatus=ProtocolError, LiveIdAuthResult=
Microsoft.Exchange.Net.MonitoringWebClient.CafeErrorPageException: An error occurred on the Client Access server while processing the request
WebExceptionStatus: Success
GET https://cas01/ecp/error.aspx?aspxerrorpath=/ecp/ HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; MSEXCHMON; ACTIVEMONITORING)
Accept: */*
Cache-Control: no-cache
Cookie: ClientId=EJNL9BZUCOUCBOMFW; $Path=/; PBack=0; $Path=; cadata=Vr1cSpzZQnZCeDsc12CPAVbTO...; $Path=/; cadataTTL=TVOpPUWwl83RbPqZUVqrdw==; $Path=/; cadataKey=kB+UhSQVw9u2loxd+mxyvARsw...; $Path=/; cadataIV=lmNHgpiEs+eTyDaU048Vw5K0P...; $Path=/; cadataSig=LrqS26oXzq38cMY+lxzOLzDDE...; $Path=/; X-BackEndCookie=S-1-5-21-1515194898-11533...; $Path=/ecp
X-FeToBeTimeout: 100
HTTP/1.1 500 InternalServerError
request-id: aca0e1d9-54a5-4c79-a55b-c22b0db77aa6
X-CalculatedBETarget: MBX02.domain
X-DiagInfo: SHR2EXMBX02
X-BEServer: SHR2EXMBX02
X-UA-Compatible: IE=10
X-FailureContext: BackEnd;500;NTAw;U3lzdGVtLk5ldC5XZWJFeGNlcHRpb246IFRoZSByZW1vdGUgc2VydmVyIHJldHVybmVkIGFuIGVycm9yOiAoNTAwKSBJbnRlcm5hbCBTZXJ2ZXIgRXJyb3IuDQogICBhdCBTeXN0ZW0uTmV0Lkh0dHBXZWJSZXF1ZXN0LkVuZEdldFJlc3BvbnNlKElBc3luY1Jlc3VsdCBhc3luY1Jlc3VsdCkNCiAgIGF0IE1pY3Jvc29mdC5FeGNoYW5nZS5IdHRwUHJveHkuUHJveHlSZXF1ZXN0SGFuZGxlci48PmNfX0Rpc3BsYXlDbGFzczJjLjxPblJlc3BvbnNlUmVhZHk+Yl9fMmIoKQ==;;ProtocolError;
X-FEServer: CAS01
Content-Length: 1766
Cache-Control: private
Content-Type: text/html; charset=utf-8
Date: Sat, 06 Jun 2015 21:08:16 GMT
Set-Cookie: X-BackEndCookie=S-1-5-21-1515194898-11533...; expires=Mon, 06-Jul-2015 21:08:16 GMT; path=/ecp; secure; HttpOnly
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET<!DOCTYPE html><html><head><title>Runtime Error</title><meta name="viewport" content="width=device-width" /><style>
body {font-family:"Verdana";font-weight:normal;font-size: .7em;color:black;}
p {font-family:"Verdana";font-weight:normal;color:black;margin-top: -5px}
b {font-family:"Verdana";font-weight:bold;color:black;margin-top: -5px}
H1 { font-family:"Verdana";font-weight:normal;font-size:18pt;color:red }
H2 { font-family:"Verdana";font-weight:normal;font-size:14pt;color:maroon }
pre {font-family:"Consolas","Lucida Console",Monospace;font-size:11pt;margin:0;padding:0.5em;line-height:14pt}
.marker {font-weight: bold; color: black;text-decoration: none;}
.version {color: gray;}
.error {margin-bottom: 10px;}
.expandable { text-decoration:underline; font-weight:bold; color:navy; cursor:hand; }
@media screen and (max-width: 639px) {
pre { width: 440px; overflow: auto; white-space: pre-wrap; word-wrap: break-word; }
}
@media screen and (max-width: 479px) {
pre { width: 280px; }
}</style></head><body bgcolor="white">
-------------------------------------------------------------------------------
States of all monitors within the health set:
Note: Data may be stale. To get current data, run: Get-ServerHealth -Identity 'CAS01' -HealthSet 'ECP'
State Name TargetResource HealthSet AlertValue ServerComponent
----- ---- -------------- --------- ---------- ---------------
NotApplicable EacCtpMonitor ECP Unhealthy None
NotApplicable MaintenanceFailureMonitor.ECP ECP Healthy None
NotApplicable MaintenanceTimeoutMonitor.ECP ECP Healthy None
States of all health sets:
Note: Data may be stale. To get current data, run: Get-HealthReport -Identity 'CAS01'
State HealthSet AlertValue LastTransitionTime MonitorCount
----- --------- ---------- ------------------ ------------
NotApplicable ActiveSync Healthy 24/05/2015 03:51:41 2
NotApplicable AD Healthy 24/05/2015 03:55:36 9
NotApplicable ECP Unhealthy 06/06/2015 20:15:30 3
Online FrontendTransport Healthy 24/05/2015 04:07:00 11
Online HubTransport Unhealthy 24/05/2015 03:53:40 14
NotApplicable Monitoring Healthy 24/05/2015 03:53:07 5
Online UM.CallRouter Healthy 24/05/2015 03:51:17 5
NotApplicable UM.Protocol Healthy 24/05/2015 03:52:34 2
NotApplicable OAB Healthy 24/05/2015 03:53:32 2
NotApplicable EAS Healthy 24/05/2015 03:51:17 2
NotApplicable Autodiscover Healthy 06/06/2015 11:41:58 4
NotApplicable Autodiscover.Protocol Healthy 24/05/2015 03:52:55 2
NotApplicable FreeBusy Healthy 24/05/2015 03:53:16 2
NotApplicable BitlockerDeployment Healthy 24/05/2015 03:50:22 2
NotApplicable ClientAccess.Proxy Healthy 24/05/2015 03:53:45 330
NotApplicable Compliance Healthy 24/05/2015 04:33:27 6
NotApplicable DLExpansion Healthy 24/05/2015 03:50:10 2
Online Transport Healthy 24/05/2015 03:53:21 2
NotApplicable EDS Healthy 24/05/2015 04:27:44 25
NotApplicable EventAssistants Healthy 24/05/2015 03:53:40 2
NotApplicable EWS Healthy 06/06/2015 11:42:00 4
NotApplicable EWS.Protocol Healthy 24/05/2015 03:52:43 2
NotApplicable ExtendedReportWeb Healthy 24/05/2015 03:53:34 2
NotApplicable FEP Healthy 24/05/2015 03:51:08 2
NotApplicable FfoRws Healthy 24/05/2015 03:50:54 2
NotApplicable Places Healthy 24/05/2015 03:52:37 2
NotApplicable RemoteMonitoring Healthy 27/05/2015 04:34:08 6
NotApplicable DataProtection Healthy 24/05/2015 03:54:23 3
NotApplicable Search Healthy 24/05/2015 03:53:24 2
NotApplicable Inference Healthy 24/05/2015 03:49:29 2
NotApplicable OWA Healthy 24/05/2015 03:51:20 3
NotApplicable MailboxSpace Healthy 06/06/2015 11:44:54 2
NotApplicable MRS Healthy 06/06/2015 11:46:46 2
NotApplicable MessageTracing Healthy 06/06/2015 11:46:28 2
NotApplicable Network Healthy 06/06/2015 11:45:56 4
NotApplicable Calendaring Healthy 06/06/2015 11:42:53 2
NotApplicable PeopleConnect Healthy 06/06/2015 11:45:14 2
NotApplicable IMAP Healthy 06/06/2015 11:44:33 2
NotApplicable POP Healthy 06/06/2015 11:44:07 2
NotApplicable ProcessIsolation Healthy 06/06/2015 11:46:26 27
NotApplicable Provisioning Healthy 06/06/2015 11:46:55 2
NotApplicable Psws Healthy 06/06/2015 11:43:22 2
NotApplicable PublicFolders Healthy 06/06/2015 11:43:47 2
NotApplicable PushNotifications Healthy 06/06/2015 11:44:24 2
NotApplicable Outlook Healthy 06/06/2015 11:46:33 3
NotApplicable RPS Healthy 06/06/2015 11:44:14 3
NotApplicable Security Healthy 06/06/2015 11:45:38 3
NotApplicable SiteMailbox Healthy 06/06/2015 11:45:31 2
NotApplicable Store Healthy 06/06/2015 11:43:38 2
NotApplicable OutlookMapiHttp Healthy 24/05/2015 03:53:43 1
NotApplicable UserThrottling Healthy 24/05/2015 04:19:32 7
Online Autodiscover.Proxy Healthy 24/05/2015 03:48:40 1
Online ActiveSync.Proxy Healthy 24/05/2015 03:48:40 1
Online ECP.Proxy Healthy 24/05/2015 03:48:40 1
Online EWS.Proxy Healthy 24/05/2015 03:48:40 1
Online OutlookMapiHttp.Proxy Healthy 24/05/2015 03:48:40 1
Online OAB.Proxy Healthy 24/05/2015 03:48:40 1
Online OWA.Proxy Healthy 24/05/2015 03:48:40 2
Online RPS.Proxy Healthy 24/05/2015 03:48:40 1
Online RWS.Proxy Healthy 24/05/2015 03:48:40 1
NotApplicable Antimalware Healthy 24/05/2015 03:51:17 2
NotApplicable MailboxTransport Healthy 24/05/2015 03:53:39 2
Online Outlook.Proxy Healthy 24/05/2015 03:48:40 1
NotApplicable EmailManagement Healthy 24/05/2015 03:52:29 2
NotApplicable FIPS Healthy 24/05/2015 03:51:18 2
NotApplicable FfoCentralAdmin Healthy 24/05/2015 03:52:03 2
NotApplicable FfoMonitoring Healthy 06/06/2015 11:46:33 6
NotApplicable OfficeGraph Healthy 06/06/2015 11:43:24 2
NotApplicable HxService.Mail Healthy 06/06/2015 11:46:56 2
NotApplicable RemoteStore Healthy 06/06/2015 11:45:19 2
Online SharedCache Healthy 06/06/2015 11:46:08 2
NotApplicable StreamingOptics Healthy 06/06/2015 11:43:35 2
NotApplicable TimeBasedAssistants Healthy 06/06/2015 11:46:47 2
Note: Subsequent detected alerts are suppressed until the health set is healthy again.
Knowledge: http://technet.microsoft.com/en-us/library/ms.exch.scom.ECP(EXCHG.150).aspx?v=15.0.1076.9
Alert view link: "http://Server02/OperationsManager?DisplayMode=Pivot&AlertID=%7b09897636-0d7e-41b2-b703-cd647763112e%7d"
Notification subscription ID generating this message: {ECB7E49D-DF13-2F63-F884-56CECD2EB2B2}
Thanks in advance...