Exception "EsepCommand failed and no fallback available."

Hi Chris, After consulting with our R&D developers, they mentioned three points that may help you troubleshot with this issue: #1. when upgrade scripts, did you also run all database scripts #2. have you also replaced "eSL-Groovy-Controller.jar" with the 6.5 version #3. have you enabled Ada for account in back office And also they mentioned that if you were seeing EsepCommand 500 error, there should be more error stack traces in back end telling the detailed reasons. If you deployed backend and platform into different servers, please make sure you have checked all available logs. Hope this could help! Duo

#1 and #2 have been confirmed to be correct. and we do get a stacktrace in the backend, but unfortunately, it doesnt seem to contain information useful to tracking down the specific problem, just that the exception was thrown and where in the code it was thrown from, which as we cant see the code, doesnt help us much. the stacktrace is attached if you can have your R&D look at it and see something we are missing. #3 i would like to get some clarification. We have not expressly set anything in back office to enable ADA. at least not on purpose. Rather, we use the .Net SDK .WithAda() flag on package creation to turn this feature on. This is both in the working environment and the environment that throws the error. Where in back office would i find the ability to explicity set this? if nothing else i can check to see of someone else set it without our knowing in the other environment. edit - i apparently cannot attach a text file, so im just putting it at the bottom of my post END: com.silanis.esl.rest.resource.DocumentResource@daf3076c.uploadDocument elapsed (ms): 1844 [2/11/19 15:24:33:772 CST] 00006b1e ThrowableMapp E EsepCommand failed and no fallback available. com.netflix.hystrix.exception.HystrixRuntimeException: EsepCommand failed and no fallback available. at com.netflix.hystrix.HystrixCommand.getFallbackOrThrowException(HystrixCommand.java:1143) at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:840) at com.netflix.hystrix.HystrixCommand.access$1400(HystrixCommand.java:81) at com.netflix.hystrix.HystrixCommand$2.call(HystrixCommand.java:721) at com.netflix.hystrix.strategy.concurrency.HystrixContextCallable.call(HystrixContextCallable.java:45) at java.util.concurrent.FutureTask.run(FutureTask.java:277) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.lang.Thread.run(Thread.java:785) Caused by: org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error; nested exception is java.lang.Throwable: Calling Thread included as the last 'caused by' on the chain. at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:94) at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:641) at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:597) at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:557) at org.springframework.web.client.RestTemplate.postForEntity(RestTemplate.java:384) at com.silanis.esl.rest.client.DefaultHttpRestClient.postForEntity(DefaultHttpRestClient.java:227) at com.silanis.esl.eslx.esep.api.service.EsepCommand.run(EsepCommand.java:44) at com.silanis.esl.eslx.esep.api.service.EsepCommand.run(EsepCommand.java:21) at com.netflix.hystrix.HystrixCommand.executeCommand(HystrixCommand.java:786) ... 7 more Caused by: java.lang.Throwable: Calling Thread included as the last 'caused by' on the chain. at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:437) at java.util.concurrent.FutureTask.get(FutureTask.java:215) at com.netflix.hystrix.HystrixCommand$QueuedExecutionFuture.performActualGet(HystrixCommand.java:1518) at com.netflix.hystrix.HystrixCommand$QueuedExecutionFuture.get(HystrixCommand.java:1448) at com.netflix.hystrix.HystrixCommand$QueuedExecutionFuture.get(HystrixCommand.java:1480) at com.netflix.hystrix.HystrixCommand.execute(HystrixCommand.java:429) at com.silanis.esl.eslx.esep.api.service.EsepGateway.executeMultipleOperations(EsepGateway.java:102) at com.silanis.esl.eslx.esep.api.service.EsepGateway.executeSingleOperation(EsepGateway.java:94) at com.silanis.esl.eslx.esep.api.service.EsepGateway.executeForTransaction(EsepGateway.java:50) at com.silanis.esl.eslx.esep.transaction.document.EsepDocumentService.addDocumentToTransaction(EsepDocumentService.java:114) at com.silanis.esl.eslx.hybrid.HybridEslxDocumentFacade.addDocumentToTransaction(HybridEslxDocumentFacade.java:44) at com.silanis.esl.document.EslDocumentService.addEslxDocumentsToTransaction(EslDocumentService.java:176) at com.silanis.esl.document.EslDocumentService.addDocuments(EslDocumentService.java:135) at com.silanis.esl.api.document.DefaultDocumentApiService.addDocumentsToPackage(DefaultDocumentApiService.java:161) at sun.reflect.GeneratedMethodAccessor964.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport$1.doInTransaction(TransactionAspectSupport.java:304) at org.springframework.transaction.jta.WebSphereUowTransactionManager$UOWActionAdapter.run(WebSphereUowTransactionManager.java:347) at com.ibm.ws.uow.embeddable.EmbeddableUOWManagerImpl.runUnderNewUOW(EmbeddableUOWManagerImpl.java:791) at com.ibm.ws.uow.embeddable.EmbeddableUOWManagerImpl.runUnderUOW(EmbeddableUOWManagerImpl.java:370) at org.springframework.transaction.jta.WebSphereUowTransactionManager.execute(WebSphereUowTransactionManager.java:290) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:298) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) at com.sun.proxy.$Proxy468.addDocumentsToPackage(Unknown Source) at com.silanis.esl.rest.resource.DocumentResource.uploadDocument(DocumentResource.java:93) at com.silanis.esl.rest.resource.DocumentResource$$FastClassBySpringCGLIB$$2abc9d4d.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85) at com.silanis.esl.rest.logging.PerformanceLoggingAspect.openSession(PerformanceLoggingAspect.java:34) at sun.reflect.GeneratedMethodAccessor159.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:620) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:609) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) at com.silanis.esl.rest.resource.DocumentResource$$EnhancerBySpringCGLIB$$9f90e8d1.uploadDocument() at sun.reflect.GeneratedMethodAccessor963.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1480) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1411) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1360) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1350) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:538) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:716) at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1233) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:782) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:481) at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:178) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97) at com.silanis.esl.canary.rest.DefaultIdsMappingFilterService.doFilterAndMapIds(DefaultIdsMappingFilterService.java:22) at com.silanis.esl.canary.rest.IdsMappingFilter.doFilter(IdsMappingFilter.java:16) at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91) at com.silanis.esl.logging.DefaultLoggingFilterService.doFilterAndLog(DefaultLoggingFilterService.java:22) at com.silanis.esl.logging.LoggingFilter.doFilter(LoggingFilter.java:16) at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91) at com.silanis.esl.security.rest.auth.AuthenticationFilter.doFilter(AuthenticationFilter.java:165) at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91) at com.silanis.esl.web.exception.ExceptionHandlingFilter.doFilter(ExceptionHandlingFilter.java:60) at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195) at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91) at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:967) at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1107) at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3980) at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:304) at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:1016) at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1817) at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:200) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:463) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:530) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:316) at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:287) at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1084) at com.ibm.ws.ssl.channel.impl.SSLConnectionLink$MyReadCompletedCallback.complete(SSLConnectionLink.java:658) at com.ibm.ws.ssl.channel.impl.SSLReadServiceContext$SSLReadCompletedCallback.complete(SSLReadServiceContext.java:1820) at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175) at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217) at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161) at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138) at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204) at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775) at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905) at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)

Hi Chris, Sorry for the misunderstanding, I have confirmed with our support that Ada is only an account/package level setting instead of a capability you can turn on/off, so since you have explicitly set Ada true in your package JSON, this doesn't look like a cause of issue to me. At the same time, I can also create a support ticket for you including all the details in this thread so that our Enterprise Support Specialist and R&D team can investigate into the issue. Duo

Thank you for the clarification on the ADA flag. I would very much appreciate the support ticket. we have run out of our own leads to invesitage and are at a stand still as to what we can do to fix this issue.