Own Cloud Plugin: "ClassNotFoundException: javax.servlet.ServletException" when running job for the first time

[Johannes Schneider]

Hi guys,

I have a strange problem:
I have created a new plugin that provides a cloud implementation.

I create new nodes/computers and connect them using JNLP.

The first time a job runs on the newly created computer it fails with that
log output:

```

FATAL: Remote call on JNLP4-connect connection from localhost/127.0.0.1:47026 failed
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to localhost/127.0.0.1:44145
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.Request.call(Request.java:202)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy6.fetch(Unknown Source)
at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:315)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at hudson.util.ProcessTree.get(ProcessTree.java:399)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
java.lang.ClassNotFoundException: javax.servlet.ServletException
at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:808)
at sun.reflect.GeneratedMethodAccessor209.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19)
at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from localhost/127.0.0.1:47026
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
at hudson.model.Run.execute(Run.java:1798)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
at hudson.util.ProcessTree.get(ProcessTree.java:399)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on JNLP4-connect connection from localhost/127.0.0.1:47026 failed
at hudson.remoting.Channel.call(Channel.java:961)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
at hudson.model.Run.execute(Run.java:1798)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
FATAL: Remote call on JNLP4-connect connection from localhost/127.0.0.1:47026 failed
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection to localhost/127.0.0.1:44145
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.Request.call(Request.java:202)
at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy6.fetch(Unknown Source)
at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:315)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at hudson.util.ProcessTree.get(ProcessTree.java:399)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
java.lang.ClassNotFoundException: javax.servlet.ServletException
at hudson.remoting.RemoteClassLoader$ClassLoaderProxy.fetch(RemoteClassLoader.java:808)
at sun.reflect.GeneratedMethodAccessor209.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at org.jenkinsci.remoting.CallableDecorator.call(CallableDecorator.java:19)
at hudson.remoting.CallableDecoratorList$1.call(CallableDecoratorList.java:21)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from localhost/127.0.0.1:47026
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
at hudson.model.Run.execute(Run.java:1798)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: java.lang.NoClassDefFoundError: javax/servlet/ServletException
at hudson.util.ProcessTree.get(ProcessTree.java:399)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1090)
at hudson.Launcher$RemoteLauncher$KillTask.call(Launcher.java:1081)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:93)
at java.lang.Thread.run(Thread.java:748)
Caused: java.io.IOException: Remote call on JNLP4-connect connection from localhost/127.0.0.1:47026 failed
at hudson.remoting.Channel.call(Channel.java:961)
at hudson.Launcher$RemoteLauncher.kill(Launcher.java:1078)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:510)
at hudson.model.Run.execute(Run.java:1798)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
```


I tried to debug it further and could narrow it down.

I think the exception is thrown the first time in the Method
hudson.util.ProcessTree#get() (Line 399).

Probably calling "SlaveComputer.getChannelToMaster()" fails.
SlaveComputer references javax.servlet.ServletException which seems to be
unavailable there.


The agent.jar that is downloaded from Jenkins does *not* contain the
javax.servlet.ServletException class.


Running the job a second time on the same computer works just fine.
The exception occurs after the job has finished.


Any idea how to solve that?


Regards,

Johannes

[Jesse Glick]

This is a longstanding JENKINS-26677 that has never been
satisfactorily resolved, though there have been workarounds. You can
see it without any custom plugin, just using `CommandLauncher` with
`jenkinsci/docker-agent`, depending on how Jenkins is being run and
what build steps you use. The problem relates to how the Servlet API
is loaded from the servlet container hosting the Jenkins master.

Anyway, if you are able to reproduce, please give feedback on
https://github.com/jenkinsci/jenkins/pull/3778. Assuming the build is
successful, it should deploy an incremental version
2.154-rc27545.0e85c9fb206a which you can use as `jenkins.version` in
your POM (or passing via `-D` to `hpi:run`).

[Johannes Schneider]

I will try

Post by Jesse Glick
2.154-rc27545.0e85c9fb206a

as soon as it is build.

One question: What are the workarounds? What can be done?

[Jesse Glick]

On Tue, Nov 27, 2018 at 11:40 AM Johannes Schneider

Post by Johannes Schneider
What are the workarounds?

I am not aware of any.