EJB inflation

Ever since the JavaEE standard introduced the Local interfaces (and especially in EJB3), I see people abusing EJBs. The logic is simple – if EJB calls are local, let’s use EJBs, and enjoy dependency injection.
Recently I assisted a customer who had over 200 EJBs in a project that had about 500 classes! I call it EJB inflation, and it’s bad. Really bad.
The reason – the EJB container does more than just proxy remote calls. It handles security, transactions, pooling and more. Using those, for every class, is paying a huge price in performance. Let’s just say that when I run a profiler on the customer code I saw that over 20% of the server time is wasted on the application server EJB related code (JTA, specifically).
I will post workarounds for this in future posts, but in the mean time – beware abusing EJBs. Don’t fall to the “if you have a hammer, everything looks like a nail” trap.

Errors while performing remote deployment from Eclipse to WebLogic Server

Using local WebLogic server with Eclipse is a piece of cake, especially when using OEPE (the Oracle Eclipse flavor, with all the relevant plugins installed). However, I run into issues while deploying an application to a remote WebLogic server, running on Linux (the machine I used run Windows).
The error was:

weblogic.deploy.api.spi.exceptions.ServerConnectionException: [J2EE Deployment SPI:260041]Unable to upload 'C:workspaceslabWorkspaceNew.metadata.pluginsorg.eclipse.core.resources.projectsHelloWorldbeadepremote_weblogicHelloWorld.war' to 't3://XXX.XXX.XXX.XXX:7041'

java.lang.Exception: Exception received from deployment driver. See Error Log view for more detail.
at oracle.eclipse.tools.weblogic.server.internal.WlsJ2EEDeploymentHelper$DeploymentProgressListener.watch(WlsJ2EEDeploymentHelper.java:1566)
at oracle.eclipse.tools.weblogic.server.internal.WlsJ2EEDeploymentHelper.deploy(WlsJ2EEDeploymentHelper.java:470)
at oracle.eclipse.tools.weblogic.server.internal.WeblogicServerBehaviour.publishWeblogicModules(WeblogicServerBehaviour.java:1346)
at oracle.eclipse.tools.weblogic.server.internal.WeblogicServerBehaviour.publishToServer(WeblogicServerBehaviour.java:803)
at oracle.eclipse.tools.weblogic.server.internal.WeblogicServerBehaviour.publishOnce(WeblogicServerBehaviour.java:623)
at oracle.eclipse.tools.weblogic.server.internal.WeblogicServerBehaviour.publish(WeblogicServerBehaviour.java:516)
at org.eclipse.wst.server.core.model.ServerBehaviourDelegate.publish(ServerBehaviourDelegate.java:708)
at org.eclipse.wst.server.core.internal.Server.publishImpl(Server.java:2690)
at org.eclipse.wst.server.core.internal.Server$PublishJob.run(Server.java:272)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Caused by: weblogic.deploy.api.internal.utils.DeployerHelperException: The source 'C:DOCUME~1trainLOCALS~1Temp1HelloWorld.war' for the application 'HelloWorld' could not be loaded to the server 'http://XXX.XXX.XXX.XXX:7041/bea_wls_deployment_internal/DeploymentService'.
Server returned HTTP response code: 403 for URL: http://XXX.XXX.XXX.XXX:7041/bea_wls_deployment_internal/DeploymentService
at weblogic.deploy.api.internal.utils.JMXDeployerHelper.uploadSource(JMXDeployerHelper.java:659)
at weblogic.deploy.api.spi.deploy.internal.ServerConnectionImpl.upload(ServerConnectionImpl.java:843)
at weblogic.deploy.api.spi.deploy.internal.BasicOperation.uploadFiles(BasicOperation.java:321)
at weblogic.deploy.api.spi.deploy.internal.BasicOperation.execute(BasicOperation.java:410)
at weblogic.deploy.api.spi.deploy.internal.BasicOperation.run(BasicOperation.java:170)
at weblogic.deploy.api.spi.deploy.WebLogicDeploymentManagerImpl.deploy(WebLogicDeploymentManagerImpl.java:364)
at oracle.eclipse.tools.weblogic.server.internal.WlsJ2EEDeploymentHelper.deploy(WlsJ2EEDeploymentHelper.java:468)
... 8 more

At first I thought this was caused by the size of the EAR file, but after creating a small HelloWorld.war and getting the same error, I traced it rather quickly.
The problem is network related. First, make sure your browser can connect to the admin console of the remote WebLogic Server (http://XXX.XXX.XXX.XXX:7001/console). If it doesn’t work – configure the proxy, or make the sys-admin open the 7001 port.
If it does work, ensure your eclipse has the proxy configured – using Window/Preferences/General/Network Connections

WebSphere 7 – deployment performance

Well, I’m currently running a migration project from Oracle’s OC4J to WebSphere 7. Why 7? Because the customer wants to use JSF 1.2, and EJB 3 – and we decided on going to WAS7 instead of WAS6.1 with the fix packs.

The migration wasn’t so smooth – the customer used some proprietary code we had to change. And let me tell you – configuring TopLink to run on WAS is a real pain, especially when some TopLink code is accessed through EJBs and some through regular Java classes from the web tier.
But the real problem was WAS7 deployment. It took forever… We tried running it from RAD7.5 or from the admin console directly – it was painful. It took around 5 minutes, and every configuration action that required application restart took ages to complete.
Turns out the problem lies in the JavaEE5 spec. When you deploy an EAR it needs to look for annotated classes that are marked as EJB and servlets. So deploying a WAR file with a large number of classes and JAR files will be very slow.
There are deployment separation solutions (putting the classes in the app-inf/lib of the EAR instead of web-inf/lib of the WAR) but a quick fix, that fits when you don’t use annotated servlets is to add the metadata-complete attribute to your web.xml file (in the web-app tag). It tells the deployer that all of your servlets and filters are configured inside the web.xml file.
See this link for the complete web_app_2_5 XSD description. http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd

WebSphere 7 book by Packtub

Well, I was asked by Packtub to review their new book on WebSphere Application Server 7. It was 3 month ago, but I finally got around to it.

The book starts kinds of slow – downloading WinSCP and Putty, installation on Linux. Not sure this is the most common developer scenario.
And I’m not sure I would learn WebSphere App Server for the first time from a book…
But then the book starts to kick off – Chapter 3 talks about Security. Insane – considering the fact that 90% of the WebSphere administrators I know don’t know how to configure security. And they go into detail – configuring LDAP and stuff.
Then they go on on Tracing, Performance – what everybody should know – and they explain it in a step-by-step manner.
So from my side – I think it’s a good book. Especially since so few people really understand how to work with WebSphere App Server.
On the down side – there is no Web Services chapter. With the new Web Service pack (available in WAS 6.1 and builtin in WAS 7) it’s impossible to configure web services in WAS, and it’s a shame the book didn’t cover this complex topic (especially web services security).

Stale Connection in WebSphere Application Server 6.1

Well, a customer of mine has a very weird bug. Sometimes he gets a StaleConnection exception when executing a statement against his Oracle Connection. The client runs WebSphere App Server 6.1, Oracle Database 10g, and uses EJBs and an XA datasource.

Finally, we discovered that there is a scenario that always raises the Stale Connection Exception, and we started investigating the root of the problem.
First of all, WebSphere throws the Stale Connection exception as a wrapper to specific SQL Exceptions received from the JDBC connection.
But why is the connection suddenly closed? A step-by-step following with the debugger found that we get the connection open from the datasource, and then, suddenly, the connection is closed.
We tried to convert the datasource to a non-XA datasource, and received allot of exceptions. This showed us that the process in question needed the XA capabilities of WAS.
We then wrote wrappers over Oracle’s XA connection manager (oracle.jdbc.xa.client.OracleXADataSource) and Oracle’s Connection (oracle.jdbc.pool.OraclePooledConnection), and seen the debug messages.
We still have no solution, but there can be 2 options:
1. There is a bug with the WAS XA handler.
2. When using statement.getConnection().close(), we close the physical connection, and not the logical connection coming from the datasource. The customer now changes his code, to see if he can get away without the statement.getConnection() bit, and close the logical connection received from the datasource.
I’ll post more details later.