Configuration based ESB

I think that anyone who read my previous posts can see that I work allot with ESBs – the blog posts were mostly dedicated to IBM, but I work with OSB, JBossESB and some other solutions as well.

I am amazed at just how much work customers invest in ESB development – they build an architecture, build a development team – and usually, 99% of the work they do – is the same across all organizations.

Because everybody are using ESB in the same way – to enforce web services policies – usually security, but also validation, version management, monitoring and a bit more.

So – what do all these off-the-shelf products give us? Not enough.

I believe that a new generation of ESB is in order – one that will not require coding. The ESB should contain a single point of entry for all services in the organization, all will follow a configured validation and enrichment path, and at the end will be directed to the actual service. No coding, no development team – just simple configuration. Should definitely suffice most customers.

I have a few ideas on an architecture of such a thing, and Aluna even has a small product based on open source technologies, that runs with both .net and Java clients.

But that is another story.

WebSphere 7 – deployment performance

Well, I’m currently running a migration project from Oracle’s OC4J to WebSphere 7. Why 7? Because the customer wants to use JSF 1.2, and EJB 3 – and we decided on going to WAS7 instead of WAS6.1 with the fix packs.

The migration wasn’t so smooth – the customer used some proprietary code we had to change. And let me tell you – configuring TopLink to run on WAS is a real pain, especially when some TopLink code is accessed through EJBs and some through regular Java classes from the web tier.

But the real problem was WAS7 deployment. It took forever… We tried running it from RAD7.5 or from the admin console directly – it was painful. It took around 5 minutes, and every configuration action that required application restart took ages to complete.

Turns out the problem lies in the JavaEE5 spec. When you deploy an EAR it needs to look for annotated classes that are marked as EJB and servlets. So deploying a WAR file with a large number of classes and JAR files will be very slow.

There are deployment separation solutions (putting the classes in the app-inf/lib of the EAR instead of web-inf/lib of the WAR) but a quick fix, that fits when you don’t use annotated servlets is to add the metadata-complete attribute to your web.xml file (in the web-app tag). It tells the deployer that all of your servlets and filters are configured inside the web.xml file.

See this link for the complete web_app_2_5 XSD description. http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd

WebSphere 7 book by Packtub

Well, I was asked by Packtub to review their new book on WebSphere Application Server 7. It was 3 month ago, but I finally got around to it.

The book starts kinds of slow – downloading WinSCP and Putty, installation on Linux. Not sure this is the most common developer scenario.

And I’m not sure I would learn WebSphere App Server for the first time from a book…

But then the book starts to kick off – Chapter 3 talks about Security. Insane – considering the fact that 90% of the WebSphere administrators I know don’t know how to configure security. And they go into detail – configuring LDAP and stuff.

Then they go on on Tracing, Performance – what everybody should know – and they explain it in a step-by-step manner.

So from my side – I think it’s a good book. Especially since so few people really understand how to work with WebSphere App Server.

On the down side – there is no Web Services chapter. With the new Web Service pack (available in WAS 6.1 and builtin in WAS 7) it’s impossible to configure web services in WAS, and it’s a shame the book didn’t cover this complex topic (especially web services security).

WebSphere Process Server – skipping steps in the process

A customer of mine has decided to build an application that can monitor his processes, instead of the regular WPS supplied tool.

His main concern was skipping steps. Turns out that when a process has executed a specific activity, and this activity is in a special position (like user task) – an outside application can cause a skip in the process.

I have developed a short demo for this, and I’m enclosing the main method here. Just make sure to disable the process security before running it – or you’ll get a non authorized exception…

// Connect to the server

Properties props = new Properties();

props.put(Context.PROVIDER_URL, “iiop://localhost:2811”);

javax.naming.InitialContext ctx = new javax.naming.InitialContext(props);

// Lookup the BusinessFlowManager. If you get casting exception – read the InfoCenter – you need to add some JARs to your code – it’s WPS/ProcessChoreographer/client/bpe137650.jar

Object lookupResult = ctx.lookup(“com/ibm/bpe/api/BusinessFlowManagerHome”);

BusinessFlowManagerHome processHome = (BusinessFlowManagerHome) PortableRemoteObject

.narrow(lookupResult, BusinessFlowManagerHome.class);

BusinessFlowManager bfm = processHome.create();

// Replace with any process id

ProcessInstanceData pid = bfm.getProcessInstance(“_PI:90030124.f71386d4.dbed54f5.5d84025c”);

// Replace with any activity name

ActivityInstanceData aid = bfm.getActivityInstance(pid.getID(),”Wait”);

// Skip to the specific activity

bfm.skipAndJump(aid.getID(), “TargetActivity”);

AspectJ 1.6 with runtime weaving and WPS 6.2

Well, I haven’t blogged in ages, and it’s not because I lacked things to share…

Anyway, almost all of my WPS customers are looking for some level of polymorphism (or process templating) and AOP. For the first – I currently have no solution. For the second – here comes…

1. Download AspectJ 1.6.2.
2. Since it’s been a long time since I worked with AspectJ, I wrote a small program in Eclipse. I just made sure that it used the WPS JDK – to make sure the -javaagent flag works. It does. However, you can’t use it when your eclipse works with a Sun JDK. So – add a -vm to your eclipse.ini file. (See here for details: http://wiki.eclipse.org/Eclipse.ini
3. Now, I opened a new WID module, wrote a short process.
4. I also created a small Java Project, which will create a JAR in the WAS/lib directory. It will include my aspects and the aop.xml file (under the META-INF directory)
5. I started my WPS server, and put the -javaagent flag in the startup options. WAIT!!! Since WPS security is enabled by default, and the aspectJ weaver is located outside the WPS libraries – java security policy won’t let it load – and you can’t start your server!!! Copy the jars into the WPS lib directory. Just make sure you don’t override existing files (aspectj.jar should be in WPS/lib – just rename it to something that doesn’t end with JAR)
6. That should be it – the aspects now work. However, how do we tie them to the process itself?
7. It seems that the way WPS works is that for every BPEL activity there is a class in the com.ibm.bpe.engine package, and the doActivate method is called. So our pointcut will need to look like this – @Before(“call(void com.ibm.bpe.engine.BpelActivityKind*.*(..))”) – this should also explain why we put the aspect in the JAR in the WAS/lib directory – it should be in the correct class-loader level.
8. Well the only thing missing is the actual pointcut – but it’s too late for that – will publish it tomorrow morning… Don’t hold it against me. I will also post the required code to get which Activity we are on. Quite cool…

WebSphere ESB and EJBs

My next post will probably be a book review – I was asked to review a new book on WAS 7.0 administration. Cool. Hopefully I’ll be able to read it in the next couple of days (did I ever mention I was a quick reader?)

In the meantime, I want to publish my findings on the use of EJBs in WebSphere ESB.

A customer of mine is using WebSphere ESB. For services, it developed EJBs (stateless, of course, but still – version 2.0).

The mediation module has imports for the EJBs.

This turns out to be a very bad architectural decision (I arrived to the project after this decision was taken 🙂 ) , and right now they are rewriting everything, to drop EJBs and use SCA Java components.

Why?

1. SCA Java components are actual EJBs, so why have an EJB call an EJB? Not a smart move, performance wise.
2. Mediation Module (6.1) EJB support is lousy. It fails generating good mapping between the Java bean parameters and Data objects. So my customer resorted into sending Strings to the EJBs, and then parsing them, at the EJB level, to POJOs, using XMLBeans. This is insane – and shouldn’t be done.

So what is the current (working!) recommendation?

Use SCA Java components, in the same module, or other modules, depending on your component visibility and deployment needs.

Inside the Java component, use a mapper to map the DataObject object into real Java POJO. and then pass it to your business logic, which is implemented in regular Java Classes.

The mapper is really easy to write:

1. Use JAXB or XMLBeans to generate mapping between the DataObject XSD and a Java class.

2. Write a method that has the following signature:

public static Object mapper(DataObject do, Class clz)

3. Use reflection on the clz – create a new instance of it, and for every field decide:

3.1 If its a primitive – transform it.

3.2 If its a Java Class – call yourself in recurssion

3.3 If its an array – run over all elements

It works fine, great performance, and the development barrier was lowered by more than a few inches.

Hope it serves someone well.

Problems installing WID 6.2 on Windows 7

I faced many problems while installing WID 6.2 on my Windows 7 machine. I won’t get into the full set of problems I encountered, but the process that did work for me was as follows:

1. Don’t run the launchpad application, but the installer inside IM_win32

2. Run the installer in compatibility mode, for WindowsXP SP2

3. Since I already tried installing – I had to install everything to a clean directory

3.1 This is a reported bug and was fixed in a fixpack already

4. Remove the file c:windows.nifregistry

I found the last tip after digging inside the installation log files, I found the following command executed -C:/Program Files/IBM/WID62/image/WPS62/iip/contrib/6.1.0-WS-WASWS/1/WinX32WEBSVinstall.exe -options C:/Users/liran.ALUNA/responsefile.WEBSV.txt -silent

And the log file specified a NullPointerException on the com.ibm.ws.install.ni.framework.product.VersionUtils.compareVersionsUpToDigit(VersionUtils.java:170)

The weird thing is that I did everything by the book, so it’s not clear to me why no one else got this error. But hey – it works now…

Stale Connection in WebSphere Application Server 6.1

Well, a customer of mine has a very weird bug. Sometimes he gets a StaleConnection exception when executing a statement against his Oracle Connection. The client runs WebSphere App Server 6.1, Oracle Database 10g, and uses EJBs and an XA datasource.

Finally, we discovered that there is a scenario that always raises the Stale Connection Exception, and we started investigating the root of the problem.

First of all, WebSphere throws the Stale Connection exception as a wrapper to specific SQL Exceptions received from the JDBC connection.

But why is the connection suddenly closed? A step-by-step following with the debugger found that we get the connection open from the datasource, and then, suddenly, the connection is closed.

We tried to convert the datasource to a non-XA datasource, and received allot of exceptions. This showed us that the process in question needed the XA capabilities of WAS.

We then wrote wrappers over Oracle’s XA connection manager (oracle.jdbc.xa.client.OracleXADataSource) and Oracle’s Connection (oracle.jdbc.pool.OraclePooledConnection), and seen the debug messages.

We still have no solution, but there can be 2 options:

1. There is a bug with the WAS XA handler.

2. When using statement.getConnection().close(), we close the physical connection, and not the logical connection coming from the datasource. The customer now changes his code, to see if he can get away without the statement.getConnection() bit, and close the logical connection received from the datasource.

I’ll post more details later.

WebSphere ESB Invalid Content Length

Well, turns out I was mistaken in my previous post. Invalid Content Length can occur when using MTOM in .net C# clients with WebSphere ESB, but that was not the case in our customer.

Invalid Content Length appeared when the client closes the socket before sending the entire request. This can happen when the process is halted during send time. WESB will sysout Invalid Content Length, but you can usually ignore it.

However, we still faced a problem with very large service calls (over 1MB in size – only XML, no attachments). Turns out that our synchronization code was messed up, and for some reason – our cache was not correctly initialized, and so we received allot of NPE (NullPointerExceptions).

So – we synched our cache, and voilla – all works.

That teaches me to blog before I see everything working in my own eyes.

One last important issue. Sending large service calls can take time. Allot of time. And so, each C# client has a Timeout property, that sets that Timeout for the service call, in milliseconds. Use it well, since you are very likely to get a Timeout exception before you get the web service response.