Open source commercialism or: how I learned to stress test and prove the app

For any perspective software you may use to run a company, you will need a guaruntee that it will be able to handle what you throw at it. In the case of a bleeding edge, open source project that shows much potential you really want to make that software work. Besides the benefits of open source, such as the code being downloadable/editable/compilable, the software is free!!! Pair that with the fact that if it is a hot project then many eyes have glanced the codebase, and many users have filed many bug reports and feature requests on issues they have. Open source projects can be very commercialy viable: just look at Tomcat.

I am working with a ambitious piece of software called Servicemix (Smix). If things go well it could really help me and my company out. While we have been using it in our production system for over 3 months I have not had time to really stretch it out. What I mean is test more of the 'out of the box' functionality in stressed ways. Smix is an ESB which implements the Java Business Integration (JBI) specification. This specification is much like the Sun servlet spec, JDBC spec etc. Those Sun guys love their specs!

To really test out this software I have created some basic service assemblies (SA). A service assembly allows you to package together functionality that comprises any number of processes. It is also part of the JBI spec. An example of a SA would be a consumer for a JMS topic queue, which passes all messages to an XSLT transformer, which in turn passes the message to an app that processes the messsage and inserts the data into a database. This would all be packaged into one file for deployment. Those familiar with J2EE would think of it as a .ear file. If an SA is analogous to a .ear, then a service unit (SU) would be to .war. In the previous example the SU's would be the JMS routing, the XSLT services, and the database service. Each would be packaged as it's own service unit in Smix.

Now I also created a configurable JMS provider, which can be tasked with many different things. I created a configurable JMS consumer, which can read from a queue. Last night I sent 75,000 small messages through an SA I created which does two separate XSLT transforms. The transfromed results go onto two separate queues. The contents of those queues are consumed by two of the consumers I created. Everything went off hitch free! Very nice for my first real foray into stress testing Smix. Here are some approx. results:

Started message providing: ~9:40 pm

Message provider rate: ~6-7 message per second
Message consumer rate: 7.8125 messages per second

Message consumer 1 finished: 2006.09.28 AD at 22:38:17:108 PDT
Message consumer 2 finished: 2006.09.28 AD at 22:38:17:108 PDT

The message payload:

<message>
    <count>(message #)</count>
    <datetime>(date-time inserted)</datetime>
</message>

An example of a transformed message:

<ecometry>2 ::: 2006.09.29 AD at 07:47:41:996 PDT</ecometry>

Working

Lot's of work with Servicemix... I have checked out the newest SNAPSHOT release and built it, deploying all the project artifacts into my companies Maven snapshot repository. I'm trying now to get all the conifguration I had for Smix before into tidy little service assemblies (SA's). These are kinda like .ear files, in that they contain service units (SU's) which are sorta like a .war. The whole service assembly idea is from the JBI specification, which Smix implements.

I'm working mostly now on learning to develop for JBI and Servicemix. The newest snapshot release has reworked examples which show how to use the maven-jbi-plugin to easily create and deploy JBI components (shared libs/binding components/service engines/service assemblies). I am posting a little Smix developer roadmap over on the Smix site; follow this link.

...also on the board is getting CVS managed projects into Subversion. Our SVN repository is now up and ready to go. We got the cvs2svn tool installed, which will allow for migrating one project at a time from CVS to SVN. The benefit in doing this is all historical information for the CVS managed project will be translated into SVN. Very nice!

With Subversion up and running I hope to soon get Continuum installed on our development server. It is a continuos integration server which works very well with Maven 2. It also works with plain old Ant, or Maven 1. Another thing on the board is getting a nice front end to our Subversion repository. I looked at fisheye, it is pretty nice but costs a bit of dough to use. I passed it up to my boss. If he is impressed then maybe we will end up writing the check

Servicemix, Spring 2.0 Maven poms in central

My work with portals has been stopped as of late. I talked to someone at the Spring project, offereing my services to get some poms together for the Spring dependencies. I found out that Ben Hale, a member of that project is going to be doing this. Can't wait for those to be up. Then I can Maven-ize the petportal app and offer it up.

I have been hacking away at Servicemix, trying to get some things done. First is using a component that comes with Smix for reading from a JMS channel (queue/topic) using JCA to handle all the resources. This component has not worked for me yet. It causes some very wierd behavior when using a durable topic subscriber. The subscriber will not get some of the messages it misses when down. This means that it is not functioning like a durable subscriber. So I gotta fix that issue.... meanwhile I am also working on learning how to create service assemblies for Smix. This allows creating SOAP services in the container, which is what I would like to do eventually. We are running our XFire SOAP services in Tomcat.

petportal Spring 2.0 Portal MVC application

This is how to get the petportal application that was supposed to be released with Spring 2.0.

First check this page:
http://sourceforge.net/cvs/?group_id=73357

That is the sourceforge site for Springframework. You can view the CVS sources that are checked in. There should be a little ditty about anonymous CVS access. I used this from my MacBook terminal to get the petportal application:

cvs -z3 -d:pserver:anonymous@springframework.cvs.sourceforge.net:/cvsroot/springframework co -P spring/samples/petportal

happy coding!

Back again....

I haven't been posting much lately (obviously). My fiance has moved up here to good old Medford, Oregon, and I have been pretty busy between helping her get settled and working. I have been playing around with a bunch of new stuff since my last post.

Ruby: I have been able to get my hands on 'Ruby Cookbook' from O'Reilly. My boss at work bought a copy for our department, bless his heart. If you plan on quickly jumping into some useful Ruby I highly recommend this book. I have already used it for developing a SOAP service client to test a Java XFire service I wrote. With this books aid I wrote the whole thing in five minutes. The benefit is in being able to test my web service with another language, and to verify the correct responses. I feel the 'cookbook' series is one of the best for becoming useful in a new language. They match a seasoned coders ability to look past the basics of a language's grammar and begin actually getting into the api's and such of a language (the nitty gritty ;).

XFire: this is a nifty Java tool for doing webservices, which get war'd up and deployed into a J2EE web container. I am using it to create web services at work now, deploying to a 5.x version of Apache Tomcat. XFire is very quick and easy to work with. I started out using MyEclipse's new built in XFire web project. Of course I wanted to use XFire in a Maven2 type project (for the enormous benefits that Maven2 provides). This is easy!!! The best part about using XFire was that the services I created worked with M$ Dynamix Ax out of the box, which is a huge deal to my bosses. I was using Axis before, but I had to customize WSDL files for each of my services. Ax's SOAP object did not like the WSDL files automatically created by Axis (thus the customizing). Creating and maintaining WSDL files was time consuming and complicated the whole web service creation process for me. XFire will also auto generate WSDL files, which then work with M$ Dynamix Ax. Hurrah!!!

Servicemix (Smix) and ActiveMQ (AMQ): these amazing open source projects have become a major deal at my work. We are currently using both to implement our cockeyed attempt at SOA. We are luckily starting out easy, but I get the feeling I will soon be going full bore into setting up our company with an Enterprise Service Architecture (ESA). My awsome boss bought more books. I don't have all of them here, but I will tell you that "Enterprise Integration Patterns" is the book to read to get your feet wet. It is a theoretical type textbook which ellicits heartwarming memories of college reading. In it you will not find long code samples. You will find in depth explanations of how and why things work in the enterprise integration world.

Portlets: I have once again taken up the burden of Java's Portlet api. It is pretty rad. I am using Liferay which is actually better for portlet development (in my opinion of course) than Pluto. Pluto is supposed to be a container used for portlet development. I think it's got a ways to go, considering the ease of deploying/redeploying portlet .war's in Liferay. Oh and not to mention how nice Liferay looks! I am hoping to write a nifty portlet which will interface with a tool we use for storing information on projects and requests for things. This tool is called QuickBase, and is an online app from Intuit. We have used it to develop a bunch of stuff for tracking projects. It was my dept's first crack at doing anything of the sort and falls a bit short (especially in the ease of use aspects). If I can get it up it will be a very nice proof of concept. Fingers crossed.

Portlet-MVC: Who wants to do plain old portlet development? Not me! I first messed around with the Struts portlet bridge framework, but quickly became bored of it. Struts may be the big kid, but the kid is obviously a beer short. I have been searching around for a better MVC for portlet development. I have found Spring 2.0's Portlet MVC. I am happy, especially since I'm a fan of Spring to begin with. The Spring portlet framework allows you to easily leverage Springs niceties. You can also work in Struts, WebWork, etc. if you really want to....

So thats it. I am trying to check out a portlet app from Spring's site. It is supposed to be packaged with the 2.0 releases of Spring, but I downloaded many different versions and none included the sample app. I'll post later on how I get it (if I can). C'ya.