More messaging fun!

In my last post I talked about using a STOMP client for Ruby. STOMP is a very light weight protocol for doing messaging. I have implemented an ESB (Servicemix) and messaging system (ActiveMQ) for my company. Our business depends on many programming languages, databases, software and such to keep things running. STOMP as I explained before allows the web guys at my work to message our system using Perl, Ruby, Python, C# and more :). This will allow for easy routing of web data through our ESB.

The goal for now is to set up orders to flow through the ESB for our web department. We are using this new piece of software to do all our CRM, product maintenance and other business functions. I have refered to this software as 'Magic' and will continue to here. Orders will need to flow into Magic somehow. Magic is M$ built, and I figured why not put the ActiveMQ NMS implementation to use. NMS is like a .Net version of JMS. The guys over on the ActiveMQ project created an implementation of NMS for ActiveMQ. This is very nice of them. I can now use either a STOMP client or NMS client to message with ActiveMQ from my C# code!

So at first I tried to create .dll libraries of the AMQ NMS implementation, then load those into Magic. Magic has CLR functionality in it's newest release. This means that the language you code with inside Magic can use objects written in C#, VB.net, Java, and on... The problem is the CLR implementation in Magic is lame! It does not support language features like events, delegates, virtual functions etc. So I cannot use the NMS code inside of Magic. Bummer. I mulled for a day about this one while working on other stuff. Then I found an idea.

Magic has a COM connector, which allows you to call pretty much any code written in Magic. This is nice for executing remote procedure calls to Magic. So my new solution is to write Windows services which do the messaging, then pass/pull data from Magic using the COM connector! I have already written my first service. It writes a simple XML doc to a queue on a separate machine every 30 seconds or so.

Now that I have this written I will be focusing on writing the COM connector code to push and pull messages from Magic. Pushing is the easy part as Asynchronous communication is easy to keep. As a message comes in, the service will grab it and push it over to Magic. Pulling data out may be harder. Right now I use a timer object in my service, to execute some functions every 30 seconds. I figure I may have to do this polling type thing to pull data. It sucks, but it is better than the alternative. In the long run it allows finer tuning of how the data will flow across our system.

Right now we are getting data out of Magic using web services. I'm not impressed with web services. The only time people ask me to write them is for improper purposes. When we need to get data out of Magic, we run batch processes which send the data to an external web service. Magic spams out large chunks of data to the web service, causing it to stress badly. But for the most part the web service sits there doing nutin :) It's not a very pretty picture. With messaging we can take the flow and streamline it. Messages flow out as they are created, not at batch time. The message consumer can pull the messages as it likes, rather than having to deal with a large volume being spamed to it. This relieves stress accross the system. very nice. I think my boss is convinced that this is the way we should go. I just need to get the many pieces of this together, so that we can do some testing.