Wednesday, November 29, 2006




Building an Enterprise Scale Mass Market AJAX Web Site using echo2, EJB3 and ServiceMix ESB
By Matthew Brooks, Technical Architect, RDF Group
mbrooks@rdfgroup.com

Introduction

I work for RDF Group as Technical Architect and in February of 2006 it was our good fortune to meet mform, a start up company aiming to launch an innovative online mortgage search and application service. RDF Group delivered an enterprise scale public facing web based application to mform which went live on the 20th November at www.mform.co.uk. This is the story of the team’s journey through some of the most cutting edge technologies in the Java space to deliver a fully web 2.0 style application backed up by a Service Oriented Architecture.

The Brief

Attending Edinburgh with my colleagues from RDF it became apparent that this fledgling company was intending to provide a unique mortgage search service on line. We felt that by adopting some of the cutting edge technologies we could provide a significant USP for the client.

Simply put the business requirement was to provide:

  • A compelling user interface to allow people to enter their mortgage details and search the whole of market to find the products which are right for them.
  • For users to select a mortgage product and provide as close an integration as is possible with mortgage providers to allow users to apply for their chosen product with the minimum of effort

The Technology

The technology stack adopted was intended to be cutting edge, whilst not compromising delivery by introducing too much risk.

The vision was to have:

  • A fully Ajax based UI, that moved away completely from the old http request/response cycle and only used the asynchronous XMLHttpRequest
  • To provide scalability using an EJB3 based farm of application servers
  • To minimise the pain of introducing new integration points with different lenders by utilising a Service Oriented Architecture.

Happily a number of key technologies had come close to maturity in all these areas and we adopted the following technology stack

The Process

Whilst this report is intended to cover the use of these technologies in delivering a fully fledged application, it is worth a brief mention of the work process we have at RDF.

Our process is based around RUP and we have adopted some of the best tools from the agile movement to help facilitate this (unit testing, uniform build management, continuous integration etc).

Whilst many software houses talk about using RUP – they tend to fall into two camps:

  • Those that pay lip service to the process – but don’t actually implement it
  • Those that try to follow the process to the nth degree and as a result fail to make progress with kind of speed that customers require.

By contrast RDF uses a strong process that is based around delivering operational slices of functionality to demonstrate progress and to test the requirements.

On the project:

  • Customer requirements were fully documented using use cases and UML model
  • Design was undertaken based on the analysis
  • Development was only undertaken on the basis of the design

As a result we were able to deliver an innovative product with fully traceable documentation and design artefacts that mean we can easily support it going forward.

User Interfaces and Echo 2

In the spring of 2006 the most extensive and well rounded Ajaxian type web framework in existence was, to my eyes, echo 2.

We implemented a great prototype of the UI and were confident of echo 2’s ability to deliver.

One of the key things about the echo 2 framework is the concept of an application.

There is one instance of an application per session and what the user sees in the browser is generated by constructing (or changing the state of) a widget lattice that is stored in the application instance.

Here is a high level sequence diagram showing the interactions between the browser and the echo 2 application


Fig 1: A sequence diagram showing the interactions between the browser and the echo 2 application

Echo 2 and CMS

One of the customer requirements was to be able to update the site content using a content management system.

We selected the Dutch based Hippo CMS system and built integration between echo 2 and Hippo CMS. Briefly we built a component that could display XHTML fragments (we could have used the one from echopointNG but in the end built our own) and the data to be displayed is pulled from Hippo using DASL.

Echo 2 the challenges

Whilst using echo 2 we discovered that whilst it was the most advanced tool for the job (at least when we started, which was before GWT came out) we did find that we had to undertake the following:

  • Adjust some of the java script in widget peers where it was not quite performing as we expected
  • Subclass the echo 2 servlet to ensure that:
    • We can trap non java script type clients and present a “non java script” type version of the page
    • We can present a more polished start up page rather than the ||| presented as default by echo 2
  • Some post back functionality does not work well with IE either under load or restricted bandwidth. Due to the way that IE polls for the post back other events on the browser were being missed.
  • Develop our own widgets where necessary if there was no suitable one available from echo or echopointNG

Echo 2 a summary

Overall echo 2 provided a fantastic framework for delivering a fully Ajaxian web site.

Its swing style approach to UI’s provided a convenient and familiar development environment.

It provides good encapsulation of JavaScript which in general works well over multiple browser versions.

Echo 2 could well do with some standard approach to layout management, however, and this has caused us some work during development.

Here are some screen shots of the finished application


Fig 2: The polished start up screen



Fig 3: The first application screen



Fig 4: An example use of an accordion pane and a modal dialogue


EJB3 and JBoss

We were keen to use EJB3 and Java 5 and JBoss 4 gave us the opportunity to use this new technology.

We broke down the layers in the EJB space in a manner that should be familiar to most people working in the JEE space:

  • A services façade layer: Consisting of EJB3 Stateless Session Beans providing coarse grain services used by the echo 2 presentation layer and transactional demarcation. All calls from the presentation tier go through this layer.
  • An Application Services Layer containing
    • Business components which encapsulate the fine grained business rules needed by the environment.
    • Domain Object Managers encapsulate complex rules and relationships between a set of related domain objects.
  • A Domain Layer containing Domain Objects mapped to the database using Hibernate.

How Spring Fits In

The only EJB’s involved in the application are in the service façade layer.
All the other layers are implemented using POJO’s wired together using Spring.

Hibernate freezes out EJB3 Entity Management

We used EJB3.0 annotations backed up by the Hibernate persistence mechanism. This bought the stability of the mature hibernate product along with some of the extra functionality that Hibernate provides.

Architectural Complexity and the new technologies

Some people could take the view that a multi layer approach as outlined above is no longer necessary in an enterprise scale app.

They ask:

  • Why have Session Beans at all when you could use Spring remoting?
  • Why have an Application Services Layer? Doesn’t this encourage the anaemic data model anti-pattern?
  • Why have Domain Object managers when you can just make Hibernate calls on a lattice of Domain Objects directly?

Session Beans and Spring remoting

This is arguably more of a style issue than anything else.

We went down the Session Bean approach for the following reasons:

  • Use of Session Beans allows for easy integration between JBoss-provided transaction management and the transactional boundaries we wanted for the system
  • Use of Session Beans makes it easy to tune particular services by adjusting the maximum number of beans in the pool for that service.
  • We were more familiar with Session Bean remoting than Spring remoting

Application Services and Anaemic Data Model.

There is an anti pattern being suggested at the moment which is the Anaemic Data Model.

This suggests that having “data carrying” objects and “data processing” components leads to a “functional” software development rather than proper object oriented development.

On the other hand, others would say that, ill used, a rich data model could lead to object bloat.

As ever we are in the architectural position of having to balance opposing forces. The solution we came up with was this.

  1. Objects may have methods that contain behaviour.
  2. These methods may only act on the objects own instance variables.
  3. When these methods wish to act on instance variables of other objects they must delegate their behaviour to methods on that objects
  4. That these methods should only ever be called (on a delegation basis) by Application Service managers.

This helps promote good encapsulation (points 1-3) and point 4 allows for easy post production code maintenance. If you know that business processes are kicked off by Application Service Managers (even if all they do is call a method on a Domain Object) then you always know where to look for the business behaviour of the system.

Domain Object Managers and Hibernate

In some ways it is arguable that, with a sophisticated object relational mapping tool such as Hibernate one can dispense with Domain Object Managers completely. After all, the job of a Domain Object Manager used to be to manage invariants between domain objects in a related logical area, a function now delivered admirably by Hibernate.

The argument for the continuing usage of Domain Object Managers is about decoupling.

Ideally I’d rather not have the Application Service Managers know anything about the persistence mechanism.

I would like to ensure that any calls to the Hibernate framework take place in one area of the code, the Domain Object Managers.

Also the mform application lends itself to very distinct, non-overlapping data domains.

This led us to use the Domain Object Manager approach – although it may well be that the decoupling of the Object Relational mapping services can, in future, be provided by a single infrastructural type class.

The mform application and the ESB

Part of mform’s key advantage is to be able to integrate with many lenders

We realised, however, that the costs of this integration could escalate and were keen that they were kept to a minimum. This is because each lender has different transport requirements. And each lender wants similar data delivered in different formats.

  • Some want XML
  • Some want a CSV file

and so on.

We were keen not to build all these integrations on a case by case basis, wanting to insulate, as far as possible, the central mform application from the services provided by lending institutions. An ESB is a key technology for delivering this.

We adopted ServiceMix ESB due to its comparative ease of configuration and its ability to be deployed within JBoss.

We defined a Web Service end point to be called by the core mform application. The Web Service definition contained as large a set of data as we could devise along with the end point for the message.

The configuration of ServiceMix was then set up to

  • Inspect the incoming XML and decide which end point it should be forwarded to.
  • Transform the XML into the format required by the Lender
  • Transport the data to the Lender.

Every time we need to send mortgage applications to a lender we invoke this web service and let ServiceMix reformat and send the details to the lender.

This will minimise the disruption of adding additional lenders – changes to the central application will only need to be made if there is some crucial data that we are not passing to the ESB that is required by a specific lender


Conclusion

RDF has been able to deliver a fully featured Web2.0 style enterprise application for mform rapidly. The technology stack of echo 2, JBoss, EJB 3 and ServiceMix ESB provides a powerful tool for providing scalable and reliable web based application with rich UI experience that users are coming to expect.




Labels: , , , , , , ,

18 Comments:

Blogger Federico said...

Hello Matt,
I'm Federico from the echo2 forums.
I wonder if you could lend me a fullscreen of the sequence diagram slide as I'm going to have a talk about echo at the local JUG.
Anyway, nice post: will forward it to some coworkers

12:49 PM  
Anonymous Rob Wilson said...

I would also like to see the sequence diagram ;-)

Very interesting blogg and it's nice to see a well written breakdown of all the technologies used to add confidence to anyone else trying to use them!

7:53 AM  
Blogger matt brooks said...

Hi Rob, Frederico,

I will get together a better version of the sequence diagram - I'm a bit rushed at the moment so it will probably be next week!

Cheers

Matt

8:30 AM  
Anonymous Anonymous said...

Matt,

Thanks for an interesting post. I'd be very interested to see a follow-up on performance and scalability issues, any volume testing you did, etc. While the technologies you used are cool, many peorple have found that several (particularly hibernate) add a lot of overhead that can be detrimental to performance an scalability when dealing with large volumes of data. Is this a concern for you, and if so how did you address it?

Cheers,

Ken, NYC

9:11 AM  
Anonymous Anonymous said...

If you could, can you please post the architectural diagram for this enterprise application? thanks

11:26 AM  
Anonymous Anonymous said...

Hi Matt,

Good blog. I would also like to see more about the scalability issue regarding ajax and clustering.

It would be better if you use ZK for the UI tier - I am a fan of it;)

Cheers,
Bob, Dubai

7:30 PM  
Anonymous Anonymous said...

Thanks for your description of the system. Could you please confirm that you used JPAI for persistence (with the Hibernate implemenation) or the Hibernate API? It wasn't clear to me - you did say you used EJB3 for Session Beans.

Thanks,
Ashley.

Ashley Aitken
mrhatken at mac dot com
Perth, Western Australia.

3:50 PM  
Anonymous Arjé Cahn said...

Great work, Matt

Good to see that you've pushed forward and have been able to tie it all together so neatly. And the fact that you've been able to integrate with Hippo CMS so easily makes me feel pretty damn good, too :)

Regards,
Arjé

3:08 AM  
Anonymous Anonymous said...

Thanks for the excellent write up. Can you elaborate on the performance characteristics of this system.

As a long time Java programmer/architect one of my biggest gripes is that our systems get bloated very quickly and that performance has been sacrificed for architectural elegance.

In the end, the customer doesn't care whether the system was written using language-X or not, or whether the design patterns have been implemented to the tee.

I would really love to hear your thoughts on the performance of this system -- what sorts of response times do you get and how much load is the system able to handle.

1:44 PM  
Anonymous Anonymous said...

don't mean to be a downer, but the main page doesn't align properly in firefox 1.5.0.8

It gets even worst as I trialed through searching through a mortgage.

just thought you might want to know -- if it were my app, i'd want to know / get testers input from the public too...

6:54 AM  
Blogger matt brooks said...

Hi Anonymous 6:54.

Thanks for this.

We have done cross browser testing - and specifically with firefox 1.5.0.8.

In fact I just went back and double checked for the errors that you have seen and it looks absolutely fine here.

What OS are you running on I wonder (my test was ff 1.5.0.8 on XP)?

I wonder if there are any particular third party firefox plugins that can cause this?

9:51 AM  
Blogger Benoit said...

Very interesting article. Thanks for sharing your experience !

Regards.

3:08 AM  
Anonymous Anonymous said...

Hi Matt,

You mention the Anaemic domain model anti pattern, presumably this means that in your domain layer you have business objects that encapsulate both state and responsibility.

Now lets say you need to make use of an infrastructure component method (e.g. a nextWorkingDay() method on a Calendar infrastructure component) from within one of the methods on one of your business domain objects

Would your domain layer business object be able to call this nextWorkingDay() method directly from the domain layer inside your component?

Hope this makes sense.....

Regards
Anon

5:22 AM  
Blogger matt brooks said...

My view of infrastructural services is that they should be stateless and available to all layers of the application (logging is a good example).
So yes a domain layer business object would be able to call nextWorkingDay().

Cheers

Matt

5:27 AM  
Anonymous Anonymous said...

I'm using ZK too after having used GWT and after looking at Echo2. There are all very good AJAX framework but I found ZK more productive: you designate your UI in XML (XUL) and not with a Swing approach, the number of components has no comparison and it is really stable. Communications between the navigator and the web server is transparent.

Really easy to learn.

Roger

11:22 AM  
Anonymous web design company,web designer,web design India,website design,web design said...

Thanks u r information

4:13 AM  
Anonymous webdesign said...

nice post mate.....


web design

5:19 AM  
Blogger somaie said...

The article was worth reading. Thank you.

www.onlineuniversalwork.com

2:35 AM  

Post a Comment

<< Home