Friday 29 February 2008

New Aussie "Oracle" bloggers

The secretive, sinister, gloomy, doomy and sightly damp Oracle Australian halls maybe opening themselves to a bit of outward inspection ;) Oracle's Barry Matthews, Carl Terrantroy, Paul Ricketts, Saul Cunningham and Steve Williamson have started the GTMi Team Blog - Australia New Zealand (ANZ).

Now to be honest GTMi sounds suspiciously like a marketing **** term, but I know enough about these guys to give them the benefit of the doubt that they know the Aussie and Kiwi crowd will turn their nose to such dribble. Instead I hope they will aim at some very interesting down to earth local content rather than towing the Oracle corporate line (which mainly involves wearing suits and attempting to talk like an American I might add ;).

Good luck to the crew and hopefully we'll see more bloggers from the Aussie Oracle halls soon?

Wednesday 27 February 2008

4 things I read more recently about JavaServer Faces

It's been a while since I published a "4 things I read more recently" post. Here's one on JSF:

Monday 25 February 2008

Web Service Development Practicalities

With a current client we've attempted to capture larger issues and practical development ideas beyond just the technical ABCs on implementing web services. We've published it here to hopefully be useful to the casual reader.

The following contains:

1. A brain dump of issues experienced in the past on web service projects that can blow out project delivery, affect the quality of the technical solution, and just plain frustrate users, developers and project managers alike. As web service projects bring in a 3rd party to supply part of the service, there appears to be an "error multiplier" (much like the military's concept of a "force multiplier") meaning that there is a greater chance of problems occurring.

2. A discussion on small ideas to help reduce the pain and build into your project plan and development beyond just building programs to consume/publish web services.

As this is a brain dump the following is certainly not a definitive guide, and not overly well structured or articulated article. In other words your mileage may vary.

Note that the discussion bends towards development from the Oracle database with mention of transactions, triggers, PL/SQL routines and the like. The discussion is also more considerate of the consumer, not publisher of web services - though both parties can take value in reading the points below to consider the other party's issues in using web services.

If you have anything to add, agree or disagree with any of the following points we'd appreciate in hearing your experience and ideas.

Development Issues

Documentation - 3rd parties publishing web services will often supply documentation on the web services to the new consumer, include descriptions of the WSDL files, their URL locations, the SOAP XML payloads and the business processes that result. New development teams should take care to confirm that the documentation supplied matches the actual services published as this is an early indicator that if they differ, the web service and external organisation are off the rails.

Network Connectivity - if you've ever sat at an organisation where you're frequently yelling at the network administrator that "I can't get to Google" or the XYZ sub-domain isn't accessible, chances are you're sitting on an unreliable network. Such network issues will become exasperating in a web services development project as you try and work out what's gone wrong this time, or are waiting once again for the network to come up.

Server Connectivity - as an extension of network connectivity, server connectivity and stability of the Application Server that publishes the web services is essential. If the web services reside on an App Server, Operating System or hardware box that goes up and down on a daily basis, it's pretty much guaranteed to go down during development.

Test Environment Verification - a good test web service needs to provide facilities beyond that of the production web service for development purposes, such that you can verify your transactions. Without this you may need to manually verify the transactions, or god-forbid actually phone somebody and check the results.

Test Environment Stability - it's common to provide a test web service environment in addition to the production web service servers. However there is a difference between providing a test environment for your own testing and development, and providing a test environment for other parties to test with; these should not be the same server. If you're currently using a published test web service from an organisation that is using that test service to do their own testing and development on, chances are you see that server go up and down, slow to a crawl, time out, change it's functionality, report garbage data, report no data, or the web service APIs will change.

Firewalls - typically at each end a firewall will exist between the consumer and publisher, and the firewalls will need to be configured to let web service traffic through. This may be both IP and Port blocking. As soon as you find out what needs to be configured in the development and test environments, request that the changes be made for the production environment so this isn't a roadblock for your production install. Because of red-tape and security constraints at your organisation and the web service publishers, this may be a long fight.

Solution Considerations and Practical Development

Error Handling - web service transmissions can fail at a number of different points and consideration on how to handle the error conditions needs to be applied. This may require flagging the unsent data and attempting to "resend" at a later time.

After a number of failed attempts for 1 or more messages the system should log the error in such a fashion that a human will be notified (either via email, in Oracle OEM logs etc). Once failed the system should stop attempting to resend and will not automatically restart but instead must be manually restarted. This requires the appropriate procedure for operations staff.

If the publishing server has been down for considerable time such that a number of waiting messages have accumulated, on the external service becoming available consideration should be given to only sending a batch of the waiting messages as to not flood the external server and network (and bring it down again via a DOS attack).

If your system fails after transmitting its payload the transaction needs to be written in such a manner that it doesn't rollback to a point where it thinks the message was not sent and doesn't erroneously send it again. Considered use of PL/SQL autonomous_transactions will take of this.

Soap Failures - the Soap web service protocol has an error reporting "fault" mechanism. Any custom code you deliver needs to be able to detect what is an expected response and what is an unexpected response, how to handle known faults or unknown faults and log them appropriately with as much detail as possible for debugging purposes. Though the Soap protocol defines a number of different soap fault responses and mechanisms to deal with these, as web services may be hand-crafted solutions you may see totally arbitrary error handling capabilities.

Network and Server Latency - the publishing server's response time can be variable dependent on network latency and server load. As such any solution should consider carefully not communicating to the external server within the same transaction that a human is part of; the wait time can become infuriating for the user. Instead write the message to be sent to a separate table/data structure then commit, with a separate independent process periodically searching for new records to be sent.

In particular be mindful of having database table triggers that call the web service routines to send messages. If a user undertakes DML on the table the operation may hang until the web service call is complete.

Test Utilities - as web services can fail at many different areas (connection, timeout, payload errors etc), it's prudent to write simple test programs that help diagnose these issues rather than depending on your final production code under development. Such utilities will help you debug and diagnose the issues without the bloat of your own code. If written properly such tools can be included in production solutions to detect issues as they occur and log the issue or notify the appropriate operational person.

Own Test Web Services
- if the external provider is having issues with providing a consistent test service, consider creating your own test web service based on the WSDL and XML payloads that they have published to keep your development going.

Service Level Agreements - care needs to be given to get the web service publisher to provide SLA on both the test and production environments. In particular uptime, not changing the specifications, not changing the business process, and notifications of system disruptions and potential future changes are essential. If you detect a casual response to this, be wary what this implies, a casual response or non concern of providing you the services.

Verification Utilities - the publisher of the web service could change their custom web service API at any time regardless of SLAs. Programs to detect changing WSDLs as well as fault handling in your programs to detect changing SOAP XML payload structures, along with reporting to a higher authority can initiate discussions about "what have you guys changed now?" rather than wasting time on "why isn't our program working now?"

Monday 18 February 2008

"Oracle's Ellison: SOA migration a slow process" no surprise

Just before Christmas, InfoWorld published an article quoting Larry Ellison on the slow uptake of SOA. Among other things, Ellison said "It takes about 10 to 20 years before [you can] rewrite all of your applications [to take advantage of SOA]" therefore implying the benefits of SOA are a long way off yet.

I'm not surprised Larry has had to go on the defensive here. Admittedly as CEO of Oracle I wouldn't like to announce the huge investment in the SOA Fusion Middleware is seeing poor growth (well, at least the criticism of poor growth).... I'm guessing my shareholders wouldn't want to hear that either.

I'm also not surprised that this sort of criticism and concerns are coming about. From my own experience, while Web Services & SOA/BPEL are great enabling technologies, they're opening a huge can of worms in inter-system and inter-organisational communications that haven't even been really broached yet. Just because the underlying Web Service communications allow you to communicate in a standard fashion, it doesn't mean the systems or organisations you communicate to are following any standards internally. And this is where the problems really start ... and you thought the technology was difficult enough?!

I'll give you an example from my past.

We built a Web Service system to take purchase-orders (PO) from numerous point-of-sale (POS) systems, and distribute them to 3 distinct suppliers to complete. The POS systems were all under our control and the POS interfaces were clearly defined.

With the suppliers it was a complete mess. While each supplier did have a web service interface and we were able to use XSLT to transform our invoices into whatever format they required, what they each did with the invoices was completely different.

One supplier did exactly what you expected. Given a PO, they raised an invoice and an out-of-stock record (if necessary) and sent them back. Easy.

Another supplier took the PO, and returned multiple updates to a single invoice as they discovered stock. Our understanding is this is against Australian laws, where an invoice is meant to be a binding document. But never mind.

The final supplier took the PO, sent an invoice record for what they could find, and an out-of-stock record for what they didn't have. However over time the PO would roll out to other warehouses where they could submit an invoice for the remaining part, or then an out-of-stock for what they didn't have. And so on. Groan.

Now the SOA advocate will yell well that's great! SOA will take care of this mess, as you can build different solutions for each supplier. True, but each supplier wasn't guaranteed to maintain these modes of operation. In fact behind the scenes they had manual processes sometimes that dynamically changed the result. And in time we wanted to add over 50 suppliers, all with "interesting" legacy system based processes. It was very conceivable we'd just need full time person to keep track of what mess 1 of those 50 suppliers had got us into this week, changing interfaces, changing processes, making mistakes and so on.

In addition only 1 of the 3 suppliers actually had test systems we could build our application against. And for the supplier that did have a test system we literally had to phone them to ask them to flush the data each time we did a test. So much for quick agile development.

You could argue they should just standardise their systems!? But that's a pretty immature attitude. Most organisations have trouble establishing standards internally, let alone have enough clout and smarts to boss around external organisations; not many of us are a Walmart. In turn those external organisations may have big $$$$s invested in their IT systems, and won't or can't or just change over night. Maybe that's years from now.

So then you say, maybe there should be industry standards! Well there are. We stumbled across some standards, but none were being used by our suppliers, and again they weren't interesting in changing.

....and so on....

So like Larry, I truly believe we're years away from the SOA revolution, and it's something that will seep in an evolutionary sense, no big revolution here I'm afraid. Coincidentally like Larry I predict the real benefits are 10 to 20 years away too, and I'm sure the technology will move on so it doesn't look like what we think it will now.

I have a further problem with Oracle here, or the way they're selling this. Oracle is marketing a technology solution that will solve all of your problems, when integrating systems internally and externally. But in reality you've only just started on this highway to hell, and you're going to have a hell of time working with your organisational peers in enabling SOA for all the reasons mentioned above and more. Oh yes SOA advocates will argue this is solvable, but in my words "good luck, you're going to need it." Several generations of your IT staff will come and go I'm sure before you see huge success.

I'm also a bit suspect of Oracle, and in particular Oracle Australia for focusing so heavily on the SOA arena recently when it's overkill (and overpriced) for most small to medium organisations (which Australia mostly contains). It's another case like the database where Oracle's solutions seem to be pricing themselves out of the market, and creating solutions very few people use (relatively to the entire IT market).

Don't get me wrong, I see a need for SOA technologies, or more precisely the goals of system integration, and I wouldn't bet against Oracle's solution being a technically savvy one, I just don't see SOA as the immediate White Knight to *all* IT's woes.

What are your thoughts on the future of SOA and all things Web Service oriented?

Thursday 14 February 2008

Grumpy Old Bloggers

As an unashamed rip off of Grumpy Old Men, following are the top 10 signs you're becoming a Grumpy Old Blogger when you:
  1. Feel like calling everyone who leaves a comment on your blog an idiot.
  2. Diss bloggers starting off for their immature and uninformed points of view.
  3. B1tch about dropping AdWords revenue or its changing policies.
  4. Whinge that people continuously don't appreciate your point of view.
  5. Complain about people are not using your blog like you want them to.
  6. Whine about needing to change the blog look and feel yet again.
  7. Track people down on their blog and leave comments that they're idiots.
  8. Fail to believe and support people in having different points of view to you.
  9. Think blogging has gone rapidly down hill since the good ol' days (2 years ago).
  10. Expunge anybody with any hint that they've joined your exclusive brethren.
I must admit I'm definitely guilty of some of these and I've only been blogging for a year. Time to sit back, chill out and restock on life's priorities I guess.

Sunday 10 February 2008

JDev 11g Declarative Components meet Designer Audit Columns

A client has asked me to demonstrate JDeveloper 11g TP3 Declarative Components and how they could be used to show the created_by, created_datetime, updated_by, updated_datetime Oracle Designer audit columns in a standard fashion.

You can download the demonstration application workspace I wrote from here.

To create this I pretty much followed section 17.6 Declarative Components from the draft Oracle Fusion Middleware Web User Interface Developer's Guide for Oracle Application Development Framework 11g Release 1 from Oracle. At this time I don't think there is anything useful I can document in this post beyond the guide besides some instructions on the demonstration below.

To install unzip the application workspace and open it in JDeveloper. Under the Schema project the createSchema.sql creates the necessary database objects for the demonstration. The application depends on a connection SageConn that you can change to connect to your favourite database account.

On running the ViewController editTable1.jspx or editTable2.jspx web pages, you'll note that clicking on the Show Audit button will display a standardised popup dialog for viewing the audit columns, all provided by a declarative component. You'll also note how the usage of the component within the web pages effectively inject the values from #{bindings.createdBy.inputValue} ... createdDatetime ... updatedBy ... updatedDatetime into the declarative component to show. This technique pops up in JSF Page Templates too and is very useful.

The declarative component was created in the SageAuditDecComponents project, exported as an ADF Library JAR, and added to the ViewController as a separate JAR file via the project properties. The declarative component then becomes available to drag and drop from the component palette into our consuming web pages.

Monday 4 February 2008

Advert: March Oracle Application Express (APEX) course in Perth

SAGE Computing Services and Penny Cookson will be running an Oracle Application Express (APEX) workshop in Perth Australia March 5th-7th.

As you may already know Penny is recognized as one of the best Oracle educators winning Oracle Educator of the Year by Oracle Magazine in 2004. Penny is also well known in the Australian Oracle community, awarded AUSOUG lifetime membership in 2003, and has won numerous conference best paper prizes. And to get the disclaimer out of the way, Penny is my boss ;)

If you're interested in all things Apex and want to pick up some down to earth training, check out the events page on our website which has more details including the agenda, location and price. We hope to see you there.

Sunday 3 February 2008

New Oracle Community groups

OracleCommunity.Net is up and kicking, and I'd like to advertise the fact that I've created the following groups that you may join:
.... + many others by other great Oracle enthusiasts. Please join and participate in some great Oracle discussions!