I am very happy to welcome Dave Shaffer into our little family of bloggers. Until recently, Dave has run Product Management for the Integration products at Oracle, i.e. SOA Suite, BPM Suite, etc., and has now established his own consultancy to provide strategic and practical advice to organizations using these products. Dave has an absolute wealth of knowledge and experience, and we are very happy to enjoy his collaboration on a series of upcoming posts on SOA architecture and best practices. I wanted to ask Dave some questions about his future, his journey to this point, and his thoughts on SOA and BPM. I am sure a lot of our readers who know Dave would also be interested to hear his views, so we are publishing our ‘interview’ here. As Dave reiterates below, we welcome you to join in the conversation, through commenting on the post for example, or by getting in touch using the details provided below.
Mark: As you have been closely connected with the Oracle SOA and BPM space, I’m sure a lot of people are curious what you are doing now that you have left Oracle. Can you share a quick update?
Dave: Sure, first let me provide just a little background for those readers who may not know me well. I came into Oracle in 2004 when Oracle acquired Collaxa, which was where the BPEL Process Manager came from. In the intervening time at Oracle, I led product management for the integration products, including SOA Suite, BPM Suite and the Governance tools. Over the past year or so, we also merged the SOA team with the product mgmt team for UPK (User Productivity Kit), a tool for creating user adoption content for both custom apps and packaged app implementations, came under my area of responsibility.
It has been great to participate in the incredible growth and success for the Oracle SOA and BPM products that we have seen over the past 7 years, but I decided in 2011 to head out on my own. The main driver was to take some time off and travel with my family (my wife and I have two kids, ages 7 and 9) but also by my nature I am drawn to change and the time seemed right to do some consulting and explore some ideas that I have.
To that end, I set up Middleworks as a small company affiliated with other services companies in the Oracle eco-system to help customers be more successful with middleware. Services that Middleworks offers now include strategy consulting for customers and partners in the Oracle eco-system and helping customers execute on their implementation plans and goals. In particular, I think that many customers could use some strategic advice, guidance and coaching from someone like myself who has a lot of knowledge of the Oracle product portfolio and organization but is independent of Oracle itself. Over time, we will see whether this develops into a short-term lifestyle change for me or a more significant entity.
Mark: OK, great. Things have certainly changed a lot since Oracle acquired Collaxa. What was it like in the early days and how has BPEL and SOA matured since then?
Dave: I think the last 10 years have been super interesting for BPM and SOA. Collaxa was founded near the end of 2000 in the midst of the dot-com bust. While it was originally intended to bring long-running and parallel processes and asynchronous service support to languages like Java, the emergence of the BPEL standard in early 2002 provided a unique opportunity to evolve the general purpose Collaxa process engine into the first BPEL execution engine. At that time, it wasn’t obvious whether BPEL was going to supplant the many other process execution languages, like XPDL, BPML, and dozens of others, which made things exciting for a small startup. Collaxa was founded by Edwin Khodabakchian and Albert Tam and though it never grew larger than 12 employees, it became the de facto standard for BPEL execution and was widely adopted in the developer community. One of the most interesting things for such a small company that was trying to land large enterprise customers was that we needed to make Collaxa feel bigger than it was. For example, we made it easy for anyone to get technical support for downloading, installing or using the BPEL engine. Everyone handled these inquiries, including Edwin, a founder and CEO of Collaxa, and myself (at that time I was responsible for all the developer use of our tools, docs and training, etc). Since a company that wants to feel larger doesn’t want their CEO answering first line technical support, and also because we wanted to give a personal feel to our support experience, we created a support persona (“Mark Ghodsian”) which Edwin and I would both respond to support threads as. Given that Edwin would work nearly all night and I would get to work fairly early in the morning, “Mark” was available nearly 24×7 and become incredibly beloved to many customers. I remember showing up at several customers who had a badge printed for Mark and were very disappointed that he wasn’t able to make it to visit them – and even more disappointed to find out that he didn’t exist at all…
Even with the level of support that Mark Ghodsian was able to provide, enterprise customers were hard to come by at that time and the real value of such an engine was much greater to a large enterprise so partnership discussions with Oracle turned into an acquisition that closed in mid-2004. An interesting observation in hindsight is that the other large Collaxa partners at that time included Siebel and Sun, both of whom are now part of Oracle anyway.
One of the other things I recall most clearly from the “early days” was the competition between models such as BPEL that were very developer friendly and those which were more business-friendly, often including BPMN design-time support and execution based on XPDL or something similar. While Business Process Management (BPM) was not new even then, there were questions of how much of the business process modeling, deployment and execution could be done by business alone and how much required IT involvement. At that time, there were many BPM pure-plays who offered the promise of enabling business to own their processes, though the dream of business being able to go it alone without IT was never truly realized. However, pitching that business and IT needed to collaborate was more difficult than selling the dream. We often described the alternative approach as “the magic pill,” comparing it with weight loss schemes. While the tried and true approach to weight loss involves improvements to diet and increased exercise, people are always looking for the “magic pill” that doesn’t require any effort to achieve weight loss. It turns out to be much easier to sell a weight-loss pill which doesn’t work than a diet+exercise program which actually does. So what we found was that we had to tone down the “diet and exercise” style message of BPM to be successful in the market. Personally, I still think this is the case (that people don’t always want to hear the unvarnished truth) but at the same time, I believe that the emergence of standards like BPMN 2.0 and the ability to have a single engine that executes both BPEL and BPMN, such as Oracle has done and other vendors say they are doing, makes it easier for IT and Business to collaborate and maybe lowers the pain and effort required to achieve the end goal – in this case, better business processes, which while not as personally satisfying as weight loss, are just as important to the health of a business.
Mark: A lot of technologies have a bit of a learning curve and sometimes knowing how to use a technology and how to use it well are two different things. Can you tell us, how important is it for new adopters of SOA Suite to spend some time learning best practices for things like application and environment design, and how to tune for best performance?
Dave: This is certainly something that I have seen repeatedly over the years with initial implementations of any new technology or standards. There are always lots of different ways to apply a technology and they don’t come with “contraindications” as medicine does. Come to think of it, if they did, that might be a good thing. For example, I remember seeing people using BPEL in ways that it shouldn’t have, for example for simple looping constructs like iterating over a large data set or as a scheduler to run indefinitely, say starting a task every night. The issue is that there is nothing in the language or product docs that says you shouldn’t do these things with it, but there are side effects which make it a problematic way to approach these requirements. For example, in the cases above, the audit trail will grow infinitely large and processes become hard to manage when they run for months or years. And of course changing the architecture or implementation approach late in an implementation is very hard to do and costly. Starting down an implementation path without being highly familiar with all the short-cuts, dead-ends and potholes along the way is a recipe for a failed or late project.
Performance tuning is indeed another similar area where it is hard to be prescriptive – the best approach for optimizing performance depends on many factors and often requires subtle knowledge about the product stack and its internals. So clearly this kind of knowledge is super important and really, the only way to learn these best practices is by experience.
However, the good news is that while the only way to learn these lessons is from hands-on experience, it doesn’t have to be your personal experience. This is where sites like yours, Mark, come in as well as content from other A-team members, partners and customers. In my time at Oracle, I found that the engineering and documentation team was always best at describing what something does and how. But it is much more difficult to release with the product the subtle best practices and anti-patterns that emerge from real-world implementations. So with my hat on as a former Oracle engineering product manager, I would like to thank you for the service that you provide to the community for taking the time to share these hidden secrets. And to encourage the community to also invest that extra time to share tidbits as they are uncovered. Finally, if there are things that the community thinks that could be done better, either to generate this information or share and index it better, please let us know and we’ll discuss it more widely.
Mark: Now that BPEL has matured, what about BPMN? Is it a competitor to BPEL and, if not, how do people decide when to use BPEL and when to use BPMN?
Dave: This is for sure an interesting question that has come up a lot especially since, as mentioned above, Oracle has now provided pretty much the same basic capabilities for the BPMN engine released with BPM Suite 11g, as are available in the BPEL engine. Historically, people would say that BPMN and related workflow languages, were good for human-centric processes and BPEL was good for integration or system-centric processes. However I think this was just a historical artifact of how the companies and products around these standards evolved. For example, BPM pure-play style engines tended to be strong at the modeling and human workflow capabilities, but less scalable, less flexible and have fewer integration capabilities like adapters and data transformation built in. On the other hand the Integration / SOA engines tended to address these weaknesses well, but have fewer capabilities around business user modeling and human workflow requirements. But in reality, the underlying standards of BPMN and BPEL in this case, didn’t impose these limitations for the most part. So now that Oracle has a single engine underneath both BPEL and BPMN and shares the same integration capabilities and scalability across them, this question becomes much more relevant.
And while even though the Oracle platform is common for both BPEL and BPMN, even within that world, BPEL and BPMN become “competitors” in the sense that customers should make a decision for a given process about whether to implement it in BPEL or in BPMN. So given that I have ruled out the legacy notion that BPEL is for system-centric and BPMN for human-centric processes, I would say that the more important question is who will be modeling and looking at the processes. BPEL as a language is very developer friendly – it looks very much like Java structurally, being a block structured, flow-based language, having try-catch style exception handling and many of the same kind of coding and looping constructs as Java. But BPEL is just going to be gobbled-gook to a business analyst or business person. BPMN, on the other hand, is a much more business friendly language. Though even here, I would suggest that true business users are not going to model with BPMN very often, even they could understand a BPMN process when looking at it – something that is not true for BPEL processes.
So, the most important question is whether a process is going to be viewed, modeled, edited, etc by business people or analysts. These kinds of processes should be modeled and implemented in BPMN. However, where a process is modeled and implemented by a developer, BPEL is a great choice. And of course, the strength of the Oracle BPM platform is that these two can be mixed and matched, for example with developers implementing some low level building blocks as BPEL processes, which can then be leveraged in higher level business processes as reusable components.
Finally, there are a few practical matters to take add to the discussion above. These include the fact that BPMN is a less structured language than BPEL and is therefore better for less structured or ad-hoc processes which can be difficult to implement in BPEL. Also, while the engine underneath is the same, the Oracle BPMN tools are newer than the BPEL tools and so there will be a few features or capabilities that have not yet materialized in BPMN, but are available in BPEL. So this is yet another area where a little architectural guidance and some real-world experience can make a big difference in making projects more successful.
Mark: Based on all the customers and projects you have seen during the past 10 years, do you have any tips for new customers or those just starting on SOA initiatives?
Dave: Sure, I have certainly seen a large number of projects that were highly successful and exceeded goals and expectations and many that did not. I think one big difference between the two really does come down to this area of getting the right expertise, training and support at the right time. For example, there are some small Integration/SOA/BPM partners of Oracle who are very focused on the Oracle Integration products where you really know exactly what you will get if you engage them. There are also very effective medium and large partners, though the particular personnel who are put on a given project will vary more widely. In this respect, the Oracle Specialized program can be very useful, as it both certifies a partner as a whole to be “specialized” in a particular product, but also includes a certification test for individuals. And while this test is not a certain litmus test for whether you are getting just the right consultant for your particular needs, it certainly can be a key piece of the puzzle to ascertain whether a particular consultant being proposed for a project has knowledge of the products.
Beyond leveraging partners, having the right relationship with the Oracle field and product management team can be useful as well. Also, there is a great deal of training and documentation out there, so creating the right training plan for your organization can make a big difference.
Finally, looking less at skills and more at architecture and the product side, once thing that I have discovered in my many years working for software vendors is that customers can be unexpectedly rigid in the options they are willing to consider. In some cases, a successful project seems not to be the primary goal or the architects/developers have preconceived notions of the best way to approach a problem, and may not be open to advice from the vendor on the best approach. There is a fine line here, of course, since the vendor may have a vested interest some approaches may use more of their technology and others may use less. Clearly the vendor should not be the sole advisor in such decisions. But once the products have been purchased and particular requirements are identified, asking the vendor the best approach or architecture for the particular products and versions in play can make a big difference especially for the more complex problems. Not only is this a good way to avoid product specific issues that may make particular approaches difficult, it also increases the skin in the game for the vendor to ensure success. As a vendor, if you ask me how to approach a problem and you follow the path I suggest, I really have to make a super-human effort to support you through any problems you encounter. On the other hand, if you forsake my advice, you may be more on your own.
These are just some thoughts I have on this matter, but if readers of this post are interested in my recommendations for partners or would like my assistance in helping with a staffing or training plan, I would be happy to discuss.
Mark: Great, thanks. And finally, if people do indeed want to get in touch with you outside of Oracle, how can they do it?
Dave: Sure. Besides any business activities I have, I really hope to stay in touch with as many of the Oracle Integration partners and customers as possible. People can always email me at email@example.com, but also I welcome any LinkedIn connections and I can be followed on Twitter at @middleworks. Drop me an email if you want me to keep you updated on Middleworks’ activities or for any other reason…
Thanks, Mark, for the great opportunity to collaborate with you!