SOA Development and Delivery

Let’s start by talking about what I mean by that.  I don’t think that ‘SDLC’ is  the right term to describe the space that I want to talk about, but it is a term that people are familiar with, so I think it makes a reasonable place to start.  There are a lot of other terms that cover this space: governance, SDLC, development, version control, continuous integration, binary management, continuous inspection, continuous delivery, automation, virtualization, configuration management, devops, integration testing, and so on.

Also, by ‘SOA’ I really mean the broader set of technologies that we use to build ‘SOA Applications’ – whatever they are, and they are more than just composites.  Real applications have many components, e.g. user interface, integration, mediation, rules, code, processes, forms, service definitions, canonical data and service models, and so on.

I don’t know about you, but I can’t think up a nice concise way to say that I mean all of those things across all of those technologies, so I am just going to call it ‘SOA Development and Delivery’ and I hope you will know what I mean.

The State of the Art

Recently, I conducted a survey, and I have spoken individually with many folks from all around the world, about how they do SOA Development and Delivery today, and how they hope to do it in the future.  I think it would be fair to say that most SOA development today is not yet ‘industrialized’ – by that I mean that it is done by people of varying skill levels, working on a best effort basis, doing what they think is the right thing to do, and doing it the way they think is right.

By ‘industrialization’ I mean the development of standards, methods, processes, tools, and best practices, and the widespread automation of common tasks.  Doing the right thing, and doing it the right way.

I think that we stand of the verge of a point of inflection in our industry where we will see SOA development and delivery become industrialized.

My feeling is that today, if I can generalize, that in terms of SOA development, people:

  • have adopted some kind of iterative, or possibly agile, development approach,
  • generally make some basic use of a version control system to manage source artifacts,
  • sometimes write unit tests, but don’t always check coverage or information content of those tests,
  • often try to automate the build process on a project by project basis,
  • generally do not try to apply any kind of quality control or inspection of their code base,
  • are maybe playing with or thinking about trying continuous integration,
  • have not recognized the need for binary management, or have, but gave up because it is too hard with the current state of tools,
  • generally try to use configuration plans (or equivalent), but often build binaries that contain topological information that is specific to a particular environment,
  • maybe have a governance process that is sometimes updated and sometimes followed,
  • don’t have a good way to manage dependencies between components,
  • use virtualization in their development and test environments, though perhaps not in concert with any type of automation,
  • don’t have a good way to manage test and development data sets,
  • have not adopted devops practices, and
  • do not have a repeatable way to execute tests and deploy the application to different environments.

The effect of all of this is that:

  • it is hard to estimate and plan SOA development projects,
  • projects often overrun their constraints,
  • often the delivered application is of poor or uncertain quality, and
  • build and deployment processes are very manual, time consuming, and error prone.

Pain all around – architects and project managers planning, developers and build engineers building, operations running, and of course the poor business customer who has to use the thing!  Projects are delivered late, over budget, and erode the confidence of the business in the whole SOA promise and approach.  So naturally, I asked myself, why is this the case?  Well I think there are a number of contributing factors here:

  • the tools have not matured to a point that enables a lot of these things in a simple way,
  • there really aren’t any standards or reference architectures to follow,
  • some people don’t think of SOA development as ‘real’ development, and therefore don’t think these kind of practices apply to SOA development,
  • some of these are still maturing in the wild.

Conversely, it does not take a lot of time on Google to find a whole bunch of people who are interested in these areas and are clearly experimenting, prototyping, and sharing their experiences.  And talking with folks who are actively engaged in SOA development, the clear, consistent message I hear is that all of this stuff is ‘really important stuff’ and we, as an industry, need it.  It is also not hard to find consulting firms around the world that are developing capabilities and offerings in this space too.

So where do we go now?

Over a series of posts, I plan to share some thoughts and experiences on a range of topics, taken from those above, and also to share some practical examples of how to apply some of the state of the art tools and techniques from Java development to SOA development.  In particular, specifically in the context of SOA/MDS/BPM/OSB/ADF, I want to talk about:

  • version control,
  • dependency management,
  • build automation,
  • continuous integration,
  • continuous inspection,
  • continuous delivery,
  • unit testing,
  • integration testing,
  • binary management,
  • environment provisioning,
  • configuration management,
  • customization (of binary artifacts to particular environments),
  • the ‘correct’ use of MDS, and
  • the relationship of SOA development and delivery to governance.

And I am planning to talk about tools like Subversion/git/gitlab, Maven, Hudson, Archiva/Artifactory, Sonar, Chef and many others.

I hope you join me on this journey, and please let me know your thoughts.

Read next post in this series.

About Mark Nelson

Mark Nelson is a Developer Evangelist at Oracle, focusing on microservices and messaging. Before this role, Mark was an Architect in the Enterprise Cloud-Native Java Team, the Verrazzano Enterprise Container Platform project, worked on Wercker, WebLogic and was a senior member of the A-Team since 2010, and worked in Sales Consulting at Oracle since 2006 and various roles at IBM since 1994.
This entry was posted in Uncategorized and tagged , , , , , , , , , , , , , , , , . Bookmark the permalink.

10 Responses to SOA Development and Delivery

  1. middleworks says:

    Hi Mark – I think the point about SOA not yet being industrialized is a great one, and I would focus especially on the fact that it is done mostly manually today (e.g. lack of automated build, lack of automated test, lack of integrated environments, lack of consistency in architecture and design decisions, and the other mature approaches that reflect older technologies, like coding in a language like Java). This makes many SOA implementations ad hoc and less successful than they would be otherwise and results in disappointed execs from an ROI perspective.

    I think your list of discussion topics will be a fantastic one for many of the customers who I talk to, andI think people will also be interested in industrializing SOA design. By that, I mean striving for more consistency within an organization around decisions like: what should be a service? What should be implemented with BPEL vs a service bus? When should canonicals be used? How should re-usable services be identified and implemented? Etc. I often see a struggle within companies around how much structure to put around decisions like this which can come under the umbrella of “Governance”, however Governance is often a bad word at fast moving organizations. So striving for enough structure to gain consistency is design, without so much structure as to bog down individual projects, is a challenge I see many customers struggle with.

  2. Pingback: Fusion Diffusion | Industrialized SOA

  3. Arun Pareek says:

    Hi Mark,

    First of all I salute your initiative here. I think as professionals and proponents of service oriented technologies, the onus is on us to take this forward in terms of maturity. The points you make are excellent and something that we should be dealing both in terms of having frameworks in place and enforcing their adoption too.
    After having worked for many years in this field, I kind of feel that the bottom of this problem is often the lack of will and desire to continuously adopt mechanisms to follow established and robust reference architecture, automation practices and testing. Part of the problem is inadequate information of all the possible tools around that can help. For instance, Craig Barr in this recent post (http://blog.rubiconred.com/2013/05/industrial-soa-where-are-you-part-1.html) explains many approaches to build automated testing but in practice I fail to see any interest to implement them.
    Starting a series on topics that you have discussed will be a great initiative and will be highly appreciated. I am sure, in the end, the outcome of all these efforts will help developers and architects carry pieces of frameworks that they can begin creating in their projects. Simply talking about industrialization will resonate as a thing to do but talking about each aspect with examples will go towards wider adoption.

    Cheers
    Arun

    • Mark Nelson says:

      Thanks for taking the time to comment Arun. I was happy to see that others are starting to write about this area too. As I said before, I think we are approaching the time when we will not be able to sweep this stuff under the rug any more 🙂

  4. Pingback: SOA Development and Delivery | Nitin's Oracle Fusion Middleware and SOA World

  5. Pingback: Additional new content SOA & BPM Partner Community | SOA Community Blog

  6. Pingback: Industrialized SOA – Where are you? – Part 1 | Rubicon Red

  7. Hi Mark, that is good stuff. I have already attempted similar SOA industrialization projects that I nicknamed “SOA-IN-A-BOX”. This framework leverages Model Driven Architecture as well as open standard and open source, which essentially provide “machine-ready” business requirements into specific models that are consumable by development tools like Eclipse and repositories. Now, I am facing a similar realization with Oracle ADF and I am finding that using the available Oracle tools difficult to integrate/automate into a seamless process. Looking forward to your articles. JP

Leave a comment