Let’s start by talking about what I mean by that. I don’t think that ‘SDLC’ is the right term to describe the space that I want to talk about, but it is a term that people are familiar with, so I think it makes a reasonable place to start. There are a lot of other terms that cover this space: governance, SDLC, development, version control, continuous integration, binary management, continuous inspection, continuous delivery, automation, virtualization, configuration management, devops, integration testing, and so on.
Also, by ‘SOA’ I really mean the broader set of technologies that we use to build ‘SOA Applications’ – whatever they are, and they are more than just composites. Real applications have many components, e.g. user interface, integration, mediation, rules, code, processes, forms, service definitions, canonical data and service models, and so on.
I don’t know about you, but I can’t think up a nice concise way to say that I mean all of those things across all of those technologies, so I am just going to call it ‘SOA Development and Delivery’ and I hope you will know what I mean.
The State of the Art
Recently, I conducted a survey, and I have spoken individually with many folks from all around the world, about how they do SOA Development and Delivery today, and how they hope to do it in the future. I think it would be fair to say that most SOA development today is not yet ‘industrialized’ – by that I mean that it is done by people of varying skill levels, working on a best effort basis, doing what they think is the right thing to do, and doing it the way they think is right.
By ‘industrialization’ I mean the development of standards, methods, processes, tools, and best practices, and the widespread automation of common tasks. Doing the right thing, and doing it the right way.
I think that we stand of the verge of a point of inflection in our industry where we will see SOA development and delivery become industrialized.
My feeling is that today, if I can generalize, that in terms of SOA development, people:
- have adopted some kind of iterative, or possibly agile, development approach,
- generally make some basic use of a version control system to manage source artifacts,
- sometimes write unit tests, but don’t always check coverage or information content of those tests,
- often try to automate the build process on a project by project basis,
- generally do not try to apply any kind of quality control or inspection of their code base,
- are maybe playing with or thinking about trying continuous integration,
- have not recognized the need for binary management, or have, but gave up because it is too hard with the current state of tools,
- generally try to use configuration plans (or equivalent), but often build binaries that contain topological information that is specific to a particular environment,
- maybe have a governance process that is sometimes updated and sometimes followed,
- don’t have a good way to manage dependencies between components,
- use virtualization in their development and test environments, though perhaps not in concert with any type of automation,
- don’t have a good way to manage test and development data sets,
- have not adopted devops practices, and
- do not have a repeatable way to execute tests and deploy the application to different environments.
The effect of all of this is that:
- it is hard to estimate and plan SOA development projects,
- projects often overrun their constraints,
- often the delivered application is of poor or uncertain quality, and
- build and deployment processes are very manual, time consuming, and error prone.
Pain all around – architects and project managers planning, developers and build engineers building, operations running, and of course the poor business customer who has to use the thing! Projects are delivered late, over budget, and erode the confidence of the business in the whole SOA promise and approach. So naturally, I asked myself, why is this the case? Well I think there are a number of contributing factors here:
- the tools have not matured to a point that enables a lot of these things in a simple way,
- there really aren’t any standards or reference architectures to follow,
- some people don’t think of SOA development as ‘real’ development, and therefore don’t think these kind of practices apply to SOA development,
- some of these are still maturing in the wild.
Conversely, it does not take a lot of time on Google to find a whole bunch of people who are interested in these areas and are clearly experimenting, prototyping, and sharing their experiences. And talking with folks who are actively engaged in SOA development, the clear, consistent message I hear is that all of this stuff is ‘really important stuff’ and we, as an industry, need it. It is also not hard to find consulting firms around the world that are developing capabilities and offerings in this space too.
So where do we go now?
Over a series of posts, I plan to share some thoughts and experiences on a range of topics, taken from those above, and also to share some practical examples of how to apply some of the state of the art tools and techniques from Java development to SOA development. In particular, specifically in the context of SOA/MDS/BPM/OSB/ADF, I want to talk about:
- version control,
- dependency management,
- build automation,
- continuous integration,
- continuous inspection,
- continuous delivery,
- unit testing,
- integration testing,
- binary management,
- environment provisioning,
- configuration management,
- customization (of binary artifacts to particular environments),
- the ‘correct’ use of MDS, and
- the relationship of SOA development and delivery to governance.
And I am planning to talk about tools like Subversion/git/gitlab, Maven, Hudson, Archiva/Artifactory, Sonar, Chef and many others.
I hope you join me on this journey, and please let me know your thoughts.