Here is the first installment – Basic Maven – you might want to watch fullscreen

Coming soon:

• Intermediate Maven
• Advanced Maven
• Using the WebLogic Maven Plugin

Let’s get started on our SOA Development and Delivery journey by talking about version control.  But before we delve right in, let’s take a moment to reflect on this axiom:

Axiom 1: Developing a SOA Composite Application is software development.

When we sit down to create a SOA Application, i.e. composites, user interfaces, services, etc., we are actually embarking on a software development exercise.  I think that some people don’t believe (or at least they don’t admit to themselves) that this is the case.  Why?  Well acknowledging that it is, in fact, software development implies a whole bunch of practices are necessary – like version control and testing for example.  And those are hard, right?

Well, they are certainly more work.  But its a bit like insurance – it is a cost you accept in the present to offset or prevent a much more significant potential cost/pain in the future.  In their quintessential book Continuous Delivery, Dave Farley and Jez Humble say:

“In software, when something is painful, the way to reduce the pain is to do it more frequently, not less.”

Putting in the extra effort upfront will save you from a lot more pain and effort later on.  Using version control fits into this category.  It can be a bit painful, but it is definitely worth it in the end.  Consider this axiom:

Axiom 2: Developing software without version control is like mixing chemicals without reading the labels – sooner or later, it is going to blow up in your face.

Why is this true?  Consider the following questions – how would you answer these if you are not using version control?

• What was the content of (some BPEL file) on (some day in the past)?
• Who changed (some endpoint URL) in (some BPEL file)?  When?  Why?
• We found a problem in production, which version of the source matches this SAR we have in production?
• Can I get a copy of that source, with no other changes or updates to any other files?
• How come I can’t build this?  It must have worked before!
• Where is the deployment script for this composite?
• How did we set up the test environment that we used to test this composite?
• Which version of the SOA configuration plans matches this version of the SAR?
• Which version of the OSB project and the ADF UI matches this version of the composite?

The answer is, of course, that you can’t answer them.  At least not with any level of confidence.  I would go so far as to say that if  you don’t have your SOA applications under version control, you should stop whatever you are doing, and go set it up right now!  Sooner or later you are going to encounter a critical problem in your production environment that you simply cannot fix.

That leads us to one more axiom:

Axiom 3: If it is not in version control, it does not exist.

If you cannot retrieve a previous revision of an artifact, then it may as well not have ever existed at all – it is of approximately the same value to you either way.

Now, what do I mean by ‘using’ version control?  Well, it is more than just checking in your changes.  That is an excellent start, and it is much better than nothing at all, but really, you should be thinking about a few more advanced ways of using your version control system as well – like branching and tagging for example.  We will come back and discuss these in some depth later in this article.

For now, let’s cover some basics.

Which version control system to use?

There are a number excellent version control systems available, both free and commercial.  I have used most of them – rcs, SCCS, CVS, Subversion, ClearCase, Perforce, Mercurial, and git to name a few, and even some proprietary ones that exist only inside the organizations that use them or are included in a particular operating system (e.g. OpenVMS) or file system (e.g. zfs).

Today I mostly use Subversion, but I am going through a transition to git.  Let’s talk about these two in a little more detail.

Firstly, both of them are free, and both are widely used.  That means that there is good tool support, a large community of people creating helpful content about how to use them, and a pretty good chance that anyone you get working on your project will have some level of familiarity with them.  They both also have excellent, freely available books to help you get started: Version Control with Subversion and Pro Git.

The other thing that is very useful about these two, specifically in the context of Oracle SOA Suite development, is that they use an atomic commit which means that you can change a number of files and commit all of those changes as a single revision.  CVS, for example, does not allow you to do this because it versions each file individually.  This capability is great in a SOA environment, as many of the changes we need to make involve making changes to more than one file.  Having the project in a state where some, but not all, of those file changes are committed, is not useful.

JDeveloper supports Subversion quite well, although you do need to invest a little time to make sure you know how to drive it.  And support for git was added in JDeveloper 11.1.1.6.

There are three key things that are driving my personal migration from Subversion to git:

• The increasing demand for distributed development, especially when multiple organizations are involved in the development lifecycle – git is one of the fleet of new distributed version control systems,
• The ability to commit, branch, merge, etc., while offline (and away from the watchful eye of the continuous integration server), and
• The availability of excellent tools (like GitLab) that provide easy visibility into the repository itself, and into the branching and merging over time.

I personally find the git workflow more complex than the Subversion one, but for me at least, the time has come to make the move.  I think that moving to distributed version control is pretty much inevitable in the modern world.

I think the most important thing to say about which version control system is the right one to use is this – any is better than none.  If you want me to recommend one, I would have to say git.

What should we put in version control?

Version control is for source artifacts, not derived artifacts like binaries, deployable packages, etc.  Source artifacts does not just mean source code – it means anything that is needed to recreate your production environment from scratch.  Craig Barr proposed an excellent list in this article which I am quoting here:

• “OSB Configuration
• SOA Projects
• Customization Files
• Composite Configuration Plans
• WebLogic Deployment Plans
• Build Scripts
• Test Scripts
• Deployment Scripts
• Release Scripts
• Start-up & Shutdown Scripts
• “Health Check” Scripts
• Application Server Configuration
• Puppet Configuration
• (Optionally) The Binaries
  Note: This is unnecessary and redundant if you follow good binary management which I’ll discuss in the next blog installment.
• And so on….”

Personally, I do not agree with putting your binaries into your version control system.  I think that binaries belong in a separate repository, because they have quite different characteristics and management needs.  We’ll talk a lot more about this in a future post on binary management, but for now a couple of examples to illustrate the point:

So what do you put into version control for SOA, OSB, ADF?

The simplest answer to this question is ‘whatever is left in the project after you have executed the clean action on the project in the IDE.’  Note that you would need to make sure you have disabled automatic builds in eclipse, otherwise it will just go ahead and build again.

We need to be careful about a couple of things here:

• First, what is a project?  For SOA, we really need to be checking in at what JDeveloper calls the SOA Application level, not at the level of what JDeveloper calls a SOA Project.  The reason for this is quite straightforward – there are a number of circumstances under which it is not possible to build a SOA Project without having access to some of the information (files) in the SOA Application.  For example, the presence of Human Tasks and Business Rules in a composite (SOA Project) are such an occasion.  In both of these cases, you need to be able to access the adf-config.xml file in the SOA Application to get the necessary MDS configuration information to build the project.
• There are some directories that your version control client may automatically hide, because their names start with a period (‘.’) – for example, there is a ‘.adf‘ directory.  These often contain important data and you need to make sure that you check them in.
• Depending on what you have done in your project, the out of the box ‘clean’ action might not do a proper clean up of your project.  If you have created extra target directories, for example, you might need to make sure that those removed too.  A good example of this is when you use Maven to build a SOA project – it will create a target directory, in addition to the normal deploy directory.  You need to make sure that the mvn clean also removes the deploy directory and/or that the IDE also removes the target directory.

The same goes for ADF and OSB projects.  Given the flexibility provided by the tools, there is really no ‘one size fits all’ answer to this question, you need to invest the time to work out the correct answer for your own projects.

At the end of the day, there are two ways you can get this wrong:

• You leave out some files that are needed.  This is a problem that you will need to go back and fix.
• You include some extra files that are not needed.  This is probably not a big deal.  It could possibly result in some extra files being in your binaries.  That may or may not be a problem.

So if you are limited for time, better to opt for too much than too little.

How should we set up the repository?

A question that comes up fairly often, particularly with Subversion, is how to structure the repository.

There are two common approaches here – one is to have a single Subversion repository with all the projects in the same repository.  The other is to have one repository per project (or development group, or whatever unit).

There are of course advantages and disadvantages to each approach.  Commonly cited issues include: the amount of administration overhead, the time taken to perform backups (which is done by dumping the whole repository), issues with revision numbers (which are shared across the whole repository) and comments (knowing which ones are for a specific project), different security requirements, or code separation requirements for different projects, and different Subversion workflow requirements across projects.

In practice though, I think that these can be handled with approximately the same amount of effort regardless of the approach chosen, assuming a suitably experienced Subversion administrator.

I believe that the one repository approach is easier, and I would recommend taking that approach unless there is some specific reason not to.  The most likely reason is that some project team does not want their code stored with another team’s code due to some kind of confidentiality or licensing issue (perceived or real).

So, If you are using Subversion, I would recommend a single Subversion repository, shared by all projects for a SOA environment.

Inside the repository, you should create zero or more levels of directories that you use to organise projects into logical groups, then under these, create a directory for each project (i.e. SOA Application, etc.) and under that create the recommended Subversion trunk, tags, and branches directories.  This is also consistent with the approach recommended in Version Control with Subversion.  So your repository might look like this:

root
  - businessUnit1
    - project1
      - composites
        - GetCustomerDetails
          - trunk
          - tags
          - branches
        - ProcessOrder
          - trunk
          - tags
          - branches
      - ui-projhects
        - ...
    - ...
  - businessUnit2
    - ...

git

With git, I am in the habit of creating a git repository for each project, as that is a more natural way to organise things in git.

Tagging

Tagging, in the context of a version control system, is essentially making a named copy at a given point in time.  (Though more than likely you will just be copying pointers, not all of the content.)

Why would you want to be able to go back to a given point in time?  There are two excellent reasons:

• You have found a problem in an older version that is deployed in a production (or other) environment that you need to fix, so you need to get back the exact version of all of the source artifacts that were used to create that particular version, and
• Things have gone very bad and you need to go back to a known good point in the past.

So, the first one tells us that you must tag whenever you are going to release.  You might also want to tag whenever you reach a significant or meaningful milestone.

The second reason can be addressed with tagging, but you might be better to use a branch in that case.  We will talk about branches in a moment.

What is a version anyway?

Often people ask about the relationship between the ‘versions’ in the version control system and the build system, and the runtime versions.  It is important to understand that there is not, and need not really be, any kind of direct relationship between them, other than for releases or release candidates.

Normally you are going to be building the latest version of the code – this is sometimes called the ‘head’, or ‘trunk’ or ‘tip’ – and executing tests against that.  So you don’t need to know which ‘version’ (‘revision’ is a more accurate word from the point of view of the version control system) it is – you can just refer to it as the latest version.  In Subversion, this is done by ending the URL with ‘/trunk‘.

The only other revisions that you are likely to build are the latest versions on a particular branch.  Again, this can be done without knowing the revision number.

You would not need to build any tagged/released version again – you could just go and get the binary from the binary repository.  Although you could easily build it again if you needed to by referring to the tag name in the URL.

And all of the other revisions are essentially old, discarded points in time that you have moved on from.

So you do need to know which tag relates to which binary version, and the easiest way to do this is to just use the binary version number in the tag.  So, for example, you might tag revision 126 as ‘VERSION-2.3‘.  If you needed to come back later and look at that version, you could just end your Subversion URL with /tags/VERSION-2.3.

That brings us to branching…

Branching

Keeping a log of all the changes over time is all very well and good, but projects don’t always follow a straight line.  What happens when you do find the problem in your old version 2.3, the one you have in production, but you have already done a heap of work on version 2.4?

This is one thing that branches are good for.  A branch lets you start a parallel stream of work from a given point in time (like a tag, but it can be from any revision).  Consider the following diagram:

Here we can see that development of the next version can continue on the trunk, while another team work on fixing the production bug in version 2.3 in the branch.  The two have no effect on each other.  We can build either the trunk or the branch, depending on what we want to do.

When you finish fixing the bug, let’s say that is with revision 131.  You should tag that one too, so you have another tag to go back to in case you find a bug in that ‘fixed’ version.

The other useful thing about branches, is that you can merge them back into the trunk (or any other branch for that matter).  This provides an ideal way to isolate potentially dangerous work.  If you are trying something new, and you are not sure how it is going to work out, you can do that work in a branch.  If it proves to be ok, you can merge the changes in that branch back into the trunk at some point in the future.  But if it is not ok, you can just forget the branch and go on as if it never happened.

A note on merging

One really important thing to know is that you can merge backwards.  This lets you essentially get rid of a bad commit.  This is definitely something that you should learn how to do in your version control system.

When to create a version

Here is another meaningful question – how often should developers commit their changes to the trunk?

Later on we are going to talk about continuous integration.  The word ‘integration’ in ‘continuous integration’ refers to the process of merging changes into a trunk.  One of the foundation principles of continuous integration is that all developers must commit regularly to the trunk.  What does regularly mean?  Well no less frequently than once every day.

This drives some desirable behaviors.  How much meaningful work can a developer do in a day?  Not much right?  Right!  That’s the whole idea!  By forcing developers to work in small iterations, we are minimizing the size of potential integration issues that can occur.  Less code to integrate – less serious problems integrating.  Less serious problems, easier to fix.  Less problems in the codebase at any given point in time – better quality!  That is what continuous integration is aiming to achieve.

When we are using continuous integration, we build every time a developer commits to the trunk (or to a branch).  But now the question arises – do we really want to build all of the developer’s little intermediate commits, that we know are not going to work anyway, since they are work in progress?

There are two ways we are going to talk about (in future articles) to address this:

• Many continuous integration servers support the notion of a pre-flight build.  This is a build that is triggered by the developer’s work in progress commit.  The sole purpose of this build is to see if the have broken anything.  It does not need to go all the way through the build process, running all of the integration tests, getting ready for release.  It is never going to be released.  This gives the developer the freedom to experiment and check the results without bogging down the continuous integration system wasting a whole bunch of time and energy on builds that don’t matter.
• The other option comes with distributed version control.  Here the developer can commit as many times as they like, but the commits will not flow through to the main repository – the one the continuous integration server is watching – until the developer does a ‘push’. When the continuous integration server sees a whole bunch of new commits, it is (usually) smart enough to just pick up the newest one and build that.  This approach includes an implicit assumption that the developer is able to build and test the software on their own environment before pushing.

More about version control

At this point, it starts to become difficult to talk about some of the strategies I want to discuss before we have moved forward a few more steps and talked about a few more topics, especially continuous integration .  So let’s put version control on hold, and come back to it later.

You can find configjar sitting under your Oracle_OSB1/tools/configjar directory in your OSB PS6 installation.  There is a readme file there that tells you how to use it from ANT and WLST.  Here, I want to show you how to use it from Maven, and therefore Hudson, etc. too.

For this post, I went into the OSB IDE and created a simple project called osbProject1 which contains a single Proxy Service called (imaginatively) ProxyService1.  It is just a plain old ‘any’ proxy service with essentially no implementation at all.  But it is enough to do what we need to do.

By the way – I have OSB and the OSB IDE running on Oracle Linux as a 64-bit application – see how here.

The configjar tools supports building sbconfig.jar files for projects and/or resources.  So you should be able to use this same approach for pretty much anything you build in OSB, possibly excepting when you have custom extensions.

If we take a look in my osbProject1 direction in my eclipse workspace, we see that it contains just a single file, ProxyService1.proxy.

We are going to add a few more files:

• A Maven POM to control the build (pom.xml)
• A settings.xml file that we will pass to configjar
• import.py and import.properties files like we had in that previous post

Let’s take a look at them now.  We will start with settings.xml.  This is a file we pass to configjar to tell it how to build the sbconfig.jar for us.  This is a relatively simple file.  Here is an example:

<?xml version="1.0" encoding="UTF-8" ?>
<p:configjarSettings xmlns:p="http://www.bea.com/alsb/tools/configjar/config">
  <p:source>
    <p:project dir="/home/oracle/workspace/osbProject1"/>
    <p:fileset>
      <p:exclude name="*/target/**" />
      <p:exclude name="*/security/**" />
      <p:exclude name="*/.settings/**" />
      <p:exclude name="*/import.*" />
      <p:exclude name="*/alsbdebug.xml" />
      <p:exclude name="*/configfwkdebug.xml" />
      <p:exclude name="*/pom.xml" />
      <p:exclude name="*/settings.xml" />
      <p:exclude name="*/osbProject1.jar" />
    </p:fileset>
  </p:source>
  <p:configjar jar="/home/oracle/workspace/osbProject1/osbProject1.jar">
    <p:projectLevel includeSystem="false">
      <p:project>osbProject1</p:project>
    </p:projectLevel>
  </p:configjar>
</p:configjarSettings>

In the source element, under project we have a dir attribute that points to the OSB project directory that was created by the OSB IDE, and we most likely check out of a version control system before starting our build.  All of those exclude elements are telling it to ignore these extra files we have added to the project, and the ones that it will generate.  Without those, your sbconfig.jar will be polluted with a bunch of unnecessary stuff.  These are all standard from project to project, except for the last one, which is the name of the sbconfig.jar itself.

The jar attribute on the configjar element names the output file.  Finally, we have the project element under projectLevel that names the projects we want to include in the sbconfig.jar that we build.  The other option is to select by resources.  You might want to go check out the OSB documentation to see how to do that.

Next, we have the import.py and import.properties files.  These are essentially the same as in the previous post so I wont repeat them. The only difference is that we need to update import.properties to contain the correct filename:

importJar=osbProject1.jar

Then we need a POM.  We are going to use the maven-antrun-plugin to execute configjar through ANT to generate the sbconfig.jar, and then the exec-maven-plugin to deploy it to our OSB server.  Here is the POM:

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.redstack.osb</groupId>
  <artifactId>osbProject1</artifactId>
  <version>1.0-SNAPSHOT</version>

  <dependencies/>

  <properties>
    <osbHome>/ciroot/product_binaries/osb11.1.1.7/Oracle_OSB1</osbHome>
  </properties>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.7</version>
        <executions>
          <execution>
            <id>deafult-cli</id>
            <phase>package</phase>
            <configuration>
              <target>
                <echo>
WARNING
-------
You must set the weblogic.home and osb.home environment variables
when you invoke Maven, e.g.:
mvn compile -Dweblogic.home=/osb11.1.1.7/wlserver_10.3
-Dosb.home=/osb11.1.1.7/Oracle_OSB1
                </echo>
                <taskdef name="configjar"
                  classname="com.bea.alsb.tools.configjar.ant.ConfigJarTask"
                  classpathref="maven.plugin.classpath"/>
                <configjar settingsFile="${basedir}/settings.xml"
                  debug="true">
                </configjar>
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
        <dependencies>
          <dependency>
            <groupId>org.apache.ant</groupId>
            <artifactId>ant</artifactId>
            <version>1.7.1</version>
          </dependency>
          <dependency>
            <groupId>org.apache.ant</groupId>
            <artifactId>ant-launcher</artifactId>
            <version>1.7.1</version>
          </dependency>
          <dependency>
            <groupId>org.apache.ant</groupId>
            <artifactId>ant-nodeps</artifactId>
            <version>1.7.1</version>
          </dependency>
          <dependency>
            <groupId>org.apache.ant</groupId>
            <artifactId>ant-apache-bsf</artifactId>
            <version>1.7.1</version>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>configjar</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/tools/configjar/configjar.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>weblogic.server.modules_10.3.6.0</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../modules/features/weblogic.server.modules_10.3.6.0.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>weblogic</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../wlserver_10.3/server/lib/weblogic.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>oracle.http_client_11.1.1</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../oracle_common/modules/oracle.http_client_11.1.1.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>xmlparserv2</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../oracle_common/modules/oracle.xdk_11.1.0/xmlparserv2.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>orawsdl</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../oracle_common/modules/oracle.webservices_11.1.1/orawsdl.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>wsm-dependencies</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../oracle_common/modules/oracle.wsm.common_11.1.1/wsm-dependencies.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>osb.server.modules_11.1.1.7</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/modules/features/osb.server.modules_11.1.1.7.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>oracle.soa.common.adapters</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>log4j_1.2.8</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/lib/external/log4j_1.2.8.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>alsb</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/lib/alsb.jar</systemPath>
          </dependency>
        </dependencies>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>deploy</id>
            <phase>pre-integration-test</phase>
            <configuration>
              <executable>/bin/bash</executable>
              <arguments>
                <argument>${osbHome}/../oracle_common/common/bin/wlst.sh</argument>
                <argument>${basedir}/import.py</argument>
                <argument>${basedir}/import.properties</argument>
              </arguments>
            </configuration>
            <goals>
              <goal>exec</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Let’s take a look at the important parts of the POM that you will need to adjust to suit your environment:

• We define a property called osbHome which is then used throughout the POM.  This needs to point to your Oracle_OSB1 directory.
• In the plugin section for maven-antrun-plugin you can see that we have a taskdef to import the configjar task from the supplied configjar-ant.xml which comes with OSB.  We also have added a bunch of dependencies to this plugin so that it has the right OSB libraries in the classpath when it executes.  Note that they all have the scope set to system which means you don’t need to import them into a Maven repository first.  You can just specify a path to locate them.  Of course, this ties your build to that one machine effectively.  You can of course just go ahead and put those jars into a Maven repository and then use them normally, though since configjar depends on a local OSB install, there is not a lot of benefit right now.
• Finally, in the plugin entry for exec-maven-plugin you can see that we execute wlst.sh and pass it our import.py script to do the deployment.

The configjar tool also requires two environment variables to be set, so you need to set these when you run Maven.  Here is an example:

mvn verify -Dweblogic.home=/ciroot/product_binaries/osb11.1.1.7/wlserver_10.3 -Dosb.home=/ciroot/product_binaries/osb11.1.1.7/Oracle_OSB1

This will build the sbconfig.jar and deploy it to OSB for us.  Here it is:

Good luck!  Enjoy!  And a big thank you to Dimitri Laloue who helped me get configjar working.

The same international team that made that video – Andrew Dorman, Tanya Williams, Carlos Casares, Joakim Suarez and James Calise – have also created a thirty minute version that walks through in much more detail and shows you, from the perspective of various business stakeholders involved in process modeling, exactly how BPM PS6 supports the end to end process lifecycle. The video centres around a Retail Leasing use case, and follows how Joakim the Business Analyst, Pablo the Process Owner, and James the Process Analyst take the process from conception to runtime, solely through BPM Composer, without the need for IT or the use of JDeveloper.

• Joakim, the Business Analyst, models the process, designs the user interaction forms, and creates business rules,
• Pablo, the Process Owner, reviews the process documentation and tests the process using the new ‘Process Player’,
• James, the Process Analyst, analyses the process and identifies potential bottle necks using ‘Process Simulation’.

Showcasing the business-focused advancements in PS6, the video illustrates the ease with which users can collaborate and quickly drive real business optimization.

Watch the video here:

Also, by ‘SOA’ I really mean the broader set of technologies that we use to build ‘SOA Applications’ – whatever they are, and they are more than just composites.  Real applications have many components, e.g. user interface, integration, mediation, rules, code, processes, forms, service definitions, canonical data and service models, and so on.

I don’t know about you, but I can’t think up a nice concise way to say that I mean all of those things across all of those technologies, so I am just going to call it ‘SOA Development and Delivery’ and I hope you will know what I mean.

The State of the Art

Recently, I conducted a survey, and I have spoken individually with many folks from all around the world, about how they do SOA Development and Delivery today, and how they hope to do it in the future.  I think it would be fair to say that most SOA development today is not yet ‘industrialized’ – by that I mean that it is done by people of varying skill levels, working on a best effort basis, doing what they think is the right thing to do, and doing it the way they think is right.

By ‘industrialization’ I mean the development of standards, methods, processes, tools, and best practices, and the widespread automation of common tasks.  Doing the right thing, and doing it the right way.

I think that we stand of the verge of a point of inflection in our industry where we will see SOA development and delivery become industrialized.

My feeling is that today, if I can generalize, that in terms of SOA development, people:

• have adopted some kind of iterative, or possibly agile, development approach,
• generally make some basic use of a version control system to manage source artifacts,
• sometimes write unit tests, but don’t always check coverage or information content of those tests,
• often try to automate the build process on a project by project basis,
• generally do not try to apply any kind of quality control or inspection of their code base,
• are maybe playing with or thinking about trying continuous integration,
• have not recognized the need for binary management, or have, but gave up because it is too hard with the current state of tools,
• generally try to use configuration plans (or equivalent), but often build binaries that contain topological information that is specific to a particular environment,
• maybe have a governance process that is sometimes updated and sometimes followed,
• don’t have a good way to manage dependencies between components,
• use virtualization in their development and test environments, though perhaps not in concert with any type of automation,
• don’t have a good way to manage test and development data sets,
• have not adopted devops practices, and
• do not have a repeatable way to execute tests and deploy the application to different environments.

The effect of all of this is that:

• it is hard to estimate and plan SOA development projects,
• projects often overrun their constraints,
• often the delivered application is of poor or uncertain quality, and
• build and deployment processes are very manual, time consuming, and error prone.

Pain all around – architects and project managers planning, developers and build engineers building, operations running, and of course the poor business customer who has to use the thing!  Projects are delivered late, over budget, and erode the confidence of the business in the whole SOA promise and approach.  So naturally, I asked myself, why is this the case?  Well I think there are a number of contributing factors here:

• the tools have not matured to a point that enables a lot of these things in a simple way,
• there really aren’t any standards or reference architectures to follow,
• some people don’t think of SOA development as ‘real’ development, and therefore don’t think these kind of practices apply to SOA development,
• some of these are still maturing in the wild.

Conversely, it does not take a lot of time on Google to find a whole bunch of people who are interested in these areas and are clearly experimenting, prototyping, and sharing their experiences.  And talking with folks who are actively engaged in SOA development, the clear, consistent message I hear is that all of this stuff is ‘really important stuff’ and we, as an industry, need it.  It is also not hard to find consulting firms around the world that are developing capabilities and offerings in this space too.

So where do we go now?

Over a series of posts, I plan to share some thoughts and experiences on a range of topics, taken from those above, and also to share some practical examples of how to apply some of the state of the art tools and techniques from Java development to SOA development.  In particular, specifically in the context of SOA/MDS/BPM/OSB/ADF, I want to talk about:

• version control,
• dependency management,
• build automation,
• continuous integration,
• continuous inspection,
• continuous delivery,
• unit testing,
• integration testing,
• binary management,
• environment provisioning,
• configuration management,
• customization (of binary artifacts to particular environments),
• the ‘correct’ use of MDS, and
• the relationship of SOA development and delivery to governance.

And I am planning to talk about tools like Subversion/git/gitlab, Maven, Hudson, Archiva/Artifactory, Sonar, Chef and many others.

I hope you join me on this journey, and please let me know your thoughts.

Read next post in this series.

For the rest of this post, I will be assuming that you are familiar with the functionality in previous releases.  If you are new to BPM, you might want to read back over these two earlier posts first:

• What BPM adds to SOA
• What’s new in BPM Feature Pack

Overview

This release adds what I like to think of broadly as two major new sets of functionality:

• First, it adds all of the tools needed for a business user to be able to use BPM extensively and without the need for a developer (assuming that all of the services and integration they need is in place) – browser based modeling, forms design, simulation, a ‘conference room pilot’/walkthrough capability, as well as enhancements to WorkSpace, BAM and Rules to make them easier for business users, and
• Secondly, it adds cases as a new first class citizen.  This release has full support for case management in the runtime engine and in JDeveloper, and exposes APIs to allow you to build your own user interface for case management.

We’ll walk through these in a bit more detail in the remainder of this post.  There is also another small new feature that it worth mentioning – PS6 will add the ability to migrate a running process instance from one revision of the process model to another.  An ANT-based utility will be available that will allow you to find process instances that are eligible for migration, and to actually perform the revision migration for you.  More on this in a later post!

Composer – Process Modeling

There are a number of enhancements to BPM Composer, the web based process modeling environment.  Here is the new project screen, let’s discuss some of the new things we see here:

• We can now edit (create and update) not just processes, but also rules, human tasks, web forms, business objects, data objects, roles, business indicators and activity guides right here in composer.  ‘What are web forms?’ you say – we will come to that shortly.
• We can define and run simulations in composer.
• We have this new ‘Process Player’ which essentially lets us conduct a test run of a process – to show a business user what it would be like.

The process editor is updated and has a few new options including editing data objects, business indicators and viewing the collaboration diagram.

Here we can see the new data associations editor.

And with this release, we get the ability for business users to author and modify rules in Composer.  Here we can see a decision table being created in Composer.  You can also define bucket sets, globals, functions, and If/Then rules.

This is the Human Task editor.  Here you can see that we can define human tasks in Composer, including the data objects in the payload.  We can use the data association editor to map data from process data objects into the human task payload.  We can also set the outcomes, priority, deadlines, etc., and choose whether the presentation (task form) will be an ADF Form or a Web Form.

This is the Business Objects editor, where we can define and modify business objects directly in Composer, including simple and complex types, and we can also add documentation.

So, overall we can see that there is a lot of new functionality in Composer, to the extent really that a business analyst could pretty much model all of the process, data and user interface components right here in their browser, without needing any IT or developer support.

Of course, we still need developers to create the business services and publish them into the Business Catalog so that they are available for use in processes!

Web Forms

A major new feature in this release is Web Forms.  This feature gives business users the ability to design the user interface for human tasks right here in Composer.  You can of course still use ADF if you prefer, or any other web framework, to design the task forms.  You can choose on a task by task basis, and you can mix and match as needed.

Web Forms are designed in Composer using a simple drag and drop approach.  You drag form components from the palette and drop them on the form, then you can set properties to control things like names, visibility, sizes, colours, and so on.

You also have the ability to write ‘form rules’ which allow you to do things like set up validation of fields, or to show and hide fields based on previous selections, or to autofill parts of the form.

You can also call external services from here.  So if you wanted to call a web service to get a list of product codes in order to populate a form on the field for example, this can also be implemented in a ‘form rule’.

Simulation

Previously, simulation was only available in JDeveloper.  Now with PS6, we have the ability for a business user to define and run simulations in Composer.

A simulation is made up of two parts:

• Simulation Models – which provide details for a single process model, like costs, times, resources, etc., and
• Simulation Definitions – which group together all of the simulation models that you want to include in a particular simulation.

You can have as many different simulation models and definitions as you like, and simulation models can be reused across as many simulation definitions as you like.

When you create a simulation model, you have the option of setting the properties for all of the interactive (human) tasks and automatic tasks in the one go:

This is a great new feature – it used to be necessary in older versions to set these one by one for each activity, which could take a long time if you have a lot of activities!

You can also define resources, assign them to roles and give them costs and productivity and availability factors:

When you run a simulation, you can see details right on the process model.  This example is showing the average number of instances that passed through each activity in the process per time unit for example:

Simulation allows business users to test one or more processes in various circumstances to understand their behaviour, costs and resource utilisation.

Process Player

The Process Player is a new feature that is designed to allow a business user to have a test run through a process to see what it would look like.  It is intended that this feature would be used to conduct a ‘conference room pilot’ of a process – to allow business users to see how the process would actually look and run before it is actually deployed to the production environment.

Player allows you to visualise how a process is going to work.  When you start a Player session, you can map some users to each role in the process, and then start an instance of the process to see how it works.

As you step through the process, you will see the task forms and you can interact with them as if the process were really deployed.

The process model is coloured in as you move through the process so that you can easily visualise where you are up to in the process – what has happened, and what will happen next.

This is intended to provide an easy way to ‘show’ and ‘explain’ the process to the business owners and stakeholders to validate it is correct before moving on to deploy it.

New Workspace

The BPM Workspace has been given a facelift to make it better looking and easier to use.  Here is what it looks like:

Some of the frequently requested features have been added in this release – for example easily customisable views (including the inbox), a ‘next task’ button, simplified wizards for task reassignment and delegation which display more user information, not just the userid, and a simplified vacation rules editor.

This new release also allows you to navigate from tasks to process instances easily.

Case Management

The other main new feature in this release is Case Management.   Cases are added as a new first class citizen – alongside BPMN, BPEL, Mediators, Rules, Human Tasks, etc.  In this release there is design time support for cases in JDeveloper, and an API to allow you to build your own Case Management applications/user interfaces.

Learn more about Case Management from this series of posts from my colleague Mark Foster:

• Part 1: An Introduction
• Part 2: Anatomy of a Project
• Part 3: Runtime Lifecycle of a Project

This can be easily managed with log rolling utilities like logrotate for Linux or logadm for Solaris. These allow you to automate the process of removing old log entries, and they work by moving the log file contents out through a series of files. This means that you will have effectively controlled the amount of space used for logs, plus you have set a time period that logs are kept online before being archived or deleted.

Let’s look at an example to understand how it works:

In this diagram we are looking at the log growth over time, with time being the vertical axis. So at the top we have a single log file – AdminServer.log – and it grows over time, as indicated by the blue bars at the top.

Then, when the first log rotation occurs (the top red dotted line), the content of AdminServer.log is moved into AdminServer.log.0, as indicated by the purple arrow, and AdminServer.log is emptied out, so as WebLogic Server continues to write into this file, we have a much smaller file now (the green one).

This process then repeats. At the next log rotation, the lower red dotted line, we get the contents of AdminServer.log.0 moved into AdminServer.log.1, the contents of AdminServer.log moved into AdminServer.log.0, and AdminServer.log emptied out again.

This continues until we get up to eight files, then the oldest one is deleted and the other seven move one to the right.

Here’s how to set it up:

First, you need to be starting the WebLogic Server instance in a way that the stdout is being appended to a file, (as opposed to just written to a file). To do this, you need to make sure you use the >> shell redirection, not >. To get stdout and stderr in the same file, you would use &>>.

Here’s an example:

/your/fmwhome/user_projects/domains/base_domain/startWebLogicServer.sh &>> /your/logs/adminserver.out &

This assumes that you have WebLogic installed in the Oracle Home /your/fmwhome and that you are storing your stdout/stderr log files in /your/logs.

Note: It is important that you use the append redirection (>>), otherwise your log files will not actually shrink in size after rotation, they will just keep growing, so that defeats the purpose of rotation in the first place.

Note: If you use the Node Manager to start WebLogic Server, it will automatically open the log files in append mode.

Next, you need to set up your log rolling utility – logrotate (on Linux) or logadm (on Solaris). Let’s look at each of these in turn.

logrotate (Linux)

The configuration for logrotate is kept in /etc/logrotate.conf. You need to add a stanza in there for each of the log files you want to rotate. Here is an example for the stdout file from above, and the server log file:

/your/fmwhome/user_projects/domains/base_domain/servers/AdminServer/logs/AdminServer.log {
  copytruncate
  daily
  rotate 8
}
/your/logs/adminserver.out {
  copytruncate
  daily
  rotate 8
}

Let’s explore these. The first line names the log file to rotate. Rotation is essentially going to move that log to a different file and create a new empty log. The line ‘rotate 8‘ tells logrotate to keep up to eight log files. The ‘daily’ means to roll them once a day, and the ‘copytruncate’ tells it which method to use – in this case to copy the file into a new file, then empty it out (as opposed to the other method which is to rename the file and create a new one – this method will not work with WebLogic Server or other JVM applications).

You can also force logrotate to run immediately, which is useful for checking you have everything set up correctly. This is done (as root) by issuing the command:

logrotate -f /etc/logrotate.conf

logrotate has many other options that allow you to specify different time periods, actions to take before and after log rotation, and size limits for when rotation should occur, to name a few. You should take a look at the logrotate documentation to see how to use it to best suit your scenario.

logadm (Solaris)

logadm provides essentially the same capabilities as logrotate, but the configuration is slightly different. To set up the same example as we saw above with logadm, we need to issue the following commands (as root):

logadm -w /your/fmwhome/user_projects/domains/base_domain/servers/AdminServer/logs/AdminServer.log -P 1d -c
logadm -w /your/logs/adminserver.out -P 1d -c

These commands will update the logadm configuration file (/etc/logadm.conf) with the necessary entries – the -w option means ‘write to configuration file’. It is not recommended to edit the file directly, but to use the logadm -w command to update it – to prevent errors.

After the -w, we see the name of the file to rotate, then -P 1d, which means period of one day – i.e. daily – and -c which tells logadm to use the copy and truncate method.

To force logadm to run immediately, you need to do two things. First, you need to tell it to assume the last run was some time in the past, this is done by issuing the same command (as root) with a timestamp on it, e.g.:

logadm -w /your/logs/adminserver.out -P 1d -c -p 'Mon Feb 25 02:00:00 2013'

This tells logadm to assume the last time it ran for this file was on that date, which is more than a day ago.  The last run timestamps are stored in /var/logadm/timestamps if you want to take a look.

You can then issue the following command (as root) to force logadm to run immediately:

logadm

Just like logrotate, logadm has a bunch of other options that let you control how often rotation is done, size limts, pre/post actions, etc. Take a look at the documentation to see what you need for your scenario.

Enjoy!

This whitepaper has been under development for quite a while now, and has been through a heap of reviews and revisions.  So it is great to finally get it out there, and hopefully you will find it useful!

Many people have contributed to this whitepaper – from reporting on tuning experiences, to writing, reviewing, and testing.  I would like to thank the following folks:

Vikas Anand, Deepak Arora, Partricio Barletta, Heidi Buelow, Christopher Karl Chan, Manoj Das, Andrew Dorman, Pete Farkas, Mark Foster, Simone Geib, Kim LiChong, Ralf Mueller, Bhagat Nainani, Sabha Parameswaran, Robert Patrick, David Read, Derek Sharpe, Sushil Shukla, Kavitha Srinivasan, Meera Srinivasan, Will Stallard and Shumin Zhao.

I sincerely hope that I have not forgotten anyone, but if I have, the error is entirely mine.

This whitepaper is meant to compliment the Performance and Tuning Guide in the Fusion Middleware documentation.  Readers should also consult the excellent whitepaper on purging SOA/BPM 11g databases by Michael Bousamra with Deepak Arora and Sai Sudarsan Pogaru which is available on OTN (here).

For those with an interest in BPM 10g, I remind you of our previously published BPM 10g Performance Tuning whitepaper, which continues to be available on OTN (here).

You can find the beta exam here.

When something goes wrong, it is important to know how to collect the diagnostic information that will be needed to analyse the problem, work out the root cause, and come up with a resolution. In some cases, you may be able to do this analysis yourself. In other cases you may need to involve specialists like network engineers, directory administrators, or Oracle Support, for example.

Let’s take a look at the kinds of diagnostic information that may be needed. Of course, it may not be necessary to collect all of these for any given issue. If you are unsure, then it is a good idea to collect them anyway, just in case you need them.

Note: The purpose of this article is to tell you how to collect the data, not how to analyse it. Sometimes that analysis requires specialist skills and experience, but even then, those specialists rely on having access to the data.

BPM server/cluster configuration files

The first thing that you will want to collect is the configuration files for your environment. There are many different types of configurations that are possible, and these files contain the information necessary for someone to understand exactly how your particular environment is configured.

These files are located inside your WebLogic domain’s home directory, in the config directory. You will see files and directories like this:

config
|-- config.xml
|-- configCache
|-- deployments
|-- diagnostics
|-- fmwconfig
|-- jdbc
|-- jms
|-- nodemanager
|-- security
`-- startup

You can just zip up this whole directory to collect the files. You might use a command like this for example:

tar xzvf bpm_config.tar.gz /home/oracle/fmwhome/user_projects/domains/base_domain/config

Note: All of the examples in this post show the Oracle Middleware home as /home/oracle/fmwhome and the WebLogic domain name as base_domain. You will need to adjust these to suit your own environment.

The next useful piece of information to capture is a list of which patches (if any) you have installed in your environment. The best way to collect this information is to capture the output of the opatch lsinventory command. You should run this twice, first with ORACLE_HOME set to the Oracle_SOA1 directory under your install directory, and second with it set to the oracle_common directory under your install directory.

The example below shows running the opatch lsinventory command for ORACLE_HOME=/home/oracle/fmwhome/Oracle_SOA1 and the output, which in this case shows that no patches have been installed. In this example, you would also run it again with ORACLE_HOME=/home/oracle/fmwhome/oracle_common.

[oracle@ps5 Oracle_SOA1]$ export ORACLE_HOME=/home/oracle/fmwhome/Oracle_SOA1
[oracle@ps5 Oracle_SOA1]$ export PATH=$ORACLE_HOME/OPatch:$PATH
[oracle@ps5 Oracle_SOA1]$ opatch lsinventory
Oracle Interim Patch Installer version 11.1.0.9.0
Copyright (c) 2011, Oracle Corporation.  All rights reserved.

Oracle Home       : /home/oracle/fmwhome/Oracle_SOA1
Central Inventory : /home/oracle/oraInventory
   from           : /home/oracle/fmwhome/Oracle_SOA1/oraInst.loc
OPatch version    : 11.1.0.9.0
OUI version       : 11.1.0.9.0
OUI location      : /home/oracle/fmwhome/Oracle_SOA1/oui
Log file location : /home/oracle/fmwhome/Oracle_SOA1/cfgtoollogs/opatch/opatch2012-12-20_11-18-33AM_1.log

Patch history file: /home/oracle/fmwhome/Oracle_SOA1/cfgtoollogs/opatch/opatch_history.txt

OPatch detects the Middleware Home as "/home/oracle/fmwhome"

Lsinventory Output file location : /home/oracle/fmwhome/Oracle_SOA1/cfgtoollogs/opatch/lsinv/lsinventory2012-12-20_11-18-33AM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1): 

Oracle SOA Suite 11g                                                 11.1.1.6.0
There are 1 products installed in this Oracle Home.

There are no Interim patches installed in this Oracle Home.

--------------------------------------------------------------------------------

OPatch succeeded.

BPM log files

The information we have collected already is generic in nature and is used to ensure the domain configuration is correct and there are no obvious problems. From this point on, we are looking at information that is used to analyse a specific problem.

The server log and ‘out’ files are often the very first place we will look when there is a problem. These files will usually contain error messages that ,will give some information about the cause of the problem.

You can use a command like this to collect the logs. Remember to collect the logs from your AdminServer and each of your managed servers.

tar xzvf soa_server1_logs.tar.gz /home/oracle/fmwhome/user_projects/domains/base_domain/servers/soa_server1/logs

This will also collect the diagnostic_images if there are any available. These provide additional information about certain problems.

It is important to understand that a problem may occur only on one server, or on a number of servers. This is why it is important to collect the logs from all of the servers. Sometimes it is necessary to analyse data from several sources in order to understand what was happening in the environment.

Sometimes, during the analysis of a problem, you may be asked to turn on some debug/trace settings and attempt to recreate the problem. If this happens, the output from those traces almost always end up in these logs.

Incident logs

WebLogic collects some data by default when various ‘incidents’ occur, for example when a ‘stuck thread’ is encountered. The data collected depends on the incident, but it usually contains things like thread dumps, logs, and error messages.

These data are stored inside the server directories in your domain directory. To collect them, you could use a command like the example below. Remember to collect the incident logs for you AdminServer and each of your managed servers.

tar xzvf incident_logs.tar.gz /home/oracle/fmwhome/user_projects/domains/base_domain/servers/soa_server1/adr/diag/ofm/base_domain/soa_server1/incident

Thread dumps

A thread dump is a snapshot of what is happening in the server at a particular point in time. It allows us to see what each thread in the server process is doing. This information is helpful to understand how the server is behaving and what it is doing.

You can take a thread dump in a variety of ways, and how you do it depends on your operating system, how you started the server, e.g. whether you started it from a command line or the node manager, and if the server has become unresponsive.

Here are some of the common ways to take a thread dump:

• Pressing Ctrl-Break on Windows, or Ctrl-\ on Linux/Solaris/etc. in the window running the WebLogic process (in the foreground),
• Sending signal 3 (SIGQUIT) to the process (kill -3 PID),
• Connecting to the process with a utility like jvisualvm and pressing the Thread Dump button:
• Requesting a thread dump in the WebLogic Server console by navigating to the server, then the Monitoring tab and the Threads sub-tab and pressing the Dump Thread Stacks button:
• Use jstack PID (or jrcmd PID print_threads for JRockit).

Most of the time, more than one thread dump will be required. A series of thread dumps over some time period are needed in order to understand how the server is behaving over time. For example, a thread dump might show that a particular thread is ‘stuck’. Another (later) thread dump will be needed to see if that thread becomes unstuck by itself later on (as commonly happens) or not. Thus the two thread dumps together would be necessary to determine if the stuck thread was a problem or not.

It is also important to take thread dumps on all of the servers that are (or could possibly be) affected by or contributing to the problem. If in doubt, take thread dumps on all of the servers.

As a general rule of thumb, you should take five thread dumps over a period of time. How do you work out a suitable period of time? If you have a specific problem, for example you see some error message and then a minute later all of your servers become unresponsive, then the time period is that minute. Take a thread dump when you first see the error message appear, then one every 20 seconds (or so). If you don’t have any way to guess the suitable time period, just take them a minute apart.

The example below shows what the output from the thread dump looks like. Note that many lines have been removed from this output.

2012-12-31 10:26:12
Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.10-b01 mixed mode):

"JMX server connection timeout 48" daemon prio=10 tid=0x00007fabf8006800 nid=0x232f in Object.wait() [0x00007fac330b4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000000f61ad8c0> (a [I)
	at com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:150)
	- locked <0x00000000f61ad8c0> (a [I)
	at java.lang.Thread.run(Thread.java:662)

(many lines deleted)

"main" prio=10 tid=0x00007facc4008800 nid=0x21db in Object.wait() [0x00007facc9f38000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000000e0bb22a0> (a weblogic.t3.srvr.T3Srvr)
	at java.lang.Object.wait(Object.java:485)
	at weblogic.t3.srvr.T3Srvr.waitForDeath(T3Srvr.java:981)
	- locked <0x00000000e0bb22a0> (a weblogic.t3.srvr.T3Srvr)
	at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:490)
	at weblogic.Server.main(Server.java:71)

"VM Thread" prio=10 tid=0x00007facc406e000 nid=0x21e4 runnable 

"GC task thread#0 (ParallelGC)" prio=10 tid=0x00007facc401b800 nid=0x21dc runnable 

"GC task thread#1 (ParallelGC)" prio=10 tid=0x00007facc401d800 nid=0x21dd runnable 

"VM Periodic Task Thread" prio=10 tid=0x00007facc40ad000 nid=0x21eb waiting on condition 

JNI global references: 1601

Heap
 PSYoungGen      total 111744K, used 71412K [0x00000000f5560000, 0x00000000fdaa0000, 0x0000000100000000)
  eden space 89856K, 76% used [0x00000000f5560000,0x00000000f985f160,0x00000000fad20000)
  from space 21888K, 12% used [0x00000000fc540000,0x00000000fc7fe198,0x00000000fdaa0000)
  to   space 23296K, 0% used [0x00000000fad20000,0x00000000fad20000,0x00000000fc3e0000)
 PSOldGen        total 174784K, used 65947K [0x00000000e0000000, 0x00000000eaab0000, 0x00000000f5560000)
  object space 174784K, 37% used [0x00000000e0000000,0x00000000e4066c60,0x00000000eaab0000)
 PSPermGen       total 131072K, used 125060K [0x00000000d0000000, 0x00000000d8000000, 0x00000000e0000000)
  object space 131072K, 95% used [0x00000000d0000000,0x00000000d7a211b8,0x00000000d8000000)

Heap dumps

Another kind of dump that may be required for some problems is a heap dump. A heap dump is essentially a copy of everything that the JVM has in memory (in the heap) at a particular point in time. These are usually going to be pretty big files – they will be at least as big as the amount of used heap. So if you are running your BPM managed server with an 8GB heap, and it is 75% in use when you take the heap dump, then the heap dump is going to be about 6GB in size.

Heap dumps are used to look at the contents of the JVM’s memory in detail. They allow us to look at every object in the JVM and see the state of those objects.

Heap dumps are often used to diagnose a class of problems called ‘memory leaks’. While a single heap dump can lead us to suspect a memory leak, two heap dumps (from the same JVM at different times) are needed to confirm that a memory leak actually exists.

Heap dumps are also useful for other kinds of problems, where we need to look at the contents of various objects to understand what the server is doing.

It is a good practice to collect heap dumps when problems occur, but you should not send them to Oracle unless they are requested. Since they are so large, you may also wish to compress them and delete them after the problem they relate to has been resolved.

You can generate a heap dump from a tool like jvisualvm (by pressing the Heap Dump button) as shown below:

You can also collect a heap dump using jmap using a command like the one below:

jmap -dump:format=b,file=heap_dump_1.bin pid

If the problem is suspected to be a memory leak, you may be asked to carry out the following steps:

• allow the server to come to a steady state after startup,
• perform six full garbage collections (by pressing the Perform GC button, next to the Heap Dump button, six times),
• take a heap dump,
• attempt to reproduce the issue, i.e. do whatever it is you do to make the problem occur,
• take another heap dump.

Another good practice is to ensure that you have configured WebLogic to automatically take a heap dump if it runs out of memory. This is done by adding the following parameter to the JVM:

-XX:+HeapDumpOnOutOfMemoryError

This setting often saves a lot of pain – if your server crashes because it ran out of memory, then this setting is pretty likely to capture the information needed to work out what went wrong. If you do not have this setting, you would need to add it, and wait for the problem to happen again. It is safe to have this setting on all of your production servers. Note that it takes some time to take a heap dump (how long depends on the size of the heap and the speed of your disks) so there is a trade-off here – collecting the information needed to fix the problem will mean that your server restart will take a bit longer, as you will have to wait for the heap dump to finish before you restart the server(s).

Garbage Collection logs

Garbage collection logs are very useful for analysing memory related issues. The JVM will not produce these logs by default, you need to tell it to produce them.

These three settings will cause the JVM to print out more detailed information about garbage collection and to produce a log (called gc.log in this example) that contains garbage collection statistics and information that is very useful when trying to do some JVM tuning:

    -XX:+PrintGCTimeStamps
    -XX:+PrintGCDetails
    -Xloggc:gc.log

And, as mentioned in the previous section, it is also a good idea to turn on this setting:

    -XX:+HeapDumpOnOutOfMemoryError

These settings are safe to leave on all the time in your production environment.

Database information – AWR reports

Many performance related issues may have to do with the underlying database. For this reason, it is important to capture some information about the database performance as well. You should collect the AWR reports for the same period during which you observed the problem in BPM. To be on the safe side, start a little earlier and end a little later. For example, if the problem occurred from 10am until noon, you might collect AWR reports from 9am to 1pm.

You can find more information about what AWR reports are and how to collect them in this post.

HTTP Server logs

For some kinds of problems, it is useful to see the logs from the HTTP Server (if any) which is in front of your BPM server or cluster. These are often useful if you are getting refused connections for example.

You should gather the following logs:

  access.log  
  error.log

If you are using Oracle Web Tier (or Oracle HTTP Server), these logs will be located in the following directory, assuming your Oracle Web Tier Home is /home/oracle/httphome and you used the default names for the instance:

/home/oracle/httphome/Oracle_WT1/instances/instance1/diagnostics/logs/OHS/ohs1

Debug logs for the WebLogic plugin may also be useful if you are seeing nodes being evicted from the cluster or if you suspect that the cluster is unbalanced – e.g. you can see a different number of sessions on each node in the cluster.

To obtain these, you need to set DEBUG=ALL in the httpd-vhosts.conf file. This will produce a log called wlproxy.log.

Operating system level information

Sometimes performance information from the operating system level can be helpful as well. You might want to consider using tools like top or prstat (with thread/’lightweight process’ support), sar, vmstat, mpstat, iostat, and netstat. If you are have a possibly network-related issue, for example loss of communications between cluster members, then tcpdump may also capture useful information.

Remember, if you are running a cluster, you would need to collect these on all nodes in the cluster at the same time.

Java information

There are also several Java tools that can help you to collect additional information. If you are not familiar with these, it might be a good idea to explore what they can do for you. I would suggest looking at jps, jstat, jinfo, jstack, jmap, and jtop.

How to send information to Oracle Support

If you need help with the problem, you should contact Oracle Support and open a Service Request (SR). The Oracle Support system will allow you to upload attachments to the SR so that you can provide information you have collected. If the files are large, like a heap dump for example, then you should upload them to Oracle Support’s FTP server instead. Support will give you instructions on how to access the ftp server and where to put your files.

We have worked with Support to fix this problem, and I am happy to say that the patch is available to download from Oracle Support now.  I hope you did not have too much inconvenience.

The Patch number is 13088538: NPE IN O.BPM.UI.LAYOUT.MIGLAYOUT:114.

Packt’s new Oracle SOA Suite 11g Administrator’s Handbook by Ahmed Aboulnaga and Arun Pareek is packed full of essential information for the Oracle SOA administrator, in fact I would go so far as to say that it should be required reading for administrators who are new the the Oracle SOA Suite platform.  I think that reading it would greatly shorten the learning curve and help new administrators avoid many common problems or points of confusion.

More so than any other single piece of content that I have seen on the topic, it provides the information that a SOA administrator needs to know in order to successfully configure, manage, monitor, troubleshoot and backup an Oracle SOA environment.

It is clear and to the point, it presents just the information that you need, and the information is easy to find.  It is not cluttered up with a whole bunch of extra information you don’t need.  It is detailed and technical – providing information that you can use.  I think the book is not only a great introduction for a new administrator who needs to get a feel for Oracle SOA Suite, but it is also a great reference volume to keep on hand, even for experienced administrators.

It is obvious when reading the book that the authors have extensive experience and that they know what is important to their audience.  I have been working with Oracle SOA Suite for several years now, since 10g days, and I am one of the authors of the official Oracle SOA Suite Certification question base, and even I learned things from this book that I did not know.

The book covers topics like managing the SOA infrastructure, managing composite applications, monitoring SOA Suite, tuning, configuration and administration, troubleshooting, security policies, managing MDS and the dehydrations store and backup and recovery.

The bonus online chapter covers important issues like patching, upgrading from 10g, cluster configuration and silent (scripted) installation.

I for one will be keeping this book on my book shelf and I highly recommend it to anyone interested in or working with Oracle SOA Suite, in an administration capacity, or who just wants to know more about the product in general.

Development is done with JDeveloper and ADF and leverages Java and HTML 5 technologies, while keeping the same visual and declarative approach ADF is known for.

You can read more about the Oracle ADF Mobile release here and learn more on its OTN page here.

To celebrate, Packt are inviting anyone already registered to www.packtpub.com, or who registers before 30th September 2012, to download any one of their eBooks for free. Packt is also opening its online library for a week for free to members, offering customers an easy to way to research their choice of free eBook.

Further details of the event can be found in the Press Release.

If you think a free eBook sounds like a good idea, head over to this link: http://www.packtpub.com/login.

Two of your humble RedStack bloggers are proud to announce our very first book – Oracle BPM Suite 11g: Advanced BPMN Topics with Packt Publishing.

This is a concise presentation of both theory and practical examples of the areas of BPMN where we have encountered the most widespread confusion and misunderstanding.  So we hope this book will help in some small way.

Here is a quick overview of the book:

Chapter 1, Inter-process Communication introduces us to the theory of how processes
can communicate with each other and with other components. A number of topics
are covered such as: conversations—what they are, the default and advanced
conversations. We discuss correlation—automatic and message based, correlation
sets and keys, and correlation inside loops and when there are multiple calls. Throw
and catch events, send and receive tasks, and when to use each are examined. We
compare messages, signals, and errors. Sub-processes are explored—embedded,
multi-instance, and reusable, and when to use each.

Chapter 2, Inter-process Communication in Practice presents a series of practical
exercises to help you to explore the theory present in Chapter 1, Inter-process
Communication. The examples include communicating between processes using
messages and correlation, using correlation inside loops, communication between
processes using signals, and reusable sub-processes.

Chapter 3, Working with Arrays presents both theory and several practical exercises on
handling arrays in BPM. Topics include data association, creating an empty array,
creating an array with empty elements, creating an initialized array, getting an element
from an array, setting an element in an array, appending elements to an array, joining
arrays, removing elements from an array, and iterating over arrays—cardinality and
collections, sequential and parallel, completion conditions, and scope.

Chapter 4, Handling Exceptions discusses the theory behind handling exceptions in
BPM. Topics include business and system exceptions, boundary events, event subprocesses, exception propagation with embedded sub-processes, call, throw and
send, and how BPM exceptions affect the SCA composite.

Chapter 5, Handling Exceptions in Practice will guide us through a number of
practical examples that help to reinforce the theory in Chapter 4, Handling Exceptions.
The examples include implementing a timeout use case with boundary events,
implementing a “cancel message” use case, using event sub-processes, and exploring
exception propagation in peer processes.

We hope you enjoy it!

I have elaborated on the above subject here.

Hope this helps.

Kavitha

You can get a preview from these slides and video on OTN.

I made it format the documents nicely with proper indentation, and with line numbers and a nicer editor.  It also will warn you if you are about to open a large document so that you know it has not crashed, but that you just have to be patient.  And I added some icons and stuff.  There is even a nice Dora the Explorer picture hiding in there for those who care to look for it

Here is an example of looking at a composite (Arrays).  You can see all the files in the composite on the left hand side, and I have a BPMN process open in the editor pane on the right.  This is connected to a database MDS under BPM 11.1.1.6.  It also allows you to connect to a file-based MDS.

Note that you can go ahead and edit the files directly in here and save the changes to MDS – if you are game!

If you want to play with this, you can grab it from my branch in Olivier’s Subversion repository on java.net – it has IDE files for NetBeans 7.2 in that branch.

Again, this is not as wonderful as I would like it to be, but here it is, in case it may help someone.  It is also in Subversion on java.net (may be newer).

# Copyright 2012 Oracle Corporation.
# All Rights Reserved.
#
# Provided on an 'as is' basis, without warranties or conditions of any kind,
# either express or implied, including, without limitation, any warranties or
# conditions of title, non-infringement, merchantability, or fitness for a
# particular purpose. You are solely responsible for determining the
# appropriateness of using and assume any risks. You may not redistribute.
#
# Please refer to http://redstack.wordpress.com/continous-integration for details.
#
# This WLST script can be used as part of a continuous integration build process
# before deploying a SCA composite, to create any necessary JMS Queues
# on the WebLogic Server.
#
# In addition to creating the queue, this script will also update the
# resource adapter and redeploy it.
#
# Parts of this script were adapted from Richard van den Berg's post at
# https://forums.oracle.com/forums/message.jspa?messageID=10131949#10131949

import time

#
# These are the parameters that you need to edit before running this script
#

# admin server url
url                  = 't3://localhost:7001'
# username to connect to the admin server
username             = 'weblogic'
# password to connect to the admin server
password             = 'welcome1'
# the name for the EIS - as defined in the JMS Adapter wizard in JDEV
eisName              = 'eis/wls/redstackQueue'
# the admin or managed server to target where the JmsAdapter is deployed
serverName           = 'soa_server1'
nmServerName         = 'soa_server1'
# the name for the queue
qName               = 'redstackQueue'
# the JNDI name for the queue
jndiName             = 'jms/redstackQueue'
# the JMS Module for the queue
jmsModule            = 'SOAJMSModule'
# the Subdeployment for the queue
jmsSubDeployment     = 'SOASubDeployment'
# the host where node manager is running
nmHost               = 'localhost'
# the port to connect to node manager (5556 is default for plain mode)
nmPort               = '5556'
# the user to connect to node manager
nmUser               = 'weblogic'
# the password to connection to node manager
nmPassword           = 'welcome1'
# the name of the weblogic domain
domain               = 'base_domain'

# don't change these ones
uniqueString         = ''
appName              = 'JmsAdapter'
moduleOverrideName   = appName+'.rar'
moduleDescriptorName = 'META-INF/weblogic-ra.xml'

#
# method definitions
#

def makeDeploymentPlanVariable(wlstPlan, name, value, xpath, origin='planbased'):
  """Create a varaible in the Plan.
  This method is used to create the variables that are needed in the Plan in order
  to add an entry for the outbound connection pool for the new data source.
  """

  try:
    variableAssignment = wlstPlan.createVariableAssignment(name, moduleOverrideName, moduleDescriptorName)
    variableAssignment.setXpath(xpath)
    variableAssignment.setOrigin(origin)
    wlstPlan.createVariable(name, value)

  except:
    print('--> was not able to create deployment plan variables successfully')

def main():

  print ' Copyright 2012 Oracle Corporation. '
  print ' All Rights Reserved. '
  print ''
  print ' Provided on an ''as is'' basis, without warranties or conditions of any kind, '
  print ' either express or implied, including, without limitation, any warranties or '
  print ' conditions of title, non-infringement, merchantability, or fitness for a '
  print ' particular purpose. You are solely responsible for determining the '
  print ' appropriateness of using and assume any risks. You may not redistribute.'
  print ''
  print ' Please refer to http://redstack.wordpress.com/continous-integration for details.'
  print ''
  print ' This WLST script can be used as part of a continuous integration build process'
  print ' before deploying a SCA composite, to create any necessary JMS queues'
  print ' on the WebLogic Server.'
  print ''
  print ' In addition to creating the queue, this script will also update the '
  print ' resource adapter and redeploy it.'
  print ''
  print ' !!! WARNING !!! WARNING !!! WARNING !!! WARNING !!! WARNING !!! WARNING !!!'
  print ''
  print ' This script will make changes to your WebLogic domain.  Make sure you know '
  print ' what you are doing.  There is no support for this script.  If something bad '
  print ' happens, you are on your own!  You have been warned.'

  #
  # generate a unique string to use in the names
  #

  uniqueString = str(int(time.time()))

  #
  # Create a JDBC Data Source.
  #

  try:
    print('--> about to connect to weblogic')
    connect(username, password, url)
    print('--> about to create a queue ' + qName)
    edit()
    startEdit()
    cd('/JMSSystemResources/' + jmsModule + '/JMSResource/' + jmsModule)
    cmo.createQueue(qName)
    cd('/JMSSystemResources/' + jmsModule + '/JMSResource/' + jmsModule + '/Queues/' + qName)
    set('JNDIName', jndiName)
    set('SubDeploymentName', jmsSubDeployment)
    save()
    print('--> activating changes')
    activate()
    print('--> done')

#
# update the deployment plan
#
    print('--> about to update the deployment plan for the DbAdapter')
    startEdit()
    planPath = get('/AppDeployments/JmsAdapter/PlanPath')
    appPath = get('/AppDeployments/JmsAdapter/SourcePath')
    print('--> Using plan ' + planPath)
    plan = loadApplication(appPath, planPath)
    print('--> adding variables to plan')

    makeDeploymentPlanVariable(plan, 'ConnectionInstance_eis/wls/' + qName + '_JNDIName_' + uniqueString, eisName, '/weblogic-connector/outbound-resource-adapter/connection-definition-group/[connection-factory-interface="oracle.tip.adapter.jms.IJmsConnectionFactory"]/connection-instance/[jndi-name="' + eisName + '"]/jndi-name')
    makeDeploymentPlanVariable(plan, 'ConfigProperty_ConnectionFactoryLocation' + uniqueString, eisName, '/weblogic-connector/outbound-resource-adapter/connection-definition-group/[connection-factory-interface="oracle.tip.adapter.jms.IJmsConnectionFactory"]/connection-instance/[jndi-name="' + eisName + '"]/connection-properties/properties/property/[name="ConnectionFactoryLocation"]/value')
    print('--> saving plan')
    plan.save();
    save();
    print('--> activating changes')
    activate(block='true');
    cd('/AppDeployments/JmsAdapter/Targets');
    print('--> redeploying the JmsAdapter')
    print(appName, planPath)
    redeploy(appName, planPath, targets=cmo.getTargets());
    print('--> done')

  except:
    print('--> something went wrong, bailing out')
    stopEdit('y')
    raise SystemExit

  #
  # disconnect from the admin server
  #

  print('--> disconnecting from admin server now')
  disconnect()

  #
  # Restart the managed server using the node manager.
  #

  try:
    print('--> about to restart server ' + nmServerName)
    nmConnect(nmUser, nmPassword, nmHost, nmPort, domain)
    print('--> stopping server...')
    nmKill(nmServerName)
    print('--> starting server')
    nmStart(nmServerName)
    print('--> done')
    nmDisconnect()

  except:
    print('--> was not able to restart server successfully')

#
#  this is the main entry point
#

main()

Enjoy!

SOA/BPM Composites can contain dependencies on resources in the runtime environment, like JDBC Data Sources and JMS Queues for example.  This poses a challenge for us, as we need to ensure that these resources exist, are properly configured, and are available before we attempt to run our SCA Tests in our composite.

So this means we need a way to automate the creation of these resources, so that we can do this as part of our build process.

This post presents a first step in that direction – a WLST script that can automate the process for JDBC.  I say ‘first step’ for a couple of reasons:

• It only handles JDBC, nothing else yet,
• It is a bit delicate – I am not really happy with the error handling (which is almost non-existent, but you have to excuse me on that point, this is my very first Python program ever), and
• I have not made it handle parameters yet, or integrated it into Maven, although this should just be ‘a small matter of programming’ as the saying goes.

It also implements a restart of the managed server, and I am not 100% sure we actually need to do that.

Anyway, here it is in all its glory – also available from Subversion on java.net (may be newer):

# Copyright 2012 Oracle Corporation.
# All Rights Reserved.
#
# Provided on an 'as is' basis, without warranties or conditions of any kind,
# either express or implied, including, without limitation, any warranties or
# conditions of title, non-infringement, merchantability, or fitness for a
# particular purpose. You are solely responsible for determining the
# appropriateness of using and assume any risks. You may not redistribute.
#
# Please refer to http://redstack.wordpress.com/continous-integration for details.
#
# This WLST script can be used as part of a continuous integration build process
# before deploying a SCA composite, to create any necessary Java EE data sources
# on the WebLogic Server.
#
# In addition to creating the data source, this script will also update the
# resource adapter and redeploy it.
#
# Parts of this script were adapted from Richard van den Berg's post at
# https://forums.oracle.com/forums/message.jspa?messageID=10131949#10131949

import time

#
# These are the parameters that you need to edit before running this script
#

# admin server url
url                  = 't3://localhost:7001'
# username to connect to the admin server
username             = 'weblogic'
# password to connect to the admin server
password             = 'welcome1'
# the name for the EIS - as defined in the DB Adapter wizard in JDEV
eisName              = 'eis/db/myDS'
# the admin or managed server to target where the DbAdapter is deployed
serverName           = 'soa_server1'
nmServerName         = 'soa_server1'
# the name for the data source
dsName               = 'myDS'
# the JNDI name for the data source
jndiName             = 'jbdc/myDS'
# the database url for the data source
dbUrl                = 'jdbc:oracle:thin:@localhost:1521:orcl'
# the database user
dbUser               = 'mark'
# the database password
dbPassword           = 'welcome1'
# the database driver to use
dbDriver             = 'oracle.jdbc.xa.client.OracleXADataSource'
# the host where node manager is running
nmHost               = 'localhost'
# the port to connect to node manager (5556 is default for plain mode)
nmPort               = '5556'
# the user to connect to node manager
nmUser               = 'weblogic'
# the password to connection to node manager
nmPassword           = 'welcome1'
# the name of the weblogic domain
domain               = 'base_domain'

# don't change these ones
uniqueString         = ''
appName              = 'DbAdapter'
moduleOverrideName   = appName+'.rar'
moduleDescriptorName = 'META-INF/weblogic-ra.xml'

#
# method definitions
#

def makeDeploymentPlanVariable(wlstPlan, name, value, xpath, origin='planbased'):
  """Create a varaible in the Plan.
  This method is used to create the variables that are needed in the Plan in order
  to add an entry for the outbound connection pool for the new data source.
  """

  try:
    variableAssignment = wlstPlan.createVariableAssignment(name, moduleOverrideName, moduleDescriptorName)
    variableAssignment.setXpath(xpath)
    variableAssignment.setOrigin(origin)
    wlstPlan.createVariable(name, value)

  except:
    print('--> was not able to create deployment plan variables successfully')

def main():

  print ' Copyright 2012 Oracle Corporation. '
  print ' All Rights Reserved. '
  print ''
  print ' Provided on an ''as is'' basis, without warranties or conditions of any kind, '
  print ' either express or implied, including, without limitation, any warranties or '
  print ' conditions of title, non-infringement, merchantability, or fitness for a '
  print ' particular purpose. You are solely responsible for determining the '
  print ' appropriateness of using and assume any risks. You may not redistribute.'
  print ''
  print ' Please refer to http://redstack.wordpress.com/continous-integration for details.'
  print ''
  print ' This WLST script can be used as part of a continuous integration build process'
  print ' before deploying a SCA composite, to create any necessary Java EE data sources'
  print ' on the WebLogic Server.'
  print ''
  print ' In addition to creating the data source, this script will also update the '
  print ' resource adapter and redeploy it.'
  print ''
  print ' !!! WARNING !!! WARNING !!! WARNING !!! WARNING !!! WARNING !!! WARNING !!!'
  print ''
  print ' This script will make changes to your WebLogic domain.  Make sure you know '
  print ' what you are doing.  There is no support for this script.  If something bad '
  print ' happens, you are on your own!  You have been warned.'

  #
  # generate a unique string to use in the names
  #

  uniqueString = str(int(time.time()))

  #
  # Create a JDBC Data Source.
  #

  try:
    print('--> about to connect to weblogic')
    connect(username, password, url)
    print('--> about to create a data source ' + dsName)
    edit()
    startEdit()
    cmo.createJDBCSystemResource(dsName)
    cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName)
    cmo.setName(dsName)
    cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDataSourceParams/' + dsName)
    set('JNDINames',jarray.array([String(jndiName)], String))
    cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDriverParams/' + dsName)
    cmo.setUrl(dbUrl)
    cmo.setDriverName(dbDriver)
    cmo.setPassword(dbPassword)
    cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCConnectionPoolParams/' + dsName)
    cmo.setTestTableName('DUAL')
    cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDriverParams/' + dsName + '/Properties/' + dsName)
    cmo.createProperty('user')
    cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDriverParams/' + dsName + '/Properties/' + dsName + '/Properties/user')
    cmo.setValue(dbUser)
    cd('/JDBCSystemResources/' + dsName + '/JDBCResource/' + dsName + '/JDBCDataSourceParams/' + dsName)
    cmo.setGlobalTransactionsProtocol('TwoPhaseCommit')
    cd('/JDBCSystemResources/' + dsName)
    set('Targets',jarray.array([ObjectName('com.bea:Name=' + serverName + ',Type=Server')], ObjectName))
    save()
    print('--> activating changes')
    activate()
    print('--> done')

#
# update the deployment plan
#
    print('--> about to update the deployment plan for the DbAdapter')
    startEdit()
    planPath = get('/AppDeployments/DbAdapter/PlanPath')
    appPath = get('/AppDeployments/DbAdapter/SourcePath')
    print('--> Using plan ' + planPath)
    plan = loadApplication(appPath, planPath)
    print('--> adding variables to plan')
    makeDeploymentPlanVariable(plan, 'ConnectionInstance_eis/DB/' + dsName + '_JNDIName_' + uniqueString, eisName, '/weblogic-connector/outbound-resource-adapter/connection-definition-group/[connection-factory-interface="javax.resource.cci.ConnectionFactory"]/connection-instance/[jndi-name="' + eisName + '"]/jndi-name')
    makeDeploymentPlanVariable(plan, 'ConfigProperty_xADataSourceName_Value_' + uniqueString, eisName, '/weblogic-connector/outbound-resource-adapter/connection-definition-group/[connection-factory-interface="javax.resource.cci.ConnectionFactory"]/connection-instance/[jndi-name="' + eisName + '"]/connection-properties/properties/property/[name="xADataSourceName"]/value')
    print('--> saving plan')
    plan.save();
    save();
    print('--> activating changes')
    activate(block='true');
    cd('/AppDeployments/DbAdapter/Targets');
    print('--> redeploying the DbAdapter')
    redeploy(appName, planPath, targets=cmo.getTargets());
    print('--> done')

  except:
    print('--> something went wrong, bailing out')
    stopEdit('y')
    raise SystemExit

  #
  # disconnect from the admin server
  #

  print('--> disconnecting from admin server now')
  disconnect()

  #
  # Restart the managed server using the node manager.
  #

  try:
    print('--> about to restart server ' + nmServerName)
    nmConnect(nmUser, nmPassword, nmHost, nmPort, domain)
    print('--> stopping server...')
    nmKill(nmServerName)
    print('--> starting server')
    nmStart(nmServerName)
    print('--> done')
    nmDisconnect()

  except:
    print('--> was not able to restart server successfully')

#
#  this is the main entry point
#

main()

To use this script, you need to go edit those parameters at the top of the program, save it somewhere, e.g. createDS.py, and then run it with WLST, for example:

cd /your/fmwhome/wlserver_10.3/common/bin/
wlst.sh /path/to/createDS.py

Please let me know if you have any success, or comments, I would love to hear them.

I must also acknowledge the following folks from whom I borrowed code and ideas: Richard van den Berg and Michel Schildmeijer and in turn those who they borrowed from, and so on.

When you are running a number of Fusion Middleware environments in the same IP subnet, as you might be in a test environment for example, the default Coherence startup parameters can cause you some issues.

A quick and easy way to avoid these issues, is to change the Coherence clusterport setting, and add the Coherence ttl setting, as shown in the example below.  You can do this by updating the EXTRA_JAVA_PROPERTIES variable in the setDomainEnv.sh script inside your domain’s bin directory.

EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES}
  -Dsoa.archives.dir=${SOA_ORACLE_HOME}/soa
  -Dsoa.oracle.home=${SOA_ORACLE_HOME}
  -Dsoa.instance.home=${DOMAIN_HOME}
  -Djava.net.preferIPv4Stack=true
  -Dweblogic.security.SSL.ignoreHostnameVerification=true
  -Dhttp.nonProxyHosts=localhost|10.240.37.80|slc01ats|*.oracle.com
  -Dtangosol.coherence.clusteraddress=227.7.7.9
  -Dtangosol.coherence.clusterport=9838
  -Dtangosol.coherence.ttl=0
  -Dtangosol.coherence.log=jdk
  -Djavax.xml.soap.MessageFactory=oracle.j2ee.ws.saaj.soap.MessageFactoryImpl
  -Dweblogic.transaction.blocking.commit=true
  -Dweblogic.transaction.blocking.rollback=true
  -Djavax.net.ssl.trustStore=${WL_HOME}/server/lib/DemoTrust.jks"
export EXTRA_JAVA_PROPERTIES

Ideally, you want to set tangosol.coherence.clusterport to a different value for each separate cluster.  Set tangosol.coherence.ttl to 0 as shown above.

I don’t particularly like the new name, since it implies that this package is only useful for installing the Oracle Database 11gR2, and that is not really the case – this package is also very useful to install before you attempt to install many Fusion Middleware products on Linux, so I must say that I like the old name better.

But, you can’t stop progress!

Anyway, the new name is as follows:

oracle-rdbms-server-11gR2-preinstall

You should install this package (using yum) before you try to install any Fusion Middleware products on Oracle Linux – it will save you a lot of trouble.  You can find more information about the package here.

Also, in case you were not aware, there is now a public yum repository for Oracle Linux – you can find config files and details here.

If you are using Maven to manage your build, it would be ideal to be able to use Maven coordinates to reference these Oracle artifacts, just like you do for any other kind of dependency.  But Oracle does not provide POM files for these artifacts – so what can we do?

Maven provides the ability to load any artifact into a Maven repository and to give it a set of coordinates.  This means that you can easily add the Oracle provided artifacts to your own Maven repository, or to a shared repository in your environment (e.g. a Maven Repository Manager) so that they can be used by yourself and other developers.

Maven will even generate the POM file for you, so all you need to do is create the coordinates.  Normally coordinates follow the same reverse URL scheme that Java packages uses, but it might be a good idea not to use com.oracle in this case – as it is possible that Oracle might one day release their own POMs, and then you could have some name clashes.  So it is probably better to use a different name, you might add ‘custom’ for example.

Suppose you want to add the wlthint3client.jar into your Maven environment, and give it the following coordinates:

• groupId: custom.com.oracle
• artifactId: wlthint3client
• versionId: 10.3.6

You could use a command like this (which would be entered all on one line):

mvn install:install-file 
    -Dfile=/fmwhome/wlserver_10.3/server/lib/wlthint3client.jar
    -DgeneratePom=true
    -DgroupId=custom.com.oracle
    -DartifactId=wlthint3client
    -DversionId=10.3.6
    -Dpackaging=jar

This will install the JAR into your local Maven repository.  You can use a different goal to install into a remote repository (like a Maven Repository Manager for example).  In this command you either need to give the repository URL in the url property, or specify the repositoryId the matches the repository definition in your Maven settings.xml.

mvn deploy:deploy-file
    -Dfile=/fmwhome/wlserver_10.3/server/lib/wlthint3client.jar
    -DgeneratePom=true
    -DrepositoryId=internal
    -DgroupId=custom.com.oracle
    -DartifactId=wlthint3client
    -DversionId=10.3.6
    -Dpackaging=jar

After running these commands, you will have this artifact in your Maven repository and you will be able to use it like any other artifact.  For example, to include it in a project as a dependency, you can just refer to it using its coordinates:

<dependency>
  <groupId>custom.com.oracle</groupId>
  <artifactId>wlthint3client</artifactId>
  <version>10.3.6</version>
  <packaging>jar</packaging>
</dependency>

Armed with this knowledge you could go and create POMs for any Oracle (or other) artifacts that you want to use in your build.  You could also create dependency hierarchies between them if you wanted or needed to.

A Maven Repository is a collection of artifacts and metadata stored in a defined hierarchy.  It could be as simple as just a collection of files in a file system with a particular directory structure, or on a simple web server.  This might work well for a single developer’s local repository (in the case of the files), or for read only artifacts (in the case of the web server) but it is not so ideal for a group of developers who are both consuming and producing artifacts.

A Maven Repository Manager is a piece of software that allows you to host and manage a number of Maven Repositories.  Why would you want more than one?  Usually people like to keep their work in progress in a separate repository to their completed artifacts – perhaps they have different backup policies or access controls.  Also, you can use a Maven Repository Manager as a proxy for external Maven Repositories (like Maven Central for example).  This allows you to cache artifacts and share them among your developers – reducing the build times and reducing your download costs too.  It is also handy if you need to be able to build without a connection to the Internet.

The diagram above shows the relationship between the various repositories.  Maven Central is the main repository.  It contains mostly free and open source artifacts.  It is basically read only to the normal Maven user.  There are also other Maven repositories on the Internet that hold both open source and commercial artifacts.

The box labeled ‘Internal Repository Manager’ represents a Maven Repository Manager deployed inside an organization.  Each developer will also have their own local repository.

There are a number of open source and commercial products available that can act as the Maven Repository Manager.  Here are a few of the common ones:

• Apache Archiva,
• Artifactoryfrom JFrog, and
• Nexus from Sonatype.

Typically you would set up a number of repositories, for example you might have repositories for:

• Mirrors/proxies/caches of external repositories like Maven Central, java.net, etc.,
• Artifacts that are being developed by your organization, this is often called a ‘snapshot’ repository, and
• Finished artifacts that were developed by your organization, this is often called an ‘internal’ repository.

You may also want to create a repository that holds artifacts that developers can use as dependencies when working on projects.  If you want to keep tight control over the artifacts that can be used, for example to ensure your project does not use third party components with incompatible license terms, or to ensure everyone is using the same version of an important dependency, then forcing dependencies to come from your internal repository is a good approach to exercise this control.

When you are using continuous integration, you may end up with a lot of builds going on, and the built artifacts will typically be stored in your snapshot repository.  This can lead to a large amount of data in a fairly short time, even in a relatively modest development organization.  So snapshot maintenance is important.  Most of the Maven Repository Managers will allow you to configure how many snapshot versions of an artifact to keep, and will automatically purge old snapshots to conserve space.

I have blogged about some BAM best practices here..

Hope that helps.

If you hit this issue, you should log a service request with Oracle Support and ask for a patch for bug number 13088538.  You will need to provide details about your JDeveloper environment, versions of plugins, etc.

Performing action Save[ from oracle.bpm.fusion.studio.editor.ProcessEditor ]
 Invoking command: [ from oracle.bpm.fusion.studio.editor.ProcessEditor ]
 Performing action Rebuild BoundaryEvents.jpr[ from oracle.bpm.fusion.studio.editor.ProcessEditor ]
 Invoking command: modify adf-config for soa
 Invoking command: [ from oracle.bpm.fusion.studio.editor.ProcessEditor ]
 Invoking command:
 Invoking command:
 Invoking command:
 Invoking command:
 Uncaught exception
 java.lang.NullPointerException
   net.miginfocom.layout.Grid.addLinkIDs()
   net.miginfocom.layout.Grid.<init>()
   net.miginfocom.swing.MigLayout.checkCache()
   net.miginfocom.swing.MigLayout.getSizeImpl()
   net.miginfocom.swing.MigLayout.preferredLayoutSize()
   o.bpm.ui.layout.MigLayout.preferredLayoutSize(MigLayout.java:114)

To find the job, use a query like the one below.  Update the schema user to match the environment:

select
  job
, schema_user
, broken
, what
, interval
from
  dba_jobs
where
  schema_user = “DEV_SOAINFRA”

Look for the job which contains something like the following in the WHAT column.  Note that it may be different if a different schema prefix was used in the environment.

dbms_refresh.refresh('"DEV_SOAINFRA"."B2B_SYSTEM_MV"');

To remove the job, take note of the job number from the JOB column and then use a command like the following, substituting in the correct job number in place of 24:

begin
dbms_job.remove(24);
end;

Alternatively, alter the materialized view to be refreshed on demand, using a command similar to this:

alter materialized view dev_soainfra.b2b_system_mv refresh on demand;

I want to consider three products that are commonly clustered, and which have somewhat different requirements – SOA (or BPM), OSB and Coherence.  Let’s take each one in turn.

SOA and/or BPM

SOA and BPM support a domain with either exactly one (managed) server or exactly one cluster.  You cannot have two (or more) SOA/BPM clusters in a single WebLogic domain.  The SOA/BPM cluster is largely defined by the database schema, in particular the SOAINFRA schema, that each server is pointing too.  All servers/nodes in a cluster must be pointing to the same SOAINFRA schema.  And a SOAINFRA schema can only be used by nodes in a single cluster.

As an aside, if you were to point nodes from two SOA ‘clusters’ to the same SOAINFRA schema for some reason, you would basically end up with just one cluster – although it would be an unsupported configuration and a lot of things would break.

SOA/BPM clusters are usually created for one of two reasons – to add extra capacity, or to improve availability.  It is important to understand that SOA and BPM do not support dual site active/active deployment, i.e. you cannot have two clusters, each with their own SOAINFRA across two data centres with any kind of database level replication.  Within a single site though, all of the nodes are active and all share work between them.

To have a cluster, you need to have a load balancer in front of the SOA servers.  This could be a hardware or software load balancer.  It needs to be capable of distributing work across all of the nodes in the cluster.  Ideally, it should also be able to collect heartbeats or response times and use that information to route new sessions to servers which seem to be least busy.

When a BPEL or BPMN process is executing in a cluster, any time there is a point of asynchronicity, e.g. an invoke, onAlarm, onMessage, wait, catch event, receive task, explicit call to checkpoint(), etc., the process instance will be dehydrated.  It is possible, and even likely, that the process will be rehydrated later on a different node in the cluster.  The design of SOA/BPM means that any node can pick up any process instance and continue from where it last dehydrated. This makes it easy to dynamically resize the cluster, by adding and removing nodes, which will automatically take their share of the work.  It also makes handling the failure of a node in a cluster particularly straightforward, as the load balancer will notice that node has failed and stop routing it work.

You also need to make sure that all of your callbacks are using the (virtual) IP address of the load balancer, not of any particular server in the cluster.  This means that all callbacks can be handled by any node in the cluster.

As your SOA/BPM workload grows, you basically want to scale up your cluster by adding more nodes.  Any decision to split the cluster would most likely be based on separation of different business workloads, perhaps for reasons of maintainability – different release cycles, timing of patches, etc., rather than any technical reason to split the cluster.

I think the most important factor here is to realize that when you run multiple SOA clusters, they will each be in a different WebLogic domain, they will each have their own SOAINFRA (and other) schemas, they will have different addresses, and they will be running different workload, i.e. different composites.

It is really important to understand that it is not possible to have a particular process instance run part way on one cluster, and then complete on a different cluster – as the two clusters would have two totally separate SOAINFRA databases, and things like callbacks, process instance ID’s, messages, etc., would all be different.

So I think from the point of view of starting a clustered deployment, the simplest approach is to just build a single SOA cluster.  This will greatly reduce the administrative overhead of dealing with multiple clusters.

OSB

Now let’s consider OSB.  It is a little different to SOA/BPM.  It does not use a database to store state, as it is designed to handle stateless, short lived operations, so we don’t need to worry about sharing database schemas.

It turns out what we do need to worry about is how many artifacts we are deploying in the OSB cluster.  When you start up OSB, it goes through a process of ‘compiling’ all of the artifacts – that is the WSDLs, XSDs and all that – essentially into objects in memory.  If there are a lot of artifacts, or if they are particular complicated – they have a lot of attributes for example – or if they contain nested references, this ‘compilation’ can start to take a long time.  And the resulting data in memory can start to take up a lot of the heap.

A good rule of thumb is to say you probably want no more than 500 or so artifacts in a single OSB cluster.  Otherwise, the startup times become so long that outages to restart are prohibitively long.  Imagine if you had to wait an hour to start up your OSB cluster for example – would that be too long?  The amount of memory in the heap required to store all of these also grows, so you would end up with a whole bunch of memory hungry servers that take forever to start – not ideal at all.

So the best approach, in my opinion, is to split up your OSB workload into several OSB clusters – with each one having a reasonable amount of artifacts on it.  You can work out what is reasonable for you by looking at the startup time and the memory needs.

Now the next logical question is how do they talk to each other?  What if a proxy services on cluster A needs to talk to a business service on cluster B?  I have heard various approaches including departmental and enterprise service buses (hierarchical), and deploying all business services on all clusters, but splitting up the proxy services, or vice versa, and so on.

I think the best approach here is to have all request to OSB route through a load balancer, and use a simple set of rules on the load balancer to route to the correct cluster based on the service URL.  If you create small enough groups of services under different URL paths, this also makes it easy for you to relocate services between OSB clusters if necessary for any reason, without any impact to any of your service consumers.  This also makes it easy for services to talk to each other regardless of which cluster they are deployed on.

A good way to split things across the clusters, in my opinion, is to put critical things on their own clusters first, then basically divide up everything else across other clusters.  Having critical things on their own clusters helps you to manage their availability, patching, performance, etc., individually and prevents the situation of being unable to update something because it is in an environment shared with a critical component that cannot tolerate the update.

Coherence

Now we come to Coherence, which is different again.  As a general rule of thumb, it is ideal to have Coherence nodes have no more than 4GB of heap.  The data in Coherence clusters tends to stay around, so they are tuned differently to SOA/BPM (for example) where most of the data is short lived and rarely tenured.

For Coherence, having more nodes in the cluster is usually a good thing.  The other question is whether to split up Coherence clusters.  Again, I think the (most) right answer here is to make that decision in terms of separating logic business functionality when it makes sense.  Unless of course you get a really big cluster, then you might start to have some technical reasons to look at splitting it up.  But I only know of a couple of organizations who have Coherence clusters anywhere near big enough for that to be a concern.

A word about the Exalogic Enterprise Deployment Guide

A lot of Exalogic customers refer to the example topology in the Enterprise Deployment Guide.  That example is well suited to a large Java EE application deployed across a cluster of WebLogic Servers and Coherence nodes.  I think the EDG makes it pretty clear that this example is not meant to be for all scenarios, and I think when we consider SOA/BPM, OSB and Coherence, there are some compelling reasons why we might choose to go with a slightly different approach.

For example, if we just blindly followed that same approach for SOA and OSB clusters, we would probably end up with resource contention issues – not enough memory available and not enough cores available to run the number of JVMs we might come up with.

Recommended approach

Let’s pretend we have six machines on which to build our environment.  This could equally be six compute nodes in an Exalogic, or just six normal machines, it does not really matter for our purposes here.  For arguments sake, let’s say each one has 12 cores and 96GB of memory.

I think now is a good time for a picture!

Here are some important things to note about this approach:

• It does not make a lot of sense to have more JVMs than cores because they will just end up competing with each other for system resources.  So in the approach above we have no more than 9 JVMs on any compute node (1 SOA, 1 AdminServer, 2 OSB, 4 Coherence and 1 NodeManager (not shown but running on every compute node).  We could probably fit more, but as we will see later memory is also an important consideration.  Also, keep in mind that the operating system is going to use some of those cores as well, so you can’t really afford to allocate them all to JVMs.
• Let’s say we allocate 16GB of heap to each SOA and OSB managed server and 4GB to each Coherence server.  That means that with just these JVMs, we are potentially consuming 64GB of memory on each compute node.  This is two thirds of the available memory, a good rule of thumb high water mark.  Remember that there is also going to be other processes using memory, including the operating system, and of course, unless you are running JRockit, the JVM is going to have a permanent generation too, which will take up more memory.  Maybe 16GB is too high – you don’t have to use up all the memory you have of course, but I guess this is really going to depend on the nature of the workload, and as I said at the beginning, I am not trying to make a one size fits all recommendation here.
• The AdminServers for the various clusters are striped across the compute nodes.  The cluster can of course survive the loss of the AdminServer and it can be restarted by a NodeManager on another compute node.  But it just makes good sense to put them on different machines, so that in the event of the failure of one compute node, you would only lose one or zero of them, not all of them at once.
• All of the URLs that consumers use point to the load balancer – whether those consumers are on these compute nodes or external.  The load balancer decides where traffic is routed.  If we found that our payments and core services could no longer fit in a single OSB cluster, we could move one to the other cluster, or to a new cluster altogether without any impact on consumers.  All we would need to do is update the routing rule in the load balancer.
• All clusters are stretched across all compute nodes.  The idea here is to be able to get the best possible use of the available resources.  Of course this could be tuned to suit the actual workload and nodes may be added or removed.  Some managed servers may not be running, but the point is that each cluster (product) has the ability to run across all nodes.  So if any node were lost, it would not matter, all nodes are essentially equal.

Let’s consider for a moment an alternative.  Suppose we rearranged the OSB clusters so that all of the managed servers in OSB Cluster A are on compute nodes CN01, CN02 and CN03, and all the managed server in OSB Cluster B are on the other three.  What would happen if we needed OSB Cluster A to have more capacity?  Or what would happen if we lost CN01, or, god forbid, CN01, CN02 and CN03 all at once?  OSB Cluster A would be under-resourced in the former case, or completely unavailable in the latter.  We could not easily just start up OSB Cluster A on the remaining nodes, or add another server on one of those nodes.  This would require some manual effort – reconfiguration at the least, and possibly redeployment as well.

I think a key measure of the quality of an architecture is its simplicity.  The simplest architectures are the best.  No need to make things any more complicated than they need to be.  Complex architectures just introduce more opportunity for error and more management cost and inflexibility.

Another good test is flexibility.  This approach does not impose any arbitrary limits on how you could deploy your applications.

Availability is another factor to consider, and I think the approach described provides the best possible availability across the whole system from the available hardware – and the best hardware utilization as well.

What about patching?

Patching is a very important consideration, that should not be backed away from when designing your architecture.  How do you patch a cluster like this?  Especially if you cannot afford a long outage.

My suggested approach here is to have two sets of binaries, the active and the standby binaries, for each cluster.  The clusters would be running on the active binaries.  When you need to apply a patch to your production environment, after you have completed testing of the patch in non-production environments, of course, you should apply the patch to the standby binaries, and therefore to those domains created from the standby binaries, the standby domains.

Now, both the active domain and the standby domain (in the case of SOA) should be pointing to the same SOAINFRA database.  You would never run them both at the same time of course.  When the patching is completed, shut down the active domain and start up the (newly patched) standby domain, which is pointing at the same SOAINFRA and therefore will start up as logically the same cluster.  Now it is the active cluster, and the other one is the standby.  If it goes bad, just swap back.  When you are ready, go patch the standby too, to keep them in sync.

[If there are any other 'old mainframe guys' reading this, you might note the similarity to the zones in SMP/E.]

Update:  It seems that this approach still has some challenges, when you think about the fact that many patches might require running a database schema update, or a script (like a WLST script for example) which may require you to start up the server(s).  So I think for now we really need to take full backups before applying patches so that we can roll back if needed.  Although even then we need to make sure that we don’t let any messages come through when we are testing the patched domain, otherwise, we might loose work if we need to roll back!

What about patches that change the database schema?  In this case you are going to have to schedule an outage to do the patching, so that you have the opportunity to backup the database before applying the patch.  Trying to do it by just swapping would deprive you of the ability to roll back the patch by just swapping domains again.

Another important consideration is that there may be some files that need to be shared/moved/copied between the active and standby domains.  It would be important to keep a tight grasp on all configuration changes to make sure that any changes made since the last swap are applied to the other domain when a swap occurs.  It might be a good idea to swap weekly, just to make sure there is a formalized process around this, and things don’t get lost.

Summary

Well, that’s it.  I would like to acknowledge that most of this was built up over the course of many conversations with many people.  I certainly do not claim that all of this is my own original ideas, but rather a summary of the position I now hold based on many conversations with a bunch of smart folks.  I would especially like to thank Robert Patrick for his ideas and many discussions on this topic.  Also a special mention to Jon Purdy for his input on Coherence

As I said in the beginning, this is just my views and I would certainly be very interested to hear your feedback and to continue the discussion.

Asking around, I discovered something called a pointcut, a feature of aspect oriented programming.  This is a little out of my area of experience, but I like to think of it essentially as a mechanism that lets you define a piece of code to run when some particular thing happens.

So for example, I can say something like ‘whenever method X on object Y is executed, do Z’ which is great – that lets me solve my exact problem.

So let’s look at a simple example.  Let’s say for the sake of an example that I want to ensure that a particular ADF view is causing the method addPartialTriggerListeners() on object org.apache.myfaces.trinidadinternal.context.RequestContextImpl to be invoked.  This ought to be invoked when we have a partial page refresh on a view.

Let’s start by building a simple ADF application.

First, we will create a managed bean that we can use as a simple data source.  Here is the code:

package test;

import java.util.ArrayList;
import java.util.List;

import javax.faces.event.ValueChangeEvent;
import javax.faces.model.SelectItem;

public class TestBean {

    private List<SelectItem> suggestions = new ArrayList<SelectItem>();
    private String theValue;

    public TestBean() {
        super();
        System.out.println("TestBean init");

        //initialise list of choices
        suggestions.add(new SelectItem("Sydney"));
        suggestions.add(new SelectItem("Melbourne"));
        suggestions.add(new SelectItem("Singapore"));
        suggestions.add(new SelectItem("Tokyo"));
        suggestions.add(new SelectItem("Beijing"));
        suggestions.add(new SelectItem("San Francisco"));
        suggestions.add(new SelectItem("New York"));
        suggestions.add(new SelectItem("Houston"));
        suggestions.add(new SelectItem("Seattle"));
        suggestions.add(new SelectItem("London"));
        suggestions.add(new SelectItem("Paris"));
    }

    public List<SelectItem> getSuggestions() {
        return suggestions;
    }

    public void setSuggestions(List<SelectItem> suggestions) {}

    public String getTheValue() { return this.theValue; }

    public void setTheValue(String value) { this.theValue = value; }

    public void selectValueChangeListener(ValueChangeEvent valueChangeEvent) {
        theValue = (String) valueChangeEvent.getNewValue();
    }
}

Then we can create our view.  To make it easier, you can copy the source from here:

<?xml version='1.0' encoding='UTF-8'?>
<jsp:root xmlns:jsp="http://java.sun.com/JSP/Page" version="2.1"
          xmlns:f="http://java.sun.com/jsf/core"
          xmlns:h="http://java.sun.com/jsf/html"
          xmlns:af="http://xmlns.oracle.com/adf/faces/rich">
  <jsp:directive.page contentType="text/html;charset=UTF-8"/>
  <f:view>
    <af:document id="d1">
      <af:form id="f1">
        <af:outputText value="This is a test page" id="ot1">
          <span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>3-5705908791</span>
        </af:outputText>
        <af:selectOneChoice label="Pick a city" id="soc1"
                            valueChangeListener="#{TestBean.selectValueChangeListener}"
                            autoSubmit="true">
          <f:selectItems value="#{TestBean.suggestions}" id="si1"/>
        </af:selectOneChoice>
        <af:outputText value="you choose #{TestBean.theValue}" id="ot2"
                       partialTriggers="soc1"/>
      </af:form>
    </af:document>
  </f:view>
</jsp:root>

Now open up the adf-config.xml and add the managed bean as shown below:

Now you can deploy the ADF application to your WebLogic Server and make sure it works.

Next, we want to enable the WebLogic Diagnostic Framework on the managed server.  This is a one off activity.  You do this in the WebLogic Console, under the Diagnostics – Diagnostic Modules menu.

There should be a module definition there that was created when you installed WebLogic, if there is not, just go ahead and create one.  You just need to give it a name and then target it to the appropriate managed server.  Then open it up and go to the Configuration – Instrumentation tab and click on the Enabled check box to turn it on.

Now we need to tell it what to monitor.  Go to the Deployments menu item and open up your ADF application that you just deployed.

Go to the Configuration – Instrumentation tab and again check the Enabled check box for this application.  Then go back in there again and click on the Add Custom Monitor button (down the bottom).

Give your monitor a name, e.g. listener, and enter the pointcut as shown below (all on one line):

execution ( * 
org.apache.myfaces.trinidadinternal.context.RequestContextImpl 
addPartialTriggerListeners(...) )

Now click on the Save button and then open up your new monitor again.  Go down to the Actions section and make sure that the TraceAction is moved across to the Chosen list.  Also make sure that the Enabled check box is selected.  Now Save your monitor.

When you are prompted, create a new Plan for the application.

Then go and update your application (select it in Deployments and click on the Update button).

If this is the very first time you have turned on WLDF, you may need to restart your managed server as well.

Now go run your application and choose something from the drop down list.

Now we can check to see if that method was run.   Go to the Diagnostics – Log Files menu and open the EventsDataArchive log.  You should see the output from your monitor right there, confirming that the method in question was in fact run.  The image below shows an example.  It also shows output from a second custom monitor.  If you are feeling adventureous, you may want to try to create that monitor too.

Well I am sure that this is only just scratching the surface of what WebLogic Diagnostic Framework can do, but I for one am inspired to go and learn more.  I hope you enjoyed this post!

Thanks to Robert Patrick and Sabha Parameswaran for teaching me about pointcuts!

Phani has also tidied up a small omission from earlier POMs, which a number of you have commented on.  This POM copies the SAR file into the target directory so that it will be published into the Maven repository as part of the build, rather than an almost empty file with the POM in it.  While we may not be able to use it as a dependency in another composite using just the Maven coordinates (like we can with Java artifacts for example), it at least makes it available to other developers.

<?xml version="1.0" encoding="windows-1252" ?>
<project xmlns="http://maven.apache.org/POM/4.0.0">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.phani..AgentDataService</groupId>
  <artifactId>GetAgentData</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <scm>
    <connection>scm:svn://x.x.x.x/repo/AgentService/trunk/SOAComposite</connection>
    <developerConnection>scm:svn://x.x.x.x/repo/AgentService/trunk/SOAComposite</developerConnection>
  </scm>
  <build>
    <sourceDirectory>src/</sourceDirectory>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>1.1</version>
        <executions>
          <execution>
            <id>add-source</id>
            <phase>generate-sources</phase>
            <goals>
              <goal>add-source</goal>
            </goals>
            <configuration>
              <sources>
                <source>SCA-INF/src</source>
              </sources>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>2.0.2</version>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.6</version>
        <executions>
          <execution>
            <id>sca-compile</id>
            <phase>compile</phase>
            <configuration>
              <target>
                <property name="parent" location=".." />
                <property name="scac.input" value="composite.xml" />
                <property name="oracle.soa.uddi.registry.inquiryUrl" value="http://gh123:7101/registry/uddi/inquiry" />
                <ant antfile="${scriptsdir}/ant-sca-compile.xml"
                     dir = "${scriptsdir}"
                     target="scac" />
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
          <execution>
            <id>sca-package</id>
            <phase>package</phase>
            <configuration>
              <target>
                <property name="parent" location=".." />
                <property name="oracle.soa.uddi.registry.inquiryUrl" value="http://gh123:7101/registry/uddi/inquiry" />
                <property name="build.compiler" value="extJavac"/>
                <property name="compositeName" value="${project.artifactId}" />
                <property name="compositeDir" value="${basedir}" />
                <property name="revision" value="${project.version}" />
                <ant antfile="${scriptsdir}/ant-sca-package.xml"
                     dir = "${scriptsdir}"
                     target="package" />
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
		  <execution>
            <id>sca-copy-jar</id>
            <phase>package</phase>
            <configuration>
              <target>
                 <copy file="${basedir}/deploy/sca_${project.artifactId}_rev${project.version}.jar" tofile="${basedir}/target/${project.artifactId}-${project.version}.jar"/>
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
          <execution>
            <id>sca-deploy</id>
            <phase>deploy</phase>
            <configuration>
              <target>
                <property name="serverURL" value="http://gh345:7011" />
                <property name="user" value="weblogic" />
                <property name="password" value="password1" />
                <property name="sarLocation" value="${basedir}/deploy/sca_${project.artifactId}_rev${project.version}.jar" />
                <property name="overwrite" value="true" />
                <property name="forceDefault" value="true" />
                <property name="partition" value="default" />
                <ant antfile="${scriptsdir}/ant-sca-deploy.xml"
                     dir="${scriptsdir}"
                     target="deploy" />
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
          <execution>
            <id>sca-test</id>
            <phase>deploy</phase>
            <configuration>
              <target>
                <property name="parent" location=".." />
                <property name="jndi.properties.input" value="sca-test.jndi.properties" />
                <property name="scatest.input" value="AgentData" />
                <property name="scatest.format" value="junit" />
                <property name="scatest.result" value="reports" />
                <ant antfile="${scriptsdir}/ant-sca-test.xml"
                     dir="${scriptsdir}"
                     target="test" />
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
    <outputDirectory>SCA-INF\classes/</outputDirectory>
    <resources/>
  </build>
  <distributionManagement>
    <!-- use the following if you're not using a snapshot version. -->
    <!--<repository>
      <id>local</id>
      <name>local repository</name>
      <url>file:///C:/Documents and Settings/Phani/.m2/repository</url>
    </repository> -->
    <snapshotRepository>
      <id>artifactory</id>
      <name>artifactory-snapshots</name>
      <url>http://portalserver:8081/artifactory/libs-snapshot-local</url>
    </snapshotRepository>
  </distributionManagement>
</project>

</project>

Thank you Phani.  Enjoy!

A little known fact about BPM is that it includes identity virtualization support through libOVD.  It is not necessary to install a full blown OVD or to set up custom DB authenticators, etc.  You can just use libOVD.

My colleague Christopher Karl Chan from Denmark has a good introduction to this topic here which I would encourage you to read if you are looking at this kind of requirement.

If you are using BPM (or SOA) and you are not using any of the B2B functionality, then you might want to consider disabling B2B on your SOA managed servers.  There are a few good reasons for doing this:

• You will save some memory,
• You will reduce your managed server start (and restart) time, and
• You avoid any potential problems that may be introduced by running unnecessary modules – think security.

So how do we do it?  There are two steps that you need to take.

First, log on the WebLogic Console and go to Deployments.  Find the b2b application and stop it.

The second step is a little more involved – we need to set the b2b.donot_initialize property to true.  But first, we need to define this property.

Log on the Enterprise Manager and navigate to soa-infra.  Then open the SOA Infrastructure menu, then SOA Administration, then B2B Server Properties.

Click on the More B2B Configuration Properties… link.

Go to the Operations tab.

Click on the addProperty operation to define a new property.

In the key field, enter b2b.donot_initialize.  In the value field, enter true.  Add a comment if you wish.

Click on the Invoke button to add the property.

Now restart your managed servers.  Viola!

Olivier describes it as:

A graphical user interface to test OBR without having to deploy anything anywhere.

You can find details and download the tool here.  If you are using Business Rules, I would highly recommend checking this out.

I have blogged (this time) on bpmtech on the above topic.

Happy coding.

Let us take this opportunity again to thank you all for visiting and we hope you found something useful here!

For now, my endeavour was to use the Activity Guide APIs to generate the data seen on the workspace Activity Guide tab. This is hopefully useful for customers who want to write their custom UI equivalent to the default Activity Guide tab off the default BPM workspace.  The screenshot is below. Please notice that my BPM process has two milestones and one Human Task under each milestone.

Assuming you have gotten the workflow context etc., fastforwarding to the meat of the issue: Using the PS4FP Activity Guide APIs, one needs to:

• Get all Activity Guide instances for a given user (API call: agQuerySvc.queryAGDisplayInfos)
• Iterate through the list of AGDisplayInfos and get an AGInstanceId
• For each AGInstance Id, get the agDisplayInfo (agQuerySvc.getAGDisplayInfoDetailsById)
• For each AGInstance Id, get the corresponding milestone (agDisplayInfo.getMilestoneDisplayInfo)
• For each Milestone, get the taskList in that milestone (milestoneDisplayInfo.getTaskDisplayInfo)
• For each task, fetch details, say status and title in this example (taskDisp1.getTask)

I have used the standard workflow java sample in Authenticate.java and enhanced it for this purpose. That is an easy starting point!
Please note the following in that context:
1. The build.xml needs another few jar files other than the one that is packaged with workflow java samples
2. As usual, the wf_client_config has connection details.
3. The bpm project MultipleTasks Project has 2 human tasks have been created, 2 milestones and each of these tasks belong to 1 milestone. The Process.Owner role is granted to jstein after deployment.

While the AGAPIs for PS4FP have been documented, please note that there are a few documentation bugs currently being worked on, including on the sample therein.
Caveat: Some code refactoring in terms of moving some constants to different jar files is expected. And so, the jars referenced here are likely to change in PS5 and beyond.

I am going to try putting the source code: The BPM Process and the java code on java.net..(If for whatever reason I cant, I will be sure to blog source code here soon…)

Happy exploring AG APIs.

(Editing post for source code etc. below)

import java.util.Date;
import java.util.HashMap;
import java.util.Map;
import java.util.*;
import java.util.List;
import oracle.bpel.services.workflow.WorkflowException;
import oracle.bpel.services.workflow.client.IWorkflowServiceClient;
import oracle.bpel.services.workflow.client.IWorkflowServiceClientConstants;
import oracle.bpel.services.workflow.client.WorkflowServiceClientFactory;
import oracle.bpel.services.workflow.query.ITaskQueryService;
import oracle.bpel.services.workflow.task.impl.TaskUtil;
import oracle.bpel.services.workflow.task.model.Task;
import oracle.bpel.services.workflow.verification.IWorkflowContext;

import oracle.bpel.services.bpm.common.IBPMContext;

import oracle.bpel.services.workflow.IWorkflowConstants;
import oracle.bpel.services.workflow.task.model.Task;
import oracle.bpel.services.workflow.verification.IWorkflowContext;
import oracle.bpel.services.workflow.query.impl.TaskQueryService;
import oracle.bpel.services.workflow.client.WorkflowServiceClientContext;
import oracle.bpel.services.workflow.metadata.config.ResourceBundleInfo;
import oracle.bpel.services.workflow.activityguide.query.IAGQueryService;
import oracle.bpel.services.workflow.activityguide.query.impl.AGQueryUtil;
import oracle.bpel.services.workflow.activityguide.query.*;
//import oracle.bpel.services.workflow.activityguide.query.impl.AGQueryService;
import oracle.bpel.services.workflow.activityguide.query.model.AGDisplayInfo;
import oracle.bpel.services.workflow.activityguide.query.model.MilestoneDisplayInfo;
import oracle.bpel.services.workflow.activityguide.metadata.IAGMetadataService;
import oracle.bpel.services.workflow.activityguide.metadata.impl.AGMetadataService;
import com.oracle.bpel.activityguide.instance.model.MilestoneInstanceType;
import oracle.bpel.services.workflow.activityguide.query.model.TaskDisplayInfoType;
import sun.security.util.Password;

import oracle.bpel.services.workflow.repos.Ordering;
import oracle.bpel.services.workflow.repos.Predicate;
import oracle.bpel.services.workflow.repos.TableConstants;

import oracle.bpel.services.workflow.IWorkflowConstants;
import oracle.bpel.services.workflow.client.IWorkflowServiceClient;
import oracle.bpel.services.workflow.client.IWorkflowServiceClientConstants;
import oracle.bpel.services.workflow.client.WorkflowServiceClientFactory;
import oracle.bpel.services.workflow.repos.Ordering;
import oracle.bpel.services.workflow.repos.Predicate;
import oracle.bpel.services.workflow.repos.TableConstants;

import oracle.bpel.services.workflow.repos.PredicateConstants;

import oracle.bpel.services.workflow.task.model.Task;
import oracle.bpel.services.workflow.verification.IWorkflowContext;
import oracle.bpel.services.workflow.activityguide.query.model.AGDisplayInfo;
import oracle.bpel.services.workflow.activityguide.query.model.MilestoneDisplayInfoType;
import com.oracle.bpel.activityguide.instance.model.MilestoneInstanceTypeImpl;
import oracle.bpel.services.workflow.task.model.Task;
import oracle.bpel.services.workflow.task.model.TaskType;

public class Authenticate
{

private static IWorkflowContext ctx;
 private static ITaskQueryService querySvc ;
 private static IWorkflowServiceClient wfSvcClient;
 public static void main(String[] args) throws Exception {
 if (args.length != 3 || !("SOAP".equals(args[0]) || "REMOTE".equals(args[0]))) {
   System.out.print("Usage java Authenticate protocol(SOAP/REMOTE) user(jcooper) password(welcome1)");
   return;
 }
 authenticate(args[0], args[1], args[2]);
}

public static void authenticate(String protocol, String user, String password)
 throws WorkflowException {

System.out.println("Authenticating user " + user + ".....");

Map<IWorkflowServiceClientConstants.CONNECTION_PROPERTY, String> properties =
 new HashMap<IWorkflowServiceClientConstants.CONNECTION_PROPERTY, String>();
 //added below and commented out from wfclientconfig.xml
 properties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_SECURITY_PRINCIPAL, "weblogic");//weblogic username
 properties.put(IWorkflowServiceClientConstants.CONNECTION_PROPERTY.EJB_SECURITY_CREDENTIALS, "welcome1");//plain pwd

 // get the client
 //IWorkflowServiceClient wfSvcClient = WorkflowServiceClientFactory.getWorkflowServiceClient(protocol, properties, Util.getLogger());
 wfSvcClient = WorkflowServiceClientFactory.getWorkflowServiceClient(protocol, properties, Util.getLogger());
 querySvc = wfSvcClient.getTaskQueryService();
 // IWorkflowContext ctx = querySvc.authenticate(user, password.toCharArray(), "jazn.com");
 IWorkflowContext ctx = querySvc.authenticate(user, password.toCharArray(), "jazn.com");
 if (ctx == null)
 {
   System.out.println("ctx is null");
 } else {
   System.out.println("Authenticated successfully");
   System.out.println("Authenticated user info from IWorkflowContext:");
   System.out.println("Context created time: " + (new Date(ctx.getStartDateTime())));
   System.out.println("User: " + ctx.getUser());
   System.out.println("User Time Zone: " + ctx.getTimeZone().getDisplayName());
   System.out.println("User Locale: " + ctx.getLocale());
 }

 try{
   //calling testquery
   testQueryAGDisplayInfos();
 }
 catch (Exception eee){
   System.out.println("error");
   eee.printStackTrace();
 }

 }

private static void testQueryAGDisplayInfos()
 throws Exception
 {
   List agQueryColumns = new ArrayList();
   // agQueryColumns.add("MILESTONE_STATE");
   // agQueryColumns.add("DEFINITION_ID");
   //List agQueryColumns = new ArrayList();
   agQueryColumns.add("IDENTIFICATION_KEY");
   agQueryColumns.add("TITLE");
   agQueryColumns.add("CREATOR");
   agQueryColumns.add("CREATION_DATE");
   agQueryColumns.add("STATUS");
   IAGQueryService agQuerySvc = wfSvcClient.getAGQueryService();
   System.out.println("after AGQuerySVC");
   //Ordering order = new Ordering(TableConstants.WFTASK_INSTANCEID_COLUMN, false, true);

   // Query for all AG instances belonging to user say jstein
   List agDisplayInfoList =
   agQuerySvc.queryAGDisplayInfos(IAGQueryService.AG_PROCESS_TYPE_BPM, ctx,
   new ArrayList(),
   IAGQueryService.AGAssignmentFilter.ADMIN,
   null, //agPredicate,
   null, //ordering,
   0,
   0);

   List taskList=null;
   for (int a=0; a<agDisplayInfoList.size();a++)
   {
     String instanceId = ((AGDisplayInfo) agDisplayInfoList.get(a)).getInstanceId();
     //AGDisplayInfo agDisplayInfo = (AGDisplayInfo) agDisplayInfoList.get(a);
     AGDisplayInfo agDisplayInfo = agQuerySvc.getAGDisplayInfoDetailsById(IAGQueryService.AG_PROCESS_TYPE_BPM,
     ctx,
     new Long(instanceId), new ArrayList(),
     IAGQueryService.AGASSIGNMENT_FILTER_ADMIN);
     System.out.println("******for AGInstancID :"+instanceId+"********");

     System.out.println("AG title:" + agDisplayInfo.getTitle());

     System.out.println("milestone display info list size:" + agDisplayInfo.getMilestoneDisplayInfo().size());
     //MilestoneDisplayInfo msDisplayInfo = (MilestoneDisplayInfo)agDisplayInfo.getMilestoneDisplayInfo();

     for (int i=0; i< agDisplayInfo.getMilestoneDisplayInfo().size(); i++)
     {
       MilestoneDisplayInfo milestoneDisplayInfo = ((MilestoneDisplayInfo) agDisplayInfo.getMilestoneDisplayInfo().get(i));
       System.out.println("-----------------for milestone name :"+milestoneDisplayInfo.getTitle()+"---------");

       System.out.println("Milestone title: " + milestoneDisplayInfo.getTitle());
       System.out.println("Milestone Name: " + milestoneDisplayInfo.getName());

       List<TaskDisplayInfoType> taskDisplayInfoList = milestoneDisplayInfo.getTaskDisplayInfo();
       System.out.println("Total number of tasks: " + taskDisplayInfoList.size());
       for(int j=0; j< taskDisplayInfoList.size();j++)
       {

         TaskDisplayInfoType taskDisp1 = taskDisplayInfoList.get(j);
         TaskType task1 = taskDisp1.getTask();
         System.out.println("^^^^^^^^^^^^^^^^^^for task Id:"+task1.getSystemAttributes().getTaskNumber()+"^^^^^^^^^^^^^^^^^^");
         System.out.println("Task Status: "+task1.getSystemAttributes().getState());
         System.out.println("Task Title: "+task1.getTitle());

       } //taskDisplayInfoList

     } //MilestoneDisplayInfoList
   } //agDisplayInfoList
 } //method
} //class

Note: the classes in build.xml (for the most part the same as in the Workflow Java samples)

<path id="client.classpath">
 <pathelement path="${bea.home}/wlserver_10.3/server/lib/wlfullclient.jar"/>
 <pathelement path="${bea.home}/wlserver_10.3/server/lib/wlclient.jar"/>
 <pathelement path="${bea.home}/oracle_common/webservices/wsclient_extended.jar"/>
 <pathelement path="${bea.home}/Oracle_SOA1/soa/modules/oracle.soa.fabric_11.1.1/bpm-infra.jar"/>
 <pathelement path="${bea.home}/Oracle_SOA1/soa/modules/oracle.soa.fabric_11.1.1/fabric-runtime.jar"/>
 <pathelement path="${bea.home}/Oracle_SOA1/soa/modules/oracle.soa.workflow_11.1.1/bpm-services.jar"/>
 <pathelement path="${bea.home}/Oracle_SOA1/soa/modules/soa-startup.jar"/>
 <pathelement path="${bea.home}/Oracle_SOA1/soa/modules/oracle.soa.bpel_11.1.1/orabpel.jar"/>

<pathelement path="./config"/>
 </path>

Note: wf_client_config.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
 <workflowServicesClientConfiguration xmlns="http://xmlns.oracle.com/bpel/services/client">
 <server default="true" name="default">
 <localClient>
 <participateInClientTransaction>false</participateInClientTransaction>
 </localClient>
 <remoteClient>
 <serverURL>t3://localhost:7001</serverURL>
 <!--userName>jstein</userName>
 <password encrypted="true">4tORP+F+3jNupTEwSeZj3A==</password-->
 <initialContextFactory>weblogic.jndi.WLInitialContextFactory</initialContextFactory>
 <participateInClientTransaction>false</participateInClientTransaction>
 </remoteClient>
 </server>
 </workflowServicesClientConfiguration>

This is basically the same as the last version of the sample, just recompiled for the new release of BPM.  I am planning to add some significant new functionality to the sample to coincide with BPM 11.1.1.7.

I am also working on a C#/.Net version of the worklist.  See some details here if you are interested.

But I recently found a great article from Daniel Amadei here.  I strongly encourage you to take a look at it if you are at all interested in test automation and/or continuous integration.

I am planning to build this kind of testing into my continuous integration project here.

I have found on these systems that changing to 1.6.0_26 fixed all problems and dramatically improved performance.  So that is what I did!  As I said, this is not scientific, I did not do extensive testing or work out why it was having problems – at the time I did not have the time to spare, I just needed it fixed.  But I am posting it anyway just in case it helps someone out.

Note that I have not seen this issue on any other operating system.

The documentation is here – note that we have the new docs.oracle.com site which is also being indexed properly by Google now (so I’m told) – addressing a long time complaint about searchability of the Oracle documentation.

And of course you can download it all from OTN.

For BPM 11g customers, this release provides a full and properly supported mechanism to get all the new features that were added in the Feature Pack patch.

Recently, I had a discussion with several colleagues and one of our large customers about when to use BPMN and when to use BPEL to model business processes.  I have discussed this topic before in this post but this conversation opened up an interesting new angle on the topic, which I wanted to share with you all.

First of all, let me be clear about the scenario here.  This is a large customer who is going to make extensive use of both BPMN and BPEL in their BPM/SOA environment.  They are not trying to decide which one to use exclusively – it is given that both BPEL and BPMN will be used extensively.  So the question here is about how to come up with a consistent approach to deciding which to use when – what to do with BPMN and what to do with BPEL.

The new perspective in this conversation was about how to choose the modeling language based on the fault/exception handling requirements of the process.  If you are not familiar with the options available for fault handling in BPMN and in BPEL, you should quickly review these sources:

• For BPMN, read Chapter 19 of the Modeling Guide
• For BPEL, read this article by my colleague Greg Mally

You will notice that there are some differences in the capabilities of the products today.  If you look back over the last few releases of BPM, you will see that there has been a significant investment in adding more fault management capabilities for BPMN processes, and it would be reasonable to assume that this will continue.

But even while the Oracle platform holds the promise of equally rich fault handling in either BPEL or BPMN, one can make a case that the fault handling capabilities in BPEL are especially suitable for system to system integration, particularly when you need to use a distributed transaction or when compensation is required.  Besides, these kinds of highly technical fault handling capabilities are a task probably best suited for more technical kinds of processes and people.  And finally, the directed graph nature and ‘alter flow’ capability in BPMN can make it more difficult (or potentially impossible) to employ the same techniques.

However, the fault handling in BPMN is rather well suited to ‘business’ faults – not enough stock to fill an order, credit check failed, order line contains a discontinued product, order cancelled, things like that.

So this leads to the following suggested approach for this customer use-case:

• Write top level (i.e. true ‘business’ processes) in BPMN,
• Do not perform any kind of system interaction in these processes – don’t use adapters, call web services, etc.,
• Use activities or embedded sub-processes with boundary events and event sub-processes to handle all business faults that may occur,
• Make sure to have a ‘catch all’ event sub-process to handle any failures that are not specifically handled,
• Theoretically there should never be a system fault in these BPMN processes,
• Whenever there is a need to do some actual work, delegate this to BPEL, i.e. use a service activity with implementation type ‘service call’ to have BPEL go do the work,
• Make the BPEL processes atomic, so that they can easily be retried, rolled back, etc.,
• Use the fault management framework to control the handling of faults in the BPEL processes, and
• Keep BPEL ‘worker’ processes in separate composites from BPMN ‘business’ processes.

This may not be perfect, but we think it offers a new, and very relevant, real-world perspective on how to decide which modeling notation is right for your processes.  We are sure interested to hear your thoughts and comments.

For a start, here are Dave’s thoughts on this idea:

First I would just like to reinforce the constraints around the scenario that this advice applies to because I think customers would come up with different approaches, and Oracle would have different best practices, when the question is asked around a greenfield project as to when to use BPEL and when to use BPMN.  In those scenarios, the advice would not be to wrap each system call in BPEL, since the unique value proposition of the Oracle platform is that the same system integration capabilities are available natively in both BPEL and BPMN.  However, I think the guidelines above make sense for a customer who will be mixing and matching BPEL and BPMN throughout their processes and applications and where it is assumed that most projects will include both.  In this case, it becomes important to strive for consistent guidelines as to where to draw the line for what to do in the BPEL part of a process and what to do in the BPMN part.  In this case, the fault handling capabilities are indeed richer in BPEL today vs what is possible in BPMN and even though they may be equivalent in the future, there are certainly many kinds of fault handling logic, compensation, etc. that will be deeper than the business would want to go.  The division of labor described above by Mark then results in not just using the best tool for the job, but also leverages the heterogeneous BPEL/BPMN architecture at this customer to make the BPMN “business view” of the process as clean as possible.

In this article, we will look at how you build a human task user interface using C# and ASP.NET.  This human task UI will show up right there in BPM workspace, just like the ADF ones do.  Here’s what it will look like when it is done:

Those red arrows wont be there!  They are just to show you where it is.  That part there where you normally see the ADF task form, that is a .Net application.

Now, as I said, you can use any framework to do this – as long as it is capable of calling Java APIs or web services and reading some data from the HTTP Request object’s Query String.

In this post we will build a UI that is specific to the task in question – like the ADF ones that you generate are.  But it would not be a long way from here to building a generator so that you could auto-generate .Net user interfaces just like you can for ADF.

What you need

To follow along this post you are going to need a couple of things:

• JDeveloper with the SOA and BPM extensions installed, at least version 11.1.1.5 plus the Feature Pack patch,
• A BPM server, the same version as JDeveloper,
• A copy of Visual Studio with the C# and web applications options installed.  I used Visual Studio 2010 Professional, but the free Visual Studio Express editions will also work if you don’t own a copy of Visual Studio.  Just make sure you get one with web applications and C# included, and
• either a lot of time and patience to type boring bolierplate code, or a copy of AutoMapper from here.  I recommend you take the AutoMapper option…

If you want to get a feel for calling the BPM/HWF APIs/web services from .Net, you might want to review this post.

Creating the composite

First thing we are going to want is a composite to play with.  We can just make a really simple one with just a human task in it.  That will be enough to do what we want to do here.  In fact, something as simple as this will do admirably:

To create this, open up JDeveloper and create a new BPM Application by selecting New from the Application menu.  In the wizard choose BPM Application in the Application Template section and give your application a name.  I called mine DotNetTest.  Then click on Next.  Give your project a name, I used the same name.  Then click on Next.  Choose Composite with BPMN Process and click on Finish.

In the next dialog, give your process a name, I called mine DotNetTest too.  Then take the defaults and continue.

Drag a User Task into your process from the component palette and drop it on the line between the Start and End nodes, as shown in the diagram above.  Notice that the line turns blue when you hover above it in the right spot.  If you don’t see the component palette, you can open it from the View menu.

Apologies that my names of things in this sample are not super imaginative…

Now we need to create some data definitions.  Open the BPM Project Navigator.  If you don’t see it, you can open it from the View menu.  Expand out your project and the Business Catalog.  Right click on the Business Catalog and choose New then Module from the popup menu.  Call the module Data.

Then right click on your new module and select New then Business Object from the popup menu.  Name your new business object.  I called mine BusinessObject1 (told you there were not very imaginative).  Then click on the green plus icon in the Attributes section to add two new attributes.  Make them both of type String.  I called mine attribute1 and attribute2. Go ahead and save your work (select Save All from the File menu.)

Now return to your process.  Click on the background of the process to make sure the structure pane shows the structure of the process.  It should look a bit like this:

Now right click on the Process Data Objects in the structure pane and select New from the popup menu.  Give it a name, I chose dataObject1, and choose <Component> for the Type.  Then click on the little magnifying glass icon and choose the business object you just defined.

Go ahead and create a second variable (process data object) of type String called dataObject2.

Now let’s set up our data in this process.  Right click on the Start node and open its Properties from the popup menu.  Go to the Implementation tab and add an argument by clicking on the little green plus icon in the Arguments Definition section. I called mine argument1.  Set the type to your business object (Data.BusinessObject1 if you used the same names as me.)

Now click on the Data Associations link.  Map your argument1 into your dataObject1 as shown in the image below.  Then right click on dataObject2 and choose Expression from the popup menu.  Enter “hello” (with the quotes) as the expression.  Your data associations should now look a little like this:

That takes care of our inputs.  When you start the process you will type in the two strings that get put into dataObject1 and dataObject2 will get set to “hello.”

Now, let’s set up the human task.  To make it interesting we are going to allow editing of some data but not of others.

Open the human task properties and go the the Implementation tab.  Click on the green plus icon to create a new human task.  Give it a name, I called mine Humantask1.  Click on the little magnifying glass icon next to Outcomes and set the outcomes to just one option – OK.  Then click on the plus icon next to Parameters to open the Browse Data Objects window that you see on the right hand side of the image below.  Drag your two data objects into the Parameters area as indicated by the red arrow.  Tick the box to make only dataObject2 editable.  Then click on OK to complete the human task definition.  We will take the defaults for everything else.

Now click on the Data Associations link.  Set up the input and output mappings as shown:

Right, now we are ready to go ahead and deploy our composite to our server instance.  You can do this back in the Application Navigator using the Deploy option in the popup menu on the project.

Follow through the wizard, I assume that you know how to do this by now if you are a regular reader.  If not, you can go ahead and take the defaults on this one.

When the deployment is finished, go to Enterprise Manager (at http://yourserver:7001/em) and login as an administrative user (like weblogic) and then navigate into the SOA folder and you should see your shiny new composite there.  Something like this:

Click on it to open the composite page, then click on the Test button and go ahead and launch a couple of instances.  We will use them later.

While we are here, let’s tell the runtime that we are planning to use our own user interface for this task.  We have not created it yet, but we have a pretty good idea what the details will be.  Go back to the composites main page.  Down the bottom you should see a list of the components in the composite, including your Humantask1.  Click on that to bring up the settings for that component.  Then open the Administration tab in these settings.

Click on the green plus icon next to Add URI.  Provide the values you see in the image above.  We are just going to use the built in test environment in Visual Studio in this post.  No need to install IIS for this.  Of course in real life you would install it and deploy your web applications to IIS.  Make sure you click on the Apply button when you are done to save your changes.

The .Net Application

Now, let’s get to work on the fun part!

Open Visual Studio and start a new project by choosing New then Project from the File menu.  In the New Project dialog box, open the Visual C# folder and select the Web category.  Make sure you have .Net Framework 4 selected (you will have installed this with Visual Studio most likely – if not, stop and go install it now) and choose the ASP.NET Web Application template.  Give your project a name, I called mine WebApplication2.  Click on OK to create your application.

We need to tell our project about AutoMapper. First of all, go extract the AutoMapper.zip into your project directory, e.g. c:\users\mark\documents\visual studio 2010\Projects\WebApplication2\WebApplication2.  This will give you three files – AutoMapper.dll, AutoMapper.pdb and AutoMapper.xml.  For a discussion of what AutoMapper is and why we want it – see this post.

In the Solution Explorer, right click on the References folder and choose Add Reference… from the popup menu.

In the Add Reference window, go to the Browse tab.  You should see the AutoMapper.dll that you just unzipped right there in your project directory.  Select it and click on OK to add a reference to your project.

Now, we need to add references to the services that we will be using.  We need to get the WSDL addresses for the TaskService and the TaskQueryService.  You can work these out using the following examples:

http://yourserver:8001/integration/services/TaskQueryService/TaskQueryService?wsdl

http://yourserver:8001/integration/services/TaskService/TaskServicePort?wsdl

To add the references, right click on the Service References folder and select Add Service Reference… from the popup menu.

Add the two web services (one at a time).  Enter the WSDL URL in the Address field, then click on the Go button.  You will see a description of the services avilable as shown in the image below.  Enter a name for the service in the Namespace field and then click on the OK button.  I called mine TaskService and TaskQueryService.

Now, because these two services use WS-Security, we need to tell .Net to use WS-Security.  This is done by editing the web.config file.  You should see it right there in the Solution Explorer, go ahead and open it and scroll down to the bottom.  Here is the part we are interested in:

 <bindings>
      <basicHttpBinding>
        <binding name="TaskQueryServiceSOAPBinding" closeTimeout="00:01:00"
          openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
          allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
          maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
          messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
          useDefaultWebProxy="true">
          <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
            maxBytesPerRead="4096" maxNameTableCharCount="16384" />
          <security mode="None">
            <transport clientCredentialType="None" proxyCredentialType="None"
              realm="" />
            <message clientCredentialType="UserName" algorithmSuite="Default" />
          </security>
        </binding>
        <binding name="TaskServiceSOAPBinding" closeTimeout="00:01:00"
          openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
          allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
          maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
          messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
          useDefaultWebProxy="true">
          <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
            maxBytesPerRead="4096" maxNameTableCharCount="16384" />
          <security mode="None">
            <transport clientCredentialType="None" proxyCredentialType="None"
              realm="" />
            <message clientCredentialType="UserName" algorithmSuite="Default" />
          </security>
        </binding>
      </basicHttpBinding>
    </bindings>
    <client>
      <endpoint address="http://ps5.mark.oracle.com:8001/integration/services/TaskQueryService/TaskQueryService2/*"
        binding="basicHttpBinding" bindingConfiguration="TaskQueryServiceSOAPBinding"
        contract="TaskQueryService.TaskQueryService" name="TaskQueryServicePortSAML" />
      <endpoint address="http://ps5.mark.oracle.com:8001/integration/services/TaskQueryService/TaskQueryService"
        binding="basicHttpBinding" bindingConfiguration="TaskQueryServiceSOAPBinding"
        contract="TaskQueryService.TaskQueryService" name="TaskQueryServicePort" />
      <endpoint address="http://ps5.mark.oracle.com:8001/integration/services/TaskService/TaskServicePort"
        binding="basicHttpBinding" bindingConfiguration="TaskServiceSOAPBinding"
        contract="TaskService.TaskService" name="TaskServicePort" />
      <endpoint address="http://ps5.mark.oracle.com:8001/integration/services/TaskService/TaskServicePortSAML/*"
        binding="basicHttpBinding" bindingConfiguration="TaskServiceSOAPBinding"
        contract="TaskService.TaskService" name="TaskServicePortSAML" />
    </client>

For each of the two bindings you will need to update the security section as shown above to use UserName credentials and the Default algorithm for message security.  Not transport, message – make sure you get the right one!

Also, which we are here, you might want to note a couple of things in the client section.  First, this is where you will go and change the endpoint addresses if you want to use a different server later on.  Second, notice that there are two endpoints for each service.  One is SAML and one is not.  We are going to use the ones that are not SAML in this example.  You can make a note of the value in the name attribute for each attribute.  We will need those later.

Ok, that takes care of our services.  Now let’s do the user interface.  We will start by customising the template (Site.Master) first.  You don’t strictly need to do this, but it is best to get rid of some of that extra stuff that might cuase confusion.  Here is the template I used:

<%@ Master Language="C#" AutoEventWireup="true" CodeBehind="Site.master.cs" Inherits="WebApplication2.SiteMaster" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head runat="server">
    <title></title>
    <link href="~/Styles/Site.css" rel="stylesheet" type="text/css" />
    <asp:ContentPlaceHolder ID="HeadContent" runat="server">
    </asp:ContentPlaceHolder>
</head>
<body>
    <form runat="server">
    <div class="page">
        <div class="header">
            <div class="title">
                <h1>
                    Sample C#/ASP.NET Task form
                </h1>
            </div>
            <div class="loginDisplay">
                        [ <a href="#" id="HeadLoginView_HeadLoginStatus">Sample</a> ]
            </div>
        </div>
        <div class="main">
            <asp:ContentPlaceHolder ID="MainContent" runat="server"/>
        </div>
        <div class="clear">
        </div>
    </div>
    <div class="footer">
    </div>
    </form>
</body>
</html>

Note that we are running this at the server, not the client, for those who know enough about .Net to care about the difference

Great, now le’s set up our main page.  Open up the Default.aspx page.  Here is how we want it to look:

If you want to, you can go and drag and drop everything into place and edit the properties.  But in the interest of making this easier for you, and to make sure the names of the UI components match the sample code below, it would be better to copy the code below into the source view.

<%@ Page Title="Home Page" Language="C#" MasterPageFile="~/Site.master" AutoEventWireup="true"
    CodeBehind="Default.aspx.cs" Inherits="WebApplication2.QueryStringRecipient" %>

<asp:Content ID="HeaderContent" runat="server" ContentPlaceHolderID="HeadContent">
</asp:Content>
<asp:Content ID="BodyContent" runat="server" ContentPlaceHolderID="MainContent">
    <h2>
        Task Details
    </h2>
<table>
  <tr><td>Task Title:</td><td><asp:TextBox ID="TextBox1" runat="server"
          Enabled="False"></asp:TextBox></td></tr>
  <tr><td>Task State:</td><td><asp:TextBox ID="TextBox2" runat="server"
          Enabled="False"></asp:TextBox></td></tr>
  <tr><td>Task Number:</td><td><asp:TextBox ID="TextBox3" runat="server"
          Enabled="False"></asp:TextBox></td></tr>
</table>
    <h2>
        Payload
    </h2>
<table>
  <tr><td>Updatable Payload Data:</td><td>
      <asp:TextBox ID="TextBox4" runat="server" AutoPostBack="True"></asp:TextBox></td></tr>
  <tr><td>Read Only Payload Data:d Only Payload Data:</td><td><asp:TextBox ID="TextBox5" runat="server"
          Enabled="False"></asp:TextBox></td></tr>
</table>
    <h2>
        Actions
    </h2>
    <asp:Button ID="Button1" runat="server" Text="OK" onclick="Button1_Click"  />
</asp:Content>

Great, that’s the UI taken care of.  Now let’s put in the code behind it.  This is where the really interesting stuff happens

Go ahead and open up your Default.aspx.cs file and put this code into it:

using System;
using System.Collections;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Web;
using System.Web.SessionState;
using System.Web.UI;
using System.Web.UI.HtmlControls;
using System.Web.UI.WebControls;

namespace WebApplication2
{

    public partial class QueryStringRecipient : System.Web.UI.Page
    {
        protected System.Web.UI.WebControls.Label lblInfo;

        private string taskId;
        private string ctxId;
        private TaskQueryService.workflowContextType ctx;
        private TaskQueryService.TaskQueryServiceClient tqs;
        private TaskQueryService.task task;

        private void Page_Load(object sender, System.EventArgs e)
        {

                // setup the automapper
                setupAutoMapper();

                // BPM will pass us the taskID and the ctx token
                // need to read these out of the http request's query string
                taskId = Request.QueryString["bpmWorklistTaskId"];
                ctxId = Request.QueryString["bpmWorklistContext"];
                //System.Diagnostics.Debug.WriteLine("Task ID:\n" + taskId);
                //System.Diagnostics.Debug.WriteLine("Context:\n" + ctxId);

                // if this is running outside of the worklist, just exit
                if (taskId == null)
                {
                    // looks like we dont have a task
                    return;
                }

                // set up the BPM context
                ctx = new TaskQueryService.workflowContextType();
                ctx.token = ctxId;

                // get the TQS
                tqs = new TaskQueryService.TaskQueryServiceClient("TaskQueryServicePort");

                // set up the request to get the task
                TaskQueryService.taskDetailsByIdRequestType getTaskRequest = new TaskQueryService.taskDetailsByIdRequestType();
                getTaskRequest.workflowContext = ctx;
                getTaskRequest.taskId = taskId;

                // get the task
                task = tqs.getTaskDetailsById(getTaskRequest);
                //System.Diagnostics.Debug.WriteLine("task title:\n" + task.title);

                // populate the UI with task details
                TextBox1.Text = task.title;
                TextBox2.Text = task.systemAttributes.state.ToString();
                TextBox3.Text = task.systemAttributes.taskNumber;

                if (!Page.IsPostBack)
                {
                    // populate the UI with current payload data
                    System.Xml.XmlNode[] payload = (System.Xml.XmlNode[])task.payload;
                    TextBox4.Text = payload.ElementAt(0).ChildNodes.Item(1).InnerText;
                    TextBox5.Text = payload.ElementAt(1).ChildNodes.Item(0).InnerText;
                }

        }

        #region Web Form Designer generated code
        override protected void OnInit(EventArgs e)
        {
            //
            // CODEGEN: This call is required by the ASP.NET Web Form Designer.
            //
            InitializeComponent();
            base.OnInit(e);
        }

        /// <summary>
        /// Required method for Designer support - do not modify
        /// the contents of this method with the code editor.
        /// </summary>
        private void InitializeComponent()
        {
            this.Load += new System.EventHandler(this.Page_Load);

        }
        #endregion

        protected void Button1_Click(object sender, EventArgs e)
        {
            System.Diagnostics.Debug.WriteLine("PRESSED OK");

            // update the task payload from the UI
            System.Xml.XmlNode[] payload = (System.Xml.XmlNode[])task.payload;
            payload.ElementAt(0).ChildNodes.Item(1).InnerText = TextBox4.Text;
            task.payload = payload;

            // get the TS
            TaskService.TaskServiceClient ts = new TaskService.TaskServiceClient("TaskServicePort");

            // update task
            TaskService.taskServiceContextTaskBaseType updateTaskRequest = new TaskService.taskServiceContextTaskBaseType();
            updateTaskRequest.workflowContext = AutoMapper.Mapper.Map<TaskQueryService.workflowContextType, TaskService.workflowContextType>(ctx);
            updateTaskRequest.task = AutoMapper.Mapper.Map<TaskQueryService.task, TaskService.task>(task);
            TaskService.task updatedTask = ts.updateTask(updateTaskRequest);

            // complete task
            TaskService.updateTaskOutcomeType updateTaskOutcomeRequest = new TaskService.updateTaskOutcomeType();
            updateTaskOutcomeRequest.workflowContext = AutoMapper.Mapper.Map<TaskQueryService.workflowContextType, TaskService.workflowContextType>(ctx);
            updateTaskOutcomeRequest.outcome = "OK";
            updateTaskOutcomeRequest.Item = updatedTask;
            ts.updateTaskOutcome(updateTaskOutcomeRequest);

            // redirect to empty page
            Response.Redirect("/Empty.htm");
        }

        private void setupAutoMapper()
        {
            // set up the automapper
            AutoMapper.Mapper.CreateMap<TaskQueryService.workflowContextType, TaskService.workflowContextType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.credentialType, TaskService.credentialType>();

            AutoMapper.Mapper.CreateMap<TaskQueryService.task, TaskService.task>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.attachmentType, TaskService.attachmentType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.callbackType, TaskService.callbackType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.customAttributesType, TaskService.customAttributesType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.documentType, TaskService.documentType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.EvidenceType, TaskService.EvidenceType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.processType, TaskService.processType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.commentType, TaskService.commentType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.identityType, TaskService.identityType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.ucmMetadataItemType, TaskService.ucmMetadataItemType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.systemAttributesType, TaskService.systemAttributesType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.actionType, TaskService.actionType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.displayInfoType, TaskService.displayInfoType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.shortHistoryTaskType, TaskService.shortHistoryTaskType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.assignmentContextType, TaskService.assignmentContextType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.assignmentContextTypeValueType, TaskService.assignmentContextTypeValueType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.collectionTargetType, TaskService.collectionTargetType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.collectionTargetActionType, TaskService.collectionTargetActionType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.preActionUserStepType, TaskService.preActionUserStepType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.systemMessageAttributesType, TaskService.systemMessageAttributesType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.flexfieldMappingType, TaskService.flexfieldMappingType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.scaType, TaskService.scaType>();
            AutoMapper.Mapper.CreateMap<TaskQueryService.UpdatableEvidenceAttributesType, TaskService.UpdatableEvidenceAttributesType>();

            // check automapper config is valid
            AutoMapper.Mapper.AssertConfigurationIsValid();
        }

    }
}

Now let’s walk through and understand what is happening here. First we declare some variables that we will use.  Notice that some of these are from the namespaces we just created with our web service references.  We are going to be using the TaskQueryServiceClient and the workflowContextType (the credentials that we use to authenticate to the BPM server).  We also will be using the task object.  We define these at the class level as we want them to be avialable for the whole lifecycle of the page.

        private string taskId;
        private string ctxId;
        private TaskQueryService.workflowContextType ctx;
        private TaskQueryService.TaskQueryServiceClient tqs;
        private TaskQueryService.task task;

As for the class, we are extending the class QueryStringRecipient and extending System.Web.UI.Page.  We say partial class because this class is already defined elsewhere (in the .Net libraries) and we are just adding some extra logic to it.  We use this class so that we can get access to the Query String from the HTTP Request.

    public partial class QueryStringRecipient : System.Web.UI.Page

Now let’s take a look at the Page_Load method.  This gets run (as you might guess) when the page is loaded.  Every time the page is loaded – even post backs.  So you have to be careful not to overwrite data from the user in here.  This method is probably doing more work than it needs to on each page load the way I have it implemented – I guess you can optimise it some more   The first thing we need to do is set up the AutoMapper.  This is done by calling a convenience method setupAutoMapper() which is down the bottom of the source file to hide all that ugliness.  As I mentioned earlier, why and how we are using AutoMapper is discussed over here.

       private void Page_Load(object sender, System.EventArgs e)
        {

                // setup the automapper
                setupAutoMapper();

Next, we need to read the data that BPM sends us in the Query String.  When we configure a URI for a human task, like we did earlier in this post, BPM will append some data into the Query String for us.  This is the data we want to read now:

• bpmWorklistTaskId is the taskId for the particular task instance we are interested in, and
• bpmWorklistConext is the BPM workflow context (security token) for the currently logged on user (logged on to BPM Workspace that is).

With these two pieces of information, we are able to do everything we need to do to that task – get its details, payload, take an action on it (system or custom actions), update it, etc.  Here is the code to grab these from the Query String:

                // BPM will pass us the taskID and the ctx token
                // need to read these out of the http request's query string
                taskId = Request.QueryString["bpmWorklistTaskId"];
                ctxId = Request.QueryString["bpmWorklistContext"];

Now we are ready to start talking to BPM.  We should first check that we actually got a task ID in the previous step, otherwise, we probably got called from outside of BPM Workspace, so we should just stop.

Then we can create our workflowContextType.  To use the details that BPM sent us, all we need to do is put them into the token property of this type, as you see below.  Then we are ready to create our TaskQueryServiceClient, note that we pass in to the constructor the name of the port we want – you wrote that down earlier, didn’t you?

                // if this is running outside of the worklist, just exit
                if (taskId == null)
                {
                    // looks like we dont have a task
                    return;
                }

                // set up the BPM context
                ctx = new TaskQueryService.workflowContextType();
                ctx.token = ctxId;

                // get the TQS
                tqs = new TaskQueryService.TaskQueryServiceClient("TaskQueryServicePort");

Next, we want to retrieve the task.  We do this by calling the getTaskDetailsById() method on the TaskQueryServiceClient.  First, we need to set up our inputs.  This is done by creating a TaskQueryService.taskDetailsByIdRequestType and setting its workflowContext and taskId properties using the values we retrieved from the Query String earlier.  Then we can call the method.  We get back a task object.

                // set up the request to get the task
                TaskQueryService.taskDetailsByIdRequestType getTaskRequest = new TaskQueryService.taskDetailsByIdRequestType();
                getTaskRequest.workflowContext = ctx;
                getTaskRequest.taskId = taskId;

                // get the task
                task = tqs.getTaskDetailsById(getTaskRequest);

Now, let’s read the data out of the task and populate our UI components.  Here we set the various text boxes to the task metadata we have chosen to show – title, status and task number.

Then we set the other group of text boxes (the ones for the payload) to those parts of the payload that we are interested in.  Of course, we don’t need to display everything, just the fields we are interested in.  In this example, we are going to take one of the two String fields from our dataObject1 and the String dataObject2.  You can see that we access the payload through a System.ml.XmlNode[] – we can just cast the payload property of the task to this type and then we can easily read the payload data.  You can see from the code here that this is specific to the payload type – so in this case we do need to know the structure of the payload.  We could of course be a bit smarter and introspect the payload to find the data we want, but for now, hardcoding it will serve our purpose here.

Notice that we only want to populate the payload UI fields from the task payload the first time we load the page.  That is why we check if (!Page.IsPostBack) – otherwise any changes that the user had made would be overwritten when they post back those very changes.

                // populate the UI with task details
                TextBox1.Text = task.title;
                TextBox2.Text = task.systemAttributes.state.ToString();
                TextBox3.Text = task.systemAttributes.taskNumber;

                if (!Page.IsPostBack)
                {
                    // populate the UI with current payload data
                    System.Xml.XmlNode[] payload = (System.Xml.XmlNode[])task.payload;
                    TextBox4.Text = payload.ElementAt(0).ChildNodes.Item(1).InnerText;
                    TextBox5.Text = payload.ElementAt(1).ChildNodes.Item(0).InnerText;
                }

That completes the Page_Load method.  Now, let’s take a look at what happens when the user clicks on the OK button on our page.  That button represents the OK (custom) action (or outcome) for that task.

The first thing we want to do is update the payload with the data that the user has entered in the form (if any).  Note that only one of the two payload fields that we are displaying is editable – you may remember when we created our task that we set only one of the two parameters to be editable.  That is why we are only taking the value from one of the TextBox components and updating the payload.  If you tried to update the non-updatable field, you would get an exception (as you might expect.)  Updating the payload is pretty mush the reverse of reading the payload:

        protected void Button1_Click(object sender, EventArgs e)
        {
            System.Diagnostics.Debug.WriteLine("PRESSED OK");

            // update the task payload from the UI
            System.Xml.XmlNode[] payload = (System.Xml.XmlNode[])task.payload;
            payload.ElementAt(0).ChildNodes.Item(1).InnerText = TextBox4.Text;
            task.payload = payload;

Now we have actually just updated the payload in our local copy of the task, we need to actually tell BPM to update the ‘real’ task on the server.  We do this by calling the updateTask() method on the TaskService.  Just like we did earlier for the TaskQueryService, we create an instance of the TaskService and pass in the name of the port we want to use.

Then we create the input for the updateTask() method, which is the oddly named TaskService.taskServiceContextBaseType.  We can then populate it with the workflowContext we got from the Query String, and our newly updated task object.

Notice how we use the AutoMapper to convert between types in the two different namespaces created for our services.

The updateTask() method returns to us a new task object, which is represents the newly updated task object on the server.  We will need to use this new object to take any further actions on this task, our old task object is now no longer of any use to us.

            // get the TS
            TaskService.TaskServiceClient ts = new TaskService.TaskServiceClient("TaskServicePort");

            // update task
            TaskService.taskServiceContextTaskBaseType updateTaskRequest = new TaskService.taskServiceContextTaskBaseType();
            updateTaskRequest.workflowContext = AutoMapper.Mapper.Map<TaskQueryService.workflowContextType, TaskService.workflowContextType>(ctx);
            updateTaskRequest.task = AutoMapper.Mapper.Map<TaskQueryService.task, TaskService.task>(task);
            TaskService.task updatedTask = ts.updateTask(updateTaskRequest);

Finally, we can complete the task by setting the outcome to “OK” using the updateTaskOutcome() method on the TaskService.  I won’t go through all the details, but you can see we create the input, populate it and the call the method.

Once this is done, we can redirect the browser to an empty page – just like the auto-generated ADF task forms do – so we maintain the normal user exerpience in the BPM Workspace.  This is done with the Response.Redirect(“/Empty.htm”) call on the last line.

            // complete task
            TaskService.updateTaskOutcomeType updateTaskOutcomeRequest = new TaskService.updateTaskOutcomeType();
            updateTaskOutcomeRequest.workflowContext = AutoMapper.Mapper.Map<TaskQueryService.workflowContextType, TaskService.workflowContextType>(ctx);
            updateTaskOutcomeRequest.outcome = "OK";
            updateTaskOutcomeRequest.Item = updatedTask;
            ts.updateTaskOutcome(updateTaskOutcomeRequest);

            // redirect to empty page
            Response.Redirect("/Empty.htm");

Now obviously we need to have such a page, so go ahead and create a new HTML page called Empty.htm and put the following code into it:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title></title>
</head>
<body>

</body>
</html>

Ok, so that completes our .Net project.  Go ahead and run itusing the ‘big green arrow’ icon.  This will start up the embedded .Net server and run the project.  It will probably open a browser window and show you the page.  Just close this, we don’t need it.

Now, go and log in to BPM Workspace as an administrative user (like weblogic) and click on the Administration link in the top right corner.  Select the role this task is assigned to, mine is DotNetTest.Role1 and then click on the little ‘new’ icon (highlighted in the image below) to assign some users to this role.

In the popup box, serach for weblogic and then move it across from Available to Selected.  Then click on OK and then on Apply to save this new mapping.  Now these tasks can be routed to the weblogic user.

Log out of the BPM Workspace and back in again so that it will refresh the mappings.  You should now see the DotNetTest tasks that you created earlier in Enterprise Manager sitting in weblogic‘s queue.  Click on one of them and your shiny new .Net task form will load.  It should look a little like this:

Try it out!  You can update the payload data, and then click on the OK button to action the task.  After you do that, go take a look at the instance flow trace in Enterprise Manager and you can verify that the payload data was in fact changed.

So there you have it, a .Net task form fully integrated into the BPM Workspace application.  Enjoy!

Thanks go to Carlos Casares for giving me the incentive to write this in the first place and to my reader in Saint Petersburg for giving me the incentive to publish it.

I recommend you take a look!