Iterating/traversing arrays in BPM

Update: See this post for an updated example.

Recently, I was working with a customer who has a requirement to create an arbitrary number of human tasks, based on the cardinality of an input data structure (array).  A bit of hunt around did not turn up any good examples or documentation on this use case, so I did a bit of research and experimentation and have come up with the following approach.

Thanks to Ali Mukadam, Sushil Shukla and Meera Srinivasan for ideas and suggestions.

The scenario we will use here is ordering a number of pathology tests.  For simplicity, we will just display each request as a human task, and pass the input data into the process using the testing framework.

To complete this example, you will need JDeveloper 11.1.1.3 with the SOA Composite Editor and BPM extensions installed (you can install them from Help – Check for Updates) and a BPM 11.1.1.3 server to deploy to.  Details of how to install BPM are in this post, note though that you will not need all the WebCenter components for this example.

To start, in JDeveloper, from the File menu we select New and then Application.  In the Create BPM Application wizard that appears, enter a name for your application, I called mine OrderTests, select the BPM Application template, and then click on Next.

On the next page, provide a name for your project, I used the same name, and click on Next.

On the last page of the wizard, take the default option to create a Composite with BPMN Process and click on Finish.

Your new application will be created, with a BPMN project in it, and the Create BPMN Process wizard will appear.  For this example, we will accept the default process template, just click on Next.

On the next page, give your actual BPMN process a name, I called mine OrderTests.  It is important that you give each process a different and meaningful name.  Then click on Finish.

Your new process will open in JDeveloper.  Switch the BPM Application view, if it is not already visible, as shown below on the left hand side.  Open up the Business Catalog node as shown.  We are going to create some data definitions to hold information about our tests, and to hold a group of tests.

Right click on the Business Catalog folder and select New and then Module from the popup context menu.

Enter a name for your new module, I called mine Data.  The module is really just a folder or container to hold the artefacts we are going to create.

Now right click on your new container and select New and then Business Object from the popup context menu.  A business object is a data type in BPM.

Enter a name for your new business object, I called mine Test.  Note that it will be placed into the module we created.  We could create it using an existing XML Schema, but in this case we will define it manually.  Click on OK to continue.

Click on the green plus icon at the right hand side of the Attributes section to add a new attribute to out business object.

In the dialog box, enter a name for the attribute, I called mine testNumber.  Then select a type for the attribute.  I used Int for an integer.  Then click on OK.

Create two more attributes in the same way, testDescription as a String, and …

… testResults as a String.

Your business object should now look like this:

Now we will create a second business object which will hold a collection of tests.  Right click on your Data module and select New and then Business Object from the popup context menu, as we did earlier.

Enter a name for this business object, I called mine SetOfTests, and click on OK.

Click on the green plus icon to add a new attribute, as we did before.  Enter a name for this attribute.  I called mine tests.  For this attribute, we want to use our other business object as the type.  To do this we click on the button with the elipsis (…) next to the Type field to open up the type browser.

From the pulldown Type select the <Array> option.

The display changes to present a new Element Type field that was not previously displayed.  This allows us to select what type the elements of the array will be.

Click on the elipses (…) next to Element Type and select <Component> from the list of options.  We are going to use the component we defined earlier as the type of the elements in our array.

From the list of components, choose Test and click on OK.

Your screen should now look like this:

Click on OK to continue.  Your screen should now look like the image below, indicating that the new attribute tests will be an array of the Test component/business object in the Data module/folder.  Click on OK.

Your new business object should now look like this:

Now that we have defined our types, it is time to create a variable in our process, using these types, to actually hold the data we want to work on in the process.  Return to the process using the tabs at the top of the main pane, or by double clicking on it in the navigator (top left).  When the process is open, you should see the structure view (bottom left) as shown in the image below. If you do not see it, you can open it from the View menu.  Right click on the Process Data Objects folder and select New from the popup context menu.

In the dialog box that appears, give your variable (process data object) a name, I called mine testData, and then click on the elipsis (…) next to the Type field to select the type for the variable.

In the type browser, select <Component> from the drop down list.

Then select Test from the list of components, and click on OK.

You should be returned to the dialog box, which should now look like the image below.  Click on OK to continue.

Now we will create a second variable for the set of tests, right click on the Process Data Objects folder again, and select New from the popup context menu.

I named this second variable setOfTestRequested.  Click on the elipsis (…) to choose the type, select <Component> and then choose the SetOfTests component, then click on OK.

Click on OK in the dialog box.  This second variable will hold the array of test data objects.

If you open up the Process Data Objects folder, you will see your two variables, as shown below.

Ok, now we have all our data and variables sorted out, let’s work on the process itself.  First, let’s define a role so we know what type of user will carry out the tasks in this process.  You can of course have multiple roles, but for our simple example we will only need one.  Double click on the swim lane header at the left hand side of the process, it is shown in a darker gray colour at the left hand side of the image below.  This will open the Role Properties dialog box.  Click on New to create a new role and enter a name for it.  I called mine Requestor.  Click on OK and then OK again.

You will notice now the the swim lane header shows the name of our new role.  Next, we will set up the inputs for our process.  Note that you do not have to have inputs for a process.  It is fine to have a human task at the beginning of the process where a user can enter the inputs directly.  By setting up inputs on the Start node, as we are here, we allow this process to be started (‘invoked’) as a service, or by sending it a message.  This will make it slightly easier for us to test.  Right click on the Start node and select Properties from the popup context menu (or double click on the Start node).

In the Properties – Start dialog box, select the Implementation tab, as shown in the image below.  The click on the green plus icon at the top right of the Arguments Definition section.

In the Create Argument dialog box, enter a name for your argument, I called mine input1 and then click on the elipsis (…) to choose the type.  Select <Component> and then SetOfTests from the component browser.  Then click on OK.

Tick the checkbox next to Use Associations and then click on the little pencil icon to the right, next to the Type: Simple drop down.  Drag the setOfTestsRequested variable from the right hand side into the Outputs box in the middle, as shown below.  This means that the inputs sent to the process (input1) will be stored in the variable setOfTestsRequested.  Click on OK to continue.

Your screen should now look like this.  Click on OK to continue.

Now repeat this process on the End node.  Add an argument called output1 of type SetOfTests, similarly to what you just did for the Start node.

In its associations, drag the setOfTestsRequested into the Inputs area as shown below.  This means the data in that variable will be sent out of the process as its output.

Now let’s implement the body of our process.  We will use the Subprocess object to handle the traversal of the array of tests.  Drag a Subprocess from the component palette on the right into the process and drop it on the line between the Start and End nodes.  You may want to move the End node across to the right first to give yourself some more room.  Notice that the line changes colour to blue when you hold a component over it.  If you cannot see the component palette, you can open it from the View menu.  Your process should now look like the image below.  You may need to click on the little plus icon to open the subprocess like this.

Now let’s set the properties of our subprocess.  We want one instance of the subprocess for each element in the input array.  Right click on the subprocess and select Properties from the popup context menu (or just double click on the subprocess).  Enter a name for the subprocess.  I called mine RequestIndividualTestSubprocess.  Then move to the Loop Characteristics tab.

In the Loop Characteristics field at the top, select the MultiInstance option, as shown below.  For the Loop Cardinality select XPath, and for the Is Sequential option (down the bottom), deselect (uncheck) the checkbox.

The MultiInstance option tells the system to create a separate instance of the subprocess for each item.  Because we unchecked Is Sequential, they will all be created at once, and execute in parallel.

Your screen should now look like the image below.  Click on the little calculator icon to the right of the the Loop Cardinality field.

The Expression Builder will open, and we can enter an XPath expression to tell the system how many instances of the subprocess we want to run, i.e. the ‘cardinality’ of the loop.

Type in count() in the expression field.  Then place your cursor between the parentheses, i.e. count(here).  In the Variables box at the bottom left, navigate to the tests part of the setOfTestsRequested variable, and highlight it, as shown below.  Then click on the Insert into Expression button to add it in between the parentheses.  Your screen should now look like this:

Note that you can just type in the whole expression if you want to.  Click on OK to return to the previous dialog box, which should now look like the image below.  This expression that we have entered tells the system to count how many tests nodes/elements there are inside the setOfTestsRequired variable, so if we pass in an array of two tests, the system will start two instances of the subprocess.

Now we move to the Arguments Definition tab.  On the Inputs tab inside this tab, click on the green plus icon to add a new argument.  Call it input1 and set the type to the component Test as we have done previously.

On the Outputs sub-tab, create an argument called output1 of type Test also.

Check/select the checkbox next to Use Associations and then click on the pencil icon to edit the data associations.  Click on the little calculator icon next to Inputs as shown below.

In the Expression Builder, enter the expression: setOfTestsRequested.tests[loopCounter] and press OK.  This expression selects just one of the tests in our variable.  The loopCounter is a special variable that the system will set to a different value for each instance of the subprocess, so in our example where we have two inputs, one instance of the subprocess will have loopCounter set to 1, and the other to 2.

Copy and paste this expression (or retype it) in the Outputs are too.  Your screen should now look like this:

Click on OK and then OK again.

Now let’s define what we want to do inside our subprocess.  We will send the each test to the user, to have them complete it an update the test with the results.  To do this, we will use a human task.  Drag a User task from the component palette and drop it on the line between the Start and End nodes inside the subprocess.  Then right click on this new task and select Properties from the popup context menu.  Enter a name for this task, as shown below.  I called mine PerformTheTest.

Now move to the Implementation tab.  Click on the green plus icon next to the Human Task field to define a new human task.

In the Create Human Task dialog box that appears, enter a name and title for the new human task.  I named mine PerformTestTask and set the title to Perform Test.  The title will be displayed to the users.  You need to make sure that the name of your task is different to the name of your process, or other components.  It is a good idea to put “Task” on the end of the names of your tasks so that they will be easily identifiable.

Next, click on the green plus icon next to Parameters to add a new data object for this task.  The Data Object browser will appear to the right, as shown below.  Drag the testData variable into the Parameters area and drop it.  Check/select the checkbox under the Editable column, so that users will be able to update this data object when they complete the task.  Your screen should now look like the image below.

Click on OK to return the the previous dialog box.  Check/select the checkbox for Use Associations and then click on the pencil icon.

In the Data Associations dialog box, set the inputs and outputs for the test variable to setOfTestsRequested.tests[loopCounter] as we did earlier.  Make sure you put it in the right input/output box.  Your screen should look like the image below.

Click on OK and then OK again to return the process.  That completes our definition of our process.  It is a simple process to illustrate our example, in real life, you would obviously have a more complicated process.

Now we need to create a user interface for our human task.  To do this we want to go to the “Composite” view.  This should have been opened automatically when we created our application.  You can open it by selecting its tab in the main pane.  If it is not there, go the the Application Navigator, by opening it from the View menu or selecting its tab in the top left pane, then navigate to the composite.xml under SOA Content in your project.  When you open it, the composite will display in the main pane, and should look like this:

This shows us that we have a business process (the blue box) that uses (indicated by the connecting line) a human task (the green box).  We can also see that our process is exposed as a service (the white box in the left hand column).  Double click on the green human task component to open it.  It will display in the main pane, and should look like this:

Click on the Create Form button at the top left of the main pane and select Auto-Generate Task Form… from the drop down menu.

A dialog box will appear, asking you to name a new project (in your same application) to store the user interface components.  Give it a name, I usually just add “UI” to the end of the task name, so I called mine PerformTestTaskUI.  Then press OK and wait.

After some time (10-60 seconds depending on your machine) you will see a few new tabs open in the main pane, and finally a “task flow” page like the one shown below will appear.  When this happens, press the Save All button (or select it from the File menu) to ensure everything is saved.

Now we have everything we need to deploy and test our process.  From the Application Navigator, right click on our original project (not the human task one we just created) and select Deploy and then OrderTests… from the popup context menu, as shown below.

The deployment wizard will appear.  On the first page, select Deploy to Application Server and then press Next.

On the Deploy Configuration page, we can accept the defaults.  If anything goes wrong and you need to deploy again, you will need to either increase the revision ID, or check/select the Overwrite any existing composite with the same revision ID option.  Click on Next.

On the Task flow deployment page, we need to include our human task user interface by selecting it from the list, as shown below.  Note that you only need to do this once.  If you do redeploy your process later on, you don’t need to redeploy the task user interface (unless you also change the task user interface of course).  Click on Next.

On the next page, select the server connection that you want to deploy to, and click on Next.

JDeveloper will connect to your server (there may be a small delay at this point) and retrieve a list of managed servers to which it can deploy.  In a typical development/test environment there will only be one, as shown below.  Make sure the correct one is selected, if you have more than one, and click on Next.

Check the details on the Summary page, then click on Finish to start the deployment.

In the Log pane, at the bottom of the screen, you can watch progress of the deployment in the Deployment tab.  Your deployment should progress as shown below.  Note that you will get error messages if you have done something wrong, and you will need to go and fix those mistakes before you can deploy your process and task user interface.

When you see Deployment finished as shown above, we are ready to test our process.

First, we need to go and map the role we created to an actual user, so that we will be able to see the tasks.  We do this in the BPM Workspace, which we can access at http://yourserver:8001/bpm/workspace.  Log on as an administrator, like weblogic.

You will probably see some other processes and tasks sitting in your queue, as shown below (unless this is your first process on the server).  Click on the Administration link at the top right.

Scroll down to find the role we defined in the list of roles.  It will be called OrderTests.Requestor, as shown – this is the name of our process and then the name of the role.  Click on it to select it.  The Details view will appear in the bottom half of the screen, as shown below.

Click on the little paper and star icon at the top of the Members box to add a new member.  In the popup box, enter a search term, like web*, and click on the Search button.  Then click on the weblogic user in the Available box and click on the little blue right arrow to move it across to the Selected box.  Your screen should now look like this:

Click on the OK button, and then click on the Apply button at the top right of the Details view (bottom half of screen).  Make sure you click on Apply or your changes will be lost.  You should now see that your role displays the user weblogic in the list at the top of the screen, as shown below.

At this point, you may want to log out and log in again, just to make sure your session will receive new tasks.

Now, we will use the Enterprise Manager to start an instance of our process and see what happens.  You can access the Enterprise Manager at http://yourserver:7001/em and log on as an administrative user, like weblogic.

Navigate to the soa-infra node in the SOA folder, as shown.  You will see the SOA dashboard, as shown below.  You may see some existing instances of other composites if this is not a new server.  In the Deployed Composites area, on the right, click on our newly deployed OrderTests composite.

This will open up the dashboard for the OrderTests composite, as shown below.  Click on the Test button to start a test instance.

Scroll down past the various security, addressing, etc. options to the payload data at the bottom.  In the Size box, next to the tests node, enter 2 and then click on the little refresh icon immediately to the right of the Size box.  This will add two tests elements to our input message.

Your screen should now appear like the one below.  Enter the test data as shown, or your own test data as you wish.  Then click on the Test Web Service button to start our composite.

The screen will move to the Response tab, as shown below.  Click on the Launch Message Flow Trace link to open up our new instance of our composite.

The Flow Trace window will open, as shown below.  Here we can see in the Trace section, that the OrderTests web service was called, which in turn started our OrderTests process, and that two PerformTestTask human tasks were started by our process.  Note that the status shows these are still Running.  Keep this window open, we will return to it shortly.

Return to your BPM Workspace window and click on the little blue circular arrows refresh icon above the task list.  You should see two new Perform Task tasks appear in the list, as shown below.  Click on the first one of these to highlight it.  After a moment, the user interface (which we generated earlier) for this taks will open in the bottom half of the screen, as shown below.  You will see the data that we entered for the first of our two tests elements displayed here.  Update the Test Results as shown below, and then click on the Approve button to complete the task.

Note that we are using the auto-generated user interface here.  You have complete control over the user interface, so you can make it look however you like, and can use all sorts of components like tables, buttons, maps, popup boxes, images, tabs, trains, etc.  We are just using the auto-generated one for simplicity.

Your screen will refresh and you will see that the first task has disappeared from your list of tasks.  Now process the second task as well, updating the status as shown below, and then completing it by pressing the Approve button.

Your screen will refresh and both tasks have now disappeared.  (They are still there, they have just moved to the “completed” view).

Now return to your Flow Trace window that we left a few moments ago.  Hit the little circular arrows refresh icon (top right) to update this page.  You will notice that the status of the process and the human tasks is now shown as Completed.  Click on the OrderTests link to drill down in to the process itself.

The Audit Trail view will open.  You can click on the little plus icons to open up the tree.  You can read from the audit trail that our process started, and created two instances of the subprocess, and that each of those created a human task.  Click on the instance left the activity link for one of the human tasks and note that you can see the updated data in the popup windows.  Click on each one and notice the differences.

This test confirms that our process worked as expected.  Feel free to go back and run a few more tests, with varying numbers of input tests.

So there you go, a simple method to iterate over (traverse) and update arrays of data in BPM.