Extending Continuous Integration to include MDS-dependent components

In this earlier post, I presented a simple approach to get started with continuous integration for SOA and BPM projects and showed how you can build and deploy an SCA composite using Hudson and Maven.  In that example, we just deployed an empty composite, and while that worked well, we usually want to put something in our composites!

If you follow that example and put say a BPEL or BPMN process in your composite, or a reference to a web service, then it will all work fine.  But, if you add a component that requires Metadata Services (MDS), then you will run into some trouble.

In this post, we will look at how we can expand on what we did previously so that we can handle components that require MDS.  Two example are Business Rules and Human Tasks.  Both of these will consume data definitions (XSDs) from MDS.  If you go ahead and add one to your project, then search for “oramds” in your project, you will see the references that were added.

I will assume for this post that you know how to add a Human Task or Business Rule to your composite.  Go ahead and do that now.  If you do not know how, then we have an example of adding a human task here.

Once you have that taken care of, you will need to add a Maven POM to you project, as we did in the earlier post.  We need to add a couple of additional items to handle building composites that include components that depend on MDS.

Here is the POM, discussion follows:

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>ScaProject2</groupId>
  <artifactId>ScaProject2</artifactId>
  <version>1.0-SNAPSHOT</version>

  <scm>
    <connection>scm:svn:https://administrator@bpm.mark.oracle.com/svn/ScaProject2/trunk</connection>
    <developerConnection>scm:svn:https://administrator@bpm.mark.oracle.com/svn/ScaProject2/trunk</developerConnection>
  </scm>

  <dependencies>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.6</version>
        <executions>
          <execution>
            <id>sca-compile</id>
            <phase>compile</phase>
            <configuration>
              <target>
                <property name="scac.input" value="${basedir}/composite.xml" />
                <property name="scac.application.home" value="${basedir}/.." />
                <ant antfile="c:/Oracle/Middleware/Oracle_SOA1/bin/ant-sca-compile.xml"
                     dir="c:/Oracle/Middleware/Oracle_SOA1/bin"
                     target="scac" />
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
          <execution>
            <id>sca-package</id>
            <phase>package</phase>
            <configuration>
              <target>
                <property name="build.compiler" value="extJavac"/>
                <property name="compositeName" value="${project.artifactId}" />
                <property name="compositeDir" value="${basedir}" />
                <property name="revision" value="${project.version}" />
                <property name="scac.application.home" value="${basedir}/.." />
                <ant antfile="c:/Oracle/Middleware/Oracle_SOA1/bin/ant-sca-package.xml"
                     dir="c:/Oracle/Middleware/Oracle_SOA1/bin"
                     target="package" />
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
          <execution>
            <id>sca-deploy</id>
            <phase>deploy</phase>
            <configuration>
              <target>
                <property name="serverURL" value="http://bpm.mark.oracle.com:7001" />
                <property name="user" value="weblogic" />
                <property name="password" value="welcome1" />
                <property name="sarLocation" value="${basedir}/deploy/sca_${project.artifactId}_rev${project.version}.jar" />
                <property name="overwrite" value="true" />
                <property name="forceDefault" value="true" />
                <property name="partition" value="default" />
                <ant antfile="c:/Oracle/Middleware/Oracle_SOA1/bin/ant-sca-deploy.xml"
                     dir="c:/Oracle/Middleware/Oracle_SOA1/bin"
                     target="deploy" />
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

  <distributionManagement>
    <!-- use the following if you're not using a snapshot version. -->
    <repository>
      <id>local</id>
      <name>local repository</name>
      <url>file:///c:/users/administrator/.m2/repository</url>
    </repository>
    <!-- use the following if you ARE using a snapshot version. -->
    <snapshotRepository>
      <id>localSnapshot</id>
      <name>local snapshot repository</name>
      <url>file:///c:/users/administrator/.m2/repository</url>
    </snapshotRepository>
  </distributionManagement>

</project>

First, notice we provide standard Maven coordinates (groupId, artifactId, version) for our project:

  <groupId>ScaProject2</groupId>
  <artifactId>ScaProject2</artifactId>
  <version>1.0-SNAPSHOT</version>

Next, we have added the scac.application.home property to both the sca-compile and sca-package ‘executions.’  This needs to point to the location where the JDeveloper Application is located.  For a bit of variety, this example was built on Windows 7 (64-bit Ultimate) so you can see the Windows-style filenames in the POM.

          <execution>
            <id>sca-package</id>
            <phase>package</phase>
            <configuration>
              <target>
                <property name="build.compiler" value="extJavac"/>
                <property name="compositeName" value="${project.artifactId}" />
                <property name="compositeDir" value="${basedir}" />
                <property name="revision" value="${project.version}" />
                <property name="scac.application.home" value="${basedir}/.." />
                <ant antfile="c:/Oracle/Middleware/Oracle_SOA1/bin/ant-sca-package.xml"
                     dir="c:/Oracle/Middleware/Oracle_SOA1/bin"
                     target="package" />
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>

Setting the scac.application.home is required when you use MDS-dependent components, as the build tasks will look for a configuration file called {scac.application.home}/.adf/adf-config.xml in order to find out where your MDS is located.  If you do not set this parameter, the (ANT) build tasks will not be able to find your MDS and the build will fail.

Here we see where the POM and ADF config are located in the project:

You also need to add some details into this file to point to your MDS.  Here is an example file, discussion follows:

<?xml version="1.0" encoding="windows-1252" ?>
<adf-config xmlns="http://xmlns.oracle.com/adf/config"
            xmlns:config="http://xmlns.oracle.com/bc4j/configuration"
            xmlns:adf="http://xmlns.oracle.com/adf/config/properties"
            xmlns:sec="http://xmlns.oracle.com/adf/security/config">
  <adf-adfm-config xmlns="http://xmlns.oracle.com/adfm/config">
    <defaults useBindVarsForViewCriteriaLiterals="true"/>
    <startup>
      <amconfig-overrides>
        <config:Database jbo.locking.mode="optimistic"/>
      </amconfig-overrides>
    </startup>
  </adf-adfm-config>
  <adf:adf-properties-child xmlns="http://xmlns.oracle.com/adf/config/properties">
    <adf-property name="adfAppUID" value="ScaTest2-9609"/>
  </adf:adf-properties-child>
  <sec:adf-security-child xmlns="http://xmlns.oracle.com/adf/security/config">
    <CredentialStoreContext credentialStoreClass="oracle.adf.share.security.providers.jps.CSFCredentialStore"
                            credentialStoreLocation="../../src/META-INF/jps-config.xml"/>
  </sec:adf-security-child>
  <adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace metadata-store-usage="mstore-usage_1" path="/soa/shared"/>
        </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_1">
            <metadata-store class-name="oracle.mds.persistence.stores.db.DBMetadataStore">
              <property name="jdbc-userid" value="dev_mds"/>
              <property name="jdbc-password" value="welcome1"/>
              <property name="jdbc-url"
                        value="jdbc:oracle:thin://@localhost:1521:bpm"/>
              <property name="partition-name" value="soa-infra"/>
            </metadata-store>
          </metadata-store-usage>
        </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config>
</adf-config>

The important section here is the metadata-store-usage part.  In the example above you can see how a database-based MDS is defined.  In this example, I have it pointing to the MDS on the SOA/BPM server where I will deploy the composite.  The MDS is defined using the JDBC details to connect to the database that stores the MDS – you will have created it during the RCU (Repository Creation Utility) phase of the installation.  You might not have noticed it or known what it was for – but you created a schema called DEV_MDS (or something similar) which contains your server’s MDS repository.

You can also see in the namespace entry it defines the path within the MDS repository as /soa/shared.  This is the area in MDS where the common XSD schema are stored.  You can browse through it and have a look at them by creating a SOA-MDS connection in JDeveloper.  If you don’t know how to do that – you can find an example in this post, towards the end of the post.

If you prefer to use a file-based MDS, your adf-config.xml will need to contain a section a little more like this:

 <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
   <property value="c:/oracle/middleware/jdeveloper/integration" name="metadata-path"/>
   <property value="seed" name="partition-name"/>
 </metadata-store>

This demonstrates using the file-based MDS repository that is included with JDeveloper.  You can take a look at it by looking under your JDeveloper directory inside integration and then seed.

Once you have these two files in place, and have edited them to match your environment, you can go ahead and check in everything to Subversion, like we did in the earlier post.  Make sure you get all the WSDL and XSD files and so on.  You might need to go into the Pending Changes view to make sure you have everything you need.  The default selections may not get everything.  You can open this view from the View menu, then Team then Pending Changes.

Open the Candidates tab, click on Refresh (the little blue arrows icon), highlight anything that you will need – basically anything that is not in the classes or deploy directories, and add it using the little green plus icon.

Then go to the Outgoing tab, highlight all the files and click on the little green tick icon to commit the files to Subversion.

Set up a Hudson job, as described in the earlier post, and then run it.  You should get your composite built and deployed to your server.  Here is an example of my Hudson build output:

Started by an SCM change
Updating https://bpm.mark.oracle.com/svn/ScaProject2/trunk
A         ScaProject2\Humantask1.componentType
A         ScaProject2\.taskeditor\Humantask1_graphics.xml
A         ScaProject2\xsd\Humantask1Payload.xsd
A         ScaProject2\xsd\Humantask1WorkflowTask.xsd
A         ScaProject2\Humantask1.task
At revision 39
Found mavenVersion 2.2.1 from file jar:file:/c:/apache-maven-2.2.1/lib/maven-2.2.1-uber.jar!/META-INF/maven/org.apache.maven/maven-core/pom.properties
Parsing POMs
[ScaProject2] $ c:\java\jdk1.6.0_25/bin/java -classpath c:\Oracle\Middleware\oracle_common\modules\oracle.mds_11.1.1\oramds.jar -cp C:\hudson\plugins\maven-plugin\WEB-INF\lib\maven-agent-2.0.0.jar;c:\apache-maven-2.2.1\boot\classworlds-1.1.jar hudson.maven.agent.Main c:\apache-maven-2.2.1 C:\hudson\war\WEB-INF\lib\hudson-remoting-2.0.0.jar C:\hudson\plugins\maven-plugin\WEB-INF\lib\maven-interceptor-2.0.0.jar 51179 C:\hudson\plugins\maven-plugin\WEB-INF\lib\maven2.1-interceptor-1.2.jar
<===[HUDSON REMOTING CAPACITY]===>channel started
Executing Maven:  -B -f C:\hudson\jobs\ScaProject2\workspace\ScaProject2/pom.xml deploy
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building Unnamed - ScaProject2:ScaProject2:jar:1.0-SNAPSHOT
[INFO]    task-segment: [deploy]
[INFO] ------------------------------------------------------------------------
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\hudson\jobs\ScaProject2\workspace\ScaProject2\src\main\resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] No sources to compile
[INFO] [antrun:run {execution: sca-compile}]
[INFO] Executing tasks

main:

scac:
Validating composite "C:\hudson\jobs\ScaProject2\workspace\ScaProject2/composite.xml"
     [scac] BPEL 2.0 validation of "testBpel2" took 189.9 milliseconds
     [scac] BPEL 2.0 validation of "testBpel2sub" took 26.0 milliseconds
     [scac] ### clazz: interface scaproject2.myServiceInterface
     [scac] warning: in Humantask1.task: Task owner not specified
     [scac] warning: in Humantask1.task: Error assignee not specified
[INFO] Executed tasks
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\hudson\jobs\ScaProject2\workspace\ScaProject2\src\test\resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] No sources to compile
[INFO] [surefire:test {execution: default-test}]
[INFO] No tests to run.
[HUDSON] Recording test results
[INFO] [jar:jar {execution: default-jar}]
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] [antrun:run {execution: sca-package}]
[INFO] Executing tasks

main:
     [echo] oracle.home = c:\Oracle\Middleware\Oracle_SOA1\bin/..
    [input] skipping input as property compositeDir has already been set.
    [input] skipping input as property compositeName has already been set.
    [input] skipping input as property revision has already been set.

clean:
     [echo] deleting C:\hudson\jobs\ScaProject2\workspace\ScaProject2/deploy/sca_ScaProject2_rev1.0-SNAPSHOT.jar
   [delete] Deleting: C:\hudson\jobs\ScaProject2\workspace\ScaProject2\deploy\sca_ScaProject2_rev1.0-SNAPSHOT.jar

init:

scac-validate:
     [echo] Running scac-validate in C:\hudson\jobs\ScaProject2\workspace\ScaProject2/composite.xml
     [echo] oracle.home = c:\Oracle\Middleware\Oracle_SOA1\bin/..
    [input] skipping input as property compositeDir has already been set.
    [input] skipping input as property compositeName has already been set.
    [input] skipping input as property revision has already been set.

scac:
Validating composite "C:\hudson\jobs\ScaProject2\workspace\ScaProject2/composite.xml"
     [scac] BPEL 2.0 validation of "testBpel2" took 210.1 milliseconds
     [scac] BPEL 2.0 validation of "testBpel2sub" took 29.5 milliseconds
     [scac] ### clazz: interface scaproject2.myServiceInterface
     [scac] warning: in Humantask1.task: Task owner not specified
     [scac] warning: in Humantask1.task: Error assignee not specified

package:
     [echo] oracle.home = c:\Oracle\Middleware\Oracle_SOA1\bin/..
    [input] skipping input as property compositeDir has already been set.
    [input] skipping input as property compositeName has already been set.
    [input] skipping input as property revision has already been set.

compile-source:
     [echo] deleting .class files of C:\hudson\jobs\ScaProject2\workspace\ScaProject2/SCA-INF/classes
    [javac] c:\Oracle\Middleware\Oracle_SOA1\bin\ant-sca-package.xml:183: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
    [javac] Compiling 2 source files to C:\hudson\jobs\ScaProject2\workspace\ScaProject2\SCA-INF\classes
    [mkdir] Created dir: C:\hudson\jobs\ScaProject2\workspace\ScaProject2\dist
     [copy] Copying 36 files to C:\hudson\jobs\ScaProject2\workspace\ScaProject2\dist
     [copy] Copying 2 files to C:\hudson\jobs\ScaProject2\workspace\ScaProject2\dist\SCA-INF\classes
      [jar] Building jar: C:\hudson\jobs\ScaProject2\workspace\ScaProject2\deploy\sca_ScaProject2_rev1.0-SNAPSHOT.jar
   [delete] Deleting directory C:\hudson\jobs\ScaProject2\workspace\ScaProject2\dist
[INFO] Executed tasks
[INFO] [install:install {execution: default-install}]
[INFO] Installing C:\hudson\jobs\ScaProject2\workspace\ScaProject2\target\ScaProject2-1.0-SNAPSHOT.jar to C:\.m2\repository\ScaProject2\ScaProject2\1.0-SNAPSHOT\ScaProject2-1.0-SNAPSHOT.jar
[INFO] [deploy:deploy {execution: default-deploy}]
[INFO] Retrieving previous build number from localSnapshot
Uploading: file:///c:/users/administrator/.m2/repository/ScaProject2/ScaProject2/1.0-SNAPSHOT/ScaProject2-1.0-20110527.112732-11.jar
2K uploaded  (ScaProject2-1.0-20110527.112732-11.jar)
[INFO] Retrieving previous metadata from localSnapshot
[INFO] Uploading repository metadata for: 'artifact ScaProject2:ScaProject2'
[INFO] Uploading project information for ScaProject2 1.0-20110527.112732-11
[INFO] Retrieving previous metadata from localSnapshot
[INFO] Uploading repository metadata for: 'snapshot ScaProject2:ScaProject2:1.0-SNAPSHOT'
[INFO] [antrun:run {execution: sca-deploy}]
[INFO] Executing tasks

main:
     [echo] oracle.home = c:\Oracle\Middleware\Oracle_SOA1\bin/..

deploy:
    [input] skipping input as property serverURL has already been set.
    [input] skipping input as property sarLocation has already been set.
setting user/password..., user=weblogic
Processing sar=C:\hudson\jobs\ScaProject2\workspace\ScaProject2/deploy/sca_ScaProject2_rev1.0-SNAPSHOT.jar
Adding sar file - C:\hudson\jobs\ScaProject2\workspace\ScaProject2\deploy\sca_ScaProject2_rev1.0-SNAPSHOT.jar
INFO: Creating HTTP connection to host:bpm.mark.oracle.com, port:7001
INFO: Received HTTP response from the server, response code=200
---->Deploying composite success.
[INFO] Executed tasks
[HUDSON] Archiving C:\hudson\jobs\ScaProject2\workspace\ScaProject2\pom.xml to C:\hudson\jobs\ScaProject2\modules\ScaProject2$ScaProject2\builds\2011-05-27_21-27-10\archive\ScaProject2\ScaProject2\1.0-SNAPSHOT\pom.xml
[HUDSON] Archiving C:\hudson\jobs\ScaProject2\workspace\ScaProject2\target\ScaProject2-1.0-SNAPSHOT.jar to C:\hudson\jobs\ScaProject2\modules\ScaProject2$ScaProject2\builds\2011-05-27_21-27-10\archive\ScaProject2\ScaProject2\1.0-20110527.112732-11\ScaProject2-1.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26 seconds
[INFO] Finished at: Fri May 27 21:27:38 EST 2011
[INFO] Final Memory: 22M/333M
[INFO] ------------------------------------------------------------------------
channel stopped
Finished: SUCCESS