Building OSB projects with Maven and removing the eclipse dependency

In this earlier post, I talked about a way to automate the build and deployment for OSB, but I did not go so far as to get that working in Maven, though you certainly could.  But, OSB PS6 has added a new tool called configjar which lets you build a sbconfig.jar file without needing to have eclipse/OEPE/OSB IDE installed on the machine where you are doing the build.  You do still need OSB, but removing that IDE dependency is a big step forward.

You can find configjar sitting under your Oracle_OSB1/tools/configjar directory in your OSB PS6 installation.  There is a readme file there that tells you how to use it from ANT and WLST.  Here, I want to show you how to use it from Maven, and therefore Hudson, etc. too.

For this post, I went into the OSB IDE and created a simple project called osbProject1 which contains a single Proxy Service called (imaginatively) ProxyService1.  It is just a plain old ‘any’ proxy service with essentially no implementation at all.  But it is enough to do what we need to do.

By the way – I have OSB and the OSB IDE running on Oracle Linux as a 64-bit application – see how here.

The configjar tools supports building sbconfig.jar files for projects and/or resources.  So you should be able to use this same approach for pretty much anything you build in OSB, possibly excepting when you have custom extensions.

If we take a look in my osbProject1 direction in my eclipse workspace, we see that it contains just a single file, ProxyService1.proxy.

We are going to add a few more files:

  • A Maven POM to control the build (pom.xml)
  • A settings.xml file that we will pass to configjar
  • import.py and import.properties files like we had in that previous post

Let’s take a look at them now.  We will start with settings.xml.  This is a file we pass to configjar to tell it how to build the sbconfig.jar for us.  This is a relatively simple file.  Here is an example:

<?xml version="1.0" encoding="UTF-8" ?>
<p:configjarSettings xmlns:p="http://www.bea.com/alsb/tools/configjar/config">
  <p:source>
    <p:project dir="/home/oracle/workspace/osbProject1"/>
    <p:fileset>
      <p:exclude name="*/target/**" />
      <p:exclude name="*/security/**" />
      <p:exclude name="*/.settings/**" />
      <p:exclude name="*/import.*" />
      <p:exclude name="*/alsbdebug.xml" />
      <p:exclude name="*/configfwkdebug.xml" />
      <p:exclude name="*/pom.xml" />
      <p:exclude name="*/settings.xml" />
      <p:exclude name="*/osbProject1.jar" />
    </p:fileset>
  </p:source>
  <p:configjar jar="/home/oracle/workspace/osbProject1/osbProject1.jar">
    <p:projectLevel includeSystem="false">
      <p:project>osbProject1</p:project>
    </p:projectLevel>
  </p:configjar>
</p:configjarSettings>

In the source element, under project we have a dir attribute that points to the OSB project directory that was created by the OSB IDE, and we most likely check out of a version control system before starting our build.  All of those exclude elements are telling it to ignore these extra files we have added to the project, and the ones that it will generate.  Without those, your sbconfig.jar will be polluted with a bunch of unnecessary stuff.  These are all standard from project to project, except for the last one, which is the name of the sbconfig.jar itself.

The jar attribute on the configjar element names the output file.  Finally, we have the project element under projectLevel that names the projects we want to include in the sbconfig.jar that we build.  The other option is to select by resources.  You might want to go check out the OSB documentation to see how to do that.

Next, we have the import.py and import.properties files.  These are essentially the same as in the previous post so I wont repeat them. The only difference is that we need to update import.properties to contain the correct filename:

importJar=osbProject1.jar

Then we need a POM.  We are going to use the maven-antrun-plugin to execute configjar through ANT to generate the sbconfig.jar, and then the exec-maven-plugin to deploy it to our OSB server.  Here is the POM:

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.redstack.osb</groupId>
  <artifactId>osbProject1</artifactId>
  <version>1.0-SNAPSHOT</version>

  <dependencies/>

  <properties>
    <osbHome>/ciroot/product_binaries/osb11.1.1.7/Oracle_OSB1</osbHome>
  </properties>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.7</version>
        <executions>
          <execution>
            <id>deafult-cli</id>
            <phase>package</phase>
            <configuration>
              <target>
                <echo>
WARNING
-------
You must set the weblogic.home and osb.home environment variables
when you invoke Maven, e.g.:
mvn compile -Dweblogic.home=/osb11.1.1.7/wlserver_10.3
-Dosb.home=/osb11.1.1.7/Oracle_OSB1
                </echo>
                <taskdef name="configjar"
                  classname="com.bea.alsb.tools.configjar.ant.ConfigJarTask"
                  classpathref="maven.plugin.classpath"/>
                <configjar settingsFile="${basedir}/settings.xml"
                  debug="true">
                </configjar>
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
        <dependencies>
          <dependency>
            <groupId>org.apache.ant</groupId>
            <artifactId>ant</artifactId>
            <version>1.7.1</version>
          </dependency>
          <dependency>
            <groupId>org.apache.ant</groupId>
            <artifactId>ant-launcher</artifactId>
            <version>1.7.1</version>
          </dependency>
          <dependency>
            <groupId>org.apache.ant</groupId>
            <artifactId>ant-nodeps</artifactId>
            <version>1.7.1</version>
          </dependency>
          <dependency>
            <groupId>org.apache.ant</groupId>
            <artifactId>ant-apache-bsf</artifactId>
            <version>1.7.1</version>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>configjar</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/tools/configjar/configjar.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>weblogic.server.modules_10.3.6.0</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../modules/features/weblogic.server.modules_10.3.6.0.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>weblogic</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../wlserver_10.3/server/lib/weblogic.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>oracle.http_client_11.1.1</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../oracle_common/modules/oracle.http_client_11.1.1.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>xmlparserv2</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../oracle_common/modules/oracle.xdk_11.1.0/xmlparserv2.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>orawsdl</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../oracle_common/modules/oracle.webservices_11.1.1/orawsdl.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>wsm-dependencies</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/../oracle_common/modules/oracle.wsm.common_11.1.1/wsm-dependencies.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>osb.server.modules_11.1.1.7</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/modules/features/osb.server.modules_11.1.1.7.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>oracle.soa.common.adapters</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>log4j_1.2.8</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/lib/external/log4j_1.2.8.jar</systemPath>
          </dependency>
          <dependency>
            <groupId>com.oracle.osb</groupId>
            <artifactId>alsb</artifactId>
            <version>11.1.1.7</version>
            <scope>system</scope>
            <systemPath>${osbHome}/lib/alsb.jar</systemPath>
          </dependency>
        </dependencies>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>deploy</id>
            <phase>pre-integration-test</phase>
            <configuration>
              <executable>/bin/bash</executable>
              <arguments>
                <argument>${osbHome}/../oracle_common/common/bin/wlst.sh</argument>
                <argument>${basedir}/import.py</argument>
                <argument>${basedir}/import.properties</argument>
              </arguments>
            </configuration>
            <goals>
              <goal>exec</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Let’s take a look at the important parts of the POM that you will need to adjust to suit your environment:

  • We define a property called osbHome which is then used throughout the POM.  This needs to point to your Oracle_OSB1 directory.
  • In the plugin section for maven-antrun-plugin you can see that we have a taskdef to import the configjar task from the supplied configjar-ant.xml which comes with OSB.  We also have added a bunch of dependencies to this plugin so that it has the right OSB libraries in the classpath when it executes.  Note that they all have the scope set to system which means you don’t need to import them into a Maven repository first.  You can just specify a path to locate them.  Of course, this ties your build to that one machine effectively.  You can of course just go ahead and put those jars into a Maven repository and then use them normally, though since configjar depends on a local OSB install, there is not a lot of benefit right now.
  • Finally, in the plugin entry for exec-maven-plugin you can see that we execute wlst.sh and pass it our import.py script to do the deployment.

The configjar tool also requires two environment variables to be set, so you need to set these when you run Maven.  Here is an example:


mvn verify -Dweblogic.home=/ciroot/product_binaries/osb11.1.1.7/wlserver_10.3 -Dosb.home=/ciroot/product_binaries/osb11.1.1.7/Oracle_OSB1

This will build the sbconfig.jar and deploy it to OSB for us.  Here it is:

osb

Good luck!  Enjoy!  And a big thank you to Dimitri Laloue who helped me get configjar working.

About Mark Nelson

Mark Nelson is an Architect ("IC6") in the Platform Architecture Team in Oracle Development. Mark's focus area is continuous delivery, configuration management and provisioning - making it simple to manage the configuration of complex environments and applications built with Oracle Database, Fusion Middleware and Fusion Applications, on-premise and in the cloud. Before joining the Platform Architecture team, Mark was a senior member of the A-Team since 2010, and worked in Sales Consulting at Oracle since 2006 and various roles at IBM since 1994.
This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

11 Responses to Building OSB projects with Maven and removing the eclipse dependency

  1. fantastic, this is a major improvement, the eclipse dependency is a MAJOR pain which costs us countless hours of frustration.
    I wonder if one can use OSB PS6 to build old OSB 10.3.5 code…

  2. This impies that despite using maven, you have the OSB installed where you do your build. Currently we have a dedicate build machine where we have no OSB installed. Currently (using older version of software) we have maven and all jars get pullled down from maven repos. We are currently upgrading to latest and greatest, and would like to have same setup. However your blog includes setting osb.home in your pom? Why is this required ? Can we not just pull down required jars by pom references?

    • Mark Nelson says:

      Hi Bernadette, Thanks for taking the time to comment. I think it is probably possible, although maybe not with this new configjar tool, to get it all working from Maven JARs without a local OSB home, I have not actually tried that for OSB. I did get it working for SOA though. Hopefully, we will get to a point where we can build for all of the FMW products without the need for the build server to have access to either a product home or an IDE, but I guess that will be a few releases down the track..

  3. Pingback: SOA Community Newsletter May 2013 | SOA Community Blog

  4. Pingback: Building OSB projects with Maven and removing the eclipse dependency by Mark Nelson | SOA Community Blog

  5. archenroot says:

    Hi Mark,
    very good continuing with your last post!

    I have my local dev on windows (don’t like it at all), and maven build is throwing me exception like it is trying to compile on linux machine:
    [INFO] Executed tasks
    [INFO]
    [INFO] — exec-maven-plugin:1.2.1:exec (deploy) @ osbProject1 —
    [INFO] ————————————————————————
    [INFO] BUILD FAILURE
    [INFO] ————————————————————————
    [INFO] Total time: 10.975s
    [INFO] Finished at: Thu Sep 12 16:40:29 BST 2013
    [INFO] Final Memory: 19M/217M
    [INFO] ————————————————————————
    [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (deploy) on project osbProject1: Command execution failed. Cannot run program “\bin\bash” (in director
    y “c:\Dev\workspaces\osb-svn\osbProject1″): CreateProcess error=2, The system cannot find the file specified -> [Help 1]
    org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (deploy) on project osbProject1: Command execution fai
    led.

    I have no idea, but will try to take a look into maven configuration little bit.

    Ladislav

    • Mark Nelson says:

      Thanks, let me know how you go. I will try to take a look too when I can :)

      • Hi Mark,
        I moved everything to linux machine. The script you provided works like a charm finally without any issue. I also found a repository created by “SOA God” :-) Edwin Biemond at https://github.com/biemond/soa_tools where he provides complete build framework ANT/Maven for different versions of OSB and SOA-infra. It looks your import.py is very similar to his last PS6 version.

        I continued by setting up Hudson open-source for automated builds and it is very easily integrate able with your setup. I also setup Artifactory open-source version for internal repositories setup, uploaded there the WL and OSB libraries and was able to remove following tags from the setup:

        And the libraries are loaded from manageable repository. Which is fine.

        Now the tricky part comes. Automatic testing. On my last project with SOA Suite we spend lots of time with manual test. Now I work for IBM on project in Ireland and the situation is very similar, so the idea came alone. Develop testing framework. I was playing around with Citrus framework, but found it too robust(which is of course good, but not in our case), and it requires to create to much configuration to make it working. Maybe I am wrong as soon as I am pretty new.

        We are building E2E data integration with very few human-workflow tasks and in our place there are main types of endpoints used:
        - HTTP – SOAP (Proxy and Businnes services)
        - JMS Server – we are using only queues(distributed), no topics
        - FTP Server – transfer messages
        - File System – archiving messages
        - Database – insert/pool

        There is for example following integration scenario used:
        DB -> Proxy pool -> Business service -> JMS Queue -> Proxy pool -> Business service -> DB insert
        Similar scenarios with different kind of services in place are used in 80-90% of integration projects.

        From this perspective I need to test DB-> Queue transport, Queue-> DB transport, and test the whole DB->DB transport.

        For this purpose I started on Monday writing down a framework which is now almost complete in Java, I am using SoapUI maven plugin and inside the SoapUI I am creating tasks as groovy scripts which instantiate the JAR classes. So the whole work is hidden and groovy scripts are really just working mostly with assertions.

        The framework is now able to do following:
        A) based on endpoint type:
        1.) generate CRUD database scripts (INSERT, SELECT, UPDATE, DELETE) now only based on one table/view object
        2.) generate dynamically using soap-ui.jar SOAP-envelope for SOAP services(wsdl)
        2.) generate queue message based on XML schema
        Files are saved into test folder and developer can edit them to put into columns/elements required values before the run

        B) Once executed first of all I disable (this is done by OSB libraries and message beans) the jms queue proxy pooling service, so I can test the DB->Queue (half of process) first, insert the row into table, than connect by QueueBrowser to queue and get the message, validate it against the provided xsd. On top of this I check if the values are correct, in simple case we are using 1:1 mapping (column -> element). If this is correct, I am enabling the pooling proxy, the message is picked up and send to the end database. I do the check value test again. I

        C) XQuery transformation test – not done yet. The heart lies in simple XML Validation. It will pickup the query, generate sample XSD instance file(XML) for input(developer can edit it before test execution of course), then it will validate input against XSD, do XQuery transform, and validate output against XSD.

        D) Generate HTML/PDF reports, so those can be commited into code repository as results. I am already using soap-ui-maven-plugin functionality to print the results in JUnit format, but this is only good for tracking build result within Hudson itself, so you can see graph of build history and see the result of test in case it failed.

        That is all for now which will save a lot of time for our team. I should run demo next week, so hope it will work :-)

        There is also to write down own maven plugin, on the other hand I can take more time to produce (I am not in development lab, but on customer side, so time is big parameter), so I am happy with soap-ui maven plugin and possibility to use SoapUI Groovy scripts within the TestCases.

        Let me know what you think, I hope I describe the approach clearly…

        I have 2 questions or idea for you and me :-):
        Can I automatically undeploy OSB project in case of the one of the step will fail?
        Can I mark some of the tests using the soap-ui plugin to be criticatl(undeploy in case of fail) and others optional(don’t undeploy in case of fail)?

        I of course can connect to managed servers trought OSB API and do it in java, but I was thinging if there isn’t already some solution completed.

        Still, thank you for your hard work and for this post, it helped me a lot.

        Best regards,

        Ladislav

      • Mark Nelson says:

        Hi Ladislav,
        Sounds great. You might like to check out Robot Framework (www.robotframework.org) as a potential integration/acceptance test framework to use. It is not too rigid, and is fairly extensible (Java, Python and XML/RPC) – will also let you define critical/not critical test suites/cases. You might consider generating your reports in some format like JUnit instead of PDF, so that you can feed them back into your CI server (hudson, jenkins, etc.) and track change over time – not sure if you were saying you already do that…
        Java is probably your best option for undeploying right now, but who knows what will come in the future :)

      • Hi Mark, thanks for response, yes you are right I already red you other article regarding the Robot Framework, this one looks really better for the first sight against Citrus Framework.

        Still I already started with my own framework :-))

        https://github.com/archenroot/soa-testing-framework

        Keep going, you have really valuable articles to read.

        Ladislav

  6. Pingback: Additional new content SOA & BPM Partner Community | SOA Community Blog

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s