Packaging Manual

Preface

This manual describes how to package applications for use in XL Deploy.

See the XL Deploy Reference Manual for background information on XL Deploy and deployment concepts.

Introduction

The XL Deploy deployment automation tool is designed to help you deploy application packages to target middleware. To make it possible to deploy your applications, they must be packaged in a format that XL Deploy understands. This manual describes XL Deploy's standard package format, the Deployment ARchive (DAR) format and various other topics related to packaging applications.

XL Deploy also offers the plug points to implement a custom importer that recognizes a different format.

Packages

XL Deploy uses the Unified Deployment Model (UDM) to structure its deployments (see the XL Deploy Reference Manual for more information). In this model, deployment packages are containers for complete application distribution, that include both the application artifacts (EAR files, static content) as well as the resource specifications (datasources, topics, queues, etc.) that the application needs to run.

Packages should be independent of the target environment and contain customization points (for instance placeholders in configuration files) that supply environment-specific values to the deployed application. This enables a single artifact to make the entire journey from development to production.

DAR Format

Out of the box, XL Deploy supports its own DAR format for packages. A valid DAR package has the following characteristics:

It's a ZIP archive.
It has either one of the two following manifest files describing the contents of the package:
- A file called deployit-manifest.xml
- Deprecated: A file called META-INF/MANIFEST.MF

Valid DAR archives can be produced using standard command line tools, such as zip, the Java jar utility, the Maven jar plugin or the Ant jar task. There is also a XL Deploy maven plugin to facilitate packaging. See the section Using the Maven plugin below.

In addition to packages in a compressed archive format, XL Deploy can also import exploded DARs or archives that have been extracted.

XML Manifest Format

The manifest file included in a DAR describes the contents of the archive for XL Deploy. When importing a package, the manifest is used to construct CIs in XL Deploy's repository based on the contents of the imported package. The format is based on XML.

A valid XL Deploy XML manifest contains at least the following tags:

<?xml version="1.0" encoding="UTF-8"?>
<udm.DeploymentPackage version="1.0" application="PetClinic">
  <deployables>
    ...
  </deployables>
</udm.DeploymentPackage>

Adding artifacts

Within the deployable tags you can add the deployables that make up your package. Suppose that we have an ear file and a directory containing configuration files in our package. We specify those as follows.

<deployables>
  <jee.Ear name="AnimalZooBE" file="AnimalZooBE-1.0.ear">
  </jee.Ear>
  <file.Folder name="configuration-files" file="conf">
  </file.Folder>
</deployables>

The element name is the type of configuration item that will be created in XL Deploy. The name attribute corresponds to the name the configuration item will get, and the file attribute points to an actual resource found in the package.

Adding resource specifications

The deployables element can contain more than just the artifacts that make up the package. You can also add resource specifications to it. For instance the specification for a datasource. These are defined in a similar manner as artifacts, but they do not contain the file attribute:

<was.OracleDatasourceSpec name="petclinicDS">
  <url>jdbc:mysql://localhost/petclinic</url>
  <username>petclinic</username>
  <password>my$ecret</password>
</was.OracleDatasourceSpec>

This will create a was.OracleDatasourceSpec with the properties url, username and password set to their corresponding values. See the Command Line Interface (CLI) Manual for information or how to obtain the list of properties that a particular CI type supports, or consult the relevant CI reference documentation.

Setting Complex Properties

The above example showed how to set string properties to a certain value. In addition to strings, XL Deploy also supports references to other CIs, sets of strings, maps of string to string, booleans and enumerations. Here are some examples.

Properties referring to other CIs

<sample.Sample name="referencing">
  <ciReferenceProperty ref="AnimalZooBE" />
  <ciSetReferenceProperty>
    <ci ref="AnimalZooBE" />
  </ciSetReferenceProperty>
  <ciListReferenceProperty>
    <ci ref="AnimalZooBE" />
  </ciListReferenceProperty>
</sample.Sample>

Set of strings properties

To set a set of strings property to contain strings "a" and "b":

<sample.Sample name="setOfStringSample">
  <setOfStrings>
    <value>a</value>
    <value>b</value>
  </setOfStrings>
</sample.Sample>

List of strings properties

To set a list of strings property to contain strings "a" and "b":

<sample.Sample name="listOfStringSample">
  <listOfStrings>
    <value>a</value>
    <value>b</value>
  </listOfStrings>
</sample.Sample>

Map of string to string properties

To set a map of string to string property to contain pairs "key1", "value1" and "key2", "value2":

<sample.Sample name="mapStringStringSample">
  <mapOfStringString>
    <entry key="key1">value1</entry>
    <entry key="key2">value2</entry>
  </mapOfStringString>
</sample.Sample>

Boolean and enumeration properties

To set a boolean property to true or false:

<sample.Sample name="booleanSample">
  <booleanProperty>true</booleanProperty>
  <anotherBooleanProperty>false</anotherBooleanProperty>
</sample.Sample>

To set an enum property to a specific value:

<sample.Sample name="enumSample">
  <enumProperty>ENUMVALUE</enumProperty>
</sample.Sample>

Embedded CIs

It is also possible to include embedded CIs in a deployment package. Embedded CIs are nested under their parent CI and property. Here is an example:

<iis.WebsiteSpec name="NerdDinner-website">
  <websiteName>NerdDinner</websiteName>
  <physicalPath>C:\inetpub\nerddinner</physicalPath>
  <applicationPoolName>NerdDinner-applicationPool</applicationPoolName>
  <bindings>
    <iis.WebsiteBindingSpec name="NerdDinner-website/88">
      <port>8080</port>
    </iis.WebsiteBindingSpec>
  </bindings>
</iis.WebsiteSpec>

The embedded CI iis.WebsiteBindingSpec CI type is an embedded CI under it's parent, iis.WebsiteSpec. The property bindings on the parent stores a list of iis.WebsiteBindingSpec instances.

For more information about embedded CIs, see the Reference Manual.

Using Placeholders in CI properties

XL Deploy supports the use of placeholders to customize a package for deployment to a specific environment. CI properties specified in a manifest file can also contain placeholders. These placeholders are resolved from dictionary CIs during a deployment (see the XL Deploy Reference Manual for an explanation of placeholders and dictionaries). This is an example of using placeholders in CI properties in a was.OracleDatasourceSpec CI:

<was.OracleDatasourceSpec name="petclinicDS">
  <url>jdbc:mysql://localhost/petclinic</url>
  <username>{{DB_USERNAME}}</username>
  <password>{{DB_PASSWORD}}</password>
</was.OracleDatasourceSpec>

Please note that placeholders can also be used in the name of a CI:

<was.OracleDatasourceSpec name="{{PETCLINIC_DS_NAME}}">
  <url>jdbc:mysql://localhost/petclinic</url>
  <username>{{DB_USERNAME}}</username>
  <password>{{DB_PASSWORD}}</password>
</was.OracleDatasourceSpec>

XL Deploy also supports an alternative way of using dictionary values for CI properties. If the dictionary contains keys of the form deployedtype.property, these properties are automatically filled with values from the dictionary (provided they are not specified in the deployable). This makes it possible to use dictionaries without specifying placeholders. For example, the above could also have been achieved by specifying the following keys in the dictionary:

was.OracleDatasource.username
was.OracleDatasource.password

Scanning for placeholders in Artifacts

XL Deploy scans files in packages for the presence of placeholders. These will be added to the placeholders field in the artifact, so that they can be replaced upon deployment of said package.

The default behavior is to scan text files only. It's also possible to scan inside archives (Ear, War or Zip files), but this option is not active by default.

You can enable or disable placeholder scanning by setting the scanPlaceholders flag on an artifact.

<file.File name="sample" file="sample.txt">
  <scanPlaceholders>false</scanPlaceholders>
</file.File>

Using this technique, you can enable placeholder scanning inside a particular archive.

<jee.Ear name="sample Ear" file="WebEar.ear">
  <scanPlaceholders>true</scanPlaceholders>
</jee.Ear>

It's also possible to enable placeholder scanning for all archives. To do this edit deployit-defaults.properties and add the following line:

udm.BaseDeployableArchiveArtifact.scanPlaceholders=true

To avoid scanning of binary files, only files with the following extensions are scanned:

cfg, conf, config, ini, properties, props, txt, asp, aspx,
htm, html, jsf, jsp, xht, xhtml, sql, xml, xsd, xsl, xslt

You can change this list by setting the textFileNamesRegex property on the udm.BaseDeployableArtifact in the deployit-defaults.properties file. Note that it takes a regular expression. It is also possible to change this on any of its subtypes which is important if you only want to change that for certain types of artifacts.

If you want to enable placeholder scanning, but the package contains several files that should not be scanned, use the excludeFileNamesRegex property on the artifact:

<jee.War name="petclinic" file="petclinic-1.0.ear">
  <excludeFileNamesRegex>.*\.properties</excludeFileNamesRegex>
</jee.War>

Note that the regular expression is only applied to the name of a file in a folder, not to its path. To exclude an entire folder, use a regular expression such as .*exclude-all-files-in-here (instead of .*exclude-all-files-in-here/.*).

Custom deployment package support

If you have defined your own type of Deployment Package, or have added custom properties to the deployment package, you can import these by changing the manifest. Say that you've extended udm.DeploymentPackage as myorg.PackagedApplicationVersion which has additional properties such as releaseDate and tickets:

<?xml version="1.0" encoding="UTF-8"?>
<myorg.PackagedApplicationVersion version="1.0" application="PetClinic">
  <releaseDate>2013-04-02T16:22:00.000Z</releaseDate>
  <tickets>
    <value>JIRA-1</value>
    <value>JIRA-2</value>
  </tickets>
  <deployables>
    ...
  </deployables>
</myorg.PackagedApplicationVersion>

Specifying the location of the application

It is possible to specify where to import the package. This is useful during initial imports when the application does not yet exist. You can specify the path to the application like this:

<?xml version="1.0" encoding="UTF-8"?>
<udm.DeploymentPackage version="1.0" application="directory1/directory2/PetClinic">
    ...
</udm.DeploymentPackage>

XL Deploy will then try to import the package into Application called PetClinic located at Applications/directory1/directory2/PetClinic. It will also perform the following checks:

The user should have the correct import rights (import#initial or import#upgrade) for the directory.
The path should exist. XL Deploy will not create it.
The Application needn't exist yet, for an initial import XL Deploy will create it for you.
If XL Deploy finds an application called PetClinic in another path, it will fail the import as application names should be unique.

Old-style Manifest Format

The new XML Manifest has been introduced in XL Deploy 3.9. In previous versions the manifest was specified using the MANIFEST.MF format described in the JAR file specification. XL Deploy can still import DAR packages containing this manifest format. This chapter is for reference purposes only as we strongly encourage you to write the manifests in the new XML format.

A valid XL Deploy manifest starts with the following preamble:

Manifest-Version: 1.0
XL Deploy-Package-Format-Version: 1.3

This identifies the Java manifest version and the XL Deploy package format version.

Specifying the Application

A deployment package contains a specific version of an application. These entries tell XL Deploy that the package contains the AnimalZoo application version 4.0:

CI-Application: AnimalZoo-ear
CI-Version: 4.0

These entries are part of the manifest preamble.

Specifying Artifacts

Artifacts are represented by files or folders in the deployment package. To import an artifact as a CI in XL Deploy, use:

Name: AnimalZooBE-1.0.ear
CI-Type: jee.Ear
CI-Name: AnimalZooBE

The standard Name entry must refer to an actual file included in the DAR. The CI-Type specifies a CI type name that is available in the system. The required CI-Name property indicates the name of the CI to be created.

Similarly, this construct can be used to create a folder CI:

Name: conf
CI-Type: file.Folder
CI-Name: configuration-files

Specifying Resource Specifications

Resource specifications are not represented by files in the deployment package. They represent resources that must be created on the target middleware when the application is deployed. Resource specifications are constructed completely based on the information in the manifest. The manifest also specifies values for properties of the CI.

For example, this manifest snippet constructs a was.OracleDatasourceSpec CI:

Name: petclinicDS
CI-Type: was.OracleDatasourceSpec
CI-driver: com.mysql.jdbc.Driver
CI-url: jdbc:mysql://localhost/petclinic
CI-username: petclinic
CI-password: my$ecret

The Name entry specifies the name of the CI to be created. In contrast to the manifest specification, the Name property in a XL Deploy manifest does not refer to a physical file present in the DAR in the case of a resource specification. The CI-Type specifies a CI type that is available in the system.

Note: the names of artifacts in your package must conform to platform requirements. For instance, a file.Folder CI with name "q2>2" cannot be deployed to a Windows host, because ">" may not be part of a file or directory name in Windows.

The other entries, CI-url, CI-username and CI-password refer to properties on the datasource CI. These properties will be set to the values specified. In general, any property on a CI can be set using the CI-<propertyname> notation. See the Command Line Interface (CLI) Manual for information or how to obtain the list of properties that a particular CI type supports, or consult the relevant CI reference documentation.

Note that it is also possible to add resource specifications to a package that is already imported in XL Deploy. See the Command Line Interface (CLI) Manual for more information.

Setting Complex Properties

Properties referring to other CIs

Name: myResourceSpecName
...

Name: my-artifact-file-name.ext
CI-Name: myArtifactCiName
...

...
CI-artifactRefProperty: my-artifact-file-name.ext
CI-resourceSpecRefProperty: myResourceSpecName

...
CI-setOfCIProperty-EntryValue-1: myResourceSpecName
CI-setOfCIProperty-EntryValue-2: my-artifact-file-name.ext

...
CI-listOfCIProperty-EntryValue-1: myResourceSpecName
CI-listOfCIProperty-EntryValue-2: my-artifact-file-name.ext

Note that references use the referent's Name property.

Set of strings properties

To set a set of strings property to contain strings "a" and "b":

CI-setOfStringProperty-EntryValue-1: a
CI-setOfStringProperty-EntryValue-2: b

List of strings properties

To set a list of strings property to contain strings "a" and "b":

CI-listOfStringProperty-EntryValue-1: a
CI-listOfStringProperty-EntryValue-2: b

Map of string to string properties

To set a map of string to string property to contain pairs "key1", "value1" and "key2", "value2":

CI-mapOfStringToStringProperty-key1: value1
CI-mapOfStringToStringProperty-key2: value2

Boolean and enumeration properties

TO set a boolean property to true or false:

CI-boolProperty: true
CI-boolProperty: false

To set an enum property to a specific value:

CI-enumProperty: ENUMVALUE

Using Placeholders in CI properties

Similar to the XML format, you can use placeholders to fill values from Dictionaries.

Name: petclinicDS
CI-Type: was.OracleDatasourceSpec
CI-driver: com.mysql.jdbc.Driver
CI-url: jdbc:mysql://localhost/petclinic
CI-username: {{DB_USERNAME}}
CI-password: {{DB_PASSWORD}}

Please note that placeholders can also be used in the name of a CI but they cannot be specified in the Name field in the MANIFEST.MF format. Use the CI-Name format instead, e.g.:

Name: petclinicDS
CI-Name: {{PETCLINIC_DS_NAME}}
CI-Type: was.OracleDatasourceSpec
CI-driver: com.mysql.jdbc.Driver
CI-url: jdbc:mysql://localhost/petclinic
CI-username: {{DB_USERNAME}}
CI-password: {{DB_PASSWORD}}

Creating a deployment package

To illustrate the way a deployment package can be created, let's describe a complete example of how to do this in both the GUI and CLI. Subsequent sections will describe how to automate this by using the XL Deploy maven plugin, the maven jar plugin or the ant jar task.

Creating a Deployment Package in the GUI

Let's say we want to create a package for version 1.0 of our brand new PetClinic application. The application contains an EAR file with the application code, a configuration folder containing configuration files and a datasource. We will create a new application and version in the GUI, upload our artifacts and then export the resulting package as a DAR archive.

First, navigate to the Repository browser in the XL Deploy GUI. Open the context menu on the Applications root node and select New -> udm -> Application. An editor tab appears where you can enter details of the new application. Name the application "PetClinic" and save it. The PetClinic Application CI appears in the repository tree.

Now, open the context menu on the PetClinic CI and select New -> udm -> DeploymentPackage, call it version 1.0 and save it. The PetClinic/1.0 package now appears in the tree.

Next, we add the resources and artifacts are package will contain. First, the EAR file. Open the context menu on the PetClinic/1.0 CI and select New -> jee -> Ear. Give the ear a name (for instance, Petclinic-ear). To attach the binary artifact, click the Browser button and look for the EAR file on your filesystem. When you've found it, upload it to XL Deploy. Press the Save button to store the CI in the repository.

Folders can be uploaded into XL Deploy as a ZIP archive. Create a ZIP file containing the configuration folder you want to upload, create the configuration folder CI and upload the ZIP file.

Finally, create a datasource in the package in the same way. Since this is a resource, not an artifact, no binary data has to be uploaded.

Once this is done, open the context menu on the deployment package and select the Export option. XL Deploy will prepare a DAR archive for the package that your browser can download. It will include a valid manifest file that you can extract and inspect.

Creating a Deployment Package in the CLI

To create the package by hand, start by creating a directory petclinic-package to hold the package content:

mkdir petclinic-package

Now, collect the EAR file and configuration directory and store them in the newly created directory:

cp /some/path/petclinic-1.0.ear petclinic-package
cp -r /some/path/conf petclinic-package

The datasource is a resource specification, not an artifact, so there is no file to include.

Now, let's create the DAR manifest for these entries.

<?xml version="1.0" encoding="UTF-8"?>
<udm.DeploymentPackage version="1.0" application="PetClinic">
  <deployables>
    ...
  </deployables>
</udm.DeploymentPackage>

Add the EAR and configuration folder:

<jee.Ear name="PetClinic-Ear" file="petclinic-1.0.ear" />

<file.Folder name="PetClinic-Config" file="conf" />

Add the datasource to the manifest as follows:

<was.OracleDatasourceSpec name="PetClinic-ds">
  <driver>com.mysql.jdbc.Driver</driver>
  <url>jdbc:mysql://localhost/petclinic</url>
  <username>{{DB_USERNAME}}</username>
  <password>{{DB_PASSWORD}}</password>
</was.OracleDatasourceSpec>

Note how the datasource uses placeholders for username and password.

The complete manifest looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<udm.DeploymentPackage version="1.0" application="PetClinic">
  <deployables>
    <jee.Ear name="PetClinic-Ear" file="petclinic-1.0.ear" />
    <file.Folder name="PetClinic-Config" file="conf" /> 
    <was.OracleDatasourceSpec name="PetClinic-ds">
      <driver>com.mysql.jdbc.Driver</driver>
      <url>jdbc:mysql://localhost/petclinic</url>
      <username>{{DB_USERNAME}}</username>
      <password>{{DB_PASSWORD}}</password>
    </was.OracleDatasourceSpec>
  </deployables>
</udm.DeploymentPackage>

Save the manifest in the package directory. Finally, create the DAR archive with the command:

    jar cf petclinic-1.0.dar petclinic-package/*

The resulting archive can be imported into XL Deploy.

Using the XL Deploy Maven Plugin

To enable continuous deployment, XL Deploy can be integrated with the Maven build system. The XL Deploy Maven plugin enables you to integrate Maven and XL Deploy. Specifically, the plugin supports:

Creating a deployment package containing artifacts from the build
Performing a deployment to a target environment
Undeploying a previously deployed application

For more information, see the XL Deploy maven plugin documentation.

Using the Maven jar Plugin

The standard maven jar plugin can also be used to create a XL Deploy package.

Create a manifest file conforming to the XL Deploy manifest standard (see section XML Manifest Format above).
Create a directory structure containing the files as they should appear in the package.

In the maven POM, configure the jar plugin as follows:

<project>
  ...
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        ...
        <configuration>
          <includes>
            <include>**/*</include>
          </includes>
        </configuration>
        ...
      </plugin>
    </plugins>
  </build>
  ...
</project>

The XL Deploy package can be generated by invoking the command:

mvn package

Using the Ant jar Task

Creating a XL Deploy package via Ant is possible using the jar task.

Create a manifest file conforming to the XL Deploy manifest standard (see section XML Manifest Format above).
Create a directory structure containing the files as they should appear in the package.

In the Ant build file, include a jar task invocation as follows:

<jar destfile="package.jar"
    basedir="."
    includes="**/*"/>

Sample Packages

XL Deploy ships with several sample packages. These examples are stored in the XL Deploy server's importablePackages directory.

Tips and Tricks

Overriding default artifact comparison

When XL Deploy imports a package, it creates a checksum for each artifact in the package. The checksum property on an artifact is a string property and can contain any string value. It is used during an upgrade to determine whether the content of an artifact CI in the new package differs from the artifact CI in the previous package. If you include information in an artifact that changes on every build, such as a build number or build timestamp, the checksum will be different even though the contents of the artifact has not changed.

In this scenario, it can be useful to override the XL Deploy-generated checksum and provide your own inside your package. Here is an example of an artifact CI with it's own checksum:

<jee.Ear name="AnimalZooBE" file="AnimalZooBE-1.0.ear">
  <checksum>1.0</checksum>
</jee.Ear>

Using the above artifact definition, even if the EAR file itself is different, XL Deploy will consider the EAR file unchanged as long as it has value 1.0 for the checksum property.

Specifying encoding for files in artifacts

By default XL Deploy will try to detect the encoding for files if a Byte Order Mark (BOM) is present. If it is not, XL Deploy will fallback to the platform encoding of Java. This behaviour can be influenced by adding values to the fileEncodings property of the artifact.

The fileEncodings property is a map of string to string, and can be specified as such in the manifest of the package. For example, suppose you have the following (internationalized) files in your file.Folder artifact:

web-content/en-US/index.html
web-content/nl-NL/index.html
web-content/zh-CN/index.html
web-content/ja-JP/index.html

You want the chinese and japanese index pages to be treated as UTF-16BE, whereas the others can be treated as UTF-8. You can now specify this in the manifest as follows:

<file.Folder name="webContent" file="web-content">
  <fileEncodings>
    <entry key=".+(en-US|nl-NL).+">UTF-8</entry>
    <entry key=".+(zh-CN|ja-JP).+">UTF-16BE</entry>
  </fileEncodings>
</file.Folder>

Using the above definition, XL Deploy will ensure that it will use those encodings when replacing placeholders in these files.