Generic Model Plugin Manual

Preface

This document describes the functionality provided by the Generic Model plugin.

See the XL Deploy Reference Manual for background information on XL Deploy and deployment concepts.

Overview

XL Deploy supports a number of middleware platforms. Sometimes, though, it is necessary to extend XL Deploy with new middleware support. The Generic Model plugin provides a way to do this, without having to write Java code. Instead, using XL Deploy's flexible type system and the base CIs from the Generic Model plugin, new CIs can be defined by writing XML and providing scripts for functionality.

Several of XL Deploy's standard plugins are also built on top of the Generic Model plugin.

Features

• Define custom containers
  • Stop, start, restart capabilities
• Define and copy custom artifacts to a custom container
• Define, copy and execute custom scripts and folders on a custom container
• Define resources to be processed by a template and copied to a custom container
• Define and execute control tasks on containers and deployeds
• Flexible templating engine

Requirements

The plugin requires:

• XL Deploy: version 4.0+

Plugin Concepts

The Generic Model plugin provides several CIs that can be used as base classes for creating XL Deploy extensions. There are base CIs for each of XL Deploy's CI types (deployables, deployeds and containers). A typical usage scenario is to create custom, synthetic CIs (based on one of the provided CIs) and using it to invoke the required behavior (scripts) in a deployment plan.

Note: since Deployit version 3.6, the deployeds in the Generic Model Plugin can target containers that implement the HostContainer interface. In addition to the Container and derived CIs, this means they can also be targeted to CIs derived from Host.

Container

A Container is a topology CI and models middleware in your infrastructure. This would typically be used to model middleware within XL Deploy that does not have out of the box support or is custom to your environment. The other CIs in the plugin can be deployed to (subclasses of) the container. The behavior of the container in a deployment is configured by specifying scripts to be executed when it is started, stopped or restarted. XL Deploy will invoke these scripts as needed.

NestedContainer

A Nested Container is a topology CI and models middleware in your infrastructure. The nested container allows for the modelling of abstract middleware concepts as containers to which items can be deployed.

Copied Artifact

A CopiedArtifact is an artifact as copied over to a Container. It manages the copying of any generic artifact (File, Folder, Archive, Resource) in the deployment package to the container. It is possible to indicate that this copied artifact requires a container restart.

Executed Script

An ExecutedScript encapsulates a script that is executed on a Container. The script is processed by the templating engine (see below) before being copied to the target container. The behavior of the script is configured by specifying scripts to be executed when it is deployed, upgraded or undeployed.

Manual Process

A ManualProcess entails a script containing manual instructions for the operator to perform before the deployment can continue. The script is processed by the templating engine (see below) and is displayed to the operator in the step logs. Once the instructions have been carried out, the operator can continue the deployment. The instructions can also be automatically emailed.

Executed Folder

An ExecutedFolder encapsulates a folder containing installation and rollback scripts that are executed on a Container. Installation scripts are executed when the folder is deployed or updated, rollback scripts are executed when it is undeployed. Execution of the scripts happens in order. Scripts are processed by the templating engine (see below) before being copied to the target container.

Processed Template

A ProcessedTemplate is a Freemarker template that is processed by the templating engine (see below), then copied to a Container.

Templating

When defining and using CIs with the Generic Model plugin, the need arises to use variables in certain CI properties and scripts. The most obvious use is to include properties from the deployment itself, such as the names or locations of files in the deployment package. XL Deploy uses the Freemarker templating engine for this.

When performing a deployment using the Generic Model plugin, all CIs and scripts are processed in Freemarker. This means that placeholders can be used in CI properties and scripts to make them more flexible. Freemarker resolves placeholders using a context, a set of objects defining the template's environment. This context depends on the type of CI being deployed.

For all CIs, the context variable step refers to the current step object and the context variable statics can be used to access static methods on any class. See the section accessing static methods in the FreeMarker manual

For deployed CIs, the context variable deployed refers to the current CI instance. For example:

<type type="tc.DeployedDataSource" extends="generic.ProcessedTemplate" deployable-type="tc.DataSource"
      container-type="tc.Server">
    ...
    <property name="targetFile" default="${deployed.name}-ds.xml" hidden="true"/>
    ...
</type>

Additionally, when performing a MODIFY operation the context variable previousDeployed refers to the previous version of the current CI instance. For example:

#!/bin/sh
echo "Uninstalling ${previousDeployed.name}"
rm ${deployed.container.home}/${previousDeployed.name}

echo "Installing ${deployed.name}"
cp ${deployed.file} ${deployed.container.home}

For container CIs, the context variable container refers to the current container instance. For example:

<type type="tc.Server" extends="generic.Container">
    <property name="home" default="/tmp/tomcat"/>
    <property name="targetDirectory" default="${container.home}/webapps" hidden="true"/>
</type>

A special case is when referring to an artifact in the placeholder. For example, when deploying a CI representing a WAR file, the following placeholder can be used to refer to that file (assuming there is a file property on the deployable):

${deployed.file}

In this case, XL Deploy will copy the referred artifact to the target container so that the file is available to the executing script. A script containing a command like the following would therefore copy the file represented by the deployable to it's installation path on the remote machine:

cp ${deployed.file} /install/path

Description of file related placeholders:

Accessing the ExecutionContext

In XL Deploy 4.5.3 and later, the Generic plugin can access the ExecutionContext and use it in a FreeMarker template. For example:

<type type="demo.DeployedStuff" extends="generic.ExecutedScript" deployable-type="demo.Stuff" container-type="overthere.SshHost">
  <generate-deployable type="demo.Stuff" extends="generic.Resource"/>
  <property name="P1" default="X"/>
  <property name="P2" default="Y"/>
  <property name="P3" default="Z"/>
  <property name="createScript" default="stuff/create" hidden="true"/>
</type>

Sample FreeMarker template:

echo "${deployed.P1}"
echo "${deployed.P2}"
echo "${deployed.P3}"
echo "${context}"
echo "${context.getClass()}"
echo "${context.getTask().getId()}"
echo "${context.getTask().getUsername()}"

echo "display metadata"
<#list context.getTask().getMetadata()?keys as k>
echo "${k} = ${context.getTask().getMetadata()[k]}"
</#list>
echo "/display metadata"

Control Task Delegates

The Generic Model Plugin has pre defined control task delegates that have the ability to execute scripts on a target host. The delegates can be used to define control tasks on any configuration item defined in XL Deploy's type system.

shellScript Delegate

The shellScript delegate has the capability of executing a single script on a target host.

Example:

<type type="tc.DeployedDataSource" extends="generic.ProcessedTemplate" deployable-type="tc.DataSource"
      container-type="tc.Server">
    <generate-deployable type="tc.DataSource" extends="generic.Resource"/>
    ...
    <method name="ping" delegate="shellScript"
        script="tc/ping.sh"
        classpathResources="tc/ping.py"/>
</type>

localShellScript Delegate

The localShellScript delegate has the capability of executing a single script on a the XL Deploy host.

Example :

<type-modification type="udm.DeployedApplication" >
    <method name="updateVersionDatabase" delegate="localShellScript"
        script="cmdb/updateVersionDatabase.sh.ftl"/>
</type>

shellScripts Delegate

The shellScripts delegate has the capability of executing multiple scripts on a target host.

Example :

<type type="tc.Server" extends="generic.Container">
    ...
    <method name="startAndWait" delegate="shellScripts"
        scripts="start:tc/start.sh,tc/tailLog.sh"
        startClasspathResources="tc/start.jar"
        startTemplateClasspathResources="tc/password.xml"
        classpathResources="common.jar"/>
</type>

localShellScripts Delegate

The localShellScripts delegate has the capability of executing multiple scripts on the XL Deploy host.

Example:

<type-modification type="udm.Version">
    <method name="udpateSCMandCMDB" delegate="localShellScripts"
        scripts="updateSCM:scm/update,updateCMDB:cmdb/update"
        updateSCMClasspathResources="scm/scm-connector.jar"
        updateCMDBTemplateClasspathResources="cmdb/request.xml.ftl"
        classpathResources="common.jar"/>
</type>

Using the deployables and deployeds

Deployable vs. Container Table

The following table describes which deployable / container combinations are possible.

Deployed Actions Table

The following table describes the effect a deployed has on its container.

Sample Usage Scenario - Deploying a new middleware platform

This section describes an example of using the Generic Model plugin to implement support for a simple middleware platform. Deployment to this platform is done by simply copying a WAR archive to the right directory on the container. Resources are created by copying configuration files into the container's configuration directory. The Tomcat application server works in a very similar manner.

By defining a container and several other CIs based on CIs from the Generic Model plugin, it is possible to add support for deploying to this platform to XL Deploy.

Defining the Container

To use any of the CIs in the Generic Model plugin, they need to be targeted to a Container. This snippet shows how to define a generic container as a synthetic type:

<type type="tc.Server" extends="generic.Container">
    <property name="home" default="/tmp/tomcat"/>
</type>

<type type="tc.UnmanagedServer" extends="tc.Server">
    <property name="startScript" default="tc/start.sh" hidden="true"/>
    <property name="stopScript" default="tc/stop.sh" hidden="true"/>
    <property name="restartScript" default="tc/restart.sh" hidden="true"/>
</type>

Note that the tc.UnmanagedServer CI defines a start, stop and restart script. The XL Deploy Server reads these scripts from the classpath. When targeting a deployment to the tc.UnmanagedServer, XL Deploy will include steps executing the start, stop and restart scripts in appropriate places in the deployment plan.

Defining a Configuration File

The following snippet defines a CI based on the CopiedArtifact. The tc.DeployedFile CI can be targeted to the tc.Server. The target directory is specified as a hidden property. Note the placeholder syntax used here.

<type type="tc.DeployedFile" extends="generic.CopiedArtifact" deployable-type="tc.File"
      container-type="tc.Server">
    <generate-deployable type="tc.File" extends="generic.File"/>
    <property name="targetDirectory" default="${deployed.container.home}/conf" hidden="true"/>
</type>

Using the above snippet, it is possible to create a package with a tc.File deployable and deploy it to an environment containing a tc.UnmanagedServer. This will result in a tc.DeployedFile deployed.

Defining a WAR

To deploy a WAR file to the tc.Server, one possibility is to define a tc.DeployedWar CI that extends the ExecutedScript. The tc.DeployedWar CI is generated when deploying a jee.War to the tc.Server CI. This is what the XML looks like:

<type type="tc.DeployedWar" extends="generic.ExecutedScript" deployable-type="jee.War"
      container-type="tc.Server">
    <generate-deployable type="tc.War" extends="jee.War"/>
    <property name="createScript" default="tc/install-war" hidden="true"/>
    <property name="modifyScript" default="tc/reinstall-war" hidden="true" required="false"/>
    <property name="destroyScript" default="tc/uninstall-war" hidden="true"/>
</type>

When performing an initial deployment, the create script, tc/install-war is executed on the target container. Inside the script, a reference to the file property is replaced by the actual archive. Note that the script files do not have an extension. Depending on the target platform, the extension sh (Unix) or bat (Windows) is used.

The WAR file is referenced from the script as follows:

echo Installing WAR ${deployed.deployable.file} in ${deployed.container.home}

Defining a Datasource

Configuration files can be deployed by creating a CI based on the ProcessedTemplate. By including a Resource in the package that is a Freemarker template, a configuration file can be generated during the deployment and copied to the container. This snippet defines such a CI, tc.DeployedDataSource:

<type type="tc.DeployedDataSource" extends="generic.ProcessedTemplate" deployable-type="tc.DataSource"
      container-type="tc.Server">
    <generate-deployable type="tc.DataSource" extends="generic.Resource"/>

    <property name="jdbcUrl"/>
    <property name="port" kind="integer"/>
    <property name="targetDirectory" default="${deployed.container.home}/webapps" hidden="true"/>
    <property name="targetFile" default="${deployed.name}-ds.xml" hidden="true"/>
    <property name="template" default="tc/datasource.ftl" hidden="true"/>
</type>

The template property specifies the Freemarker template file that the XL Deploy Server reads from the classpath. The targetDirectory controls where the template is copied to. Inside the template, properties like jdbcUrl on the datasource can be used to produce a proper configuration file.

Discovery

The Generic Model plugin supports discovery in any subtype of Container, Nested Container and Deployed. Extenders of the plugin provide shell scripts that interact with the discovery mechanism, via the standard out, with specially formatted output representing the inspected property or discovered configuration item.

<!-- Sample of extending Generic Mode plugin -->
<type type="sample.TomcatServer" extends="generic.Container">
    ...
    <property name="inspectScript" default="inspect/inspect-server" hidden="true"/>
</type>

<type type="sample.VirtualHost" extends="sample.NestedContainer">
    <property name="server" kind="ci" as-containment="true" referenced-type="sample.TomcatServer"/>
    ...
    <property name="inspectScript" default="inspect/inspect-virtualhost" hidden="true"/>
</type>

<type type="sample.DataSource" extends="generic.ProcessedTemplate" deployable-type="sample.DataSourceSpec"
      container-type="sample.Server">
    <generate-deployable type="sample.DataSourceSpec" extends="generic.Resource"/>
    <property name="inspectScript" default="inspect/inspect-ds" hidden="true"/>
    ...
</type>

Encoding

The discovery mechanism uses URL encoding as described in RFC3986 to interpret the value of an inspected property. It is the responsibility of the plugin extender to perform said encoding in the inspect shell scripts.

Sample of encoding in a BASH shell script

function encode()
{
    local  myresult=$(printf "%b" "$1" | perl -pe's/([^-_.~A-Za-z0-9])/sprintf("%%%02X", ord($1))/seg')
    echo "$myresult"
}
myString='This is a string spanning many lines and with funky characters like !@#$%^&*() and \|'"'"'";:<>,.[]{}'
myEncodedString = $(encode "$myString")
echo $myEncodedString

Property Inspection

The discovery mechanism identifies an inspected property when output with the following format is sent to the standard out.

INSPECTED:propertyName=value

The output must be prefixed with INSPECTED: followed by the name of the inspected property, an = sign and then the encoded value of the property.

Sample :

echo INSPECTED:stringField=A,value,with,commas
echo INSPECTED:intField=1999
echo INSPECTED:boolField=true

Inspecting Set properties

When an inspected property is a set of strings, the value must be comma separated

INSPECTED:propertyName=value1,value2,value3

Sample :

echo INSPECTED:stringSetField=$(encode 'Jac,q,ues'),de,Molay
# will result in the following output
# INSPECTED:stringSetField=Jac%2Cq%2Cues,de,Molay

Inspecting Map properties

When an inspected property is a map of strings, entries must be comma separated and key values must be colon separated

INSPECTED:propertyName=key1:value1,key2:value2,key3:value3

Sample :

echo INSPECTED:mapField=first:$(encode 'Jac,q,ues:'),second:2
# will result in the following output
# INSPECTED:mapField=first:Jac%2Cq%2Cues,second:2

Configuration Item Discovery

The discovery mechanism identifies a discovered configuration item when output with the following format is sent to the standard out.

DISCOVERED:configurationItemId=type

The output must be prefixed with DISCOVERED: followed by the id of the configuration item as stored in the XL Deploy repository, an = sign and the type of the configuration item

Sample :

echo DISCOVERED:Infrastructure/tomcat/defaultContext=sample.VirtualHost

Mail server setup

The generic plugin also adds support for mail servers to XL Deploy. A Mail server is a configuration item mail.SmtpServer defined under the /Configuration root node. A udm.Environment environment configuration item can have a reference to a mail server. If it does not have a reference, a default mail server named 'defaultSmtpServer' will be used to send configured mails. Using the mail server, configuration items such as the generic.ManualProcess can send mails notifying you of manual actions that need to be taken.

Here's a CLI snippet showing you how to create a mail server CI:

mailServer = factory.configurationItem("Configuration/MailServer","mail.SmtpServer")
mailServer.host = "smtp.mycompany.com"
mailServer.username = "mymailuser"
mailServer.password = "secret"
mailServer.fromAddress = "noreply@mycompany.com"
repository.create(mailServer)

The mail.SmtpServer uses Java Mail to send emails. You can specify additional Java Mail properties in the smtpProperties attribute. Refer to the Javadoc for JavaMail for a list of all properties.

Configuring TLS

To configure the mail server to send emails using TLS, set the following property in the SMTP properties:

mailServer.smtpProperties = {}
mailServer.smtpProperties["mail.smtp.starttls.enable"] = "true"
repository.update(mailServer)