Upgrade Manual

Upgrade process

Overall, the upgrade process consists of the following steps:

• Obtain a new version of the XL Deploy software (the main product and/or plugins) from XebiaLabs.
• Read the new version's release notes so you are aware of the new functionality and possible upgrade considerations.
• Read the new version's upgrade manual (this document) so you are aware of possible upgrade considerations.
• Stop the current version of XL Deploy if it is still running, making sure there are no running tasks active.
• Create a brand-new installation directory for the new version of XL Deploy so the existing version is still available in case of problems.
• Extract the new XL Deploy software release into the new installation directory.
• Copy the data from the previous XL Deploy installation directory into the new installation directory.
• Start the new version of XL Deploy.

See the section Performing the Upgrade below for a detailed explanation of these steps.

Upgrade notes

• It is possible to skip XL Deploy versions when upgrading. XL Deploy will sequentially apply any upgrades for the intermediate (skipped) versions. Please read the specific upgrade instructions for each version carefully.
• The new version of XL Deploy may not be compatible with the existing version of your plugins. If this is the case, you'll need to download an updated version of your plugin as well. Please read the specific upgrade instructions for each version carefully.
• If a repository upgrade is required, XL Deploy will detect that it is running against an old repository and will automatically execute an upgrade when it is first started. The server log will contain extensive logging of the repository upgrade process. You may want to save this log for future reference.
• Plugin versions are related to the version of XL Deploy (or Deployit) that they are compatible with. For instance, WAS plugin version 3.6.0 requires at least Deployit server version 3.6.0. This version of the WAS plugin should also work in Deployit 3.7.0 unless stated otherwise in this document or in the release notes.

Performing the upgrade

Before upgrading carefully read the Release Notes and make note of anything that may apply to your situation.

To begin upgrading XL Deploy, first unpack the distribution archive. The distribution archive contains the following:

• A server archive.
• A CLI archive.

Upgrading the Server

To upgrade an existing XL Deploy server installation, do the following:

1. Extract the server archive. It creates an installation directory called, for example, xl-deploy-4.0.0-server.
2. Stop the Deployit/XL Deploy server.
3. Copy the contents of the conf directory from the previous installation into the new installation directory.
4. If necessary, update the product license conf/deployit-license.lic you can download your current licenses from https://tech.xebialabs.com/download/license.
5. Copy the entire repository directory from the previous installation into the new installation directory.
6. Copy the contents of the importablePackages directory from the previous installation into the new installation directory.
7. Unless new versions of your plugins were provided with the new XL Deploy version, copy the contents of the plugins directory from the previous installation into the new installation directory. (See also the version-specific upgrade notes below for plugin incompatibilities.)
8. Copy the contents of the ext directory from the previous installation into the new installation directory.
9. DO NOT copy the contents of the hotfix directory (unless instructed) because hotfixes are version-specific.
10. If you added libraries to XL Deploy's lib directory (for instance, database drivers), copy the additional libraries from the previous installation into the new installation directory.
11. If you have made any changes to the XL Deploy server startup scripts ( server.sh or server.cmd ), manually re-do these changes in the new installation directory.
12. Verify that libraries in the lib folder do not also appear in the plugins directory, even if their versions are different. For example, if lib contains a guava-16.0.1.jar then the plugins directory should not contain any guava-x.x.x.jar file (such as guava-13.0.jar). In this case, you must remove the library from the plugins directory. For example:
    
13. Start the XL Deploy server interactively to allow any automatic repository upgraders to run.
14. If you normally run the XL Deploy server as a service, stop it again and start it as you normally do.

Note: please make sure that the plugins and extensions in the old XL Deploy installation are compatible with the new XL Deploy server version.

This completes upgrading of the XL Deploy server.

Upgrading the CLI

To upgrade an existing XL Deploy CLI installation, do the following:

1. Create a directory for the new XL Deploy CLI installation, including the new XL Deploy CLI version number in the directory name.
2. Extract the CLI archive in this directory.
3. Unless new versions of your plugins were provided with the new XL Deploy version, copy the contents of the plugins directory from the previous installation into the new installation directory.
4. Copy the contents of the ext directory from the previous installation into the new installation directory.
5. DO NOT copy the contents of the hotfix directory (unless instructed) because hotfixes are version-specific.
6. If you have made any changes to the XL Deploy CLI startup scripts (cli.sh or cli.cmd), copy these from the bin directory in the previous installation into the new installation directory.

This completes upgrading of the XL Deploy CLI.

Specific upgrade notes

This section describes specific considerations for migrating from or to a particular XL Deploy/Deployit version:

• Upgrading to XL Deploy 4.0.2
• Upgrading to XL Deploy 4.0.0
• Upgrading to Deployit 3.9.90
• Upgrading to Deployit 3.9.5
• Upgrading to Deployit 3.9.4
• Upgrading to Deployit 3.9.3
• Upgrading to Deployit 3.9.2
• Upgrading to Deployit 3.9.0
• Upgrading to Deployit 3.8.4
• Upgrading to Deployit 3.8.3
• Upgrading to Deployit 3.8.0
• Upgrading to Deployit 3.7.0
• Upgrading to Deployit 3.6.0
• Upgrading from Deployit 3.0.x or earlier

Upgrading to XL Deploy 4.0.2

Security upgrade

• Spring and Spring Security were upgraded to version 3.2, so if you use deployit-security.xml, you must update it by removing the version specification of XSD files:
  • Change .../spring-beans-3.1.xsd to .../spring-beans.xsd
  • Change .../spring-security-3.1.xsd to .../spring-security.xsd

Upgrading to XL Deploy 4.0.0

New license necessary

We made changes to the license system as a result a new license file is needed to run XL Deploy. You can download your license from https://dist.xebialabs.com/customer/licenses/. The license needs to be installed into the conf directory.

Note: Ensure the license file is not changed in any way, that will corrupt the license. Some mail programs may corrupt the license file unless wrapped in a ZIP.

Java 7

XL Deploy requires Java 7 to run.

Plugin compatibility

The following old plugins are not compatible with XL Deploy 4.0.0:

• glassfish-plugin 3.9.x
• ec2-plugin 3.9.x
• cloud-plugin 3.9.x
• wls-plugin 3.9.x
• jbossdm-plugin 3.9.x
• wp-plugin 3.9.x
• was-plugin-extensions 3.9.x

Use of cp, tar, copy and xcopy

XL Deploy 4.0.0 uses version 2.4.2 of the remoting framework Overthere that introduces a new feature to copy files on a remote machine in an efficient manner. Overthere 2.4.2 does this by using the following commands to copy file and directories (where {0} is replaced with the path of the source file or directory and {1} is replaced with the path of the target file or directory):

• File copy on Unix and z/OS: cp -p {0} {1}
• Directory copy on Unix and z/OS:
  • XL Deploy 4.0.0: tar cC {0} . | tar xmC {1} .
  • XL Deploy 4.0.1 and up: cd {1} ; tar -cf - -C {0} . | tar xpf -
• File copy on Windows: copy {0} {1} /y
• Directory copy on Windows: xcopy {0} {1} /i /y /s /e /h /q

This functionality is used in three places in XL Deploy:

• When a file or directory is staged, it will be uploaded to the staging directory at the start of the deployment and then, when a step uses it, it will be copied to the target directory or the temporary directory (depending on the plugin) using one the commands specified above.
• The behaviour of the File Plugin has been modified to more atomic:
  1. If staging is enabled, the file or directory will be uploaded to the staging directory at the beginning of the deployment and then copied from the staging directory to the target directory (using one the commands specified above) during the actual deployment.
  2. If staging is not enabled, the file or directory will be uploaded to the temporary directory and immediately copied from there to the target directory (using one the commands specified above) in a single step.
• The behaviour of the generic.CopiedArtifact type of the Generic Model Plugin has been modified in a similar manner. This means all plugins that extend this behaviour, e.g. the Tomcat Plugin or the JBossAS plugin for 5.x and 6.x, are also affected by this.

Prior to XL Deploy 4.0.0, the behavior of the File Plugin and of generic.CopiedArtifact was to copy the file or directory directly to the target path. From XL Deploy 4.0.1 and up, this behavior can be re-enabled by setting the hidden property copyDirectlyToTargetPath to true

For example, to revert the File Plugin to its pre-4.0.0 behavior, add the following line to the conf/deployit-defaults.properties file and restart the XL Deploy server:

file.DeployedArtifactOnHost.copyDirectlyToTargetPath=true

Orchestrators

The kind of the orchestrator property on the udm.DeployedApplication type has been changed from STRING to LIST_OF_STRING, because it is now possible to compose orchestrators. If you have CLI scripts that assign a single String value to this property this will still work, because it will be put into the list as the only element. If you have scripts that depend on reading the value, the script must be changed to expect a list of string when reading the property.

Deprecated orchestrators

The following orchestrators have been deprecated and will be removed in an upcoming (major) release.

• container-by-container-serial, replaced by the sequential-by-container orchestrator.
• composite-package, replaced by the sequential-by-composite-package orchestrator.
• group-based, replaced by the sequential-by-deployment-group orchestrator.

UI functionality has been moved

The pause button and the skip button in task execution screen were moved into a context menu on the steps.

Upgrading to Deployit 3.9.90

Java 7

From this version onwards Deployit will require Java 7 to run. All pre-existing plugins are compatible with this version of Java and it should not be needed to rebuild them.

Upgrading to Deployit 3.9.5

Security upgrade

• Spring and Spring Security were upgraded to version 3.2, so if you use deployit-security.xml, you must update it by removing the version specification of XSD files:
  • Change .../spring-beans-3.1.xsd to .../spring-beans.xsd
  • Change .../spring-security-3.1.xsd to .../spring-security.xsd

Upgrading to Deployit 3.9.4

Support for deployment package level properties

Support for deployment package level properties has been added in Deployit 3.9.4 to make it possible to set values for deployment properties from the udm.DeploymentPackage.

Support for native Windows WinRM

Support for the native Windows WinRM implementation, i.e. the winrs command, has been implemented in Deployit 3.9.4 to make the setup of WinRM easier for users running the Deployit server on a Windows platform. The existing Java implementation that is internal to Deployit is still available for users running the Deployit server on Unix or Windows.

Because of this, the WINRM connection type has been renamed to WINRM_INTERNAL and the new WINRM_NATIVE connection type can be used to select the native Windows WinRM implementation. The WINRM_HTTP and WINRM_HTTPS connection types that were duplicates of the WINRM connection type have been removed.

Please update any scripts or procedure that create overthere.CifsHost CIs will have to be updated to reflect the new connection types:

• Replace WINRM with WINRM_INTERNAL.
• Replace WINRM_HTTP with WINRM_INTERNAL and winrmEnableHttps set to false.
• Replace WINRM_HTTPS with WINRM_INTERNAL and winrmEnableHttps set to true.

You do not have to upgrade the contents of your repository. The UpgradeToRemotingPlugin394 upgrader mentioned below will take care of that.

Automatic repository upgrade

The first time this version of Deployit is started with a repository created with an older version of Deployit, the following upgraders will automatically upgrade the repository:

• Deployit394Checksums: Deployit 3.9 introduced the checksum property which stores SHA-1 checksums of artifacts. The SHA-1 algorithm in some cases generated too short checksum strings. This upgrader will recalculate all incorrect checksums.
• UpgradeToRemotingPlugin394: This upgrader converts the connection type properties of all overthere.CifsHost properties in the repository as follows:
  • Replaces WINRM with WINRM_INTERNAL.
  • Replaces WINRM_HTTP with WINRM_INTERNAL and sets winrmEnableHttps to false.
  • Replaces WINRM_HTTPS with WINRM_INTERNAL and sets winrmEnableHttps to true.
  • Replaces WINRS (only available in hotfix versions of Deployit) with WINRM_NATIVE.

Upgrading to Deployit 3.9.3

Automatic repository upgrade

The first time this version of Deployit is started with a repository created with an older version of Deployit, the following upgraders will automatically upgrade the repository:

• Deployit393ReportsView: Deployit 3.9.3 introduces the new global report#view permission to control how is allowed to view the reports tab. This upgrader will grant all roles that have the login permission the report#view permission.

Upgrading to Deployit 3.9.2

DEPL-4652

An autoPrepareDeployeds method has been added to the deployment object in the CLI and the corresponding REST API that works as follows:

• For an initial deployment without any deployeds, it is identical to a call to generateAllDeployeds.
• For an initial deployment with at least one deployed, the behaviour is as below.
• For an update deployment, keep existing mappings but:
  • Add mappings for new deployables, if allowed by types and tags.
  • Add mappings for new containers, if allowed by types and tags.
  • Remove mappings for old deployables, without a warning.

Upgrading to Deployit 3.9.0

Upgrading plugins

Plugins provided by XebiaLabs for Deployit version 3.8 will work in Deployit 3.9. There is no need to upgrade plugins when you are upgrading to Deployit 3.9.

Custom plugins built in Java against the Deployit plugin API will need to take the following change into account:

• class ReadOnlyRepository returned from the DeploymentPlanningContext.getRepository() call has been replaced by class Repository. The new Repository class provides an extension on the original ReadOnlyRepository class.

Recompiling the plugin Java source code against the new plugin API should bring your plugin up-to-date.

Checksum property

In Deployit 3.9, a new system property called checksum has been introduced on configuration items of type udm.BaseDeployableArtifact. This property influences how Deployit will calculate upgrades of deployments (See: Packaging Manual). If you've previously defined a property called checksum on your configuration items, please rename it to avoid clashes with the newly introduced checksum property.

Importing (old style manifests)

On importing it is now possible to pass in a CI-Name for non-artifact configuration items. The behaviour for both Artifacts and non-Artifacts is now the same, both will be stored under the CI-Name value if it is given, and fallback to the Manifest entry name Name.

Upgrading to Deployit 3.8.4

DEPLOYITPB-4188

In versions of Deployit prior to 3.8.4, the envVars property on the generic.Container type of the generic-plugin (and therefore also on the sql.SqlClient type of the database-plugin and its subtypes) was resolved using FreeMarker. From Deployit 3.8.4 onwards this is no longer done as their values should refer to Unix or Windows environment variables which will be resolved by the shell.

Upgrading to Deployit 3.8.3

Automatic repository upgrade

The first time this version of Deployit is started with a repository created with an older version of Deployit, the following upgraders will automatically upgrade the repository:

• Deployit383AclClear: The upgrader that runs when upgrading from a pre-3.7.0 version of Deployit to a version below Deployit 3.9.3 (Deployit37Security, see below) left behind incorrect rep:policy nodes in the repository (DEPLOYITPB-4141). This upgrader removes those nodes.

Upgrading to Deployit 3.8.0

Automatic repository upgrade

The first time this version of Deployit is started with a repository created with an older version of Deployit, the following upgraders will automatically upgrade the repository:

• Deployit38ConfigurationRoot: This upgrader creates the new Configuration root that is supported by Deployit 3.8.0 and up.

Release Dashboard

The Release Dashboard feature has changed to make it easy to reuse a deployment pipeline for many applications. A new CI has been introduced, release.DeploymentPipeline which is stored under the new Configuration root node. Each application that should be used in the Release Dashboard can link to the required deployment pipeline.

When starting Deployit 3.8 for the first time, any deployment pipelines configured in 3.7 will be converted to 3.8 deployment pipeline CIs.

Placeholder scanning

Placeholder scanning in archives (EAR, WAR, etc.) is now disabled by default. To enable it, edit deployit-defaults.properties and add the following line:

udm.BaseDeployableArchiveArtifact.scanPlaceholders=true

Custom Java plugins

If you have custom Java plugins written against the Deployit 3.7 API, you may need to make some changes to take advantage of the new features in Deployit 3.8.

Deployit 3.8 has changed the API to manage deployments and tasks. Java plugins that use the UDM Plugin-API from Deployit 3.7 will continue to work using a legacy layer. However, if your Java plugin extends classes from the Generic plugin or Python plugin, you may need to update your Java code, since these plugins now use the new API. Here's a table of replacements (read 'c.x.d.p' as 'com.xebialabs.deployit.plugin')

Upgrading to Deployit 3.7.0

• Deployit 3.7 contains an updated security implementation. Data from a pre-3.7 repository will automatically be converted to the new security model. See the section below for a complete description of the security upgrade.
• New versions of the following plugins are needed:
  • WAS
  • WLS
  • Tomcat
  • JBoss

Security upgrade

Permissions and repository

Starting with Deployit 3.7, Deployit needs to know the admin user's password to access the repository. This must be entered in the deployit.conf file prior to starting the new Deployit server. Note that the password in the configuration file is encrypted by the Deployit server when it starts.

The main change in the Deployit 3.7 security model is that local security permissions are stored only on root nodes and core.Directory CIs. During the upgrade, Deployit will convert pre-3.7-style permissions to the new permission system and automatically create directories to attach the relevant permissions to. The server log contains a description of the old security information that was found and how this was translated to the new permission system.

Permissions in Deployit are now assigned to roles instead of directly to principals. Deployit itself contains the definitions of roles and the principals they map to. Deployit will create a role for every user it finds during the upgrade.

It is also no longer possible to globally grant certain permissions. Permissions deploy#initial, deploy#upgrade, deploy#undeploy, import#initial, import#upgrade, task#skip_step, task#move_step can now only be granted on a directory CI. Deployit will convert these global permissions into directory permissions during the upgrade.

Deployit 3.7 no longer allows deny permissions. Instead, permissions should be segregated using the new core.Directory grouping CIs. The deny permissions are not migrated by Deployit and an equivalent configuration must be created manually.

Permission repo#edit no longer allows deleting of packages. From 3.7 onwards, new permission import#remove is required to be allowed to delete packages.

In summary, these are the security changes in Deployit 3.7:

LDAP

Configuration of LDAP security has also changed. The configuration in 3.7 is specified in a file called deployit-security.xml. The 3.6-style LDAP configuration in the jackrabbit-jaas.config file can be migrated easily to the 3.7 format. The following table describes the mapping of the 3.6-style LDAP setup to the new 3.7 setup:

See the System Administration Manual for more details on the available LDAP configuration options.

If you used Deployit 3.6 with LDAP, you will have modified the conf/jackrabbit-repository.xml file. For 3.7, the Security snippet must be:

<Security appName="Jackrabbit">
        <SecurityManager class="org.apache.jackrabbit.core.DefaultSecurityManager" workspaceName="security" />
        <AccessManager class="org.apache.jackrabbit.core.security.DefaultAccessManager" />

        <LoginModule class="org.apache.jackrabbit.core.security.authentication.DefaultLoginModule">
                <param name="anonymousId" value="anonymous" />
                <param name="adminId" value="admin" />
        </LoginModule>
</Security>

Additional upgrade steps

After completing the upgrade procedure described in Upgrading the Server above, perform the following additional steps:

1. Edit the deployit.conf file and enter the admin password to property admin.password.
2. If you use LDAP, migrate the LDAP configuration to the deployit-security.xml configuration file.
3. Create a backup of your repository.
4. Start the Deployit server. When the server starts, Deployit will migrate the security information from the 3.6 repository to the 3.7 structure. In this process, Deployit will automatically create a role for each user in Deployit as well as special core.Directory CIs to assign permissions to.
5. Log into Deployit as the admin user and inspect the changes to the repository.
6. Open the Admin tab to create meaningful role assignments.
7. Open the Repository tab to create meaningful directory names.

Completing the migration

After the Deployit automated upgrade completes, log in to the Deployit GUI with various credentials to ensure that the security setup was correctly migrated. It is now possible to see the security permissions on directory CIs in the Repository Browser.

Upgrading to Deployit 3.6.0

• Deployit 3.6 is a drop-in replacement for Deployit 3.5. No repository migration is needed.
• New versions of the following plugins are needed:
  • WAS plugin
  • WLS plugin

Upgrading from Deployit 3.0.x or earlier

Deployit and the plugins were changed extensively in version 3.5. As a consequence, it is not possible to automatically migrate from a 3.0 or earlier version to version 3.5 or later. If you want to perform this type of upgrade, please contact the XebiaLabs support team.