/ AUTOMATION, OPERATIONS

Operationalizing vRealize Automation - Release

One of the key challenges of operating any cloud management platform such as vRealize Automation is managing the release process, and introducing new content and services without user impact or down time. vRealize Suite Lifecycle Manager (vRSLCM) was introduced with the aim to address day zero (deployment) and day two operations for the vRealize Suite products including content release. In this blog article we will take a look at the content release process with vRSLCM.

vRealize Suite Lifecycle Manager provides native content release management allowing automation teams to treat vRealize Automation (vRA) content as infrastructure as code with support for version control in an external Git repository as part of the release process.


Pre-Requisites for content release

Some of the key prerequisites to prepare for when considering content release with vRealize Suite Lifecycle Manager are outlined below. These prerequisites hold true even if your not using vRSLCM for content release and building your own content release management solution.

Structuring Code for Release

A critical and often overlooked piece to content release is structuring you workflow / code to allow for a modular and iterative release. If your workflow design is modular and on a feature basis this significantly simplifies the release process, rather than monolithic workflow with many unrelated integrations. The end goal of well structured workflow on a feature basis is the ability to release regularly with an iterative approach. Composite Blueprints should be able to be released independently of workflow in most scenarios. A vRO workflow package release should make up a single feature and EBS subscription if applicable.

The benefits of well structured workflow to release management include:

  • reduced chance of a merge conflicts
  • iterative release process allowing for targeted releases

Workflow Design Patterns

These are some high level design patterns to enable an iterative release process.

Modular Workflow

A workflow integration thats covers a single system integration and use case. Example in this case is applying a vSphere tag.

Loosely coupled event broker integrations

Event broker subscriptions include a conditional property toggle, to turn on or off a feature based on the existence of a defined property and value. Ensuring Event Broker (EBS) subscription are always conditional allows for separation of release cycle for blueprints and workflow, new workflow and subscriptions can be introduced but will only be consumed based on the existence of conditional properties.

All Condition Value Example
Lifecycle State Name Equals {State Name} VMPSMasterWorkflow32.MachineActivated
State Phase Equals Pre Pre
data~machine~properties~{property} Equals “true” enable.servicenow=true

Workflow Design Anti-patterns

These are some workflow design and integration patterns to avoid to prevent complexity when managing vRA/vRO release process.

Monolithic Workflow

A workflow that combines many unrelated systems with complex decision/tree logic built into the workflow. A complex parent workflow with many unrelated downstream integrations. This pattern causes large releases even for minor changes forcing updates to all linked components, significantly increasing complexity and risk of the release.

Tightly coupled event broker integrations

Event broker integrations are configured with simple mandatory conditions that will always be hit, or bound to static values only such as blueprint name, such that the integration can not be turned on or off without unpublishing the subscription.

Decoupling Code from Configuration

To ensure code can be moved between environments, code should be loosely coupled with references to environment configuration such as endpoints, passwords and anything object based. Anything that is object based will be assigned a unique identifier by vRealize Orchestrator and as such will not have the same identifier when moved between different vRO instances. If code is tightly coupled with hard references this will cause objects to detach and workflow failure as a result of content migration between multiple instances.

For more details on this see Aaron’s article for an example of decoupling configuration via action based lookup, decoupling-configuration-from-code

Also consider the release process during development for items that are bound to the vRO inventory and limit their use, this includes items such as REST operations, instead consider dynamic creation of the REST operations in code on the fly preventing the need to move them at all.

Git Repository

A pre-requisite to managing vRealize Automation content as code with external version control is a Git repository. Currently as of the 2.0 release the only supported repository for vRealize Suite Lifecycle Manager (vRSLCM) is GitLab. Currently Gitlab Community Edition or Enterprise Edition are supported from vRSLCM 2.0.

Create a new repository in gitlab under a group, ensuring lifecycle manager has a svc account with access to the group.

You will need to create a personal token under the vRSLCM user in GitLab enabling api access, a token will be generated which will be required later when configuring source control in vRSLCM. Store the token in a secure location as it can’t be retrieved.

Note: Its still currently possible to move content directly without using an external source control system with vRSLCM 2.x, you just lose some of the traceability an external source repository provides.


Environment Architecture

A key consideration for release process is the deployment environment architecture and ensuring this can meet the needs of the release process with an understanding of how the product operates and the roles within an organisation aligned to the release process, including blueprinters, vRO developers, testers, consumers, and release managers. Some of the these roles may be covered by the same team/people or may be disparate often depending on the organisation size, and complexity of the environment and integrations.

The mapping of roles to environments, along with an understanding of provisioning concurrency and scale is critical to defining your vRA architecture, both in number environments required, and for alignment to the reference architecture deployment size. As a minimum starting point development and production environments should be on separate instances to support the ongoing blueprint/development activities. The number of vRA/vRO environments required beyond this, will vary based on organisation and roles as discussed above. A simple example of mapping functions and roles to environments is below, in this case a minimum of 4 environments is required to meeting the needs with potential for 5 environments depending on how the teams operate and interact.

A few technical considerations in regards to how vRA and vRO integrate when considering environment architecture are outlined below.

  • Whilst vRO has the concept of workflow versioning, only the latest or current version is executable.
  • Each tenant within vRA can natively be connected to a single instance of vRO embedded or external.
  • vRO workflow is executed and identified natively via an EBS subscription based on workflow id.

Configuring vRSLCM for Content Management

Identity Manager Integration

To support role based access control in vRSLCM, an external VMware Identity Manager instance must be configured as an authentication source. vRSLCM can deploy an instance of vIDM, or an existing instance can be leveraged. In this case we will connect to an existing shared instance.

Note: Ensure Active directory configuration matches exactly if the AD source is already pre-configured in identity manager, vRSLCM will validate against the existing configuration.

Settings > User Management

Once vIDM is configured successfully, access control can be applied to identity manager groups or users. The key roles within the vRLCM with regards to content are the Content Developer, and Release Manager. The Content Developer role in particular has a restricted access dependent on the endpoint policies applied to each endpoint during configuration.

Setting > User Management > Access Control

Content Endpoints

To allow for content release between environments we first need to add endpoints for the relevant vRA/vRO systems for content capture and release. Endpoints need to be created for all vRA tenants, vRO orchestrator servers and all branches to be used for source control in your git repository.

New Endpoint

Configure vRA Endpoints

Note: When configuring a vRA endpoint with an embedded vRO orchestrator, the orchestrator endpoint will be automatically discovered and configured.

vRA Content Endpoint

Configure Endpoint Policy

Endpoint Policies provide governance controls around content endpoints with alignment to the roles of developer and release manager within vRSLCM.

The table below outlines the content impact of endpoint policy based on the assigned vRSLCM role.

Content Settings Role Behaviour
Content version is production ready Release Manager can view only production endpoints
Content version is production ready Developer Can only test endpoints that have allow Tests set,
and don’t include the production ready policy
Content version is NOT marked
production ready
Release Manager
Developer
Can view the test endpoints
that have the allow test policy set
Content version is NOT marked
SourceControlled
Release Manager
Developer
Can view the content endpoints
that do not have the Source Control
policy set on the content endpoint
Content version is marked
SourceControlled
Release Manager
Developer
All the content endpoints are displayed
based on other conditions in this table

Adding GitLab Endpoints

As a prerequisite to adding a GitLab endpoint, the source control token generated for the vRSLCM user should first be configured . Authentication to GitLab uses the previously generated GitLab personal access token.

Content Management >

Content Settings > Source Control Access > Add Source Control Server

Specify the FQDN of your Gitlab server.

Once added, edit, the Git Source Control configuration to apply the access token key.

Creating GitLab Endpoints

A GitLab endpoint is required for each branch within your git repository that will used for content source control, multiple branches can help with aligning releases to environments as part of the release process.

Content Management > Endpoints > GitLab

GitLab Endpoint Details

Note: Repository is in the format shortname group/repositoryName

GitLab Endpoint Policy Settings

Enabling the code review policy setting will result in a separate branch created per commit for each dependent component, the merge review is used for code review in git before merging to the target endpoint branch, in this case the development branch.

Code Review

You will need to create endpoints for each of your git branches. In the below example we have a development, test and master branch in Git and corresponding endpoints for each in vRSLCM.


Content Release Process

The content release process can be broken in the following sub components

  1. Content Capture
  2. Test
  3. Checkin
  4. Deploy

We will run through a simple use case of capturing a blueprint and its downstream dependencies. We will cover the testing process and custom pipelines in a followup blog post.

Content Capture and Checkin

Content Management >

Endpoints > Content > Add Content

In this example we will capture the CentOS blueprint content and its dependencies and check in to the development branch for code review. These steps can be performed as a single pipeline execution, or alternatively the content could be first captured, then a day 2 action performed on the content to checkin to a repository.

Content Capture Details

In this case we are capturing a CentOS blueprint and its downstream dependencies such as property groups and definitions from content endpoint which is our test vRA environment, ensuring we get the latest content from the endpoint. Tags can be used to help filtering, as well as group content for deployment. Marking the content as production ready or not will control what endpoints this content can be deployed to based on endpoint policy settings configured.

Check In Details

As we chose to both capture and checkin, we need to specify the GitLab endpoint or branch in which the content capture will be checked in. In this case we are are committing to the development branch in git.

Content Summary

Content Pipeline Execution

On submitting the content capture, the progress can be monitored under Content Pipelines

GitLab Code Review

Once complete, as we chose to “checkin” as part of the content capture process, the content is immediately committed to Git, also as the “enable code review” option was specified for this git endpoint, we have a new branch created for each change, with a merge request to the development branch within git, this can be merged to development once the changes have been code reviewed.

Once merged to development. The content can be recaptured directly from Git. In this scenario, I merged the development branch to the test branch externally in Git, then recaptured.

Endpoints > Content > CentOS 7

Recapturing content from GitLab

We are now updating vRSLCM version of the content by capturing directly for git (test branch) in preparation for release to the test environment.

Endpoints > Content > CentOS 7 > Capture

Deploy

With each capture the internal content version within vRLCM will also be incremented. In the below image we can see I have 4 versions with the latest copy sourced from the git master branch ready for deployment to our production environment.

Release Content

To deploy a version, we choose the version we want based on the versions above, clicking the Deploy action. We can then choose the destination endpoint to deploy, filtering endpoints by assigned tags.

Content > CentOs 7 > Activity Log

We can see the successful deployment status of the successful deployment in the activity log, including previous pipeline executions.

Deployed Blueprint

Thats it, our content has now been deployed to our production instance of vRA, including dependencies that makes up the blueprint such as property groups and definitions.

This post was just meant to give you an intro into moving content vRSLCM, we will drill into integrating testing and custom pipelines as part of a follow post to the series.