Getting Started for Engineers (DevBox Version)
NOTE: This document describes how to get Agile Cloud Manager working on a DevBox. Do this first because your CICD process will begin with working DevBoxes for each of your engineers.
After reading some of the architecture documentation, an engineer can get all 4 working demos up and running very quickly.
You can run the Demos on a Windows or Linux DevBox.
We have standardized an automated process of setting up a Windows DevBox using an AWS EC2 instance, with easy instructions that you can follow on this page.
After you get the standardized EC2 DevBox working, you can subsequently use the same provisioning scripts to turn your Windows laptop into a DevBox, or you can modify to set up your own Linux or Mac DevBox.
The steps for setting up a standardized Windows DevBox are:
- Step One: Create the DevBox
- Step Two: Login to the DevBox
- Step Three: Confirm that acm was added to PATH
- Step Four: Assemble keys.yaml and config.yaml
- Step Five: Copy keys.yaml and config.yaml into the EC2 DevBox
- Step Six: Run the CloudFormation Demo
- Step Seven: Run the Terraform/Packer Demo
- Step Eight: Run the ARM Template Demo
- Step Nine: Run the Custom Controller Demo
- Step Ten: Stop or Destroy DevBox Instance
- Step Eleven: (Optional) Restart DevBox Instance
Each of these steps is described in detail in the following paragraphs.
Step One: Create the DevBox
Log In To AWS And Set Region
Log in to AWS as root.
In AWS, set the region to us-west-2 Oregon as shown in the following screen shot.
Work In CloudShell
Then open up CloudShell by searching for it in the AWS Services search tool and right clicking it in the results to open up a CloudShell in a new tab as shown in the following screen shot.
Download the acm_devbox_windows_ec2.yaml CloudFormation template by pasting the following command in CloudShell.
wget https://github.com/AgileCloudInstitute/aws-building-blocks/blob/master/cf/acm_devbox_windows_ec2.yaml?raw=true -O acm_devbox_windows_ec2.yaml
Examine contents of cloudshell by running:
ls -al
Template should be present and there should NOT be any MyKeyPair.pem in the directory.
Run the acm_devbox_windows.yaml CloudFormation template from CLI.
aws cloudformation create-stack --stack-name acmWindowsDevboxStack --template-body file://acm_devbox_windows_ec2.yaml
Work In CloudFormation
Open the CloudFormation service by going back to the first tab from which you set the region etc above, and this time search for “CloudFormation” in the AWS Services search tool and this time right click on CloudFormation to open the CloudFormation service in yet another new tab as shown in the following screen shot.
In the resulting CloudFormation service tab, find the acmWindowsDevBox stack and wait until it is in the “CREATE_COMPLETE” state, as shown in the following screen shot.
Get KeyPair In CloudShell
Next, back in the CloudShell service tab, get the id of the key pair by running the following command.
aws ec2 describe-key-pairs --filters Name=key-name,Values=MyKeyPair --query KeyPairs[*].KeyPairId --output text
Paste the result of the preceding command into the following command as the key pair id. For example, if the key pair id were “key-123e4rrrtrt56u7r8” , then the next command you would run would look like:
aws ssm get-parameter --name /ec2/keypair/key-123e4rrrtrt56u7r8 --with-decryption --query Parameter.Value --output text > MyKeyPair.pem
Confirm that the keypair has been downloaded by typing the following command and looking for MyKeyPair.pem in the results:
ls -al
Then open the contents of MyKeyPair.pem with the following cat command:
cat MyKeyPair.pem
-----BEGIN RSA PRIVATE KEY-----
Long-alphanumeric-key-many-lines
Long-alphanumeric-key-many-lines
Long-alphanumeric-key-many-lines
Long-alphanumeric-key-many-lines
Long-alphanumeric-key-many-lines
-----END RSA PRIVATE KEY-----
Select the entire contents of the MyKeyPair.pem file, beginning with the -----BEGIN RSA PRIVATE KEY----- line and ending with the -----END RSA PRIVATE KEY----- line. You may need to shrink the size of the window contents in order to reduce the entire contents of the key into the visible window. Copy using Ctrl-C, this will put the contents of the KeyPair into the clipboard. The following screenshot shows how to reduce the size of the contents of the browser window and then select the entire key into the clipboard as follows:
Step Two: Login to the DevBox
Navigate To EC2 Instance In AWS Console
Back in the initial browser tab from which you opened up the other tabs, search for the EC2 service in the AWS Services search tool and right click on it to open a new tab with the EC2 service as shown in the following screen shot.
Then, on the resulting EC2 service tab, click on the “instances (running)” button as shown.
Then, on the resulting list of instances, click on the instance to navigate to the instance as shown in the next screen shot.
Navigate To RDP Client SetUp Tool
Click on the “Connect” button on the resulting instance detail screen.
Then click the “RDP client” tab on the connect page.
Convert Key Pair Into Password
Click on the “Get Password” link on the resulting screen.
Paste the contents of the MyKeyPair.pem from the clipboard by Ctrl-V into the text area that asks for the key pair. Remember, you put the contents of MyKeyPair.pem into the clipboard in a preceding step above. The text area should now contain a multi-line keypair with the first line reading -----BEGIN RSA PRIVATE KEY----- and the last line reading -----END RSA PRIVATE KEY-----.
Click on the “Decrypt Password” button to decrypt the password.
The following screenshot illustrates pasting in the key and then clicking on “Decrypt Password.”
The decrypted password will now be visible on the page you will be directed to. Copy the password to a safe place like a text file where you can have it on hand for subsequent steps.
Download Then Open The RDP File
Click on the “Download remote desktop file” button to download the RDP file. Then, after the RDP file has been downloaded, click the new link at bottom to open the RDP file.
The following screen shot illustrates:
- The decrypted password.
- The button to download the RDP file.
- And then the new link at bottom to open the RDP file after it has been downloaded.
Connect Using RDP File
Click “Connect” on the resulting dialog box to log in to the remote desktop of the new DevBox as shown in the following screen shot.
You will need to paste in the password you just decrypted in a preceding step into the login screen shown in the following screen shot.
Then click to authorize connection to the new unknown ec2 instance as shown in the following screen shot.
Authorize Discoverability
The result will be that you will log in to the Windows desktop of the ec2 devbox that you created with the create-stack cli command above. When a dialog box asks you to authorize allowing the EC2 instance to be discoverable by other computers on the same network, click yes to approve, as shown in the following screen shot taken inside the RDP client.
Step Three: Confirm that acm was added to the PATH of the EC2 instance
The EC2 instance’s UserData startup script installs acm and other required programs.
The t2.micro instance size we are using is part of the AWS free tier, but is somewhat slow and sometimes takes longer to complete tasks than a larger instance size would require.
Therefore, you should check to ensure that acm has finished installing by doing the following:
Open PowerShell as administrator inside the RDP client of the new DevBox and run the following commands in the subsequent sections of this document.
PS C:\Users\Administrator>acm version
1.2
If the result is NOT a valid version such as 1.2 shown in the example above, then try the following:
PS C:\Users\Administrator>$env.PATH
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\Users\Administrator\AppData\Local\Programs\Python\Python311\;C:\Users\Administrator\AppData\Local\Programs\Python\Python311\Scripts\;C:\Program Files\Amazon\AWSCLIV2\;C:\Users\Administrator\acmhome;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files\Amazon\cfn-bootstrap\;C:\Program Files\Git\cmd;C:\Program Files\Git\bin;C:\Users\Administrator\AppData\Local\Programs\Python\Python311\Scripts\;C:\Users\Administrator\AppData\Local\Programs\Python\Python311\;C:\Users\Administrator\AppData\Local\Programs\Python\Launcher\;C:\Users\Administrator\AppData\Local\Microsoft\WindowsApps;
If you find C:\Users\Administrator\acmhome in the PATH output as shown above, then type the following dir command to confirm that acm.exe is among the many contents of the C:\Users\Administrator\acmhome directory.
PS C:\Users\Administrator>dir C:\Users\Administrator\acmhome
…
acm.exe
…
Next, try the acm version command again as follows:
PS C:\Users\Administrator>acm version
1.2
If that still does not work, create a ticket on GitHub and we will gladly help you.
Step Four: Assemble keys.yaml and config.yaml
Now that you have created your DevBox, you will need to put keys and config into it so that the Agile Cloud Manager can be used. You can assemble all the required inputs using the reliable process described in our article “Assemble keys.yaml and config.yaml”.
After you have assembled your keys.yaml and your config.yaml, return here and continue to Step Five below.
Step Five: Copy keys.yaml and config.yaml into the EC2 DevBox
Next, take the keys.yaml and config.yaml you created and place them in C:\Users\Administrator\acm\keys\starter
You can copy and paste from your laptop into the EC2 instance because the RDP connection program is working as a Windows program.
So you simply open keys.yaml in your laptop and copy the contents into the Windows clipboard.
Then you navigate into the RDP client and open notepad.exe in the EC2 instance through the RDP client, and you then paste the contents of the clipboard into the notepad instance in the RDP client and you save the file as “all files” and name it “keys.yaml” and specify its location to be C:\Users\Administrator\acm\keys\starter
Copy and paste config.yaml from your laptop into the EC2 instance’s C:\Users\Administrator\acm\keys\starter using the same process described in the preceding paragraph.
Step Six: Run the CloudFormation Demo
Create Working Directory
Back in PowerShell, create a directory C:\acm_demos\cf and then navigate to it in Powershell.
mkdir C:\acm_demos\cf
cd C:\acm_demos\cf
dir
Run SetUp
Then run the following command to set up:
acm setup on sourceRepo=https://github.com/AgileCloudInstitute/acm-demo-cloudformation.git
The result should look like this:
Turn On Appliance And Monitor Progress
Then turn on the appliance as follows:
acm appliance on
From your laptop, navigate to the AWS console web site, and then navigate to the CloudFormation service within the AWS console website to view the stacks being created.
You can navigate in the CloudFormation interface to view the resources as they are being created within each stack.
Also review the local logs on the DevBox by navigating to C:\Users\Administrator\acm\logs and opening up log-verbose . You can review how to interpret the logs by reading the articles about how to use logging in the Agile Cloud Manager.
Turn Off Appliance
Wait until after the resources have been created.
Then turn off the appliance by typing the following command in PowerShell within the EC2 instance RDP client:
acm appliance off
Wait for the acm appliance off command to successfully finish running. Confirm that the run was successful both in PowerShell and in the AWS console. In PowerShell, you can check that there were no error messages and you can wait until the command prompt returns. In the AWS console, you can view the CloudFormation service’s list of stacks, and you can confirm that each of the new stacks was deleted.
Dig Deeper
If you encounter any errors, or if you want to experiment, dig deeper, and potentially cleanup after running “acm appliance on” and “acm appliance off”, you can try reading the instructions at this link
Step Seven: Run the Terraform/Packer Demo
Create Working Directory
Back in PowerShell, create a directory C:\acm_demos\tf_packer and then navigate to it in Powershell.
mkdir C:\acm_demos\tf_packer
cd C:\acm_demos\tf_packer
dir
Run SetUp
Then run the following command to set up:
acm setup on sourceRepo=https://github.com/AgileCloudInstitute/acm-demo-terraform-packer.git
Turn On Appliance And Monitor Progress
Then turn on the appliance as follows:
acm appliance on
In your web browser, open up portal.azure.com and navigate to the “Resource groups” service. You will be able to see new resource groups being created as the “acm appliance on” command runs, and you can navigate inside the resource groups to see the individual resources.
If for some reason your portal.azure.com interface does not show your resource groups being created, you can also navigate to the subscription in Azure, and the resources will be shown as they are being created and then later destroyed.
Also review the local logs on the DevBox by navigating to C:\Users\Administroator\acm\logs and opening up log-verbose . You can review how to interpret the logs by reading the articles about how to use logging in the Agile Cloud Manager.
Turn Off Appliance
Wait until after the resources have been created.
Then turn off the appliance by typing the following command in PowerShell within the EC2 DevBox instance RDP client:
acm appliance off
Wait for the acm appliance off command to successfully finish running. Confirm that the run was successful both in PowerShell and in the Azure Portal. In PowerShell, you can check that there were no error messages and you can wait until the command prompt returns. In the Azure Portal, you can view the Resource Groups to confirm that each of the new resources was deleted, and you can review the Subscription to see that the status of each of the resources has been changed to indicate deleted.
Dig Deeper
If you encounter any errors, or if you want to experiment, dig deeper, and potentially cleanup after running “acm appliance on” and “acm appliance off”, you can try reading the instructions at this link
Step Eight: Run the ARM Template Demo
Create Working Directory
Back in PowerShell, create a directory C:\acm_demos\arm and then navigate to it in Powershell.
mkdir C:\acm_demos\arm
cd C:\acm_demos\arm
dir
Run SetUp
Then run the following command to set up:
acm setup on sourceRepo=https://github.com/AgileCloudInstitute/acm-demo-arm.git
Turn On Appliance And Monitor Progress
Then turn on the appliance as follows:
acm appliance on
In your web browser, open up portal.azure.com and navigate to the “Resource groups” service. You will be able to see new resource groups being created as the “acm appliance on” command runs, and you can navigate inside the resource groups to see the individual resources.
If for some reason your portal.azure.com interface does not show your resource groups being created, you can also navigate to the subscription in Azure, and the resources will be shown as they are being created and then later destroyed.
Also review the local logs on the DevBox by navigating to C:\Users\Administroator\acm\logs and opening up log-verbose . You can review how to interpret the logs by reading the articles about how to use logging in the Agile Cloud Manager.
Turn Off Appliance
Wait until after the resources have been created.
Then turn off the appliance by typing the following command in PowerShell within the EC2 DevBox instance RDP client:
acm appliance off
Wait for the acm appliance off command to successfully finish running. Confirm that the run was successful both in PowerShell and in the Azure Portal. In PowerShell, you can check that there were no error messages and you can wait until the command prompt returns. In the Azure Portal, you can view the Resource Groups to confirm that each of the new resources was deleted, and you can review the Subscription to see that the status of each of the resources has been changed to indicate deleted.
Dig Deeper
If you encounter any errors, or if you want to experiment, dig deeper, and potentially cleanup after running “acm appliance on” and “acm appliance off”, you can try reading the instructions at this link
Step Nine: Run the Custom Controller Demo
Create Working Directory
Back in PowerShell, create a directory C:\acm_demos\custom and then navigate to it in Powershell.
mkdir C:\acm_demos\custom
cd C:\acm_demos\custom
dir
Run SetUp
Then run the following command to set up:
acm setup on sourceRepo=https://github.com/AgileCloudInstitute/acm-demo-custom-api.git
Then turn on the appliance as follows:
acm appliance on
Turn On Appliance And Monitor Progress
Review the local logs on the DevBox by navigating to C:\Users\Administroator\acm\logs and opening up log-verbose . You can review how to interpret the logs at this link.
The custom controller example application created a local API on the DevBox running on localhost and then communicated with that local API. The logs will give you a clear picture of what happened as the appliance was being created. You can also examine the source code to better understand how the source code is writing to the logs.
Turn Off Appliance
Wait until after the resources have been created.
Then turn off the appliance by typing the following command in PowerShell within the EC2 DevBox instance RDP client:
acm appliance off
Wait for the acm appliance off command to successfully finish running. Confirm that the run was successful in PowerShell, which should print out something very similar to what gets stored in the logs. In PowerShell, you can check that there were no error messages, you can review the workflow status reports that get printed to the shell during runtime, and you can wait until the command prompt returns.
Dig Deeper
If you encounter any errors, or if you want to experiment, dig deeper, and potentially cleanup after running “acm appliance on” and “acm appliance off”, you can try reading the instructions at this link
Step Ten: Stop or Destroy DevBox Instance
Then either stop the EC2 DevBox instance or destroy it.
Option 1: Stop Instance
To stop the instance do this:
- In AWS the Console, navigate to EC2 instance.
- In Instance State drop-down menu, select “Stop”.
- Then click the “Stop” button on the resulting pop up dialog box.
Option 2: Destroy Instance
To destroy the DevBox instance instead of stopping it, do the following:
Make sure that you have cleaned up anything important. For example, you could copy the logs if you need the logs. And you could delete the secrets if you wish to.
Go back into AWS CloudShell in your web browser and refresh if necessary. Then simply delete the stack by running the following in the CloudShell from which you originally ran the commands at the start above that created the CloudFormation stack:
aws cloudformation delete-stack --stack-name acmWindowsDevboxStack
Consider Modifying Your DevBox Process
If you decide to continue using EC2 instances as DevBoxes instead of your own laptop, you can modify the CloudFormation template to better fit your needs including:
• Keep config, secrets, and logs in other secure file shares whose lifecycle is managed separately, so that you can eliminate the manual steps associated with config, secrets, and logs.
• Protect your EC2 DevBox instances from being terminated.
• Better integrate with networking and IAM for your enterprise.
If you want to use your own laptops as DevBoxes, then simply run the UserData script that is included in the CloudFormation Template that this tutorial used to create your EC2 DevBox. The UserData script creates all the required directories, installs all the prerequisites, and adds acm to the PATH. You can modify the script as needed if you already have some of the prerequisites installed, or if you need a different operating system. Note that the required directories in Linux are different than in Windows. The Linux required directories are demonstrated in the scripts that the Agile Cloud Manager Pipeline Demo repository uses to provision Linux agents.
Step Eleven: Re-Start or Re-Create DevBox Instance
Your next use of the DevBox will depend on whether you chose to stop or delete the DevBox.
If you stopped the DevBox, then simply go into the AWS Console and find the instance and restart it.
If you deleted the DevBox, then begin the instructions on this page from the top again to re-run the create-stack command in CloudFormation. Use your backed up copies of keys.yaml and config.yaml instead of creating new keys.yaml and config.yaml. But make sure to create your own standards for secrets lifecycle and secrets management. The process of creating and destroying keys and config can be completely automated and can be well secured. We are simply offering the approach in this documentation to get you started.
After you have followed the instructions in this document to get a working DevBox, the next step will be to get the demos running in GitHub by following the instructions at this other link. You can use the Agile Cloud Manager with any pipeline tool, including Jenkins, Azure DevOps, GitLab, and many others. The GitHub example is offered only because it is widely used.