Assembling keys.yaml and config.yaml
We have automated most of the work involved in creating the keys.yaml and config.yaml files that you will need in order to run the demos.
The two major items are first to create Azure credentials and then second to create AWS credentials. Doing it in this order reduces the number of manual steps for this particular demo and therefore improves the reliability of the process.
Cloud Shell is also used to create the credentials in each cloud so that you do not have to concern yourself with the extra work of installing the Azure CLI and AWS CLI if all you want to do is run the demos in an environment like GitHub runners that come preinstalled with those CLI tools. But you must log into each cloud as the highest super user to create these credentials in order to get the demos working the first time. Later, you can experiment with tightening the security after the demos are working for you.
This article is divided into 5 sections as follows:
- Section One: Configuring Azure
- Section Two: Azure Subscription Owner Role
- Section Three: Azure Active Directory Global Administrator Role
- Section Four: Azure Active Directory Application Administrator Role
- Section Five: Configuring AWS
- Section Six: Validate The Complete keys.yaml and config.yaml
Section One: Configuring Azure
Step One:
Log into portal.azure.com as an account owner.
Step Two:
Create a new subscription. Then paste the subscriptionId and subscriptionName into a text file.
Step Three:
Navigate to the Azure Active Directory associated with the new subscription. Then paste the tenantId for that Active Directory into the same text file.
Step Four:
Open a cloud shell by clicking the cloud shell icon at the top of the screen, as shown in the following screen shot.
Step Five:
Select bash as the shell for simplicity, as shown in the following screen shot.
Step Six:
Type “az –version” in the cloudshell terminal. This was tested in version 2.48, with Python 3.9. If you later encounter problems, please note the version with the exact steps you followed, and open an issue so that we can help you.
Step Seven:
Download this script and a dependent arm template into your Azure cloud shell by typing the following into the cloud shell terminal:
wget https://github.com/AgileCloudInstitute/acm-demos-github/blob/main/acmDemoSetup.py?raw=true -O acmDemoSetup.py
wget https://github.com/AgileCloudInstitute/acm-demos-github/blob/main/subscriptionScopeRole.json?raw=true -O subscriptionScopeRole.json
Step Eight:
Confirm that the files have been successfully downloaded by typing “ls -al” and looking for the file name in the results. Make sure you do not have multiple copies remaining from previous downloads. If you have redundant copies, then delete all of them with the rm command and then re-run the two “wget” commands from step seven to get the newest versions. If there are old copies of config.yaml and keys.yaml, then back the old copies up somewhere else and delete them from the cloudshare with the rm command to avoid downstream errors below.
Step Nine:
Paste the subscriptionId, subscriptionName, and tenantId values from steps 2 and 3 above into the following command. If your subscriptionName contains spaces, make sure to put quote marks “” around the subscriptionName. Then, after you have put the correct values for subscriptionId, subscriptionName, and tenantId in place, run the following command to automatically create a keys.yaml and config.yaml populated with valid azure credentials that will be needed for the demo.
python acmDemoSetup.py create azurefiles subscriptionId=valid-subscription-id-guid subscriptionName=validSubscriptionName tenantId=valid-active-directory-tenant-id appRegistrationName=myappregtest1 clientSecretName=mycert2 clientSecretEndDate=2024-12-31
Step Ten:
Confirm the command completed without errors by reviewing the results as shown in the following screen shot. Things to check include:
- All required fields are present in config.yaml
- All fields in config.yaml have values
- All required fields are present in keys.yaml
- Three specific AWS keys have missing values as shown in the screen shot, because this is the Azure shell and you will have to populate the AWS values in the AWS cloud shell separately as shown in subsequent steps later in this tutorial.
- ErrorCount is 0
- config.yaml and keys.yaml should be created now in the current directory.
Step Eleven:
Click the download button as shown in the next screen shot.
Step Twelve:
Type in “config.yaml” in the resulting modal to select the config.yaml file for download, as shown in next screen shot.
Step Thirteen:
Then click on the “Click here to download file” link that will appear as shown in the following screen shot.
Step Fourteen:
Confirm that config.yaml has been downloaded into the Downloads folder of your local computer.
Do not alter the contents of config.yaml, because their precise machine-generated format is required for the demos to work properly.
Step Fifteen:
Click the download button in the Azure Cloud Shell again as you did in step 11 above.
Step Sixteen:
Type in “keys.yaml” (without quotes) in the resulting modal to select the keys.yaml file for download, as shown in the next screen shot.
Step Seventeen:
Click on the “Click here to download file” link which will appear at the bottom right of the screen, as shown in the next screen shot.
Step Eighteen:
Confirm that the keys.yaml file has been downloaded into the downloads file on your local computer. You will see that it contains valid Azure values, but that it is still missing the three AWS values.
DO NOT ALTER THE CONTENTS OF keys.yaml BECAUSE THE PRECISE, MACHINE-GENERATED FORMAT IS NECESSARY FOR THE DEMOS TO RUN PROPERLY.
Step Nineteen:
Move keys.yaml and config.yaml to a safe location outside the Downloads folder.
Section Two: Azure Subscription Owner Role
Confirm whether the subscription Owner role has been assigned to the app registration you created.
Step One:
Open a separate tab in your browser and navigate inside portal.azure.com to the subscription in which you created the resources.
Step Two:
On the Subscription’s main page, look on the left-hand navigation panel for “Access Control (IAM)” and click on it. Then on the resulting main panel details page, click on the “Role assignments” link to make the role assignments search tool visible. Type the first several letters of the app registration name you entered in the cli command and, if necessary, click on the “Refresh” tab on top to update the search.
The app registration will have the same name given in your keys.yaml file for clientName.
If you do not see the app registration listed with the “Owner” role for the subscription, then you must manually assign the subscription owner role.
The following diagram shows all of the parts of Step Two, and illustrates a situation where the subscription owner role is not showing as assigned even though the Azure CLI reports that the role was assigned. The words “No user assignments exist” in the screen shot indicate that the Owner role was not assigned to any app registration whose name begins with the “myapp” string you used in the search box.
If you get similar results showing that the “Owner” role for the subscription has not been assigned, you must manually assign the subscription Owner role in the portal.azure.com GUI.
Step Three:
You can manually assign the subscription Owner role by clicking on the “Add” button and choosing the “Add role assignment” option as shown in the following screen shot.
Step Four:
On the resulting “Add role assignment” page, click on the “Privileged administrator roles” tab, and then type “owner” into the resulting search box to filter the available roles. Select the “Owner” role from the resulting list of roles, and then click the “Next” button as shown in the following screen shot.
(Note that these are temporary credentials you can destroy after running the demo each time. You need a super user here to make sure that the demos work perfectly. Later, you can experiment with lower role assignments to tighten security after you have gotten the demos to work.)
Step Five:
On the resulting “Members” subtab, make sure that “Owner” was selected, then click the “Select members” link and, in the resulting modal, type the first few letters of the name of the app registration that you created. Then select it in the results and click the “Select” button on the bottom left to select the member to assign the role. All these actions are illustrated by red highlighted sections of the following screen shot.
Step Six:
The next screen shot illustrates how the screen will look after you select the app registration. Click on the “Select” button after you select the app registration by name as shown in the following.
Step Seven:
On the resulting screen, confirm that the app registration’s name is listed in the “Members” and click on the “Next” button as shown in the following screen shot.
Step Eight:
On the resulting “Review+Assign” page, confirm that the “Owner” role and the correct app registration name were selected, and then click the “Review+Assign” button as shown in the following screen shot.
Step Nine:
On the resulting main “Role assignments” page, confirm that the “Owner” role has been assigned to the proper name of the app registration that you created. The following screen shot shows how this should look.
Section Three: Azure Active Directory Global Administrator Role
Confirm that Active Directory roles have been assigned to the app registration you created. The required roles are “Global Administrator” and “Application Administrator”.
Step One:
Start by navigating to the Active Directory Tenant’s main page and clicking on the “Roles and administrators” tab as shown in the following screen shot.
Step Two:
On the resulting page, search for “Global” and click on the “Global Administrator” role as shown in the following screen shot.
Step Three:
On the resulting Global Administrator page, check to see if the name of the app registration that you created is listed among the assignees. If it is NOT listed, then click on the “Add assignments” link as shown in the following screen shot.
Step Four:
On the resulting “Add Assignments” page, search for the name of the app registration you created, then click to select it from the results, and then click the “Add” button, as shown in the following screen shot.
Step Five:
Your new app registration should now be listed among the members of the “Global Administrator” role on the resulting “Global Administrator | Assignments” page as shown in the following screen shot.
Section Four: Azure Active Directory Application Administrator Role
Confirm that the “Application Administrator” role has been assigned to your app registration in the Active Directory tenant.
Step One:
Navigate to the “Roles and administrators | All roles” page, then search for the “Application Administrator” role and then click on the role in the results, as shown in the following screen shot.
Step Two:
If your app registration’s name is not included in the list of assignees on the resulting “Application Administrator | Assignments” detail page, then click on the “Add assignments” button as shown in the following screen shot.
Step Three:
On the resulting “Add assignments” modal, search for the name of your app registration, then select it, and then click on the “Add” button as shown in the following screen shot.
Your app registration’s name should now be listed in the list of assignees on the “Application Administrator Assignments” page as shown in the following screen shot.
You are now ready to proceed to the AWS setup process.
Section Five: Configuring AWS
Step One:
Log in to the AWS GUI console as root. You will need to be a super user to create PowerUsers and perform other high-authority actions required to create the credentials required for this demo. Later on, after you have gotten the demo running successfully as root, you can later on experiment with tightening security for subsequent ongoing work.
Step Two:
Set the region to us-west-2 for the demo, as shown in the next screen shot. (Later, after you successfully complete the demo, you can experiment with other regions)
Step Three:
Search for “CloudShell” in the AWS services as shown in the following screen shot, and click the link to navigate to open a cloudshell terminal.
Step Four:
Type “aws –version” in the cloudshell terminal. This is tested in version 2.11. If you later encounter problems, note the version. and please report back to us.
Step Five:
Download the demo setup script into your cloudshell by typing the following into the terminal:
wget https://github.com/AgileCloudInstitute/acm-demos-github/blob/main/acmDemoSetup.py?raw=true -O acmDemoSetup.py
Step Six:
Confirm that the file has been successfully downloaded by typing “ls -al” and looking for the file name in the results.
Step Seven:
Run the following command to create IAM resources including keys:
python3 acmDemoSetup.py create aws userName=ACMUser_xyz groupName=SuperUserACM_xyz keyPairName=ACMKeyPair_xyz
Note that the values for userName, groupName, and keyPairName can later be changed to be any valid values. But start with these values because they work, assuming you do not already have resources with the same names created in your account.
Also note that we are using the “python3” command because we require Python 3, and because the AWS cloud shell uses Python 2 unless you explicitly specify “python3”.
Step Eight:
Examine the cloudshell terminal output to confirm there were no errors reported, and that the keys were printed to the terminal. The command and its output might look like:
[cloudshell-user@ip-10-8-123-108 ~]$ python3 acmDemoSetup.py create aws userName=ACMUser_xyz groupName=SuperUserACM_xyz keyPairName=ACMKeyPair_xyz
Beginning to run command.
Copy the following three key/value pairs to your keys.yaml to replace the placeholders:
KeyName: ACMKeyPair_xyz
AWSAccessKeyId: AKIAYT5RE4U89OIT456S
AWSSecretKey: dr45t67uy789oikmh76yhgtr45de32ws56tgb78k
[cloudshell-user@ip-10-8-123-108 ~]$
NOTE THAT YOUR AWSAccessKeyId AND AWSSecretKey WILL BE REAL. NOTE THAT THE VALUES ABOVE ARE MADE UP FOR SECURITY REASONS, BUT AT LEAST SHOW YOU THE PROPER FORMAT.
Step Nine:
Copy the following three lines of yaml after “Copy the following three key/value pairs to your keys.yaml to replace the placeholders:” at the end of the terminal output and paste them into the keys.yaml that was created when you ran the Azure command and saved the resulting keys.yaml to your local computer.
KeyName: ACMKeyPair_xyz
AWSAccessKeyId: AKIAYT5RE4U89OIT456S
AWSSecretKey: dr45t67uy789oikmh76yhgtr45de32ws56tgb78k
Note that the 3 preceding lines will have actual secrets that you will need to copy into keys.yaml
Also note that you will be replacing the empty/default lines that were written for keyName, AWSAccessKeyId, and AWSSecretKey when you ran the command that created keys.yaml from the Azure cloud shell. Note that later you can experiment with creating the files from the AWS cloudshell, but for this demo do it the way instructed here to reduce the number of manual steps to keep this demo simplified.
Step Ten:
Save a backup copy of the keys.yaml someplace safe, so you have access to it to delete the resources later.
DO NOT CHANGE THE FORMAT OF keys.yaml. THE MACHINE-GENERATED FORMAT MUST REMAIN UNCHANGED IN ORDER FOR THE DEMOS TO WORK PROPERLY. THE ONLY CHANGES YOU MAKE ARE TO PUT THE THREE MACHINE-GENERATED AWS LINES IN TO REPLACE THE EMPTY PLACEHOLDERS FOR THE AWS VARIABLES THAT WERE CREATED BY THE AZURE PROCESS.
Section Six: Validate The Complete keys.yaml and config.yaml
Do one more validation of the complete keys.yaml and config.yaml files by completing the following steps:
Step One:
Navigate to the directory into which you copied keys.yaml and config.yaml . If you are in Windows, use PowerShell so that the same commands will work.
Step Two:
Download “acmDemoSetup.py” by running the following command:
wget https://github.com/AgileCloudInstitute/acm-demos-github/blob/main/acmDemoSetup.py?raw=true -O acmDemoSetup.py
Step Three:
Confirm that the three required files are in the directory by running the “dir” command, which will generate the following output:
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 5/12/2023 4:59 PM 54523 acmDemoSetup.py
-a---- 5/12/2023 4:42 PM 830 config.yaml
-a---- 5/12/2023 4:50 PM 263 keys.yaml
Step Four:
Run the validation command as follows:
python acmDemoSetup.py validate-config-and-keys
Confirm that the validation script succeeded by checking to see that the console output includes the following lines:
All required fields are present in keys.yaml
All of the fields in keys.yaml have values.
All required fields are present in config.yaml
All of the fields in config.yaml have values.
ErrorCount is: 0
You are now ready to proceed with either the GitHub demo or the DevBox demo using the keys.yaml and config.yaml you created here.