Increase application visibility and governance using AWS Service Catalog AppRegistry

Many customers deploy applications with a multitude of resources using AWS CloudFormation templates.  As customers begin to scale, these templates are often re-used across multiple applications.  At this point, important tasks like identifying deployed applications and understanding which CloudFormation stacks are associated with an application become more difficult. Visibility is an important component of a mature DevOps / SecOps practice. Until today, customers have been using complex custom tooling or third-party solutions to address this problem.

Today, AWS Service Catalog released a new feature call AppRegistry to help customers natively build an AWS resources inventory that has insights into uses across applications.

AWS Service Catalog AppRegistry allows you to create a repository of your applications and associated resources.  You can then define and manage your application metadata.  This allows you to understand the context of your applications and resources across your environments. These capabilities enable enterprise stakeholders to obtain the information they require for informed strategic and tactical decisions about cloud resources.

AppRegistry provides a way to group your applications and application resources. You define applications within the AppRegistry by providing a name, description, associations to CloudFormation stacks, and associations to application metadata. Finance, security, and IT teams create and manage attributes that capture the important application metadata. These application attributes support an open JSON schema, providing you flexibility to capture the complexity of your enterprise metadata taxonomy. Example application attributes include: the application security classification, organizational ownership, application type, cost center, and support information. Developers can then associate required attributes with their applications.

AppRegistry is intended for use by a broad range of enterprise stakeholders:

Cloud Centers of Excellence (CCOE): AppRegistry allows the team to understand the full set of applications and resources provisioned by development teams and lines of business. This could help answer governance questions like: “Was the application deployed from an approved template?”

Builders: AppRegistry allows builders to find all of the resources associated with an application. They can search by different criteria that they provide, such as application instance, Region, account, or other dimensions enabled by tagging and metadata.  This helps answer questions like: “How many application versions are running in the development environment?”

DevOps:  AppRegistry allows them to search and query across domains for approved operations on resources.  This helps answer questions like:  “Which applications require updating?”

C-level executives (CIO, CTO, CISO): AppRegistry allows them to get a full view of cloud deployments including all application instances and resources across IT environments.

Security: AppRegistry allows security teams to identify resources involved in a security incident.  This helps answer the question:  “How many applications with this data classification are using a non-supported version of a specific component or resource?”

Compliance: AppRegistry allows risk and compliance team members to obtain a view of all resources within an application that currently meet specific compliance certifications. This helps answer the question:  “Which applications are tagged as HIPAA, FISMA, or FedRAMP compliant but have resources that do not meet that classification?”

Register an application in AppRegistry

First I’ll walk you through the steps required to create attribute groups and applications.  We will then associate an application with attribute groups and AWS CloudFormation stacks.

Step one

The CCOE administrator sets-up company-wide shared attribute groups.

The creation of these groups sets standards so that application can consistently be found using the same criteria.  As such, these groups should have broad applicability, for example defining the organizational placement of the application, application classification, and so on.

Here are AWS CLI examples of how to create an Attribute Group:

aws servicecatalog-appregistry create-attribute-group --name "Corp_DataScience_SalesAnalytics" --description "Sales analytics applications managed by the data science teams." --attributes '{"Department":"datascience","CostCenter":"cc1","Team":"salesanalytics"}'

aws servicecatalog-appregistry create-attribute-group --name "Corp_Application_Classification_High" --description "Applications classified as high." --attributes '{"ApplicationResilience":"high","DataSecurity":"high","DataSensitivity":"high"}'

Output example:

{ "attributeGroup": { "id": "0bf2qf4xa67ghtip6xzfnwonpr", "arn": "arn:aws:servicecatalog:us-east-1:XXXXXXXXXX:/attribute-groups/0bf2qf4xa67ghtip6xzfnwonpr", "name": "Corp_Application_Classification_High", "description": "Applications classified as high.", "creationTime": "2020-11-13T15:12:29.958404+00:00", "lastUpdateTime": "2020-11-13T15:12:29.958404+00:00", "tags": {} } } 

Step two

The development team sets up attribute groups.

This is done to track team- and application-specific information necessary for downstream reporting and management. The applicability of these groups should be restricted to the given application.

Here are AWS CLI examples of how to create an Attribute Groups:

aws servicecatalog-appregistry create-attribute-group --name "Commerce_AnalyticsEngine" --description "Real-time sales analytics applications managed by the data science teams." --attributes '{"Team":"developers","app-type":"processing","SLA":"1h","Runtime":"Python-3.6","Support":{"Phone":"XXX-XXX-XXXX","Email":"support@app.com"},"Compliance":["SOC-1","ISO-27018"]}}'

aws servicecatalog-appregistry create-attribute-group --name "Commerce_Payments" --description "24X7 real-time payments processing." --attributes '{"Team":"payments","app-type":"processing","SLA":"0.1h","Runtime":"Java-12","Support":{"Phone":"XXX-XXX-XXXX","Email":"support@app.com"},"Compliance":["SOC-1","PCI"]}}'

Notice that attributes can have a nested structure or include lists.

Output example:

{ "attributeGroup": { "id": "0bwdgnibevsc5clgtm7hehuljh", "arn": "arn:aws:servicecatalog:us-east-1: XXXXXXXXXX:/attribute-groups/0bwdgnibevsc5clgtm7hehuljh", "name": "Commerce_Payments", "description": "24X7 real-time payments processing.", "creationTime": "2020-11-13T15:20:19.248896+00:00", "lastUpdateTime": "2020-11-13T15:20:19.248896+00:00", "tags": {} } } 

Step three

The development team creates applications in the AppRegistry.

Here are AWS CLI examples of how to create an Application:

aws servicecatalog-appregistry create-application --name "CC_Recommendation_App" --description "Real-Time Recommendation Engine Application." --tags Name=RecommendationEngine,ApplicationId=AppXYZ,ProductLine=PR10,BusinessUnit=BU1

aws servicecatalog-appregistry create-application --name "CC_Payments_App" --description "Real-time payments service for processing customer orders." --tags Name=ProcessingEngine,ApplicationId=AppXYZ,ProductLine=PR11,BusinessUnit=BU1

Output example:

{ "application": { "id": "021n7zom01q573k7pfv404oj19", "arn": "arn:aws:servicecatalog:us-east-1: XXXXXXXXXX:/applications/021n7zom01q573k7pfv404oj19", "name": "CC_Payments_App", "description": "Real-time payments service for processing customer orders.", "creationTime": "2020-11-13T15:31:05.117977+00:00", "lastUpdateTime": "2020-11-13T15:31:05.117977+00:00", "tags": { "ApplicationId": "AppXYZ", "BusinessUnit": "BU1", "Name": "ProcessingEngine", "ProductLine": "PR11" } } } 

Step four

The development team associates attribute groups to their applications.

Here is an AWS CLI example of how to associate applications with attribute groups:

aws servicecatalog-appregistry associate-attribute-group --application "CC_Recommendation_App" --attribute-group "Commerce_AnalyticsEngine"

aws servicecatalog-appregistry associate-attribute-group --application "CC_Payments_App" --attribute-group "Commerce_Payments"

aws servicecatalog-appregistry associate-attribute-group --application "CC_Recommendation_App" --attribute-group "Corp_DataScience_SalesAnalytics"

aws servicecatalog-appregistry associate-attribute-group --application "CC_Payments_App" --attribute-group "Corp_Application_Classification_High"

Output example:

{ "applicationArn": "arn:aws:servicecatalog:us-east-1: XXXXXXXXXX:/applications/021n7zom01q573k7pfv404oj19", "attributeGroupArn": "arn:aws:servicecatalog:us-east-1: XXXXXXXXXX:/attribute-groups/0bf2qf4xa67ghtip6xzfnwonpr" } 

Step five

The development team associates existing AWS CloudFormation stacks with their applications.

Here is an AWS CLI example of how to associate existing an AWS CloudFormation stack with an application:

aws servicecatalog-appregistry associate-resource --application CC_Recommendation_App --resource-type CFN_STACK --resource cc_recomendation_app-CloudFormation-stack-name

aws servicecatalog-appregistry associate-resource --application CC_Payments_App --resource-type CFN_STACK --resource cc_payment_app_cfn_stack

Output example:

{ "applicationArn": "arn:aws:servicecatalog:us-east-1: XXXXXXXXXX:/applications/021n7zom01q573k7pfv404oj19", "resourceArn": "arn:aws:cloudformation:us-east-1: XXXXXXXXXX:stack/cc_payment_app_cfn_stack/9b137f20-e081-11e9-b842-12069a319f62" } 

Query AppRegistry

In this section, I’ll show you how to query AppRegistry to list applications that met specific criteria.

First, “Show me my applications so that I know what I have in the registry.”

aws servicecatalog-appregistry list-applications

Output example:

{ "applications": [ { "id": "021n7zom01q573k7pfv404oj19", "arn": "arn:aws:servicecatalog:us-east-1: XXXXXXXXXX:/applications/021n7zom01q573k7pfv404oj19", "name": "CC_Payments_App", "description": "Real-time payments service for processing customer orders.", "creationTime": "2020-11-13T15:31:05.117000+00:00", "lastUpdateTime": "2020-11-13T15:31:05.117000+00:00" }, { "id": "02rsoggaiagoa64bo4pd41yvkc", "arn": "arn:aws:servicecatalog:us-east-1: XXXXXXXXXX:/applications/02rsoggaiagoa64bo4pd41yvkc", "name": "CC_Recommendation_App", "description": "Real-Time Recommendation Engine Application.", "creationTime": "2020-11-13T15:25:39.683000+00:00", "lastUpdateTime": "2020-11-13T15:25:39.683000+00:00" } ] } 

Secondly, you can “Describe my applications and their attributes so that I can understand them.”

#!/bin/bash
listApps=$(aws servicecatalog-appregistry list-applications --query 'applications[*].[name]' --output text)
for app in ${listApps[*]}
do
    attrGroup=$(aws servicecatalog-appregistry list-associated-attribute-groups --application $app --max-results 1 --query 'attributeGroups[0]' --output text)
    printf "Attributes for $app\n"
    aws servicecatalog-appregistry get-attribute-group --attribute-group $attrGroup --query 'attributes'
done

Output example:

Attributes for CC_Payments_App
"{\"ApplicationResilience\":\"high\",\"DataSecurity\":\"high\",\"DataSensitivity\":\"high\"}"
Attributes for CC_Recommendation_App
"{\"Department\":\"datascience\",\"CostCenter\":\"cc1\",\"Team\":\"salesanalytics\"}"

“Show me the stacks and resources in my application.”

aws servicecatalog-appregistry list-associated-resources --application CC_Recommendation_App --query 'resources[*].[name]' --output text

Output example:

cc_recomendation_app-CloudFormation-stack-name

Lastly, you can search for an application associated with group.

#!/bin/bash
searchGroupName=Corp_Application_Classification_High
listApps=$(aws servicecatalog-appregistry list-applications --query 'applications[*].[name]' --output text)
for app in ${listApps[*]}
do
    attrGroup=$(aws servicecatalog-appregistry list-associated-attribute-groups --application $app --max-results 1 --query 'attributeGroups[0]' --output text)
    groupName=$(aws servicecatalog-appregistry get-attribute-group --attribute-group $attrGroup --query 'name' --output text)
    if [ $groupName = $searchGroupName ]; then printf "$app\n"; fi
done

Output example:

CC_Payments_App

Conclusion

In this blog post I introduced a new AWS Service Catalog feature called AppRegistry.  This feature allows organizations to build an inventory of applications and resources provisioned with AWS.  We have shown many examples where applications can be grouped with multiple attributes.  This allows many stakeholder teams to identify applications based on criteria specific to their use.  It also allows them to identify AWS CloudFormation stacks used together to create resources.

AWS What’s Next Livestream

If you would like to see a live demo of how AWS Service Catalog AppRegistry works, be sure to tune into
the AWS What’s Next Livestream on December 16th from 2:00 PM – 2:30 PM (PST).

Remek is a Senior Cloud Infrastructure Architect with Amazon Web Services Professional Services. He works with AWS financial enterprise customers providing technical guidance and assistance for Infrastructure, Security, DevOps, and Big Data to help them make the best use of AWS services. Outside of work, he enjoys spending time actively, and pursuing his passion – astronomy.