Use AWS Service Catalog to build a custom catalog of products from AWS Marketplace

Some AWS customers I work with have dedicated analytics/AI research teams who use AWS infrastructure to run their own code or use products from the AWS Marketplace to perform computations. Typically, the AI research team deploys multiple solutions. They bundle their own home-grown solutions into an AWS CloudFormation template along with some AWS Marketplace products, such as the AWS Deep Learning AMI. You want to make it easy for your AI research team to deploy approved AI-based software packages. You can enable your AI research team to innovate while ensuring that governance is built into the process. You (or your procurement team) can procure the software that the research team needs from AWS Marketplace. You can ensure that data scientists can use a wizard to launch the AI software that you distribute to them.

You can build a custom catalog of products from AWS Marketplace for different combinations of stakeholders, such as all or some users from a business unit, or a specific set of users attached to a federated role, or even a specific user. Use the AWS Service Catalog to create and manage custom catalogs of products that you approve for use in your organization. In addition to AWS Marketplace products, you can add your own AWS CloudFormation products to your custom catalog.

Walkthrough: Using AWS Service Catalog to build your custom catalog of AMIs from AWS Marketplace

In this blog post, I’ll show you how you can build a custom catalog of Amazon Machine Image (AMI)-based products procured from AWS Marketplace, using AWS Service Catalog.

First I’ll walk you through a use case to show you how to build a custom catalog of software products. Let’s say that you want your AI Research team to use the free AWS Deep learning Marketplace AMI instead of any other paid software. You recently noticed a few unauthorized AWS Marketplace software charges (due to unauthorized software launches by your AI Research team) on your AWS bill. So you want to build a catalog specifically for your AI research team to control what they can launch from AWS Marketplace.

Before I get into how it works, let’s first review a few key AWS Service Catalog concepts:

A product is a blueprint for building your AWS resources that you want to make available for deployment on AWS along with the configuration information. You create a product by importing a CloudFormation template, or, in case of AWS Marketplace-based products, by copying the product to the AWS Service Catalog. A product can belong to multiple portfolios. To know more about the product, see the AWS Service Catalog documentation.
A portfolio is a collection of products, together with the configuration information. You can use portfolios to manage the user access to specific products. You can grant portfolio access at an IAM user, IAM group, and IAM role level. To know more about the portfolio, see the AWS Service Catalog documentation.
A provisioned product is a CloudFormation stack that is, the AWS resources that are created. When an end user launches a product, the AWS Service Catalog provisions the product in form of a CloudFormation stack. To know more about the provisioned product, see the AWS Service Catalog documentation.
Constraints control the way users can deploy a product. With launch constraints, you can specify a role that the AWS Service Catalog can assume to launch a product from the portfolio. To know more about constraints, see the AWS Service Catalog documentation.

The following solution diagram illustrates how you can build a catalog of AWS Marketplace products, using AWS Service Catalog in a single-account-based organization.

If your organization has multiple AWS accounts to manage, it is a best practice to set up a master/child account relationship to distribute the procured software in your organization. The master account is the central account in which you create (or copy) products, add them to portfolios, and from which you finally share portfolios with the child accounts. In a child account, you import portfolios from the master account and launch software. To learn more about the hub-and-spoke model, see AWS Service Catalog Hub and Spoke Model. The following solution diagram illustrates how you can build a catalog of AWS Marketplace products, using AWS Service Catalog in a multi-account hub-and-spoke-model-based solution.

Note

In each step, I’ve included specific instructions to follow if you want to distribute the AMI-based products in either the single- or multi-account model.

Before you start building your custom catalog, terminate the unauthorized software.

Step 1: Revoke AWS Marketplace permissions and grant AWS Service Catalog permissions to your catalog users

To remove the AWS Marketplace subscription and AWS Marketplace-based AMI launch access from your IAM users, replace Update_Your_Account_ID_here with your account ID in the following example policy document and attach it with your team’s (In this case, the AI research team’s) IAM group or federated role. By associating this policy with your IAM users, you are denying them Run Instance permission for any AMI that is not owned by your account or Amazon (the Deep Learning Base AMI is owned by 898082745236).
```
{
	"Version": "2012-10-17",
	"Statement": [{
			"Sid": "DenyAWSMarketplaceWriteAccess",
			"Effect": "Deny",
			"Action": ["aws-marketplace:*"],
			"Resource": "*"
		},
		{
			"Sid": "DenyMarketplaceAMIAccess",
			"Effect": "Deny",
			"Action": ["ec2:RunInstances"],
			"Resource": "arn:aws:ec2:*::image/ami-*",
			"Condition": {
				"ForAnyValue:StringNotEquals": {
					"ec2:Owner": [
						"amazon",
						"self",
						"898082745236",
						"Update_Your_Account_ID_here"
					]
				}
			}
		}
	]
}
```
To know more about how to create an IAM policy and attach it to a principal, see the documentation for:
- Creating an IAM Policy.
- Attaching IAM Policies.
Note

If you have created private AMIs from AWS Marketplace-based AMIs, you need to also attach a private paid-AMI-specific Deny-Effect policy for RunInstances-Action for your users. Here is an example of such policy that you can use after replacing Update_Your_AMI_ID_Here with your private paid AMI ID.
```
{
	"Version": "2012-10-17",
	"Statement": [{
		"Sid": "DenyPrivateAMIAccess",
		"Effect": "Deny",
		"Action": ["ec2:RunInstances"],
		"Resource": "arn:aws:ec2:*::image/Update_Your_AMI_ID_Here"
	}]
}
```
Next, you need to allow the AI research team to launch the AWS Service Catalog product. To do so, attach the Amazon managed AWSServiceCatalogEndUserFullAccess policy with AI Research team. Next, your procurement team needs to subscribe to the AWS Marketplace AMI-based product that you want to distribute.

Step 2: Subscribe to AWS Marketplace AMI product you want to distribute

Note: If you want to distribute the product using the hub-and-spoke model in your multi-account AWS setup, you need to do this step from both of the accounts—the child as well as the master account.

In this case, your procurement team needs to subscribe to the AWS Deep Learning AMI from the AWS Marketplace console. To subscribe to the product:

Sign-in to the AWS Management Console and then open AWS Marketplace console at https://thinkwithwp.com/marketplace/.
In the Search text box, type the product name (In this case, Deep Learning Base AMI (Amazon Linux)), and press Enter.
Choose the appropriate result from the search results.
Choose Continue to Subscribe.
The listing detail page will display three tabs with headers as 1-click launch, Manual Launch, and Service Catalog. Choose the Service Catalog tab.
Note
If the AWS Marketplace AMI product you want to distribute does not has the Service Catalog option in the AWS Marketplace listing detail page of the product, you can create a CloudFormation template and manually upload it as an AWS Service Catalog product. In this case, you need to skip Step 3 and continue from Step 4. For more information, see Managing and Adding AWS Marketplace Products Manually.
Since the Deep Learning Base AMI is sold by Amazon Web Services, it does not ask you to accept the software terms again. If your procurement team is procuring AMI based software sold by a vendor that is not Amazon Web Services, they need to choose Accept Software Terms. (They need to check with your organization’s legal department before choosing this button.)

Next, you need to make the software product available for distribution in the AWS Service Catalog.

Step 3: Copy the software product to the AWS Service Catalog

Note: If you want to distribute the product using the hub-and-spoke model in your multi-account AWS setup, you need to do this step from the master account.

To make the software product available in the AWS Service Catalog, follow these steps:

Browse to the product listing page by following the instructions in Step 2.1 to Step 2.5.
Select an appropriate AWS Region and a version in the Copy to Service Catalog panel and then choose Copy to Service Catalog.
This will copy the Deep Learning Base AMI (Amazon Linux) product to the AWS Service Catalog in the selected Region.

Note

It is a best practice to standardize the AWS Marketplace-based AMI before you distribute the AMI to your business units. The standardization process involves hardening, Inspection, and validation of an AMI before distribution. To know more, see Building a Secure, Approved AMI Factory Process Using Amazon EC2 Systems Manager (SSM), AWS Marketplace, and AWS Service Catalog. (Note that subsequent to this whitepaper the service name changed from Amazon EC2 Systems Manager to AWS Systems Manager.)

Step 4: Create a portfolio to distribute the product to your IAM users

Note: If you want to distribute the product using the hub-and-spoke model in your multi-account AWS setup, you need to do this step from the master account.

Next, you need to create an AWS Service Catalog portfolio with the name as AI Research Team Portfolio. In this walkthrough, we will be naming our portfolio AI Research Team Portfolio. After you have created a portfolio, add Deep Learning Base AMI (Amazon Linux) to the AI Research Team Portfolio that you created. (The product should become available in your catalog, after your procurement team chooses Copy to Service Catalog in Step 3.) To learn more about adding a product to a portfolio, see the documentation at Adding Products to Portfolios.

If you have other CloudFormation templates to which you want to give the AI Research team access, you can do so by uploading your CloudFormation template as a product to the AI Research Team Portfolio. After your portfolio is ready, you can distribute it to your end users.

Step 5: Distribute the catalog

Next, you need to grant the AI research team access to the portfolio. If you have a single AWS Account that the entire organization uses, you can grant the AI research team’s users access to the portfolio that you created in Step 4. Ensure that you have also associated the role/group selected, with the AWSServiceCatalogEndUserFullAccess policy, as suggested in Step 1. To know more about how to grant users access to the portfolio, see the documentation at Grant End Users Access to the Portfolio.

If you want to distribute the product using the hub-and-spoke model, follow these steps:

From the master account, share the portfolio created with the child account. For more information on portfolio sharing, see Portfolio Sharing. Next, you need to sign-in to the child account and then import the portfolio shared by the master.
Next, create a local portfolio in the child account and Add the product imported from the master account’s AWS Service Catalog to the portfolio created in the preceding step.
Grant the AI research team access to the local portfolio from the child account.

You can also associate tags (for a cost center, a business unit, etc.), using the Tag Options feature, with a local portfolio to ensure that resources deployed from the portfolio have tags automatically associated with them. To know more about the Tag Options feature, see the documentation.

Step 6: Create a launch constraint

Next, you create an AWS Service Catalog specific IAM role that will have permissions to create a CloudFormation stack and to launch an EC2 instance. You need to create the role in the account from which the AI research team would be launching an AMI instance. To create the role for the launch constraint, use the following policy document.

{
	"Version": "2012-10-17",
	"Statement": [{
		"Sid": "LaunchSCMPAMI",
		"Effect": "Allow",
		"Action": [
			"ec2:AuthorizeSecurityGroupEgress",
			"ec2:AuthorizeSecurityGroupIngress",
			"ec2:DeleteSecurityGroup",
			"ec2:TerminateInstances",
			"ec2:CreateSecurityGroup",
			"ec2:RunInstances",
			"ec2:createTags",
			"catalog-user:*",
			"cloudformation:CreateStack",
			"cloudformation:DeleteStack",
			"cloudformation:DescribeStackEvents",
			"cloudformation:DescribeStacks",
			"cloudformation:GetTemplateSummary",
			"cloudformation:SetStackPolicy",
			"cloudformation:ValidateTemplate",
			"cloudformation:UpdateStack",
			"ec2:describe*",
			"ec2:get*",
			"s3:GetObject",
			"sns:*"
		],
		"Resource": [
			"*"
		]
	}]
}

After you have created the role, you need to associate the role with the Deep Learning Base AMI (Amazon Linux) product as a launch constraint inside AI Research Team Portfolio. To know more about how to create a role and associate it as a launch constraint, see the documentation at Add a Launch Constraint to Assign an IAM Role.
You will notice that this IAM-based service role has permission to launch any AMI if your account has subscribed to it. AWS Service Catalog assumes this role for launching the product that you specify, so you don’t need to provide these permissions to your IAM user. You might need to review and add additional permissions to the role you created if you are using the launch role to provision a more complex CloudFormation template.

Step 7: Test the solution

Finally, here is how you can test whether the AI research team can provision the Deep Learning AMI through the AWS Service Catalog:

Sign-in to the AWS Management Console using AI research team’s IAM user and then open https://console.thinkwithwp.com/servicecatalog/.
In the Products section of the console, choose the Deep Learning Base AMI (Amazon Linux).
Choose the Launch product to start the wizard for configuring your product.
On the Product version page, for Name, type Deep-Learning-AMI, choose a version.
Choose Next.
Provide appropriate parameters on the parameters page.
On the TagOptions page, choose Next.
On the Notifications page, choose Next.
On the Review page, choose Launch.
Note
Before you choose Launch, ensure that the Launch role has the necessary permissions.

The AI research team should be able to launch the Deep Learning Base AMI through the AWS Service Catalog. If the AI Research team tries to launch an AWS Marketplace instance of the Deep Learning Base AMI using the EC2 console, the instance will not launch. Since the Deep Learning Base AMI is owned by Amazon, the end user will be able to launch an instance of the AMI through the AWS Command Line Interface (CLI) or AWS CloudFormation if they have necessary permissions to run an EC2 instance. However, the users will not be able to launch any paid AMI that has owner attribute set as aws-marketplace, through the CLI, CloudFormation, or through Amazon EC2 console. The only way to launch the instance would be through the AWS Service Catalog. This approach helps you tighten the security around your paid purchases.

AWS Service Catalog recently released a feature called AutoTags. AutoTags are tags that identify the portfolio, product, and user that launched a product, and are automatically applied by the AWS Service Catalog to provisioned resources. AutoTags do not count towards your 50 Tags limit. AutoTags enable you to do following:

Identify which principal (IAM user, federated user, or assumed-role user) launched the Service Catalog product. Since there is a one-to-one mapping between an AWS Marketplace product and a service catalog product, you can easily extract the information about the principal that launched the marketplace product.
Which portfolio was the product launched from – This gives user an insight into the context behind the launch. If portfolios are created based on an Environment/Sprint and the paid-AMI product was distributed through multiple portfolios, the Portfolio ID Autotag would let user identify the context in which it was launched.
Use Resource Groups to discover all AWS Service Catalog-based Marketplace AMI products launched by a specific principal.

Conclusion

AWS Service Catalog allows you to build custom catalogs to suit different business needs. It makes it easier to manage access of the paid (or custom) software products. This blog post demonstrates how you can build a catalog of products for organizations that are single-account-based as well as multi-account-based. With AWS Service Catalog, you can tighten the security around your paid AWS Marketplace AMI-based purchases while enabling their distribution in an organized manner.

If you have questions about implementing the solution described in this post, you can start a new thread on the AWS Service Catalog Forum or contact AWS Support.

About the Author

Kanchan Waikar is an AWS Marketplace Solutions Architect at Amazon Web Services. She enjoys helping customers build architectures using AWS, AWS Marketplace products, and AWS Service Catalog.

AWS Cloud Operations Blog