How Internal Developer Platforms Built with AWS Proton Help Achieve DevOps Best Practices

By Rizwan Patel, CTO – Redapt
By Tatiana Cooke, Principal PMT – AWS
By Adam Keller, Sr. Developer Advocate – AWS
By Srini Bala, Sr. Container Specialist – AWS

Redapt

Organizations may have audited infrastructure for compliance in the past, but now teams want to “shift left” and make it easier for application developers to adhere to a recommended path.

In response to this, platform engineering is an area of growing interest for many organizations as they seek development models that make it easy for application developers to proactively implement best practices.

The platform engineering function also helps improve the developer experience by providing the right set of tools, technologies, and templates in a self-service portal. The introduction of a self-service portal reduces operational friction and allows developers to discover, build, and deliver value within guardrails configured by their platform team.

Redapt is an AWS Advanced Tier Services Partner and AWS Marketplace Seller that helps customers create their internal development platforms (IDPs), which empower application developers while providing well-defined, consistent, and secure environments that are cost effective, shared, and monitored.

Redapt is excited to work with customers on IDPs using AWS Proton, an AWS managed service that helps implement IDP features for ongoing Day 2 operations by creating the right integrations across platform and development teams through templates and automation.

In this post, we show how IDPs can be built using AWS Proton to help organizations achieve DevOps best practices. We also show an example of how IDPs deliver on the value of DevOps best practices by creating visibility across a mix of CI/CD tools and workflows.

IDPs and DevOps Maturity

Creating IDPs using tools like AWS Proton can help increase the DevOps maturity of an organization by helping align people, processes, and technology pillars to improve metrics.

This can include reducing the lead time for deployment changes, increasing deployment frequency, reducing mean time to resolution of issues, and addressing other key operational metrics. This DevOps maturity translates into benefits for customers’ business objectives and their end customers.

As an illustration, implementing a solution like AWS Proton coupled with Redapt’s solution catalogs, assets, and blueprints helps elevate application lifecycle maturity starting from basic visibility to higher levels (by providing governance, guardrails, and automation). The figure below shows the progression through the different levels of DevOps maturity.

Figure 1 – Steps of the DevOps maturity model.

Let’s see how the different levels get incorporated into application (DevOps) lifecycle.

Visibility

• The assessment component of the Redapt Technology Evolution Playbook (RTEP) is a customized take on the maturity model and helps create baselines. Use RTEP to gain visibility on security, compliance, operational levels, multi-cloud cost, usage, performance, configuration, and secure data in different environments.
• Analyze metrics such as deployment frequency, lead time for change, time to restore service, change failure rate, and operational performance (reliability and resiliency) for any business unit, team, department, application, or project for better business context.

Optimization

• Identify ways to optimize current standards for enhanced security, compliance, and operational efficiency across teams and clouds.
• Incentivize teams to optimize by increasing awareness and accountability related to cost and configuration efficiencies.
• Document best practices to educate others and maintain optimization in the future.

Governance, Automation, and Enforcement

• Improved showback/chargeback accuracy.
• Automated policy compliance and non-compliance alerts/reports.
• Automated policy enforcement.
• Automated policy drift detection and alerts on policy drift.
• Automated remediation.

Scaffolding

• Catalog of pre-approved patterns with the ability to create a new application or component from a template.

Guidance

• Dashboards and direct links for each important step or tool that developers use. These are used to synthesize information about version control, documentation, resource status (build/deploy/operating), monitoring, support ticketing, and others.

Automation

• Optimize the way a team operates by automating processes that may be manual today, including:
  • Create standard templates for resources (infrastructure) using infrastructure as code (IaC) for all teams.
  • Automatically checking new apps in to source control.
  • Source control changes are built to a container and posted to your registry by default.
• Depending on the release strategy, a new release could be automatically deployed or go into a workflow deployment that includes a manual approval.
• Implement and automate a stack of tools in the following areas: continuous integration, version controls, continuous monitoring and testing, containerization, orchestration and configuration management, and deployment for end-to-end DevOps benefits.
• Policy as code is used for all resources/assets for all teams and is regularly reviewed and audited.
• Automation drives higher release velocity, as well as repeatability and quality.

IDP in Action: Consolidating Toolsets for Improved Observability

As an illustration of how an internal development platform powered by AWS Proton can deliver on the promise of DevOps benefits, Redapt built an example application and took on the challenge of building a sophisticated pipeline across tools with good observability features.

Using AWS Proton, Redapt has been able to easily extend the pipeline by adding components such as observability, security, and blueprints.

To demonstrate the way different tools can be brought together, Redapt identified existing observability services from AWS transcending the metrics, events, logs, traces (MELT) paradigm, such as Amazon DevOps Guru, AWS X-Ray, Amazon CloudWatch, and Amazon CloudTrail.

Redapt created the following reference architecture, which provides real-time metrics around AWS CodeBuild, AWS CodeDeploy, and AWS CodePipeline, as well as reactive and prescriptive guidance like Insight Opened, Insight Cleared, Recommendation Created, Anomaly Associated created (from Amazon DevOps Guru).

This was done via Amazon QuickSight dashboards to help improve delivery quality and improving business metrics by reducing MTTX— Detect, Diagnose, Remediate, Recover.

AWS Proton helps create templates with all of the observability tools needed across application pipelines. With IDPs using AWS Proton templates, observability gets standardizes across application pipelines with best practices included.

By shifting left, developers won’t need to worry about setting up and configuring observability tools for their application pipelines.

Figure 2 – IDP using AWS Proton (templates).

Redapt’s Reference Architecture

An overview of Redapt’s reference architecture and the process is shown in Figure 3 below:

• Process is kicked off by developers committing code to repository, deploying their services (including serverless and containers), and/or initiating promotion across environments.
• Observability MELT activities are captured through various services including Amazon CloudWatch, AWS X-Ray, CloudWatch ServiceLens, Amazon Managed Prometheus, Amazon Managed Grafana, and AWS Distro for OpenTelemetry.
• Activities that match predefined Amazon EventBridge event patterns and threshold trigger an alarm and/or are emitted to Amazon Kinesis.
• Amazon QuickSight reports are created to depict real-time (and historical) metrics on events that are transformed through AWS Glue and stored in Amazon Athena.
• Kibana dashboards are presented for events from sources like Amazon OpenSearch Service.
• Amazon DevOps Guru (AIOps component) metrics like New Insight Open, New Anomaly Association, Insight Severity Upgraded, New Recommendation Created, and Insight Closed are captured and presented in OpsCenter.
• Events can be transformed and sent to external Configuration Management Database (CMDB) systems (like ServiceNow).

Figure 3 – Redapt architecture and DevOps observability.

Included below from Redapt’s reference architecture are a few snippets from the Amazon QuickSight dashboard that depicts key DevOps metrics which can be compared over a timeframe to show improvement in key areas:

• Resource change volume.
• Time and number of environments spun up.
• Change failure rate metrics including outages—number and time frame, associated across builds.
• Deployment metrics across applications, services, and environments.
• Build activity metrics— grouping across services, projects, account, and environments.
• Pipeline activity metrics.
• Source control metrics—access, commits, and deployment status.
• Churn rate.
• Customer retention.
• Customer lifetime value.
• Net promoter score.

Figure 4 – Pipeline activity.

Conclusion

Redapt’s experience building out an internal developer platform (IDP) using AWS Proton helped identify an architecture with DevOps benefits that includes:

• Automated infrastructure as code provisioning and deployment of serverless and container-based applications.
• Standardized infrastructure.
• Integrated deployments with CI/CD pipeline continuously and seamlessly in an AWS account across multiple environments.
• Standardized observability integrated using AWS Proton to provide insights into DevOps process.
• Triggers based on events/anomalies, and machine learning-based actionable recommendations to mitigate them.

AWS Proton adheres closely with the product paradigm for an IDP, meeting the needs of Redapt’s customers with flexible ways to create and update templates. This allows for ongoing collaboration between developers, operations/platform, and customer stakeholders, and helps Redapt elevate the maturity level for each customer’s unique DevOps journey.

You can learn more about Redapt on AWS Marketplace.

.

.

Redapt – AWS Partner Spotlight

Redapt is an AWS Advanced Tier Services Partner that helps customers create their internal development platforms (IDPs), which empower application developers while providing well-defined, consistent, and secure environments that are cost effective, shared, and monitored.

Contact Redapt | Partner Overview | AWS Marketplace