How SGK reduced IT operating costs by 83% with NoOps and serverless microservices on AWS

SGK Inc. (SGK) provides packaging, ecommerce solutions, and brand experiences for more than 1,500 clients around the world. SGK’s products and services provide near-real-time asset management, workflow integrations, and analytics for brand strategy and omnichannel marketing as well as asset design and production. The analytics in particular drive supply chain and product management decisions. Overall, SGK manages more than 1.6 million consumer packaged goods (CPG) and retail customer assets and supports 8,500 stores across North America.

As SGK’s requirements rapidly evolve to keep pace with the shifting retail landscape, the company’s solutions need to be flexible and scalable. With that in mind, SGK develops agile, modular applications to support client brand, packaging, and user-experience go-to-market strategies. Its clients can focus on brand creativity without making large up-front investments. Whether SGK develops technology as a standalone software-as-a-service (SaaS) application with its own pricing model or augments an existing solution, rapid application development is key to SGK’s revenue.

Why SGK went serverless

SGK started building applications on Amazon Web Services (AWS) in 2013. As demand for new applications increased, SGK embraced serverless services from AWS to provide rapid product development for its clients and to control costs. These eight key factors led SGK to shift from server-centric to serverless designs:

• Rapid deployment—Serverless is ideal for continuous integration and continuous delivery (CI/CD) pipelines because developers can quickly deploy new product features and fixes to production services in minutes, usually without application downtime. SGK can focus on coding business value rather than dealing with server and network configuration issues and upgrades.
• Microservices-driven architecture—SGK saw an opportunity to improve productivity by breaking down large monolithic application code into smaller separate components and services. This made it easier for SGK to test, operate, and maintain large-scale applications and improve workflows. Serverless services in microservices architecture was the next natural step for SGK. AWS Lambda, a serverless compute service, has functions that run small segments of the code in response to events, making them an excellent choice for microservices-driven architecture. This includes autonomous elastic environments that facilitate independent scaling and fault isolation.
• Near-real-time scalability—Serverless architectures can be provisioned rapidly and can scale on demand, without developer intervention, in response to sudden spikes or steady use growth.
• DevOps/NoOps—Underlying service updates and maintenance is performed by AWS, paving the way for NoOps, which means SGK can focus on serving clients instead of managing infrastructure.
• Pay-for-value—There’s no up-front cost. You pay for only what you use and only when you use it; there is no cost during idle time with serverless architectures.
• Lower total cost of ownership—In 2021, SGK collaborated with the AWS team on a study that examined savings over a 3-year period when using an AWS serverless architecture. This study covered SGK’s larger portfolio of 35 products and services and found that SGK saved 83 percent in compute, storage, network, and support costs.
• Challenges with on-premises data centers—SGK couldn’t create a rapid development and test environment in a traditional data center that used the latest technologies and agile development processes. The company also faced these two major challenges:
  • CapEx vs. OpEx—Because of the significant investment cost to procure and set up new infrastructure, SGK often had to choose between developing new services or covering data center costs.
  • Lines of control—SGK development teams often had to compete for data center space, which led to product development delays as well as lost opportunities and revenue.

What SGK has accomplished using AWS

Over the past 5 years, SGK created a process to quickly deploy minimum viable products (MVPs) using a microservices architecture so that CPG and retail clients could develop and fine-tune marketing content for new technology solutions without making expensive infrastructure investments. So far, the company has created more than 35 products and services that span every aspect of the content creation environment.

Although SGK is primarily a marketing agency, by shifting to an AWS cloud-based, serverless architecture, the company removed infrastructure management dependencies—undifferentiated heavy lifting—so it could quickly develop software solutions for its clients that could easily scale to reach global audiences. SGK has reduced IT infrastructure costs by 83 percent and deployment times by 96 percent.

SGK serverless application architectures overview

Let’s dive into the architectures of some of the applications SGK developed using AWS managed and serverless services.

Dot-authentication—SGK built an in-house single sign-on (SSO) service that provides login for multiple internal applications based on a user’s permissions. Dot-authentication uses Amazon Cognito—a simple and secure user sign-up, sign-in, and access control service.

All the applications run on the Amazon Elastic Container Service (Amazon ECS), a fully managed container orchestration service, on an AWS Fargate, a serverless compute engine, cluster. Based on a user’s permissions, dot-authentication grants access to the user for only the authorized applications.

When a user initiates the session, the domain name is resolved by Amazon Route 53, a highly available and scalable cloud domain name system (DNS) web service, and the user request is routed to Amazon CloudFront, Amazon’s content delivery network (CDN) solution. Amazon CloudFront serves the web user interface (UI) content from a bucket in Amazon Simple Storage Service (Amazon S3), an object storage service built to retrieve data from virtually anywhere. The Amazon S3 bucket hosts the static web interface. On the login page, the user provides credentials and logs in with Amazon Cognito to complete authentication. The Amazon Cognito user pool authenticates the API requests and assigns a valid JWT session token. AWS Lambda routes API requests to Amazon DynamoDB, a flexible NoSQL database service, which maps authenticated users to their permissions. Amazon DynamoDB is an excellent choice for this architecture because it is a key-value database that delivers single-digit millisecond performance at virtually any scale and is serverless. Based on users’ authorization, they are directed back to the UI, which dynamically fetches applications from AWS Fargate. The user can see all the available applications and can choose the ones that they want to use. The applications run on a serverless microservices architecture on AWS Fargate, which serves as serverless compute for containers, eliminating the need to configure and manage control panels, nodes, and instances.

Asset management application—Using AWS serverless architectures, SGK has facilitated a multitenant brand inventory personalization supply chain solution delivery to its customers in a quick-to-market fashion. SGK maintains a near-real-time inventory of all assets present in over 20,000 store locations for marketing purposes. This application is accessed by more than 3,000 users globally and stores nearly four million items based on the store location and menu.

As in dot-authentication, the UI is decoupled from the API, establishing loosely coupled components that make updates much easier. The user gets to the UI, hosted in an Amazon S3 bucket, by way of Amazon CloudFront. The content is dynamically pulled from AWS Fargate. AWS Fargate is SGK’s solution of choice in AWS because it is easily scalable—horizontally and vertically—and compared to physical on-premises infrastructure, it is significantly more cost effective. SGK uses Amazon DynamoDB for most of its application configuration settings instead of capturing them in the YAML file as a part of the build. The configuration can be updated while the application is running, using dynamic reporting. All the parts of the query are maintained in Amazon DynamoDB, and new fields in the report can be added. So, dynamic queries from AWS Fargate will include the newly added fields in the search. The images for the application are stored in an Amazon S3 bucket, exceeding 65,000, without a need to maintain dedicated file servers that can have compute constraints as data volume increases. The application is also using an instance from Amazon Aurora, a MySQL- and PostgreSQL-compatible relational database, so AWS is responsible for the maintenance, backups, and replications. The solution also uses various AWS Lambda functions to insert records in an Excel format to Aurora. The application is very stable and does not need maintenance or operational support.

Recalculating business rules—As input variables change, the application recalculates 270 business rules for every retail store, creating over 750,000 updates every day.

To avoid impacts on application performance, the solution uses AWS Lambda functions. These functions are initiated by scheduled events on Amazon EventBridge, a serverless event bus, on a fixed daily schedule to extract all values that change from an Aurora relational database and import them into a data warehouse in Amazon Redshift, which analyzes structured and semistructured data. Amazon EventBridge scheduled events initiate nine other AWS Lambda functions that apply the business rules and recalculate the data. After the calculation is complete, Amazon EventBridge scheduled events initiate two more lambdas, which extract data from Amazon Redshift. The lambdas transform into messages in Amazon Simple Queue Service (Amazon SQS), a fully managed message queuing service, and place the values in the queue. In the last step, Amazon EventBridge scheduled events initiate another lambda multiple times, which imports the calculated values from Amazon SQS to Aurora. Aurora is updated with the new values. The business rule calculations run overnight, so new values are updated by the morning.

Serverless code deployment—This deployment automation creates artifacts in Bitbucket without starting and stopping any instances. The application is divided, based on the decoupled architecture in the UI and API. In this case, these two parts are created in completely different programing languages. The project has two hot branches: PROD and staging, which can initiate deployment upon merging with a feature branch.

When a developer pushes or merges code into the hot branch, the script creates the UI part and deploys it into the UI Amazon S3 bucket. Also, if there is a change in the API being created, tagging and pushing a new Docker image into Amazon Elastic Container Registry (Amazon ECR), a fully managed container registry, and restarting the AWS Fargate services takes the new image by default. The entire process takes approximately 4 minutes, which is very efficient compared to the deployments in a typical on-premises environment that can take several hours.

NoOps & serverless—the future of application development

We mentioned NoOps previously, and it quite literally means a “no operations” environment that has a fully automated underlying infrastructure—through applied technologies—so there’s no need for an in-house dedicated team to manage the software. It’s easier to build new applications with this approach than to rebuild existing ones on premises. But it requires a cultural shift in the organization and a new mindset that NoOps is possible. Also, if you choose to buy third-party software instead of building a solution, be sure to ask yourself how much you can automate your processes. The release of AWS Fargate in late 2017 was a decisive moment to endorse NoOps as a great option for a fully scalable API. SGK launched its first application using a NoOps approach at the beginning of 2018, and since then, all its solutions are NoOps and serverless.

Modernize your application development process

IT organizations need to streamline software development for rapid delivery, without vulnerabilities and defects, while reducing costs—to support evolving business needs. This can be quite a challenge for enterprise organizations. However, by using managed services from AWS, you can gain efficiencies, accelerate delivery timelines, and reduce costs by reusing certified infrastructure and recertifying only the differentiating components.

Using a platform-as-a-service (PaaS) development model on AWS, SGK built a multitenant software delivery pipeline to rapidly build, test, scan, and deploy applications on a global scale while adhering to IT security and compliance requirements.

If you’re ready to modernize your software development processes, AWS is here to help you overcome this and other critical challenges to improve IT operations. Contact your AWS account team to get started today.