Introduction

While Amazon Web Services (AWS) customers such as Bizzy, Invista, Zalando, and Engie have implemented data and analytics solutions on AWS in support of their SAP workloads, many more are working with AWS to see how they can gain further insights by exploring trends in data. The large amount of data generated by business transactions processed in SAP, when properly harnessed by data and analytics solutions, can enable innovative decision making in many areas, such as customer engagement, cost management, and product roadmaps. One of the first steps in this journey is choosing the right tools to integrate data and analytics solutions with your SAP workloads.

In this blog I’m going to show how to integrate SAP systems with AWS services, using the SAP Integration Suite, available on the SAP Business Technology Platform (BTP). To cover the network and connectivity aspect to various SAP solutions on AWS, please have a look at my blog “How to connect SAP solutions running on AWS with AWS accounts and services“.

We frequently get asked by our customers about how to integrate SAP systems with AWS services. SAP BTP is a common platform for SAP customers to build integrations and extension scenarios and it also fits the need to integrate with AWS services. Popular use cases include building feeds of SAP data into machine learning or analytics services in AWS. Or enabling analytics for large volumes of data by using Amazon Simple Storage Service (Amazon S3), which is the object storage service of choice for data lakes, for performant access to structured and unstructured data.

For extracting data from SAP systems, it is important to keep the application context. While data extraction on database level would lose the application context, unless additional tools are used, extractors using OData, IDocs etc. maintain data relationships and integrate on the application layer. There are multiple solutions available to integrate and extract SAP data. Using native AWS tools like AWS Lambda and AWS Glue are explained in the blog Building data lakes with SAP on AWS.

In this blog, I want to focus on SAP Integration Suite, running on the SAP Business Technology Platform (BTP), to showcase an approach using SAP tools and services to extract data to AWS, without writing code.

SAP provides the Amazon Web Services Adapter for SAP Integration Suite to connect to AWS services, without writing or maintaining code. The AWS Adapter enables data exchange between the SAP Integration Suite and AWS services, where the AWS services can act as sender or receiver. The following AWS services are supported:

Sender Adapter:

• Amazon Simple Storage Service (Amazon S3)
• Amazon Simple Queue Service (SQS)

Receiver Adapter:

• Amazon Simple Storage Service (Amazon S3)
• Amazon Simple Queue Service (SQS)
• Amazon Simple Notification Service (SNS)
• Simple Workflow Service (SWF)

Using SAP BTP and the Integration Suite offers various benefits. Most SAP customers are already using BTP services and have created extensions on top of SAP BTP. The integration is at the application level and multiple communication channels like HTTP/HTTPS, IDoc, OData, etc. are supported. For a full list please refer to SAP documentation – communication channels. In addition to the AWS service integration, integration flow functionality like message conversion, IDoc splitter, encryption, etc. are provided. With that, you can extend existing integration scenarios and applications with AWS services.

Prerequisite

1. SAP Support User (S-User)
2. SAP BTP Account – You can create a trial account to test the walkthrough described in this blog post
3. Integration Suite subscription – or see “setup BTP account” below
4. Deployment of SAP S/4HANA. The easiest way to deploy this is using AWS Launch Wizard for SAP

Solution Architecture

Example walk through: S/4HANA IDoc extraction to Amazon S3

In this example walk through, I’m going to extract an IDoc from an SAP S/4HANA System, convert it to JSON format and store it in Amazon S3.

1. Setup BTP account

Subscribe and enable SAP Integration Suite in your BTP account

Step by step documentation how to setup the Integration Suite is described in the Tutorial “Set Up SAP Integration Suite Trial”. Trial accounts are available in the AWS regions eu-central-1 and us-east-1 and you can select the region during the account setup.

2. Download AWS Adapter

You can download the AWS Adapter from the SAP Software Center by navigating to

SUPPORT PACKAGES & PATCHES –> By Alphabetical Index (A-Z) –> C –> SAP CP IS ADAPTER BASE PACK –> SAP CPIS AWS ADAPTER.

Note: SAP Software Download Center requires your SAP S-User ID.

Download and extract the file with the latest version. The zip file contains also the installation guide, which describes how to implement the adapter in the SAP Integration Suite.

3. Install AWS Adapter

Open SAP Integration Suite and open “Design, Develop, and Operate Integration Scenarios”. Click on “Design” on the Menu and create a new package.

Navigate to the “Artifacts” tab and click on Add > Integration Adapter

Select the integration adapter file (AmazonWebServices.esa) which is part of the files you’ve downloaded in the previous step.

Deploy the adapter, by clicking on the actions button and select “Deploy”.

4. Configure access to AWS services

To access AWS services, you need to store the AWS user with programmatic access to your account in the SAP Integration Suite. It’s recommended to create a new user in AWS Identity and Access Management (IAM), for accessing services through the SAP Integration Suite and define least privilege access to the required AWS resources. For this example, it is sufficient to grant access to the Amazon S3 bucket, which is used to store the extracted SAP data.

You need to store the IAM user and password as secure parameter in the Integration Suite. Select “Monitor” in the menu and navigate to “Manage Security” –> “Security Material” and create a new “Secure Parameter”.

Please note that the secure parameter is not the password or the secure access key of the IAM user, but the access key itself. You need to create an additional secure parameter for the secure access key. For example, with the name “AWS_IAM_Secret_Access_Key”.

5. Create Integration Flow

In the “Design” section of the SAP Integration Suite, select the package you’ve created before and navigate to the “Artifacts” tab. Create a new integration flow.

Once you’ve clicked on the integration flow you’ve just created, you can enter the graphical designer, where you can define your sender, receiver and the integration process.

Define the sender, by providing a name – “S4HANA” as in my example. Create a connection from the sender to “Start” and select IDoc as Message Flow in the pop-up window. In the connection tab of the IDoc communication insert “/FLIGHTBOOKING_CREATEFROMDAT01” as address and leave authorization and user role with the default values.

Just for the demo purposes, I’ve implemented a message conversion from IDoc XML to JSON in the flow.

The receiver in this example is Amazon S3. From the “End Message” to the “Receiver” you can select the AWS Adapter installed in step three. Under the connections tab of the receiver, you can define the Amazon S3 bucket in your AWS account and the access and secret access key, configured in step four.

In the processing tab you need to define the “content type” depending on the MIME type (“application/xml” or “text/plain”) For this example, use “application/xml”. The complete flow should look like this:

Once the flow is defined, you can save and deploy the configuration. Navigate in the monitoring menu to “Manage Integration Content” and wait until your flow has the status “Started”. Please copy the endpoint URL of the integration flow. You’ll need this later for the RFC configuration.

6. Configure S/4HANA System

Define a logical system in transaction BD54 for the AWS target resources. For example, “AWSS3” for the Amazon S3 bucket. You also require a logical system for the sender, which is the client 100 of the S/4HANA System in this example.

In transaction SALE execute “Maintain Distribution Model and Distribute Views” and create a new model view:

Next step, create a new BAPI for the created model and define the sender and receiver according to the logical systems defined in the previous steps. For this demo, I’ve selected the FlightBooking object which is available by default in the S/4HANA system:

To enable a secure communication, the certificates from Integration Suite are required in the S/4HANA system. Download the certificates from the Integration Suite, by clicking on the lock symbol next to the URL in the browser. Download all three available certificates and upload all of them in the transaction STRUST, under SSL client SSL Client (Anonymous), by adding them to the certificate list. After that, you can create a new RFC connection in transaction SM59 of the type HTTP. Use the endpoint URL of your Integration Suite integration flow as host and port 443. In the tab “Logon & Security” select Basic Authentication and enter your user for the SAP BTP. Scroll down and change the status of secure protocol to SSL Certificate: Anonym SSL Client (Anonymous).

Note: please consider client certificate authentication for your productive workloads.

Now, you need to create a port in transaction WE21. Create a new port for IDoc processing and select the RFC destination created earlier. Define “application/x-sap-idoc” as content type and enable SOAP protocol.

In transaction WE20 define a partner profile of the type “LS” (logical system). Select a user under post processing and create the following new outbound parameter:

Message Type: FLIGHTBOOKING_CREATEFROMDAT
Receiver port: Port created in previous step
Basic type: FLIGHTBOOKING_CREATEFROMDAT01
Select “Pass IDoc Immediately”

7. Test the integration workflow

Go to transaction WE19 and execute the test by using the message type “FLIGHTBOOKING_CREATEFROMDAT”

Double click on EDIDC and define the port and partner no. for the receiver, as the parameters created before.

Double click on E1BPSBONEW and put in some test data.

Finally start the outbound processing by clicking on “Standard Outbound Processing”. You can monitor the IDoc processing within SAP or in the monitoring section in the Integration Suite.

For a simple validation of the data in Amazon S3, you can list your configured bucket, using AWS Command Line Interface (AWS CLI) command: aws s3 ls s3://<your-S3-bucket> --summarize | sort

This is an example of an IDoc, which was successfully transformed to JSON format:

{
  "FLIGHTBOOKING_CREATEFROMDAT01": {
    "IDOC": {
      "@BEGIN": "1",
      "EDI_DC40": {
        "@SEGMENT": "1",
        "TABNAM": "EDI_DC40",
        "MANDT": "100",
        "DOCNUM": "0000000000200021",
        "DOCREL": "755",
        "STATUS": "30",
        "DIRECT": "1",
        "OUTMOD": "2",
        "IDOCTYP": "FLIGHTBOOKING_CREATEFROMDAT01",
        "MESTYP": "FLIGHTBOOKING_CREATEFROMDAT",
        "STDMES": "FLIGHT",
        "SNDPOR": "SAPS2B",
        "SNDPRT": "LS",
        "SNDPRN": "S2BCLNT100",
        "RCVPOR": "SAPBTPIS",
        "RCVPRT": "LS",
        "RCVPRN": "AWSS3",
        "CREDAT": "20210621",
        "CRETIM": "150054",
        "ARCKEY": "urn:sap.com:msgid=02D45F160CA71EEBB4D316D6C4AA2C47"
      },
      "E1SBO_CRE": {
        "@SEGMENT": "1",
        "E1BPSBONEW": {
          "@SEGMENT": "1",
          "AIRLINEID": "LH",
          "CONNECTID": "345",
          "FLIGHTDATE": "15.07.21",
          "CUSTOMERID": "R44324xxx",
          "CLASS": "1",
          "COUNTER": "34",
          "AGENCYNUM": "3562",
          "PASSNAME": "ARNE KNOELLER",
          "PASSFORM": " Mr",
          "PASSBIRTH": "06.07.1992"
        },
        "E1BPPAREX": {
          "@SEGMENT": "1"
        }
      }
    }
  }
}

Summary

SAP Integration Suite with the AWS adapter provides an easy and efficient integration to AWS services. Customers who are already using SAP BTP and the Integration Suite for other requirements, can extend their existing platform to cover AWS service integration for data and analytics use cases. Especially for RISE with SAP and S/4HANA Cloud customers this is a good way to integrate the SAP solution with AWS services.

SAP Integration Suite keeps the application context by accessing IDocs or OData services. Hence the context is also available in the data stored on Amazon S3 for example and can be processed further. It’s the customer’s choice how to integrate SAP data with AWS services and it depends on the use-case. With the native integration, SAP Integration Suite and third-party tools, we provide flexibility and choice for our customers.

To learn why AWS is the platform of choice and innovation for 5000+ SAP customers, visit thinkwithwp.com/sap.