AWS Database Blog

Build Oracle Enterprise Manager with a repository in an Amazon RDS Custom for Oracle database

As you migrate Oracle workloads to AWS, you may want to implement Oracle Enterprise Manager (OEM) Cloud Control, Oracle’s management platform, which provides a single pane of glass for managing Oracle environments.

In this post, we provide the architecture and process to implement OEM 13.5 with high availability (HA) with an Amazon Relational Database Service (Amazon RDS) Custom for Oracle database as Oracle Management Repository (OMR). You can implement this architecture to achieve high availability for Oracle Management Service (OMS) servers behind a Network Load Balancer and the RDS Custom for Oracle database with a read replica. This architecture provides Oracle’s OEM Level 3 high availability architecture, which provides high availability at the application and database tiers. This solution uses native AWS services such as Amazon Elastic Compute Cloud (Amazon EC2) instances, Amazon Elastic Block Storage (Amazon EBS), Amazon Elastic File System (Amazon EFS), and a Network Load Balancer.

For more information about installing OEM, adding additional OMS servers, and deploying OEM management agents, see Cloud Control Advanced Installation and Configuration Guide.

You can monitor your Amazon Relational Database Service (Amazon RDS) for Oracle instances using OEM. For more information, see Oracle Management Agent for Enterprise Manager Cloud Control and Implement high availability for OEM in AWS Cloud.

OEM HA architecture

The downtime or outage of any OEM service component impacts the monitoring of all targets, which includes on-premises databases and databases running in the AWS Cloud that you’re monitoring.

If you have more than 25 concurrent OEM user sessions, more than 100 OEM agents, or more than 1,000 OEM targets, then Oracle recommends using a medium or large OEM configuration, which requires high availability.

For a large organization, you can efficiently monitor thousands of targets, because each component of the monitored target becomes a separate target, such as a database, listener, database host, or OEM agent. If you undersize your OEM environment, you may experience performance problems with your monitoring solution.

OEM licensing

Oracle provides a Restricted Use License for OEM to run on EC2 instances. Based on Oracle’s OEM sizing guide, you can establish an evaluation or small OEM installation and use Oracle’s Restricted Use Licenses.

However, if you use the traditional OEM Level 3 HA on AWS architecture under the Restricted Use License for OEM, then it may not cover the licensing required for high availability features. Based on Oracle’s recommended sizing, you may need to purchase Oracle Enterprise Edition Database. Additional Oracle database licenses may be required due to the implementation of an Oracle Data Guard environment.

We recommend that you send database backups directly to Amazon Simple Storage Service (Amazon S3) via the Oracle Secure Backup (OSB) Module for Amazon S3.

You are responsible for making sure you have the proper Oracle licenses to run your workloads.

AWS native OEM HA architecture

The following diagram illustrates AWS native OEM HA architecture.

For OEM medium and large environments, this architecture contains the following components:

  • Network Load Balancer – Comes with HA in two Availability Zones
  • Two OMS servers – An application tier on EC2 servers in two Availability Zones
  • One OMR database – Uses an RDS Custom for Oracle database with a Data Guard read replica in separate Availability Zones
  • Amazon EFS – A shared file system between two OMS servers in two Availability Zones
  • Amazon Route 53 – We use Amazon Route 53 as the DNS for setting up transparent OMR database HA
  • Amazon RDS for Oracle – The target database in AWS
  • On-premises Oracle database – The target database

Unlike the traditional OEM Level 3 HA AWS architecture, this architecture doesn’t require an administrator to configure a Server Load Balancer (SLB) with the OEM OMS. Tasks such as creating SLB monitors, SLB pools, SLB TCP profiles, SLB persistent profiles, SLB rules, and SLB virtual servers aren’t required with AWS native OEM HA architecture.

Architecture overview

An RDS Custom for Oracle database is created along with a read replica in another Availability Zone in the same Region to support OMR HA. The data in OMR is replicated using Oracle Data Guard to protect against database failure.

OMS is installed on two EC2 servers to provide HA for the OMS application layer. Both servers connect to the same OMR database. They share an Amazon EFS mount to share software library files.

A Network Load Balancer is provisioned to support the workload of OEM agent requests and OEM console requests to both OMS servers. A Network Load Balancer is a network device that comes with HA in two Availability Zones.

The OEM agents from the RDS for Oracle database targets and OEM agents from on-premises Oracle databases upload data to the OMS via the Amazon Network Load Balancer.

The OEM console can access OEM via the Network Load Balancer, providing HA for the console.

If one OMS server fails, the OEM agent uploads to the surviving OMS server and OEM console access continues via the surviving OMS server. The Network Load Balancer directs traffic to the surviving OMS server automatically as targets in its target group are marked as unhealthy.

A current generation EC2 instance type is recommended for the OEM EC2 instances. Current generation instance types, such as instances built on the AWS Nitro System (see Instances built on the Nitro System), support hardware virtual machines (HVMs). The HVM Amazon Machine Images (AMIs) are required to take advantage of enhanced networking, and they also offer increased security. The following virtualized instances are built on the Nitro System and are recommended for OEM, OMS and OMR EC2 instances: C5, M5, R5, R5b, T3 and z1d. As a reminder, the EC2 instance types should be selected based on Oracle’s OEM sizing guide.

Architecture deployment

Complete the following steps to deploy the OEM HA architecture:

  1. Create an RDS Custom for Oracle database in one Availability Zone of the Region chosen for the deployment of the OEM HA architecture.
  2. Create a read replica of the RDS Custom for Oracle database you created in a different Availability Zone of the same Region where the primary database is hosted from step 1.
    Following screenshot shows RDS Custom for Oracle primary and read replica databases.
  3. Create a private zone in Route 53 DNS to add a CNAME entry for the OMR database you created.
    You use the DNS name of the repository database while configuring OMS servers in later steps. This provides flexibility of switching or failing over between the read replica and primary repository database without changing the OMS configuration.The following screenshot shows routing entry in Route 53 DNS.
  4. Set up two OMS EC2 servers in two Availability Zones and attach EFS shared mount to them.
    The following snapshot shows EFS filesystem mount.
  5. Change the sys password of the RDS Custom for Oracle database and note it down because OEM installation will prompt for the sys access.
  6. Open access using the security group between the OMS servers and the RDS Custom database OMR for the following ports:
    1. 1521 – OMR database SQLNet access
    2. 7803 – OEM console port
    3. 4903 – OEM secure agent upload
    4. 7102 – OEM Admin server
    5. 3872 – OEM Agent registration
  7. Follow the instructions for OEM installation and add a second OMS as per the instructions in Installing additional Oracle Management Service.
  8. Apply necessary OEM 13.5 patches to the latest patch level.
  9. Make sure that you are able to access the OEM console from both OMS server URLs before provisioning a Network Load Balancer.
  10. Provision a Network Load Balancer in the same Availability Zones where the OMS EC2 servers are located.
    The following screenshot shows settings for the NLB.
  11. Add the listeners to the NLB for ports 7803, 4903 and 3872, and create a target group for the load balancer that maps to the OMS servers running on two EC2 instances. Use the following configuration:
    1. Use the IPs option to choose the OMS EC2 instances.
    2. Create two target groups: one for agent uploads and one for the OEM console.
    3. Provide a name for the target group and choose the VPC where the EC2 instances are running.
    4. Choose HTTP1 protocol for target communication.
    5. Use the default health check.
    6. Choose EC2 instances that are running OMS and use ports 7803, 4903 and 3872 to register targets.
    7. For the OEM-agents target group, set Stickiness as Enabled and Preserve client IP addresses as Disabled.
    8. For the OEM-console target group, set Stickiness as Enabled and Preserve client IP addresses as Enabled.

    The following screenshot shows registered targets in a NLB target group.

    The following screenshot shows oem-console target group with healthy targets.

    Following screenshot shows listener set up in the NLB.

    At this time, OEM agents running on OMS servers are uploading directly to the OMS and not through the Network Load Balancer.

  12. Secure OMS on each OMS server along with agent configuration changes: 
    $ $OMS_HOME/bin/bin/emctl status oms -details

$ $OMS_HOME/bin/emctl secure oms -host NLB URL -secure_port 4903 -slb_port 4903 -slb_console_port 7803 -slb_jvmd_https_port 4903 -- Refer to OMS set up with load balancer

$ $OMS_HOME/bin/emctl stop oms -all

$ $OMS_HOME/bin/emctl start oms

$ $OMS_HOME/bin/emctl status oms -details

    Verify that OMS is now locked to accept data via the Network Load Balancer URL.

    Unlock the console for uploads.

    $ $OMS_HOME/bin/emctl secure unlock – this only unlocks ports for uploads, and OMS remains secure

$ $OMS_HOME/bin/emctl stop oms -all

$ $OMS_HOME/bin/emctl start oms

$ $OMS_HOME/bin/emctl status oms -details
  13. After you secure OMS on both servers, the agents running on them need to be re-secured with the new upload URL of the Network Load Balancer: 
    $ $AGENT_HOME/bin/emctl secure agent -emdWalletSrcUrl “https://NLB URL:4903/empbs/upload” -- Refer to OMS set up with load balancer
$ $AGENT_HOME/bin/emctl status agent – verify that the upload URL has changed and test the upload

$ $AGENT_HOME/bin/emctl upload

    The OEM console should now be accessible via the Network Load Balancer URL and all targets should show as green or online on the console.

  14. Change all agents for all targets to secure them using the same command to verify that they’re uploading data to the OMS via the Network Load Balancer.

Test HA for Oracle OEM OMS and OEM repository database

To test high availability for OEM OMS and the OEM repository database in RDS Custom for Oracle, complete the following steps:

  1. Stop one OMS running on the EC2 server and continue to access the OEM console via the Network Load Balancer URL: 
    $ $OMS_HOME/bin/emctl stop oms

    The OEM agents from the targets will continue to upload the data via the Network Load Balancer, and they will show as online on the OEM console.

    Now you can restart the OMS that was shut down, shut down the other OMS, and observe the same behavior to test HA.

  2. To test HA for the OEM repository database in RDS Custom for Oracle, shut down both OMS servers before switching over to the secondary OEM repository database: 
    $ $OMS_HOME/bin/emctl stop oms

    For more details, refer to Working with Oracle replicas for RDS Custom for Oracle and Enabling High Availability with Data Guard on Amazon RDS Custom for Oracle.

  3. After the switchover to the mounted replica of the OEM repository database in the RDS custom for Oracle database, switch the endpoint address in the Route 53 DNS hosted zone.
    Following screenshot shows Route 53 DNS record with switched endpoint address.
  4. After the DNS change, start both OMS servers: 
    $ $OMS_HOME/bin/emctl start oms

After the OMS servers start, log in to the console from the Network Load Balancer URL to confirm that it’s working normally. Additionally, all OEM agents should continue to upload data via the Network Load Balancer, which flows to the standby OEM repository database, which has now become the primary database.

Clean up

If you followed along and won’t be using this architecture, make sure you remove the following resources to avoid costs :

  • Amazon EC2 servers used for OMS
  • Amazon EFS mount
  • Amazon RDS Custom database used for OMR
  • Network Load Balancer
  • Private zone in Route 53 that is used by OEM URL

Conclusion

In this post, we showed you how to build a robust architecture with AWS to run Oracle Enterprise Manager, which can provide high availability for Oracle Management Service and Oracle Management Repository. This architecture not only provides HA but is also scalable as your organization grows its workload of monitored targets.

About the Author

Sachin Vaidya is a Database Specialist Solutions Architect. He works with AWS customers to provide guidance and technical assistance on database projects, helping them improve the value of their solutions when using AWS.