AWS Developer Tools Blog
Serverless Service Discovery: Part 4: Registrar
In this, the last part of our serverless service discovery series, we will show how to register and look up a new service. We will add these components:
AWS Lambda Registrar Agent
In Docker, it is common to have container agents that add functionality to your Docker deployment. We will borrow from this concept and build a Lambda registrar agent that will manage the registration and monitoring of a service.
def component_status(lambda_functions, rest_api_id):
"""Checking component status of REST API."""
any_existing = False
any_gone = False
client = boto3.client('lambda')
for lambda_function in lambda_functions:
try:
logger.info("checking Lambda: %s" % (lambda_function,))
client.get_function_configuration(
FunctionName=lambda_function)
any_existing = True
except botocore.exceptions.ClientError:
any_gone = True
client = boto3.client('apigateway')
try:
logger.info("checking Rest API: %s" % (rest_api_id,))
client.get_rest_api(restApiId=rest_api_id)
any_existing = True
except botocore.exceptions.ClientError:
any_gone = True
if (not any_existing):
return "service_removed"
elif (any_gone):
return "unhealthy"
else:
return "healthy"
def lambda_handler(event, context):
"""Lambda hander for agent service registration."""
with open('tmp/service_properties.json') as json_data:
service_properties = json.load(json_data)
logger.info("service_name: %s" % (service_properties['service_name'],))
logger.info("service_version: %s" % (service_properties['service_version'],))
status = component_status(service_properties['lambda_functions'],
service_properties['rest_api_id'])
register_request = {
"service_name": service_properties['service_name'],
"service_version": service_properties['service_version'],
"endpoint_url": service_properties['endpoint_url'],
"ttl": "300"
}
if (status == 'healthy'):
logger.info('registering healthy service')
register_request["status"] = 'healthy'
response = signed_post(
service_properties['discovery_service_endpoint']+"/catalog/register",
"us-east-1",
"execute-api",
json.dumps(register_request))
elif (status == 'unhealthy'):
logger.info('registering unhealthy service')
register_request["status"] = 'unhealthy'
response = signed_post(
service_properties['discovery_service_endpoint']+"/catalog/register",
"us-east-1",
"execute-api",
json.dumps(register_request))
else:
logger.info('removing service and registrar')
deregister_request = {
"service_name": service_properties['service_name'],
"service_version": service_properties['service_version']
}
response = signed_post(
service_properties['discovery_service_endpoint'] +
"/catalog/deregister",
"us-east-1",
"execute-api",
json.dumps(deregister_request))
client = boto3.client('lambda')
client.delete_function(
FunctionName=service_properties['registrar_name'])
The Lambda registrar agent is packaged with a property file that defines the Lambda functions and Amazon API Gateway deployment that are part of the service. The registrar agent uses the component_status function to inspect the state of those parts and takes action, depending on what it discovers:
- If all of the parts are there, the service is considered healthy. The register function is called with the service information and a healthy status.
- If only some of the parts are there, the service is considered unhealthy. The register function is called with the service information and an unhealthy status.
- If none of the parts are there, the service is considered to have been removed. The deregister function is called, and the Lambda agent will delete itself because it is no longer needed.
Subsequent register function calls will overwrite the information, so as the health status of our services changes, we can call the function repeatedly. In fact, when we deploy the agent with our Hello World service, we will show how to put the Lambda registrar agent on a five-minute schedule to continuously monitor our service.
Deploy the Hello World Service with the Lambda Agent
We will first implement our simple Hello World Lambda function:
def lambda_handler(api_parameters, context):
"""Hello World Lambda function."""
return {
"message": "Hello "+api_parameters['name']
}
We will create a Swagger file for the service:
{
"swagger": "2.0",
"info": {
"title": "helloworld_service",
"version": "1.0.0"
},
"basePath": "/v1",
"schemes": ["https"],
"consumes": ["application/json"],
"produces": ["application/json"],
"paths": {
"/helloworld/{name}": {
"parameters": [{
"name": "name",
"in": "path",
"description": "The name to say hello to.",
"required": true,
"type": "string"
}],
"get": {
"responses": {
"200": {
"description": "Hello World message"
}
},
"x-amazon-apigateway-integration": {
"type": "aws",
"uri": "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/$helloworld_serviceARN$/invocations",
"httpMethod": "POST",
"requestTemplates": {
"application/json": "{\"name\": \"$input.params('name')\"}"
},
"responses": {
"default": {
"statusCode": "200",
"schema": {
"$ref": "#/definitions/HelloWorldModel"
}
}
}
}
}
}
},
"definitions": {
"HelloWorldModel": {
"type": "object",
"properties": {
"message": {
"type": "string"
}
},
"required": ["message"]
}
}
}
Now we are ready to pull everything we have done in this blog series together: we will deploy this service with a Lambda registrar agent that registers and deregisters it with our serverless discovery service. First, we need to add the requests Python module to the directory we are deploying from because our Lambda registrar agent is dependent on it.
pip install requests -t /path/to/project-dir
Second, we deploy the Hello World service and the Lambda registrar agent:
ACCOUNT_NUMBER = _your aws account number
######################################
# Deploy Hello World Service
######################################
create_deployment_package("/tmp/helloworld.zip", ["helloworld_service.py"])
hello_world_arn = create_lambda_function(
"/tmp/helloworld.zip",
"helloworld_service",
"arn:aws:iam::"+ACCOUNT_NUMBER+":role/lambda_s3",
"helloworld_service.lambda_handler",
"Hello World service.",
ACCOUNT_NUMBER)
replace_instances_in_file("swagger.json",
"/tmp/swagger_with_arn.json",
"$helloworld_serviceARN$",
hello_world_arn)
api_id = create_api("/tmp/swagger_with_arn.json")
rest_api_id, stage, endpoint_url = deploy_api(api_id, "/tmp/swagger_with_arn.json", "dev")
######################################
# Deploy Lambda Registrar Agent
######################################
with open('/tmp/service_properties.json',
'w') as outfile:
json.dump(
{
"lambda_functions": ["helloworld_service"],
"rest_api_id": rest_api_id,
"stage": stage,
"endpoint_url": endpoint_url,
"service_name": "helloworld",
"service_version": "1.0",
"discovery_service_endpoint":
"https://1vvw0qvh4i.execute-api.us-east-1.amazonaws.com/dev",
"registrar_name": "registrar_"+rest_api_id
}, outfile)
create_deployment_package("/tmp/helloworld_registrar.zip",
["registrar.py", "/tmp/service_properties.json",
"requests"])
registrar_arn = create_lambda_function(
"/tmp/helloworld_registrar.zip",
"registrar_"+rest_api_id,
"arn:aws:iam::"+ACCOUNT_NUMBER+":role/lambda_s3",
"registrar.lambda_handler",
"Registrar for Hello World service.",
ACCOUNT_NUMBER)
After we have deployed the Hello World service, we create a JSON file (service_properties.json) with some of the outputs from that deployment. This JSON file is packaged with the Lambda registrar agent.
Both the service and the agent are now deployed, but nothing is triggering the agent to execute. We will use the following to create a five-minute monitoring schedule using CloudWatch events:
client = boto3.client('events')
response = client.put_rule(
Name="registrar_"+rest_api_id,
ScheduleExpression='rate(5 minutes)',
State='ENABLED'
)
rule_arn = response['RuleArn']
lambda_client = boto3.client('lambda')
response = lambda_client.add_permission(
FunctionName=registrar_arn,
StatementId="registrar_"+rest_api_id,
Action="lambda:InvokeFunction",
Principal="events.amazonaws.com",
SourceArn=rule_arn
)
response = client.put_targets(
Rule="registrar_"+rest_api_id,
Targets=[
{
'Id': "registrar_"+rest_api_id,
'Arn': registrar_arn
},
]
)
Now we have deployed a service that is being continuously updated in the discovery service. We can use it like this:
############################
# 1. Do service lookup
############################
request_url="https://yourrestapiid.execute-api.us-east-1.amazonaws.com/"\
"dev/catalog/helloworld/1.0"
response = requests.get(request_url)
json_response = json.loads(response.content)
############################
# 2. Use the service
############################
request_url=("%s/helloworld/Magnus" % (json_response['endpoint_url'],))
response = requests.get(request_url)
json_response = json.loads(response.content)
logger.info("Message: %s" % (json_response['message'],))
We should get the following output:
INFO:root:Message: Hello Magnus
Summary
We have implemented a fairly simple but functional discovery service without provisioning any servers or containers. We can build on this by adding more advanced monitoring, circuit breakers, caching, additional protocols for discovery, etc. By providing a stable host name for our discovery service (instead of the one generated by API Gateway), we can make that a central part of our microservices architecture.
We showed how to use Amazon API Gateway and AWS Lambda to build a discovery service using Python, but the approach is general. It should work for other services you want to build. The examples provided for creating and updating the services can be enhanced and integrated into any CI/CD platforms to create a fully automated deployment pipeline.