Creating AWS RDS resources using JSON CloudFormation template

This article is not going to provide complete template to create RDS database cluster as the code examples for resources creation is already available at AWS documentation, but this will give some code snippets and important points that might help while creating the JSON templates. If the requirement is to create resources both in AWS and AMS(Amazon Managed Services) then, JSON is better choice instead of YAML as AMS accepts only JSON format.

While creating RDS Database cluster, it is also required to create its depending resources as mentioned in below pic. Apart from the resources mentioned in the pic, some other resources also required like event subscription, cluster parameter group, database parameter group etc…

AWS RDS Single Region Framework

Parameters:

Parameters enables to input custom values to your template each time you create or update a stack.

• If the input parameter value start with either digit or a letter(case insensitive) then Allowed Pattern is: "^[0-9a-zA-Z\\-]*$"
• If the input value is an email address then Allowed Pattern is: "^(([^<>()\\[\\]\\\\.,;:\\s@\"]+(\\.[^<>()\\[\\]\\\\.,;:\\s@\"]+)*)|(\".+\"))@((\\[[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}])|(([a-zA-Z\\-0-9]+\\.)+[a-zA-Z]{2,}))$"
• If the input value starts with a letter(case insensitive) and remaining are either digit or letter(case insensitive) then Allowed Pattern is : "^[a-zA-Z]+[0-9a-zA-Z_]*$"
• If input values are specific then Allowed Values should be like: [
  "true",
  "false"
  ]
• If input values are a list with comma separated, then the Type is: “CommaDelimitedList”.

“VpcSubnets”: {
 “Type”: “CommaDelimitedList”,
 “Description”: “VPC Subnets for lambda in comma separated”
 }

• No need to pass aws account id and region as input parameters, because they can be substituted directly like below:

“S3Bucket”: {“Fn::Sub”: “${AWS::AccountId}-s3-bucket-rds-${AWS::Region}”}

Mappings:

The optional Mappings section matches a key to a corresponding set of named values. Mappings can be declared like below:

"Mappings": {
 "DBFamilyMap": {
   "12.4": {
    "family": "aurora-postgresql12"
   },
   "12.6": {
    "family": "aurora-postgresql12"
   },
   "12.7": {
    "family": "aurora-postgresql12"
   }
  }
}

Mapping usage in the template for above declaration is like below:

"Family": {
  "Fn::FindInMap": [
    "DBFamilyMap",
    {"Ref": "DBEngineVersion"},
    "family"
  ] 
}

Conditions:

Conditions can be declared like below:

"Conditions": {
 "AttachReplica": {
  "Fn::Equals": [
   {"Ref": "AddReplica"},
   "Yes"
  ]
 }
}

Condition usage: If the condition is true then the resource will be created.

"DBPrimaryReplicaInstance": {
  "Type": "AWS::RDS::DBInstance",
  "DependsOn": "PgDBPrimaryInstance",
  "Condition": "AttachReplica",

Secret Manager:

To configure a secret manager key, use below syntax for generating a secret string. DBUsername is the input parameter value here:

"GenerateSecretString": {
  "SecretStringTemplate": {"Fn::Sub":"{\"username\": \"${DBUsername}\"}"},
  "GenerateStringKey": "password",
  "ExcludeCharacters": "\"@/\\",
  "PasswordLength": 16
}

To extract the secret user and password in the cluster creation from above secret string is as below. MasterSecret is a secret manager resource used to generate a secret string in the template:

"MasterUsername": {
  "Fn::Sub": [
  "{{resolve:secretsmanager:${userreplace}:SecretString:username}}", 
  {
    "userreplace": {
     "Ref": "MasterSecret"
    }
   }
 ]
},
"MasterUserPassword": {
 "Fn::Sub": [
  "{{resolve:secretsmanager:${preplace}:SecretString:password}}", 
  {
   "pwdreplace": {
    "Ref": "MasterSecret"
   }
  }
 ]
}

Join

The intrinsic function Fn::Join appends a set of values into a single value, separated by the specified delimiter. If a delimiter is the empty string, the set of values are concatenated with no delimiter.

“Description”: {
  “Fn::Join”: [
    "",
    [“Aurora Database Instance Parameter Group for cfn Stack “,
      {“Ref”: “DBName”}
    ]
  ] 
}

Boolean

Boolean values can be used directly like below:

"DeletionProtection": true

DependsOn

When you add a DependsOn attribute to a resource, that resource is created only after the creation of the resource specified in the DependsOn attribute.

“DependsOn”: [ “SecretClusterAttachment”, “DBPrimaryInstance” ]

Lambda Invocation from template:

Lambda function can be called from the cfn stack by using the custom resources in the template. For that, it is required to adjust event settings of the lambda, then invoke the lambda and adjust the lambda code to handle the response.

If event settings for lambda function is not adjusted, the template waits till the default timeout of the lambda function in case of any failure, so it is better to adjust these settings like below:

"EventConfigLambda" : {
   "Type" : "AWS::Lambda::EventInvokeConfig",
   "DependsOn": "RDSPostDBLambdaFunction",
   "Properties" : {
    "FunctionName" : "Function Name",
    "MaximumEventAgeInSeconds" : 100,
    "MaximumRetryAttempts" : 2,
    "Qualifier" : "$LATEST"
   }
  }

After Lambda event settings are done, then Lambda can be invoked using the custom resource like below. The properties section has the input values for the event.

"InvokeLambda" : {
   "Type": "AWS::CloudFormation::CustomResource",
   "DependsOn": "EventConfigLambda",
   "Version" : "1.0",
   "Properties" : {
      "ServiceToken": {
        "Fn::GetAtt": ["LambdaFunction","Arn"]
      },
      "input" : "Input value to the event",
      "ClientRequestToken": "1",
      "Step" : "None"
    }
  }

Once the function is invoked, it should return with proper response for either success or failure, else the template stack waits till the cloud formation stack time out.
For example, if cfn stack requires a lambda to run some post database activities after RDS database is created, then function should be adjusted to handle the response of the event like below:

import socket
import urllib3
http = urllib3.PoolManager()
responseData = {}
success = "SUCCESS"
failed = "FAILED"
def lambda_handler(event, context):
  input = event['ResourceProperties']['inputvalue']
  token = event['ResourceProperties']['ClientRequestToken']
  step = event['ResourceProperties']['Step']
  if event['RequestType'] == 'Delete':
    print("Request Type:",event['RequestType'])
    send_response(event, context, success, responseData)
  else:
    post_creation_steps(event, context, service_client, input, token)
def post_creation_steps(event, context, service_client, input, token):
  try:
    # Now execute the post DB stuff 
    send_response(event, context, success, responseData)
  except Exception as e:
     logger.error("Post DB creation stuff failed")
     send_response(event, context, failed, responseData)
     raise e
 
def send_response(event, context, responseStatus, responseData, noEcho=False):
  responseUrl = event['ResponseURL']
  stid = event['StackId'].split(":")[5].split("/")[1]
  physicalid = stid + "-" + event['LogicalResourceId']
  responseBody = {
   'Status': responseStatus,
   'Reason': 'None',
   'PhysicalResourceId': physicalid,
   'StackId' : event['StackId'],
   'RequestId' : event['RequestId'],
   'LogicalResourceId' : event['LogicalResourceId'],
   'NoEcho': noEcho,
   'Data': responseData
 }
 json_responseBody = json.dumps(responseBody)
 headers = {
   'content-type': '',
   'content-length': str(len(json_responseBody))
 }
 try:
    response = http.request('PUT', responseUrl, headers=headers, body=json_responseBody)
 except Exception as e:
    print("send(..) failed executing http.request(..):", e)

Hope this helps !!