Skip to content

Latest commit

 

History

History
237 lines (170 loc) · 18 KB

README.md

File metadata and controls

237 lines (170 loc) · 18 KB

Example AWS SAM app - raw mode function

This document is a comprehensive walkthrough for the process of deploying a simple piece of Wolfram Language code to AWS Lambda as a raw-mode function. After installing the necessary tools, you will configure Wolfram Engine licensing for your function, create an Amazon ECR container image repository, and finally deploy the function using the AWS SAM CLI.

The Examples/aws-sam/raw-mode directory contains source code and supporting files for an example Wolfram Language-based serverless application that you can deploy with the AWS SAM CLI. It includes the following files and folders:

The application uses several AWS resources, including a Lambda function. These resources are defined in the template.yaml file in this project. You can update the template to add AWS resources through the same deployment process that updates your application code.

Clone the repository

If you have not done so already, you should clone this Git repository so that the code for the example is available on your local filesystem:

$ git clone https://github.com/WolframResearch/AWSLambda-WolframLanguage

Install dependencies

If you already have some or all of these tools installed, you can skip the appropriate steps.

To follow this walkthrough, you will need the following tools:

This walkthrough assumes that you have installed these tools, have access to an AWS account and security credentials, and have access to the Wolfram Language through a product like Wolfram Mathematica or Wolfram Engine.

Create a Wolfram Engine on-demand license entitlement

In order for the Wolfram Engine kernel inside the Lambda function's container to run, it must be activated using on-demand licensing.

On-demand licensing is a pay-as-you-go licensing method whereby Wolfram Engine usage is billed against your Wolfram Service Credits balance at a per-kernel-hour rate. This method allows you to run arbitrary numbers of concurrent Wolfram Engine kernels for pennies per kernel per hour, and to scale up and down in a cost-effective manner. You may use the starter Service Credits quota available with a free Wolfram Cloud Basic account for initial experimentation before purchasing more Service Credits. With the instructions below, usage will be charged at 4 Service Credits per kernel per hour.

An on-demand license entitlement is a reusable license key that can be used to activate one or more Wolfram Engine kernels. Creating an entitlement requires access to the Wolfram Language. If you do not have Wolfram Mathematica, a Wolfram|One subscription or another Wolfram Language product, you can sign up for a free Wolfram Cloud Basic subscription and create an entitlement from within a cloud notebook.

Use the CreateLicenseEntitlement function to create a new license entitlement linked to your Wolfram Account:

In[1]:= entitlement = CreateLicenseEntitlement[<|
    "Policy" -> "WLMBDA",
    "StandardKernelLimit" -> 15,
    "LicenseExpiration" -> Quantity[1, "Week"],
    "EntitlementExpiration" -> Quantity[1, "Years"]
|>]
Out[1]= LicenseEntitlementObject["O-WLMBDA-DA42-5Z2SW6WKQQL", <|
    "PolicyID" -> "WLMBDA", "PolicyName" -> "AWS Lambda runtime",
    "BillingInterval" -> Quantity[900, "Seconds"],
    "KernelCosts" -> <|
        "Standard" -> Quantity[4., "Credits"/"Hours"],
        "Parallel" -> Quantity[4., "Credits"/"Hours"]
    |>,
    "KernelLimits" -> <|"Standard" -> 15, "Parallel" -> 0|>,
    "CreationDate" -> DateObject[{2021, 4, 28, 16, 50, 49.}, "Instant", "Gregorian", -4.],
    "ExpirationDate" -> DateObject[{2022, 4, 28, 16, 50, 49.}, "Instant", "Gregorian", -4.],
    "LicenseExpirationDuration" -> Quantity[MixedMagnitude[{7, 0.}], MixedUnit[{"Days", "Hours"}]]
|>]

In[2]:= entitlement["EntitlementID"]
Out[2]= "O-WLMBDA-DA42-5Z2SW6WKQQL"

Take note of the returned entitlement ID (O-WLMBDA-DA42-5Z2SW6WKQQL above); you will need it when you deploy your application in a subsequent step. This entitlement ID should be treated as an application secret and not committed to source control or exposed to the public.

The meanings of the specified entitlement settings are:

  • "Policy" -> "WLMBDA": Use the WLMBDA licensing policy, which is tailored for use with AWS Lambda. The associated on-demand license fee is 4 Service Credits per kernel per hour.
  • "StandardKernelLimit" -> 15: Up to 15 kernels may run concurrently. (This means 15 instances of your Lambda function.)
  • "LicenseExpiration" -> Quantity[1, "Week"]: Each kernel may run for up to one week at a time.
  • "EntitlementExpiration" -> Quantity[1, "Years"]: The entitlement expires one year after creation. (This means you must create a new entitlement and replace it in your application once a year.)

You may adjust these settings as needed for your use case. For more information, see the documentation for CreateLicenseEntitlement.

Create an ECR repository

The instructions in this section are based on the AWS blog post "Using container image support for AWS Lambda with AWS SAM".

Before deploying your application, you must create an Amazon Elastic Container Registry (ECR) repository in which to store the container image for your function.

NOTE: If you wish, you can use one repository with multiple applications. Doing so can greatly reduce the time spent on the initial push, because large layers (e.g. the OS and Wolfram Engine) can be shared between images. If you have already created the example-wl-sam-apps repository during the HTTP-mode walkthrough, you can use the repositoryUri from before and skip this step.

To create the repository, run the following in your shell:

$ aws ecr create-repository --repository-name example-wl-sam-apps

This will return a JSON document like:

{
    "repository": {
        "repositoryArn": "arn:aws:ecr:us-east-1:123456789012:repository/example-wl-sam-apps",
        "registryId": "123456789012",
        "repositoryName": "example-wl-sam-apps",
        "repositoryUri": "123456789012.dkr.ecr.us-east-1.amazonaws.com/example-wl-sam-apps",
        "createdAt": "2021-04-28T17:27:48-04:00",
        "imageTagMutability": "MUTABLE",
        "imageScanningConfiguration": {
            "scanOnPush": false
        },
        "encryptionConfiguration": {
            "encryptionType": "AES256"
        }
    }
}

Take note of the repositoryUri; you will need it when you deploy your application in the next step.

Ensure that your local Docker daemon is authenticated to your account's ECR registry:

$ aws ecr get-login-password | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com

(Replace 123456789012.dkr.ecr.us-east-1.amazonaws.com with the domain name component of the repositoryUri from the previous command result.)

You can also install the Amazon ECR Docker Credential Helper to facilitate Docker authentication with Amazon ECR.

Deploy the example application

The Serverless Application Model Command Line Interface (SAM CLI) is an extension of the AWS CLI that adds functionality for building and testing Lambda applications. It uses Docker to build a container image containing your function code, and it interfaces with AWS CloudFormation to deploy your application to AWS. For more information on using container image-based Lambda functions with AWS SAM, see the AWS blog post "Using container image support for AWS Lambda with AWS SAM".

To build and deploy your application for the first time, run the following in your shell from within the Examples/aws-sam/raw-mode directory of the cloned Git repository:

$ cd Examples/aws-sam/raw-mode
$ sam build
$ sam deploy --guided

The first command will build a container image from the Dockerfile. The second command will package and deploy your application to AWS after a series of prompts:

  • Stack Name: The name of the stack to deploy to CloudFormation. This should be unique to your account and region. In this example, we will use example-raw-wl-sam-app.
  • AWS Region: The AWS region you want to deploy your app to.
  • Parameter OnDemandLicenseEntitlementID: Your license entitlement ID from the previous step. This parameter is masked, so the text you type/paste will not be echoed back to you.
  • Image Repository for ExampleRawFunction: The ECR repositoryUri from the previous step.
  • Confirm changes before deploy: If enabled, any change sets will be shown to you for manual review before execution. If disabled, the AWS SAM CLI will automatically deploy application changes without prompting for review.
  • Allow SAM CLI IAM role creation: Enter y. Many AWS SAM templates, including this example, create AWS IAM roles required for the included AWS Lambda function(s) to access AWS services. By default, these are scoped down to minimum required permissions. To deploy an AWS CloudFormation stack that creates or modifies IAM roles, the CAPABILITY_IAM value for capabilities must be provided. If permission isn't provided through this prompt, to deploy this example you must explicitly pass --capabilities CAPABILITY_IAM to the sam deploy command.
  • Save arguments to configuration file: Enter y. If enabled, your choices will be saved to a samlconfig.toml configuration file in the current directory, so that in the future you can just re-run sam deploy without parameters to deploy changes to your application.
  • SAM configuration file and SAM configuration environment: If you enabled the previous option, these options allow you to configure how the configuration file is saved. You may leave these options at their default values.

For more information, see the documentation for sam deploy.

The initial deployment of your application may take several minutes, as Docker will have to push the entire container image - including the Wolfram Engine base layers - to your ECR repository. Subsequent deployments will be faster, as only the changed portions of the image will be pushed.

You can find the ARN of your deployed Lambda function in the output values displayed after deployment:

Key                 ExampleRawFunction
Description         Example raw-mode Lambda function
Value               arn:aws:lambda:us-east-1:123456789012:function:example-raw-wl-sam-app-ExampleRawFunction-AKWQGDATPPTP

You can invoke this function from the AWS CLI using aws lambda invoke.
On Linux and macOS (Bash-like shells):

$ aws lambda invoke \
  --function-name arn:aws:lambda:us-east-1:123456789012:function:example-raw-wl-sam-app-ExampleRawFunction-AKWQGDATPPTP \
  --payload '{"input": "Hello World"}' \
  --cli-binary-format raw-in-base64-out \
  /tmp/output.json && cat /tmp/output.json

{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}
{"reversed":"dlroW olleH"}

In Powershell on Windows:

PS C:\> aws lambda invoke --function-name arn:aws:lambda:us-east-1:123456789012:function:example-raw-wl-sam-app-ExampleRawFunction-AKWQGDATPPTP --payload '{""input"": ""Hello World""}' --cli-binary-format raw-in-base64-out output.json
{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}

PS C:\> cat .\output.json
{"reversed":"dlroW olleH"}

In addition to the AWS CLI, raw-mode functions can be invoked using the AWS Lambda API; AWS SDKs; and other AWS services such as Lex, S3 and SNS. Wolfram Language clients can invoke arbitrary Lambda functions using the AWS service connection.

NOTE: If you wish to minimize invocation latency, consider enabling provisioned concurrency as described in the HTTP-mode walkthrough.

Add a resource to your application

The application template uses the AWS Serverless Application Model (AWS SAM) to define application resources. AWS SAM is an extension of AWS CloudFormation with a simpler syntax for configuring common serverless application resources such as functions, triggers, and APIs. For resources not included in the SAM specification, you can use standard AWS CloudFormation resource types.

Fetch, tail, and filter Lambda function logs

To simplify troubleshooting, the SAM CLI has a command called sam logs. sam logs lets you fetch logs generated by your deployed Lambda function from the command line.

NOTE: This command works for all AWS Lambda functions; not just the ones you deploy using SAM.

$ sam logs -n ExampleRawFunction --stack-name example-raw-wl-sam-app
2021/04/28/[$LATEST]b0d33af52c2644e6ae9c6d76a6cda134 2021-04-28T23:02:02.084000 START RequestId: f27d9984-efa0-4220-aec9-fdb863e0244e Version: $LATEST
2021/04/28/[$LATEST]b0d33af52c2644e6ae9c6d76a6cda134 2021-04-28T23:02:02.507000 Wolfram Language 12.3.0 Engine for Linux x86 (64-bit)
2021/04/28/[$LATEST]b0d33af52c2644e6ae9c6d76a6cda134 2021-04-28T23:02:05.726000 Copyright 1988-2021 Wolfram Research, Inc.
2021/04/28/[$LATEST]b0d33af52c2644e6ae9c6d76a6cda134 2021-04-28T23:02:07.424000 END RequestId: f27d9984-efa0-4220-aec9-fdb863e0244e
2021/04/28/[$LATEST]b0d33af52c2644e6ae9c6d76a6cda134 2021-04-28T23:02:07.424000 REPORT RequestId: f27d9984-efa0-4220-aec9-fdb863e0244e    Duration: 169.94 ms    Billed Duration: 6767 ms    Memory Size: 512 MB    Max Memory Used: 234 MB    Init Duration: 6596.33 ms

You can add the --tail option to stream logs to your terminal in near-real time. You can find information and examples about filtering Lambda function logs in the SAM CLI documentation.

Cleanup

To delete the sample application that you created, use the AWS CLI to delete the application's CloudFormation stack:

$ aws cloudformation delete-stack --stack-name example-raw-wl-sam-app

Resources

See the AWS SAM developer guide for an introduction to SAM specification, the SAM CLI, and serverless application concepts.

This file is derived from an AWS-provided SAM CLI application template. The original document from which this walkthrough has been modified is located here. The repository containing the original document is licensed under the Apache-2.0 License, and carries the following notice: Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.