AIBS Informatics AWS Lambda

This is a base package that can be used standalone with some core lambda functionality or as a dependency.

Package Overview

The package contains several classes and functions that make it easy to create strongly typed lambda functions with many nice-to-have features (serialization/deserialization, easy to add metrics, utilities to create batch sqs and dynamo db event bridge processing ). In addition to these base classes, you can also use a collection of general purpose lambda handler classes.

Base Classes and Functions

LambdaHandler

The LambdaHandler class provides a base class for creating strongly typed lambda functions with features like serialization/deserialization, logging, and metrics.

ApiLambdaHandler and ApiResolverBuilder

These classes extend the LambdaHandler class and provide a way to create strongly typed lambda functions that can be used as API Gateway endpoints.

• ApiLambdaHandler: A base class for API Gateway handlers.
• ApiResolverBuilder: A utility class for building API Gateway resolvers.

Standalone Lambda Classes in this Package

Most of these lambda functions are found under src/aibsinformaticsaws_lambda/handlers/

AWS Batch Functions

• CreateDefinitionAndPrepareArgsHandler: Handles the creation and preparation of AWS Batch job definitions.
• PrepareBatchDataSyncHandler: Prepares data synchronization tasks for AWS Batch.

Data Sync Functions

Data Sync Operations (Writing, Reading and Syncing Data)

• GetJSONFromFileHandler: Retrieves JSON data from a file.
• PutJSONToFileHandler: Writes JSON data to a file.
• DataSyncHandler: Simple data sync task.
• BatchDataSyncHandler: Handles batch of data sync tasks.
• PrepareBatchDataSyncHandler: Taking a data sync request, it analyzes and generates multiple batches of data sync tasks to evenly distribute the load across multiple batch data sync tasks.

Data Sync File System Functions (Managing Data Paths)

• GetDataPathStatsHandler: Retrieves statistics about data paths.
• ListDataPathsHandler: Lists data paths.
• OutdatedDataPathScannerHandler: Scans for outdated data paths.
• RemoveDataPathsHandler: Removes data paths.

Demand Functions

• PrepareDemandScaffoldingHandler: Prepares scaffolding for demand execution.

Notification Functions

• NotificationRouter: Routes notifications to the appropriate notifier.
• SESNotifier: Sends notifications via Amazon SES.
• SNSNotifier: Sends notifications via Amazon SNS.

ECR Image Replicator

• ImageReplicatorHandler: Handles the replication of ECR images between repositories using the ECRImageReplicator.

CLI Invocation

With this package, you can also invoke lambda functions from the command line. The CLI executable is installed as handle-lambda-request and can be used to invoke lambda functions with payloads that can be specified as JSON, files, or S3 objects.

``` usage: handle-lambda-request [-h] [--handler-qualified-name HANDLERQUALIFIEDNAME] [--payload PAYLOAD] [--response-location RESPONSE_LOCATION]

CLI AWS Lambda Handler

options: -h, --help show this help message and exit --handler-qualified-name HANDLERQUALIFIEDNAME, --handler-name HANDLERQUALIFIEDNAME, --handler HANDLERQUALIFIEDNAME handler function qualified name. If not provided, will try to load from ('AWSLAMBDAFUNCTIONHANDLER', 'HANDLER') env variables --payload PAYLOAD, --event PAYLOAD, -e PAYLOAD event payload of function. If not provided, will try to load from AWSLAMBDAEVENTPAYLOAD env variable --response-location RESPONSELOCATION, -o RESPONSELOCATION optional response location to store response at. can be S3 or local file. If not provided, will load from AWSLAMBDAEVENTRESPONSE_LOCATION env variable. ```

Examples

Invoking a Lambda Function with a JSON Payload

bash handle-lambda-request --handler-qualified-name aibs_informatics_aws_lambda.handlers.data_sync.operations.GetJSONFromFileHandler --payload '{"path": "/path/to/file.json"}' --response-location /tmp/response.json

Invoking a Lambda Function with a JSON Payload from a File

bash handle-lambda-request --handler-qualified-name aibs_informatics_aws_lambda.handlers.data_sync.operations.GetJSONFromFileHandler --payload-file /path/to/payload.json --response-location /tmp/response.json

Invoking a Lambda Function with a JSON Payload from S3 and Saving the Response to S3

bash handle-lambda-request --handler-qualified-name aibs_informatics_aws_lambda.handlers.data_sync.operations.GetJSONFromFileHandler --payload-file s3://bucket/key/payload.json --response-location s3://bucket/key/response.json

Invoking a Lambda Function with environment variables

bash AWS_LAMBDA_EVENT_PAYLOAD='{"path": "/path/to/file.json"}' AWS_LAMBDA_EVENT_RESPONSE_LOCATION='/tmp/response.json' handle-lambda-request --handler-qualified-name aibs_informatics_aws_lambda.handlers.data_sync.operations.GetJSONFromFileHandler

Testing

The package includes comprehensive tests for all handlers, which can be found under the test directory.

Contributing

Any and all PRs are welcome. Please see CONTRIBUTING.md for more information.

Licensing

This software is licensed under the Allen Institute Software License, which is the 2-clause BSD license plus a third clause that prohibits redistribution and use for commercial purposes without further permission. For more information, please visit Allen Institute Terms of Use.