Data

Tools

Indexes

Applications

Experiments

Open Source Science

lambdr

Run R containers on AWS Lambda

https://github.com/mdneuzerling/lambdr

Science Score: 13.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (17.2%) to scientific vocabulary
Last synced: 11 months ago · JSON representation

Repository

Run R containers on AWS Lambda

Basic Info
Statistics
  • Stars: 140
  • Watchers: 7
  • Forks: 13
  • Open Issues: 11
  • Releases: 7
Created almost 5 years ago · Last pushed 12 months ago
Metadata Files
Readme Changelog License

README.md

lambdr

R-CMD-check CRAN status CRAN downloads Last commit Codecov test coverage license <!-- badges: end -->

This package provides an R runtime interface client for the AWS Lambda serverless compute service. It makes it possible to deploy code written in R as Lambda functions when using container images.

This package is unofficial. Its creators are not affiliated with Amazon Web Services, nor is its content endorsed by Amazon Web Services. Lambda, API Gateway, EventBridge, CloudWatch, and SNS are services of Amazon Web Services.

The runtime

Any programming language can be used to create a container-based Lambda function. All that's required is a language-specific runtime. AWS provides such runtimes for a handful of languages, e.g. Python, Go, and Ruby - but not for R.

As a runtime for R, lambdr's job is to coordinate the translation and transfer of inputs/responses between the outside world and an invoked Lambda's handler function.

The default behaviour is to convert the body of the received event from JSON into arguments for the Lambda's handler function using the jsonlite package.

For example, a raw event body of {"number": 9} will be converted to list(number = 9). The handler function will then receive the arguments directly after unlisting, eg. number = 9. This works for direct invocations, as well as situations where the user wishes to implement behaviour specific to a trigger.

Some invocation types have their own logic for converting the event body into an R object. This is useful for say, using an R function in a Lambda behind an API Gateway, so that the R function does not need to deal with the HTML elements of the invocation. The below invocation types have custom logic implemented. Refer to the vignettes or the package website for more information.

Alternatively, user-defined functions can be provided for parsing event content and serialising results. The user can also use the identity function as a deserialiser to pass the raw event content --- as a string --- to the handler function. Refer to ?lambda_config for more information.

invocation type | implementation stage |:---------------|:---------------------| direct | API Gateway (REST) | API Gateway (HTML) | EventBridge | SNS |

Installation

The package can be installed from CRAN with:

r install.packages("lambdr")

The development version is available with:

r remotes::install_github("mdneuzerling/lambdr")

Running

In a runtime.R file, source all functions needed and then run:

{r} lambdr::start_lambda()

This runtime.R file should be executed by the Docker image containing your Lambda code.

The lambdr::start_lambda() function relies on environment variables configured by AWS. It will fail if run locally. In particular, the handler as configured by the user through AWS will determine which function handles the Lambda events. For debugging and testing, values can be provided to the function in the absence of environment variables. See ?lambdr::lambda_config for details.

Example

Consider the following runtime.R file:

```r parity <- function(number) { list(parity = if (as.integer(number) %% 2 == 0) "even" else "odd") }

lambdr::start_lambda() ```

The parity function accepts a number argument and returns its parity as a named list, for example:

```r parity(5)

$parity

[1] "odd"

parity(8)

$parity

[1] "even"

```

This function can then be placed into a Docker image. An example Dockerfile is provided below, but the key components are:

  • Start from a minimal Rocker parent image (not the Rocker tidyverse image)
  • Install package dependencies using pak, which also handles system dependencies.
  • Copy across runtime.R and any other necessary files
  • Set the handler as the CMD. The lambdr package interprets the handler as the name of the function to use, in this case, "parity". The CMD can also be set (or overriden) when setting up the Lambda in AWS.

```dockerfile FROM docker.io/rocker/r-ver:4.4

curl is required for {pak}

RUN apt-get update && apt-get -y install --no-install-recommends curl

options(warn=2) will make the build error out if package doesn't install

RUN Rscript -e "options(warn = 2); install.packages('pak')"

Using {pak} to install R packages: it resolves Ubuntu system dependencies AND

the R dependency tree. Other required packages can be installed here.

RUN Rscript -e "pak::pak('lambdr')"

RUN mkdir /R COPY runtime.R /R RUN chmod 755 -R /R

ENTRYPOINT Rscript R/runtime.R CMD ["parity"] ```

The image is built and uploaded to AWS Elastic Container Registry (ECR). First, a repository is created:

bash aws ecr create-repository --repository-name parity-lambda --image-scanning-configuration scanOnPush=true

This provides a URI, the resource identifier of the created repository. The image can now be pushed:

bash docker build --platform linux/amd64 -t r-on-lambda . docker tag mdneuzerling/r-on-lambda:latest {URI}/parity-lambda:latest aws ecr get-login-password | docker login --username AWS --password-stdin {URI} docker push {URI}/parity-lambda:latest

In either the AWS console or the command line, a Lambda can be created from this image. Call the Lambda "parity" to match the function name. Tests can be executed within the console. Alternatively the Lambda can be invoked from the CLI:

bash aws lambda invoke --function-name parity \ --invocation-type RequestResponse --payload '{"number": 8}' \ /tmp/response.json --cli-binary-format raw-in-base64-out

The output is now available in the generated file:

bash cat /tmp/response.json

bash {"parity":"even"}

A detailed guide on how to create these Dockerfiles, as well as a guide for those new to Docker, can be found in the vignettes.

Hex logo by Phizz Leeder

Owner

  • Name: David Neuzerling
  • Login: mdneuzerling
  • Kind: user
  • Location: Melbourne, Australia

Data scientist and R engineer. Also Python and Julia. He/him.

GitHub Events

Total
  • Issues event: 8
  • Watch event: 8
  • Delete event: 1
  • Issue comment event: 20
  • Push event: 6
  • Pull request event: 3
  • Fork event: 1
  • Create event: 2
Last Year
  • Issues event: 8
  • Watch event: 8
  • Delete event: 1
  • Issue comment event: 20
  • Push event: 6
  • Pull request event: 3
  • Fork event: 1
  • Create event: 2

Committers

Last synced: almost 3 years ago

All Time
  • Total Commits: 122
  • Total Committers: 2
  • Avg Commits per committer: 61.0
  • Development Distribution Score (DDS): 0.016
Past Year
  • Commits: 3
  • Committers: 1
  • Avg Commits per committer: 3.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
mdneuzerling d****d@n****m 120
David Neuzerling m****g 2
Committer Domains (Top 20 + Academic)
neuzerling.com: 1

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 31
  • Total pull requests: 13
  • Average time to close issues: 3 months
  • Average time to close pull requests: 1 day
  • Total issue authors: 20
  • Total pull request authors: 3
  • Average comments per issue: 3.1
  • Average comments per pull request: 1.23
  • Merged pull requests: 12
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 4
  • Pull requests: 5
  • Average time to close issues: 4 days
  • Average time to close pull requests: 2 days
  • Issue authors: 3
  • Pull request authors: 2
  • Average comments per issue: 1.5
  • Average comments per pull request: 1.6
  • Merged pull requests: 4
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • mdneuzerling (8)
  • jimjam-slam (3)
  • fmmattioni (2)
  • sckott (2)
  • arkadi-aigora (1)
  • grayskripko (1)
  • jimgar (1)
  • oesah (1)
  • flochaz (1)
  • mrismailt (1)
  • dwbelliston (1)
  • mpettis (1)
  • manhcuogntin4 (1)
  • rpodcast (1)
  • sebsilas (1)
Pull Request Authors
  • mdneuzerling (11)
  • jimgar (4)
  • anisfeld (2)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads:
    • cran 474 last-month
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 2
    (may contain duplicates)
  • Total versions: 12
  • Total maintainers: 1
proxy.golang.org: github.com/mdneuzerling/lambdr
  • Versions: 7
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 5.5%
Average: 5.6%
Dependent repos count: 5.8%
Last synced: 11 months ago
cran.r-project.org: lambdr

Create a Runtime for Serving Containerised R Functions on 'AWS Lambda'

  • Versions: 5
  • Dependent Packages: 0
  • Dependent Repositories: 2
  • Downloads: 474 Last month
Rankings
Stargazers count: 3.5%
Forks count: 7.9%
Average: 16.7%
Dependent repos count: 19.2%
Downloads: 24.3%
Dependent packages count: 28.7%
Maintainers (1)
david@neuzerling.com
Last synced: 11 months ago

Dependencies

DESCRIPTION cran
  • httr * imports
  • jsonlite * imports
  • logger * imports
  • knitr * suggests
  • lifecycle * suggests
  • rmarkdown * suggests
  • testthat >= 3.0.0 suggests
  • webmockr * suggests
  • withr * suggests
.github/workflows/R-CMD-check.yaml actions
  • actions/checkout v3 composite
  • actions/upload-artifact main composite
  • r-lib/actions/check-r-package v2 composite
  • r-lib/actions/setup-r v2 composite
  • r-lib/actions/setup-r-dependencies v2 composite
.github/workflows/full-check-on-release.yaml actions
  • actions/checkout v3 composite
  • actions/upload-artifact main composite
  • r-lib/actions/check-r-package v2 composite
  • r-lib/actions/setup-pandoc v2 composite
  • r-lib/actions/setup-r v2 composite
  • r-lib/actions/setup-r-dependencies v2 composite