Writing Lambda functions in Rust!

Luciano Mammino (@loige)

loige.link/lambda-rust

AWS User Group Dublin 2023-09-13

👋 I'm Luciano (🇮🇹🍕🍝🤌)

👨‍💻 Senior Architect @ fourTheorem

nodejsdp.link

📔 Co-Author of Node.js Design Patterns  👉

Let's connect!

linktr.ee/loige

loige.link/lambda-rust

Grab the slides

$ ~ whoami

Always re-imagining

We are a pioneering technology consultancy focused on AWS and serverless

loige

✉️ Reach out to us at  hello@fourTheorem.com

😇 We are always looking for talent: fth.link/careers

We can help with:

Cloud Migrations

Training & Cloud enablement

Building high-performance serverless applications

Cutting cloud costs

awsbites.com

loige

What is Rust? 🦀

  • A (relatively) new programming language

  • Most loved... for 7 years in a row!

  • Low-level, but general-purpose

  • Performant & memory-safe

loige

Why do I like it ❤️

  • A lovely mascot

  • Strongly typed with a really good type-system

  • Takes inspiration from Haskell, C++, OCaml, JavaScript, Ruby

  • Great (built-in) package manager (Cargo)

  • Good ecosystem of libraries

  • Pattern matching

  • No null, Option & Result types

loige

use std::env;

fn main() {
    let region = env::var("AWS_REGION");
}

loige

Result<String, VarError>
use std::env;

fn main() {
    let region = env::var("AWS_REGION");
    
    match region {
        Ok(region) => println!("Selected region: {}", region),
        Err(_) => println!("Error: AWS_REGION not set"),
    }
}

loige

use std::env;

fn main() {
    let region = env::var("AWS_REGION")
        .expect("AWS_REGION environment variable not set");
}

loige

String
use std::env;

fn main() {
    let region = env::var("AWS_REGION")
        .unwrap_or_else(|_| "eu-west-1".to_string());
}

loige

Rust makes it very hard for you to ignore possible errors or the absence of values.

Serverless, in a nutshell 🥜

  • A way of running applications in the cloud

  • Of course, there are servers... we just don't have to manage them

  • We pay (only) for what we use

  • Small units of compute (functions), triggered by events

loige

Serverless... with benefits 🎁

  • More focus on the business logic (generally)

  • Increased team agility (mostly)

  • Automatic scalability (sorta)

  • Not a universal solution, but it can work well in many situations!

loige

AWS Lambda

Serverless FaaS offering in AWS

Can be triggered by different kinds of events

  • HTTP Requests
  • New files in S3
  • Jobs in a Queue
  • Orchestrated by Step Functions
  • On a schedule
  • Manually invoked

loige

Some use cases 🛠️

  • HTTP APIs backend with API Gateway
  • Custom logic step in a Step Function
  • Process new S3 objects (Create picture thumbnails)
  • Scrape/Synchronise data on a schedule (Import data from an FTP)
  • Process jobs from a queue (Generate PDF invoices and send them)
  • Execute code when a business event happens (Send welcome email)
  • Define the logic to rotate secrets
  • Analyse logs and react to suspicious activities
  • etc ...

loige

(some) Limitations 😖

  • Maximum execution time is 15 minutes...
  • Payload size (request/response) is limited
  • Doesn't have a GPU option (yet)

loige

... so again, it's not a silver bullet for all your compute problems! 🔫

AWS Lambda Pricing 💸

Cost = Allocated Memory 𝒙 time

loige

aws.amazon.com/lambda/pricing/

AWS Lambda Pricing 💸

Cost = Allocated Memory 𝒙 time

loige

aws.amazon.com/lambda/pricing/

512 MB = $0.0000000083/ms

Executing a lambda for 15 mins...

0.0000000083 * 900000 = 0.007 $

AWS Lambda... what about CPU? 🙄

You don't explicitly configure it:

CPU scales based on memory

loige

loige.link/lambda-cpu

AWS Lambda... what about CPU? 🙄

You don't explicitly configure it:

CPU scales based on memory

loige

Memory vCPUs
128 - 3008 MB 2
3009 - 5307 MB 3
5308 - 7076 MB 4
7077 - 8845 MB 5
8846+ MB 6

loige.link/lambda-cpu

🏃‍♂️ Lambda execution model

  • It's serverless: it should run only when needed
  • Lambda code is stored in S3
  • event-based: an event can trigger a lambda execution
  • if no instance is available, one is created on the fly (cold-start)
  • if an instance is available and ready, use that one
  • if an instance is inactive for a while, it gets destroyed

loige

🏃‍♂️ Lambda execution model

loige

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

event (JSON)

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

response or
error

in detail

🏃‍♂️ Lambda execution model

loige

Runtime

Handler (logic)

Poll for events

event (JSON)

execute

response or
error

response (JSON)
or error

in detail

Why Rust + Lambda = ❤️

  • Performance + Efficient memory-wise = COST SAVING 🤑
  • Very fast cold starts! (proof) ⚡️
  • Multi-thread safety 💪
  • No null types + Great error primitives = fewer bugs 🐞

loige

Supported Lambda runtimes

  • Node.js

  • Python

  • Java

  • .NET

  • Go

  • Ruby

  • Custom

loige

Supported Lambda runtimes

  • Node.js

  • Python

  • Java

  • .NET

  • Go

  • Ruby

  • Custom

loige

RUST?!

[deprecated]

Rust Runtime for Lambda

loige

github.com/awslabs/aws-lambda-rust-runtime

OK, Where do we start?

loige

install cargo-lambda

Cargo Lambda

  • A third-party command for Cargo that makes it easier to author, test and deploy Lambdas in Rust
  • Mostly built by an AWS employee (@calavera)
  • It can cross-compile for Linux ARM (on Win/Mac/Linux)
  • Integrates well with SAM and CDK for IaC

loige

cargo-lambda.info

loige

use aws_lambda_events::event::s3::S3Event;
use lambda_runtime::{run, service_fn, Error, LambdaEvent};

async fn function_handler(event: LambdaEvent<S3Event>) -> Result<(), Error> {
    for record in event.payload.records {
        tracing::info!(
            "[{}] Bucket={} Key={}",
            record.event_name.unwrap_or_default(),
            record.s3.bucket.name.unwrap_or_default(),
            record.s3.object.key.unwrap_or_default()
        );
    }
    Ok(())
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing_subscriber::fmt()
        .with_max_level(tracing::Level::INFO)
        .with_target(false)
        .without_time()
        .init();

    run(service_fn(function_handler)).await
}

loige

loige

Event & Context

loige

async fn function_handler(event: LambdaEvent<S3Event>)
  -> Result<(), Error> {
    // let event = event.payload;
    let (event, ctx) = event.into_parts();
    println!(
        "This execution will expire at {}", 
        ctx.deadline
    );

    for record in event.records {
        // ...
    }
    Ok(())
}

loige

Request & Response types

loige

async fn function_handler(event: LambdaEvent<S3Event>) 
  -> Result<(), Error> {
    // ...
    Ok(())
}

loige

Request

Response

What if we want to use different types? 🤨

Option 1

use type definitions in the aws_lambda_events crate

loige

loige

Processing jobs from SQS

loige

Example

Jobs

Event source mapper

# Cargo.toml

[dependencies]
aws_lambda_events = { 
  version = "0.10.0",
  default-features = false,
  features = [
    "sqs",
  ]
}

loige

use aws_lambda_events::event::sqs::{BatchItemFailure, SqsBatchResponse, SqsEvent};
// ...

async fn function_handler(event: LambdaEvent<SqsEvent>)
  -> Result<SqsBatchResponse, Error> {
    let mut failed_jobs = Vec::with_capacity(event.payload.records.len());

    for record in event.payload.records {
        // process the job
        // ...
        // if the job failed, add it to the failed_jobs list
        failed_jobs.push(BatchItemFailure {
            item_identifier: record.message_id.unwrap_or_default(),
        });
    }

    Ok(SqsBatchResponse {
        batch_item_failures: failed_jobs,
    })
}

// ...

loige

Option 2

Create custom request and response types

loige

loige

Custom logic in Step Function

Example

loige.link/fsb-refactor

# Cargo.toml

[dependencies]
serde = "1.0.183"
serde_json = "1.0.104"

loige

// ...

#[derive(serde::Deserialize)]
struct Request {
    url: String,
}

#[derive(serde::Serialize)]
struct Response {
    issue_number: u32,
}

async fn function_handler(event: LambdaEvent<Request>) 
  -> Result<Response, Error> {
    println!("I am going to scrape {}", event.payload.url);
    // TODO: actual scraping logic here
    Ok(Response { issue_number: 333 })
}

// ...

loige

loige

Option 3

Use arbitrary JSON values!

loige

// ...

async fn function_handler(
    event: LambdaEvent<serde_json::Value>,
) -> Result<serde_json::Value, Error> {
    let url = event
        .payload
        .as_object()
        .unwrap()
        .get("url")
        .unwrap()
        .as_str()
        .unwrap(); // 🤮
    println!("I am going to scrape {}", url);
    // TODO: actual scraping logic here
    Ok(serde_json::json!({ "issue_number": 333 }))
}

// ...

loige

HTTP-based lambdas

loige

loige

use lambda_http::{run, service_fn, Body, Error, Request, RequestExt, Response};

async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
    // Extract some useful information from the request
    let who = event
        .query_string_parameters_ref()
        .and_then(|params| params.first("name"))
        .unwrap_or("world");
    
    let message = format!("Hello {who}, this is an AWS Lambda HTTP request");

    // Return something that implements IntoResponse.
    // It will be serialized to the right response event 
    // automatically by the runtime
    let resp = Response::builder()
        .status(200)
        .header("content-type", "text/html")
        .body(message.into())
        .map_err(Box::new)?;
    Ok(resp)
}

loige

use lambda_http::{run, service_fn, Body, Error, Request, RequestExt, Response};

async fn function_handler(event: Request) -> Result<Response<Body>, Error> {
    // Extract some useful information from the request
    let who = event
        .query_string_parameters_ref()
        .and_then(|params| params.first("name"))
        .unwrap_or("world");
    
    let message = format!("Hello {who}, this is an AWS Lambda HTTP request");

    // Return something that implements IntoResponse.
    // It will be serialized to the right response event 
    // automatically by the runtime
    let resp = Response::builder()
        .status(200)
        .header("content-type", "text/html")
        .body(message.into())
        .map_err(Box::new)?;
    Ok(resp)
}

loige

These are just abstractions! 🧐

Lambda is still using JSON behind the scenes.

For HTTP you generally use the Lambda-Proxy integration.

Building & Deploying

cargo lambda build --release && cargo lambda deploy

loige

loige

loige

NO TRIGGER CONFIGURED! 🙄

SAM

loige

Serverless Application Model

IaC with...

AWS SAM

  • YAML-based Infrastructure as code (IaC) tool focused on serverless apps
  • Great when you have to go beyond just one lambda
  • ... or when you need more advanced integrations
    (e.g. API Gateway)
  • It supports everything that is natively supported with CloudFormation, but with a slightly simpler syntax
  • Deploys through CloudFormation!

loige

aws.amazon.com/serverless/sam

# template.yaml

AWSTemplateFormatVersion : '2010-09-09'
Transform:
  - AWS::Serverless-2016-10-31

Description: |
  A sample Serverless project triggered from S3 CreateObject events
Resources:
  ExampleFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs18.x
      Handler: index.handler
      Events:
        S3CreateObject:
          Type: S3
          Properties:
            Bucket: !Ref MyPhotoBucket
            Events: s3:ObjectCreated:*

  MyPhotoBucket:
    Type: AWS::S3::Bucket

loige

AWS SAM + Cargo Lambda

SAM Works with Cargo Lambda (beta feature):

  • Define IaC with the full power of SAM
  • Build and run your Rust lambdas with Cargo Lambda
  • Can simulate API Gateway locally!

loige

Note: Cargo Lambda also works with CDK
(github.com/cargo-lambda/cargo-lambda-cdk)

# template.yaml

AWSTemplateFormatVersion: "2010-09-09"
Transform:
  - AWS::Serverless-2016-10-31

Resources:
  ExampleHttpLambda:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: rust-cargolambda
    Properties:
      CodeUri: .
      Handler: bootstrap
      Runtime: provided.al2
      Architectures:
        - arm64
      Events:
        HttpPost:
          Type: Api
          Properties:
            Path: /
            Method: get

loige

# samconfig.toml

version = 0.1

[default]
[default.global]
[default.global.parameters]
stack_name = "rust-http-lambda"

[default.build.parameters]
beta_features = true
[default.sync.parameters]
beta_features = true

Tells SAM to build using Cargo Lambda

Selects a "custom runtime"

Defines an HTTP trigger
(API Gateway)

Enables SAM beta features

Building, Local testing & Deploying

 

sam build
sam local start-api
sam deploy

loige

loige

loige

Closing notes

  • Lambda is great (most of the time)
  • Writing Lambdas in Rust is fun and it can be very cost-efficient
  • Still not very common to write Lambdas in Rust, but the tooling is already quite good (Cargo Lambda + SAM)
  • Go, have fun, share your learnings!

loige

BONUS: SAM + Cargo Lambda
a complete example

loige

github.com/lmammino/earthquake-notifier

Cover photo by Ahmed Sobah on Unsplash

Thanks to @gbinside, @conzy_m, @eoins, and @micktwomey for kindly reviewing this talk!

THANKS!

nodejsdp.link

loige

loige.link/lambda-rust

Grab these slides!