Let’s revisit our pipeline once again.

We want to send notifications from S3 to our Lambda whenever we put a file into our S3 bucket, and in this tutorial, we are using AWS CDK in Typescript to achieve that.

1. Ensure logging within the AWS Lambda

This is important to prove that the Lambda successfully receives the S3 notification from the bucket, and we can trace it within AWS Cloudwatch.

import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)

def lambda_handler(event, context):
    # Outputs the incoming event into CW logs
    logger.info(event)
    
    return {
        "statusCode":200
    }
# For direct invocation and testing on the local machine
if __name__ == '__main__':
    print(lambda_handler(None,None))

Otherwise, there is no way we would know if our pipeline is working.

2. Exporting the Lambda function as an object

For the S3 notifications to work, the S3 bucket needs to know which lambda function to send its notifications over.
This is one of the main strengths of AWS CDK where we can export our Lambda function as an object for other infrastructures to leverage on.

In lib/basic-lambda-stack.ts:

import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';
export class basicLambdaStack extends cdk.Stack{
  // Making the object accessible for reuseability
  public readonly lambdaFunction: lambda.Function;

  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);
    const function_name = 'gefyra-basic-lambda';
    const lambda_path = 'src/lambda/basic_lambda';

    // Initialization of the lambda function
    this.lambdaFunction = new lambda.Function(this, function_name, {
        functionName: function_name,
        runtime: lambda.Runtime.PYTHON_3_8,
        code: lambda.Code.fromAsset(lambda_path),
        handler: "lambda_function.lambda_handler"
    });
  }
}

As you can see:

  • Within the class itself, we would expose an object via public readonly lambdaFunction: lambda.Function;
  • Use the same object to initialize the Lambda function.

From the above code snippet, we allow our Lambda function to be reused within the stack.

3. Get the S3 bucket ready for the Lambda

Install @aws-cdk/aws-s3-notifications with npm install @aws-cdk/aws-s3-notifications. Make sure to update all your AWS CDK libraries at the same time to avoid conflicts and deployment errors.

We are going to modify the lib/s3-bucket-stack.ts to receive the Lambda object and attribute S3 event notifications to the Lambda.

import * as cdk from '@aws-cdk/core';
import * as s3 from "@aws-cdk/aws-s3";
import * as s3n from '@aws-cdk/aws-s3-notifications';
import * as lambda from '@aws-cdk/aws-lambda';

// Allows the stack to receive a lambda.function object
export interface S3Props extends cdk.StackProps{
  readonly lambdaFunction: lambda.Function;
}

export class S3BucketStack extends cdk.Stack {
  public readonly bucket: s3.Bucket;

  // Replace the props with our new interface
  constructor(scope: cdk.Construct, id: string, props: S3Props) {
    super(scope, id, props);

    // The code that defines your stack goes here
    this.bucket = new s3.Bucket(this, "gefyra-data-collection-dev",{
      versioned: false,
      bucketName: "gefyra-data-collection-dev",
      publicReadAccess: false,
      blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
      removalPolicy: cdk.RemovalPolicy.DESTROY
    });

    // Assigning notifications to be sent to the Lambda function
    this.bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(props.lambdaFunction));
  }
}  

So let’s break this down one by one.

Creating an interface to receive the Lambda function object

// Allows the stack to receive a lambda.function object
export interface S3Props extends cdk.StackProps{
  readonly lambdaFunction: lambda.Function;
}

The above code snippet creates a dependency between the Lambda stack and the S3 stack. This also breaks the bin/gefyra-cdk-demo.ts script where we run our code to deploy the infrastructure because it is now expecting a lambdaFunction object as input.

We will rectify it later.

Assign the newly-created interface as part of the constructor

  // Replace the props with our new interface
  constructor(scope: cdk.Construct, id: string, props: S3Props){

Typescript is strongly-typed, and when we assign our interface as part of the constructor, the props object will not accept any objects that are foreign to the interface.

This reduces human error where a developer might wrongly attribute an incorrect class or object to the variable.

Attribute the S3 notifications to the Lambda function.

    // Assigning notifications to be sent to the Lambda function
    this.bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(props.lambdaFunction));

With the s3.EventType.OBJECT_CREATED, every time a user puts a file within the bucket, it would send a notification to the Lambda function that we assign to above.

Also, you can fine-tune your event notifications with s3.EventType.OBJECT_REMOVED, or any other configuration based on your use case. We will only use this generic event type here.

We don’t have to worry about IAM permissions between the S3 bucket and the Lambda because AWS CDK would do that for us. That’s the beauty of AWS CDK, and it removes the complexity of finding out the necessary permissions between resources.

However, fine-tuning permissions is still the best practice for security.

4. Prepare the executable script for deployment

In the bin/gefyra-cdk-demo.ts:

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from '@aws-cdk/core';
//import { GefyraCdkDemoStack } from '../lib/gefyra-cdk-demo-stack';
import { S3BucketStack } from '../lib/s3-bucket-stack';
import { basicLambdaStack } from '../lib/basic-lambda-stack';

const app = new cdk.App();
//new GefyraCdkDemoStack(app, 'GefyraCdkDemoStack');

// Deploying basic Lambda function
const basic_lambda_stack = new basicLambdaStack(app, 'basicLambdaStack');

// Creating an S3 bucket stack
const s3_bucket_stack = new S3BucketStack(app, 'gefyraS3Stack', {
  lambdaFunction: basic_lambda_stack.lambdaFunction
});

// Re-using assets
const bucket = s3_bucket_stack.bucket;

As previously indicated, all files within the lib folder are scripts that initialize the infrastructure, and all files within the bin folder execute the deployment.

Nothing much changed with the basic_lambda_stack object. However, you’ll realize that the s3_bucket_stack now requires a Lambda function object as input. Typescript will throw an error if you missed that out.

We can now deploy our infrastructure.

5. Deploying the pipeline

When you run npm run build && cdk synth, you’ll face a decision of what to deploy first.

If you’re don’t know AWS CDK, you would be a little worried because the order of operations could mess up deployment sequences. For S3 notifications to work, it would need an existing Lambda ARN to use.

However, AWS CDK doesn’t care. If we deploy the S3 bucket first, AWS CDK is smart enough to deploy the Lambda function first before deploying the S3 bucket.

And if you deploy the Lambda function first, it wouldn’t create conflicts if we deploy the S3 bucket separately.
We can see visual proof that the S3 bucket has successfully linked itself to the Lambda function:

6. Testing the pipeline with a file.

  1. Drop a file into the S3 bucket.
  2. Check the Cloudwatch logs for that Lambda function to see if the notification was successful.

You should be able to see that the Lambda function receives a notification from S3 whenever a file was dropped into the bucket.

Pro-Tip

It’s a hassle to keep testing the Lambda by putting files into the S3 bucket all the time if we know it was going to work.

Saving the S3 notification into a JSON file could ease testing practices, but the Cloudwatch log uses single quotes instead of double quotes within the payload. This doesn’t allow us to save the payload by copying and pasting.

There is a quick and simple way to fix that within your browser:

  1. Open the console of your browser (I tested with Chrome and Firefox).
  2. Type console.log(JSON.stringify(<copy and paste the S3 notification here>));

It would return a proper JSON string where you can save it as a file or reuse it as a test case within the AWS console.

When we create a tutorial for unit and/or integration tests in the future, we can mock the payload with the same JSON file.

Git Repo Reference

All the above code can be found in my repo: https://github.com/jonathan-moo/gefyra-cdk-demo/tree/CDK-T4

Related: is an obelisk an asherah pole, lettre d’information du syndic enedis, ted williams signed baseball, homestuck class personalities, cia paramilitary operations officer age limit, what does cumulative damage on an iowa title mean, mk11 hardest characters, keene, ca haunted hospital, brianna maitland skull, aftermarket steering wheel laws qld, does sweet sweat cause cancer, maryland driver’s license restriction card, summit medical group livingston lab phone number, old ged score conversion chart, apartments under $500 in ocala, fl,Related: notre dame assistant coaches salaries, used crownline boats for sale, colin montgomerie current wife, as wavelength increases what happens to the energy, jefferson county tn septic records, abeka consumer math quiz 17, southwestern baptist theological seminary job board, world record for stabbing a bear, mongols texas shooting, 5 letter words with arm in them, wrko advertisers list, apartments in germantown, tn under $700, publish fictitious business name in newspaper, boris becker and steffi graf relationship, zaxby’s pay period,Related: portable battery operated cash register, used hydrovane wind vane for sale, certainteed landmark driftwood vs weathered wood, analyseur syntaxique d’une phrase en ligne, 2007 state quarter errors, if i hadn’t met you ending explained, doubletree event pricing, disadvantages of salt water bath, les neuf regions de l’abdomen et leurs organes, is robinson pond hudson nh open, knoxville tn obits last 3 days, why is stockton called mudville, why did bazzini restaurant closed, eclipse rp fear roleplay rule, interactive authagraph world map,Related: how many children did carol burnett have, double cross vodka vs grey goose, scavenger hunt clues for adults around the house, nathaniel jackson obituary, mnsu student hockey tickets, intake court kenosha, wi, ryan clark house baton rouge, tamika palmer buys house and bentley, crosman c11 magazine problem, leticia gardner and samantha gibson, nightbird singer dies, edison, nj police news, rs3 do augmented items degrade, the most dangerous game freytag, 14 year old boy falls from ride full video,Related: silha funeral homes obituaries, michael norell health, sersi boyfriend eternals, how to connect mp3 player to computer windows 11, derick amos madden obituary, albuquerque men’s baseball league, how to clean skip hop activity center seat, write at least six sentences describing any type of parade, porter airlines jobs, savour restaurant menu, little nomad net worth 2020, rancho cucamonga dmv driving test route, smith funeral home lancaster, ohio obituaries, water dragon jutsu words, bob mortimer family,Related: data science book by zeeshan ul hassan, hc one notice period, how many calories in 2 scrambled eggs, onee max kangvape charger, condos for sale at the lodge no wildwood, nj, sandwell council parking fines contact number, bartow housing authority application, what did mclean stevenson do after mash, prince george’s county police news, knox county inmate messaging, was fred thompson ever on gunsmoke, kim plath naturopathic doctor education, shooting in concord, nh today, long beach transit 131 bus schedule, stearns county public defender,Related: munchkin bottle warmer button not working, john y brown net worth 2020, mexico national team chicago, add email alias in active directory powershell, sam carlson port protection bio, bud’s meat market on cullen menu, lisa harris allen ex husband, inverwood golf course 3 year deal, how to stop a cuckoo clock from cuckooing at night, nitro multi species boats for sale, michelle joy cannons net worth, callaway county warrants, police incident in derby today, richard silva obituary, jacob jones obituary 2022,
3 2 votes
Article Rating