Skip to main content

Getting to know the S3 Event Object

In this post i am going to show you how you can get into the content of the file that trigged your lambda.

As we have seen in the previous post, the event object is passed as an argument to our lambda function, like this:


exports.handler = (event, context, callback) => {

};

When you place a trigger on a Lambda function based on a S3 event, S3 will pass all the context information that you need inside of the event object. In this case, our lambda will be trigged by an S3 event and the event object that will be passed to it will have a Record property with an array of objects inside of it like is shown below:

{
   "event":{
      "Records":[
         {
            "eventVersion":"2.0",
            "eventSource":"aws:s3",
            "awsRegion":"us-east-1",
            "eventTime":"2018-09-22T14:25:20.411Z",
            "eventName":"ObjectCreated:Put",
            "userIdentity":{
               "principalId":""
            },
            "requestParameters":{
               "sourceIPAddress":""
            },
            "responseElements":{
               "x-amz-request-id":"",
               "x-amz-id-2":""
            },
            "s3":{
               "s3SchemaVersion":"1.0",
               "configurationId":"",
               "bucket":{
                  "name":"",
                  "ownerIdentity":{
                     "principalId":""
                  },
                  "arn":""
               },
               "object":{
                  "key":"",
                  "size":226,
                  "eTag":"",
                  "versionId":"",
                  "sequencer":""
               }
            }
         }
      ]
   }
}

You will want to pay a special attention to the S3 property here. This property contains an object that have two important properties for us, the bucket and the key. Using those information, that is, the bucket name and the file name, that we can get into the content of our S3 file like this:


var bucket = event.Records[0].s3.bucket.name;
var file = event.Records[0].s3.object.key;

var params = {
  Bucket: bucket, 
  Key: file
 };
 s3.getObject(params, function(err, data) {
   if (err) console.log(err, err.stack);
   else     console.log(JSON.parse(data.Body.toString('utf-8'))); 
 });

This will return a data object containing your object properties, assuming you hava a json object on it. The JSON.parse() here is used to convert it back to an object, instead of having it as string.

Comments

Popular posts from this blog

Log Aggregation with ELK stack and Spring Boot

Introduction In order to be able to search our logs based on a key/value pattern, we need to prepare our application to log and send information in a structured way to our log aggregation tool. In this article I am going to show you how to send structured log to ElasticSearch using Logstash as a data pipeline tool and how to visualize and filter log information using Kibana. According to a definition from the Wikipedia website: Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. According to Elasticsearch platform website , Elasticsearch is the heart of the Elastic stack, which centrally stores your data for lightning fast search. The use of Elasticsearch, Kibana, Beats and Logstash as a search platform is commonly known as the ELK stack. Next we are going to start up Elasticsearch, Kibana and Logstash using docker so we can better underst...

How to create a REST API Pagination in Spring Boot with Spring HATEOAS using MongoDB

Introduction In this post we are going to see how we can create a REST API pagination in Spring Boot with Spring HATEOAS and Spring Data MongoDB . For basic queries, we can interact with MongoDB using the MongoRepository interface which is what we are going to use in this tutorial. For more advanced operations like update and aggregations we can use the MongoTemplate class. With Spring applications we start adding the needed dependencies to our pom file if using Maven as our build tool. For this project we are going to use the following dependencies: Spring Web , Spring Data MongoDB and Spring HATEOAS . To quickly create your Spring Boot project with all your dependencies you can go to the Spring Initializr web page. This is how your project should look like: As with any MVC application like Spring there are some minimal layers that we need to create in our application in order to make it accessible like the Controller , Service , Model and Repository layers . For this...

Understanding RabbitMQ

Introduction RabbitMQ is a centralized message broker based on the AMQP (Advanced Message Queuing Protocol) protocol, acting as a Middleware between Producers and Consumers of different systems. In a message system, Publishers sends a message to a message broker where messages are consumed some time later by one or more Subscribers. By introducing a message brokeer between systems we are decoupling the sender application from the receiver. In this case the service that is responsible for sending or publishing the message does not need to know about any other service. All it needs to care about is the message and its format. With a message system you send a message to a message broker first and when the consumers or listeners of it become online they can start consuming from the message queue. This means you can keep sending messages without even care if the other application is online or if they had any failures. RabbitMQ Architecture Exchange, queue and bindings are the ...