Daily Resource For Devs

Introduction In our development environment, besides being easy and fast to use, above all, we need a lightweight version of it to run locally, right? I have tested different Oracle images and this was the option that best work for me. In this post i will show you how you can run the Oracle Database Express Edition version on your local machine quickly and easily and also how to connect to it using Oracle SQL Developer IDE and SQL*Plus. How to do it Basically we just need two simple steps to get our Oracle database running on our local environment. First we need to login to the Oracle Container Registry so we can pull the image from it and then we just need to run it in a container. You can find more information by going to the Oracle Container Registry , under the "Browse Containers", click into Database and next click in the express Repository for the Oracle Database Express Edition. I am going to skip the docker pull step, because docker run will do it for us ...

Introduction As a software engineer, we are quite used to deal with logs in our daily lives, but in addition to ensuring that the necessary logs are being sent by the application itself or through a service mesh, we often have to go a little further and interact with some log tool to extract more meaningful data. This post is inspired by a problem I had to solve for a client who uses Splunk as their main data analysis tool and this is the first in a series of articles where we will delve deeper and learn how to use different Splunk commands. Running Splunk with Docker To run Splunk with docker, just run the following command: docker run -d —rm -p 8000:8000 -e SPLUNK_START_ARGS=--accept-license -e SPLUNK_PASSWORD=SOME_PASSWORD --name splunk splunk/splunk:latest Sample Data We are going to use the sample data provided by Splunk. You can find more information and download the zip file from their web site . How does it work? In order to be able to interact with Splunk t...

Introduction In this post we are going to see how we can create a Spring Configuration Server to externalize our Spring Boot application configuration files. In order to do that we need one Spring Boot application running as a configuration server, one Spring Boot application running as a configuration client and one external repo like GitHub to keep our configuration yml files. Config Server Let´s start by going to the Spring Initializr web site to create a Spring Boot project. For this project we just need one dependency, the Config Server from Spring Cloud Config as shown in the image below: This is all you need. You can go ahead and click in generate to download your Spring Boot project. Next, open your application.yml file and add the code below. We just need to add this configuration to let Spring know where to get our yml configuration files. Under repos we can add configurations for all the environments we need. I am just adding a section for my developme...

Introduction In this post we are going to see how we can create a REST API pagination in Spring Boot with Spring HATEOAS and Spring Data MongoDB . For basic queries, we can interact with MongoDB using the MongoRepository interface which is what we are going to use in this tutorial. For more advanced operations like update and aggregations we can use the MongoTemplate class. With Spring applications we start adding the needed dependencies to our pom file if using Maven as our build tool. For this project we are going to use the following dependencies: Spring Web , Spring Data MongoDB and Spring HATEOAS . To quickly create your Spring Boot project with all your dependencies you can go to the Spring Initializr web page. This is how your project should look like: As with any MVC application like Spring there are some minimal layers that we need to create in our application in order to make it accessible like the Controller , Service , Model and Repository layers . For this...

Introduction In this post you are going to learn the core principles of Kubernetes and how to create a k8s cluster in Google Cloud Project Creation and Confiuration The first thing you need to do on GCP in order to create a Kubernetes cluster is to create a project so you can later be able to add services to it. So, go to your Google Cloud Console and create a new project now if you haven´t done it yet. With our project created we need to enable the Kubernetes Engine API for the project so we can start using the gcloud client to configure and create our cluster. So, those are the steps we will need to cover after our project creation: 1. Enable Kubernetes Engine API in the Google Cloud Console 2. Install gcloud client 3. Install kubectl client 4. Logging to GCP 5. Setting a Project ID 6. Creating a Cluster 7. Creating a Pod 8. Creating a Service 9. Increase replica count 1. Enable Kubernetes Engine API in the Google Cloud Console Go to th...

If you need an in-memory database through REST API that can load and aggregate external data into a single api with support for persistence to a json file, a middleware hook ready to log api requests while also updating your relational database, you might find this post interesting. We all know database operations are costly and is always a good idea to have some sort of cache of it near our frontend application for faster access and operations and is also much better to have to interact with it through a rest api without the need to interact with all that drivers and connections stuffs. What problem does i think it solve? This post is going to be a long one. I am going to show and talk about my open source project of an in-memory rest database made in NodeJS with no addition of external libraries so ever, just raw NodeJS to handle all API´s requests, that can be used for cache as a service, mock, database aggregator for a rest service and can even update your database with out-o...

If you are running your applications on a microservices architecture you might want to trace which instances are getting called, how often and which one may have performance issues. In this post we are going to add a distributed tracing to monitor our Quarkus application so we can easily identify which instances are being called. In this post we will see how to log all received requests by adding Jaeger as the Distributed Tracing system of an existing Quarkus application. First we need to add the smallrye-opentracing extension to our project by running the Quarkus add-extension command: Hopefully, you should get a success message printed out in your terminal telling you that your extension has been installed and as usual we now need to import our new maven dependency into the project. The next thing we need to do is configure our application properties file to set, among other things, the name of the service for the Jaeger UI in which all of our logs will be saved to. ...

If you came from a Java background you may be interested to know how you can build a rest api in kotlin. In this post we will create one using Ktor , but wait, what is ktor? According to their web site: Ktor is an asynchronous framework for creating microservices, web applications, and more. It’s fun, free, and open source. Just like any Java Spring Boot Application, Ktor also has its own Application file . So lets start from it by comparing a spring boot application file with a Ktor. A Ktor application file: As you can see, both files have a main method that receives as an argument an array of strings, but Ktor uses its main method to create an embedded netty application. In a Spring Application we would create a controller class responsible to handle all http requests to an endpoint by making use of java annotations. In Ktor, we need to create a kotlin file to represent our Application Routes by using the Ktor Route class . As you can see we make use of the route metho...

From the introduction of the openpolicyagent.org site: OPA generates policy decisions by evaluating the query input against policies and data. In this post i am going to show you an easy and fast way to test your policies by running OPA in Docker. First, make sure you have already installed Docker and have it running: docker ps Inside your choosen directory, create two files. One called input.json file for your system representation and one file called example.rego for your rego policy rules. Add the following content to your json file: Add the following content for the example.rego: Each violation block represents the rule that you want to validate your system against. The first violation block checks if any of the system servers have the http protocol in it. If that is the case, the server id is added to the array. In the same way, the second violation block checks for the servers that have the telnet protocol in it and if it finds a match the server id is also...

An http interceptor can become handy if you need to log all your requests or add a bearer token to every request when calling a protected resource with an access token in OAuth 2.0 . To create an interceptor you basically need to create an OkHttpClient by calling the OkHttpClient Builder and add an interceptor to it. OkHttpClient okHttpClient = new OkHttpClient.Builder() You can have access to the current request by calling chain.request(). Once you get the request you can then add or build on top of it by adding an additional header to send your Bearer Token. Request auth_request = chain.request() The last step is to return a Response object by calling chain.proceed, passing the new request. return chain.proceed(auth_request); And the final code: OkHttpClient okHttpClient = new OkHttpClient.Builder() .addInterceptor(new Interceptor() { @Override public Response intercept(Chain chain) throws IOException { Request auth_request = cha...

To save your data in RedShift you first need to download the pg module because in fact RedShift is a Postgres database. npm install pg If you have not already done so, install the amazon sdk with the following command: npm install aws-sdk We can now write our first lines of code to import these modules into our file: var AWS = require('aws-sdk'); const { Pool, Client } = require('pg'); Creating a connection pool with the database: let pool = new Pool({ user: user, host: 'your-endpoint.redshift.amazonaws.com', database: database-name, password: 'your-password', port: 5439, }); You need to create a table in the RedShift public schema with a column for each property in your json file. Your file must be in RJSON (relaxed json) format, that is, without commas between your objects. Let's create our table in RedShift with the following sql command : CREATE TABLE public.product ( title varchar(256) NOT NULL, brand var...

Amazon ES Service is a fully managed system that makes it easy to deploy Elastic Stack to AWS servers in an integrated way. Some features like installing Kibana plugins are not yet available. ElasticSearch is part of the Elastic Stack, a group of tools/services from the Elastic Company (elastic.co) Elastic Stack: * Kibana * ElasticSearch * Beats * Logstash ElasticSearch is a NoSQL document database and is most common used with Kibana , a UI tool to visualize data from the ES database. ElasticSearch is used for high speed text search. It was previously known as the ELK Stack because of the tools/services from Elastic Company that are used togheter: * ElasticSearch * Logstash * Kibana In elasticsearch db you post data to an index in the same way you insert data into tables in a RDBMS (Relational Database Management Systems). We use index to separate and group different types of information (data) in the same way we use tables in a database. SO INDEXES ARE FOR ES...

If you are having a 403 Forbidden error from CloudFront , that means your domain name is not linked to your CloudFront distribution and because CloudFront stays in front of your API GATEWAY you need to create a CNAME record pointing your domain name to your CloudFront target domain name in order for it to work. So, If you need to point your api to a custom domain name, all you have to do is following those 2 easy steps: 1 - CREATING YOUR CUSTOM DOMAIN NAME Go to the API GATEWAY console and click on the Custom Domain Name menu. Click on the Create Custom Domain Name button. Next, assign a certificate matching the same domain name you are creating and map to the root path and destination of your desired api. Lastly, copy the CloudFront Target Domain Name . You will need to paste that in your Route 53 record. 2 - CREATING A CNAME RECORD ON ROUTE 53 Create a CNAME record on Route 53 for the same custom domain name, assigning to it the CloudFront Target Domain. CONCLUSION ...

In this post i am going to show you how you can get into the content of the file that trigged your lambda. As we have seen in the previous post, the event object is passed as an argument to our lambda function, like this: exports.handler = (event, context, callback) => { }; When you place a trigger on a Lambda function based on a S3 event, S3 will pass all the context information that you need inside of the event object. In this case, our lambda will be trigged by an S3 event and the event object that will be passed to it will have a Record property with an array of objects inside of it like is shown below: { "event":{ "Records":[ { "eventVersion":"2.0", "eventSource":"aws:s3", "awsRegion":"us-east-1", "eventTime":"2018-09-22T14:25:20.411Z", "eventName":"ObjectCreated:Put...

With any kind of http request, being it a simple web server or an api gateway, we will usually need to log different kind of information regarding the received request. But how can we do that with Lambda and how can we get all available information from it? Thats when the event object comes in place. The API GATEWAY will send all information about the request encapsulated in the event object passing it as an argument to our lambda function, like this: exports.handler = (event, context, callback) => { }; The event object have the following nested objects as shown below: 1. resource 2. path 3. httpMethod 4. headers 5. multiValueHeaders 6. queryStringParameters 7. multiValueQueryStringParameters 8. pathParameters 9. stageVariables 10. requestContext 11. body 12. isBase64Encoded { "resource": "/", "path": "/", "httpMethod": "GET", "headers": {}, "multiValueHe...

If you stringify the event you will notice that the queryStringParameters is actually an object rather than an array. So in order to get all queryStrings params passed in an API Gateway request you can simply make a for each key value pair in the object like this: var element = {}; for (var key in event.queryStringParameters) { element[key] = event.queryStringParameters[key]; }

In this post, we are going to see how to bring data from mysql to RDS. The first thing you gonna need to do is a dump of your db to a sql file so we can than bring it into the RDS. So, here we go. If you have not done it already go to the RDS console and create or launch a new mysql db instance. Take note of the following infos: db name, db user, db passwd and the db endpoint. We gonna use all that with mysql command to bring all tables of our mysql into the RDS. So, next we need to create our dump file. Log into your instance to get access to mysql cli. You can login like this: ssh -i file.pem user@IP To get inside mysql cli: mysql -u root Create a dump of your database: mysqldump -u root db_name > file.sql This will create a dump of your db on your user location. Exit the mysql cli to check if the file was created. We can take advantage of the mysql cli of our previously db to run our last command to bring the file into our new RDS instance. mysql -h yo...

One of these days i had to rename a resource name on the AWS API GATEWAY, so here are the steps needed. In order to change your resource name you need to run 3 commands on the cli. One to get your API ID, one to get the Resource ID and one last command to change its name. First lets gonna find the ID of the desired API. Using the aws cli type the following command: aws apigateway get-rest-apis The API will return a json with an array of items containing all of your apis. Look into the name property to find the api that have the resource you need to change. Right above the name property you have the id property of your api. Take note of this id and run the following command with it to get the resource id. aws apigateway get-resources --rest-api-id 1010101010 Get the id matching the resource path that you are looking for to run the final command thats going to change your resource name. aws apigateway update-rest-api --rest-api-id 1010101010 --patch-operations op=replace,pat...