ClouDigest ClouDIY ClouDebrief

ClouDIY: AutomatIaC

-

                                             Run Infra as Code with Jenkins


Let us distribute this process into two parts.

Part-1: We are going to install Jenkins on EC2 with one click. Then we are going to install plugins in Jenkins. Finally we will run a cloudformation from github in Jenkins.

Part-2: We are going to install AWS CLI into jenkins, then learn about Jenkinsfile and then create a pipeline to deploy cloudformation from GIT.


Part:1

Search for ‘Jenkins’ in AWS Marketplace products. Choose an AMi according to the OS you want. I chose one with Ubuntu in it.

Now quickly choose the required settings and instance configuration that you need and launch it. We chose t2.micro as it is free.

Our EC2 instance has started to spin up. Rename it to avoid confusion.

SSH into this instance. To get password to login into jenkins:

cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the public ip of instance.

And paste it on the browser following the 8080 port. That is ip_address:8080.

Enter the password here that you got after using the cat command in the above steps.

Later you will be prompted to install the plugins page. We will install default plugins for now and install the plugins we need later.

After the plugins are installed. Enter username password and other details.

Now you will be prompted to this  page. 




To install the plugin that we need.

Click on Manage Jenkins>>Manage Plugins>> Available and then search for ‘Cloudformation’.Install this plugin. 

Now let us run cloud formation from github into jenkins.

For this we have created a file in json format. Let us look into it first.

```

{

    "AWSTemplateFormatVersion": "2010-09-09",

    "Resources": {

        "S3Bucket": {

            "Type": "AWS::S3::Bucket"

        }

    },

    "Outputs": {

        "BucketName": {

            "Value": {

                "Ref": "S3Bucket"

            },

            "Description": "Name of the sample Amazon S3 bucket."

        }

    }

}

```

Here's what the different parts of the template do:

"AWSTemplateFormatVersion": This specifies the version of the CloudFormation template format being used. In this case, it's the 2010-09-09 version.

"Resources": This section contains a list of resources that will be created when the CloudFormation stack is deployed. In this template, there is only one resource, an S3 bucket.

"S3 Bucket": This is the logical ID of the S3 bucket resource. The "Type" attribute specifies the type of AWS resource to create, in this case, an S3 bucket.

"Outputs": This section contains a list of output values that can be retrieved from the CloudFormation stack after it has been created.

"BucketName": This is the logical ID of the output value. The "Value" attribute is set to the "Ref" function, which retrieves the value of the S3 bucket's logical ID. This means that the output will contain the name of the S3 bucket that was created by this CloudFormation stack. The "Description" attribute provides a description of the output value.

Overall, this CloudFormation template creates an S3 bucket resource and outputs the name of the bucket. When the CloudFormation stack is created from this template, the S3 bucket will be provisioned and the bucket name will be available as an output value.



Let us get back to jenkins and create a freestyle project using this json file.

Click on create job>>Freestyle Project and name your project.

Now, it has to get code from github repo.

Choose ‘Git’ in ‘Source Code Management’ for this to happen.

Copy paste this repo URL.

In credentials add your github credentials.

Now in ‘Build Environment’ choose ‘Create AWS Cloud Formation Stack’. This option is up there because of the plugin that we installed before.

Fill the details accordingly.

In ‘Cloud Formation recipe file/S3 URL. (.json)’ section give your json file name from github repo.

Here you need to enter Access Key and Secret Key.

The need for these keys for Jenkins is to call Cloudformation API.

Let us head towards AWS to get Access Key and Secret Key

.

Goto IAM>>Users>>Add user from AWS Console.

Name the user.

Now, attach a policy to the user.

You need to attach ‘CloudFormation FullAccess’ and ‘S3 bucket FullAccess’ policies to it.

Check the policies attached and Create User.

Open the user that you just created and create an Access Key.

Copy the Access Key and Secret Key.

And paste the Access Key and Secret Access Key here.



Now save the project. Then select ‘Build Now’ to build the project.

You can see logs and the project has been built.Also in AWS console

CloudFormation>>Stack you can see the Stack has been created.



Part:2

Now, we need to install AWS CLI. We can do it in multiple ways like SSH into instance and install or we can also use AWS SSM Session Manager to install without keys. 

We will be just SSHing into it and then download using commands.

First SSH into it. Then update using: sudo apt-get update

Then install awscli using: sudo apt-get install awscli

We will be using following jenkins file to pass on command on CLI:

pipeline {

    agent any

    stages {

        stage('Submit Stack') {

            steps {

                sh "aws cloudformation create-stack --stack-name s3bucket --template-body file://infraasacodewithjenkins.json --region 'us-east-1'"

            }

        }

    }

}


This is a Jenkins pipeline script that uses the AWS CLI to create an AWS CloudFormation stack from a template file. The pipeline has one stage called "Submit Stack" that consists of a single step.

Here's what the different parts of the pipeline script do:

"pipeline": This is the top-level block of the Jenkins pipeline script that defines the entire pipeline.

"agent any": This specifies that the pipeline can run on any available agent. In other words, it allows the Jenkins master to allocate an executor on any available Jenkins node to run this pipeline.

"stages": This section contains a list of stages that the pipeline will execute. In this case, there is only one stage called "Submit Stack".

"stage('Submit Stack')": This defines a stage in the pipeline called "Submit Stack".

"steps": This section contains a list of steps that the pipeline will execute as part of the "Submit Stack" stage.

"sh": This is a Jenkins step that allows running shell commands. In this case, it runs the AWS CLI command to create a CloudFormation stack with the name "s3bucket" using the "infraasacodewithjenkins.json" template file in the "us-east-1" region.


Overall, this pipeline script automates the creation of an AWS CloudFormation stack using the AWS CLI. When executed, the pipeline will create the stack named "s3bucket" in the "us-east-1" region using the "infraasacodewithjenkins.json" template file.


Unlike the previous part, here we will use IAM Role.

Let us create and attach an IAM role to EC2 with full S3 and CloudFormation Access.

Select the EC2 instance and go to:

 Actions Drop Down Menu>>Security>>Modify IAM Roles

Create an IAM ROle with following policies attached.

Attach that IAM role to EC2 instance

Now let us create a Jenkins Pipeline Job.

First name your job and choose pipeline as type of job.

Straight away go to change ‘Pipeline Configuration’. Here choose ‘pipeline script from SCM’. Then choose GIT. Fill the details accordingly. We will be using this repo with Jenkins file and CloudFormation File.

Details has been filled according to the resources in GIT.

Then choose “Build now’ to build the job.





In logs you can see the job has been built successfully. AlsoThe command in ‘CLI-Jenkins’ file is being executed.

The s3 Bucket has been created which was our end goal.


Conclusion: Experience Seamless Infrastructure Deployment, Simplify the process by modifying the code in "infraasacodewithjenkins.json" with the desired infrastructure code on GitHub, and let the pipeline handle the rest.

Comments

Post a Comment

Popular posts from this blog

ClouDesign: Architecting Serverless

-
Image
  Serverless Solution Architecture Introduction AWS provides a comprehensive suite of serverless services that allow developers to build secure, scalable, and cost-efficient applications. In this article, we will design a serverless architecture. We will look at common packages that usually come together for a given use case. ClouDatum: DynamoDB global tables replicate data across multiple AWS Regions in less than one second. API Gateway can process over 10,000 requests per second per API. DynamoDB DAX can improve read performance by up to 10x compared to standard DynamoDB reads. Cognito user pools can scale to handle millions of users without manual intervention. The goal here is to design architecture using serverless services offered by AWS. Also, the architecture might vary from each individual's POV. We should try to understand the core use case, which service is usually good to opt for and where. Let us start with our first case. 1. Mobile App: MyTodoList  Understandin...
Read more

ClouDIY: Bootstrapping Linux Servers – Part 2

-
ClouDIY: Bootstrapping Linux Servers – Part 2 Advanced tools and services for Linux system administration (based on RH134) 1. Shell Scripting Basics Shell scripts allow you to automate tasks such as backups, user setup, or software installation. They're simply text files containing a sequence of shell commands. #!/bin/bash echo "Welcome" whoami Explanation: The first line tells Linux to use the Bash shell. echo displays text, and whoami prints the current user. Tip: To run the script, first make it executable: chmod +x script.sh ./script.sh  Loops and Conditions Scripts often need to make decisions or repeat actions. You can do this using if statements and loops . If statement: Checks if a file exists before taking action: if [ -f "/etc/passwd" ]; then echo "File exists" fi For loop: Repeats a command for each item in a list: for i in 1 2 3 do echo $i done This loop prints the numbers 1 to 3, one per line. grep, awk,...
Read more

ClouDebrief: Database Savings Plans

-
Image
Every year at AWS re:Invent, the spotlight naturally goes toward AI , bigger models, faster chips, smarter agents.  But amidst all the AI buzz, there was one announcement that made existing AWS customers sit up, lean forward, and whisper: “Finally… something that helps me save money today.” And that crowd favorite was the introduction of Database Savings Plans (DSP),  a flexible new pricing model that can reduce database costs by up to 35% when you commit to a consistent amount of usage (in $ per hour ) over a one-year term, with no upfront payment . What Is Database Savings Plans? The Problem Before DSP Before Database Savings Plans, saving money on AWS databases required juggling multiple, different mechanisms: Reserved Instances for Amazon RDS Aurora Reserved Capacity Reserved capacity for DynamoDB Region-specific and engine-specific reservation rules Different pric...
Read more