Posts

AWS Command line Tasks : Create EC2 instance and EC2 instance with 500GB root volume.

To create an EC2 instance from command line. aws ec2 run-instances  --image-id ami-efe09bf8 --count 1 --instance-type m4.4xlarge --key-name mysshkey --security-group-ids sg-126adasd2--subnet-id subnet-148fd971 --block-device-mapping /dev/sda1=:500:false --region us-east-1 --image-id : The AMI ID which we are going to create the instance. --count : The number of instances that we need to create. --instance-type : The type of EC2 instance that we are going to use it. --key-name : The pem that we are going to use to connect to this server. --secrutiy group : The security group this instance is getting used. --subnet-id : The subnet in which we want the ec2 instance. --block device mapping - device:space in gb : to terminate volume [true or false.] To create an EC2 instance with 500GB root volume. aws ec2 run-instances --image-id ami-efe09bf8 --block-device-mappings  '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":500,"DeleteO...

AWS command line tasks - Create 500 GB EBS Volume, to attach and detach the volume

To create a volume of 500 GB   aws ec2 --region us-east-1 create-volume --size 500 --availability-zone us-east-1d --volume-type gp2 size : in GB availability zone : This needs to be in the same availability zone as that of the instance that we are going to attach the EBS volume. To attach an EBS volume to the ec2 instance. aws  ec2 attach-volume --volume-id vol-9dfjhsdkfj93 --instance-id i-1234567899oasd  --device /dev/xvdb --region us-east-1 After attaching format the drive and mount it like what we do for a normal hard disk drive. To detach an EBS volume from the ec2 instance. aws ec2 detach-volume --volume-id vol-9dfjhsdkfj93 --region us-east-1

AWS Command line Tasks - Create SSH key pair, security group, Allow connection to a port

We need a working aws cli configured to use the following commands. To create an SSH pem file. aws ec2 create-key-pair --region us-east-1 --key-name mysshkey --output text > mysshkey.pem  region :  The region in which the ssh pem file is getting created. Here we are saving the pem file in our linux machine as the name mysshkey.pem  Modify the permission of the pem file. chmod 400 mysshkey.pem  To create a security group. aws ec2 create-security-group --region us-east-1 --group-name work_project --description "Project_for_work" --vpc-id vpc-12345  group name - It is the name that we are going to give for the security group description - The description that we are going to give for the security group To allow connection to a port aws ec2 --region us-east-1 authorize-security-group-ingress --group-id sg-12345 --protocol tcp --port 22 --cidr 10.100.0.0/16 aws ec2 --region us-east-1 authorize-security-group-ingress --group-id sg-12345 --pr...

High process count notification - shell script

#Author : Jaison # Purpose : To send email to the user when a user hit a specific process count list. process_count="$(ps -u root  -L | wc -l)" if [ "$process_count" -gt "10000" ]; then  Email_Body='Test"   echo "$Email_Body" | /usr/bin/mail -s "Server $HOSTNAME Process Count : $process_count IS A LARGE VALUE FOR  USER. CHECK IMMEDIATELY. " <email_account> fi

useradd shell script - bulk read username and password from file.

# Author : Jaison #Purpose : Add user and set their password from the text file all.txt file_name="all.txt" while read user pass do     #useradd ${user} -p ${pass}     useradd ${user}     echo "${pass}" | passwd --stdin ${user}     echo "Adding user "${user}"  with the password  "${pass} done < $file_name

DNS - A simple explanation.

Hi All, Let me try to throw some pebbles regarding the DNS, one of the simplest concepts on the internet, but if explained properly, it can go into a high level of complexity. DNS is something which is used to identify the devices which are connected to the internet.Before jumping into the concept of DNS, I hope everyone who is reading this chapter is aware of IP Adress. If not, IP Address or in short IP is a set of numbers and symbols used to identify machines connected on the internet. So we already have IP so you might be thinking what is the need of DNS in the world?? Well, that is the problem with the humans, we are good with names but not so good with the numbers. To overcome this problem, we are using DNS. Domain Name System in the point of view of a System Administrator is something unavoidable when it comes to hosting the websites on the server. The simple concept of how it works is explained in my propensity as shown below. When a website is accessed on a web b...

Cheat sheet for Hardware RAID health check - Megaraid, Adaptec, 3wareraid and HPraid.

Hi Folks, In another post " raid type " I've mentioned how to find if the server is running with Software or Hardware RAID. Here, we can go ahead and check the health of the Hardware RAID array in the server. The first thing we need to detect is the controller in which the raid is configured and you can easily get it from the another post that I mentioned earlier. However, I'm providing it here again for your convenience. 1. Login to the server as the root user. 2. Execute the following command. /sbin/lspci -vv | grep -i raid 3. This will show the raid controller running on the server. The different types of controller I've worked with includes a. Megaraid which uses megacli b. Adaptec which uses arcconf . c. 3ware raid which uses tw_cli d. HPraid which uses hpacucli . How to check the Megaraid array health status? The Megaraid is usually installed in the /opt/MegaRAID and you can execute the following command to verify the health of the arr...

Logical volume vmxxxx_img is used by another device - Error on LVM removal

Hi Folks, I've faced error while trying to remove an LVM from the server. The exact LVM error will be "Logical volume vmxxxx_img is used by another device" on executing the lvremove command. Feel free to remove the following steps to remove the LVM from the server. Note: Please remember to replace <id> with your  VMID in the below section. ---------------------------------------------------------- dmsetup ls dmsetup info -c xen-vm<id>_img dmsetup remove xen-vm<id>_img lvremove -f /dev/xen/vm<id>_img ----------------------------------------------------------

File system corrupted on the OpenVZ container.

It happens rarely but when it does, it is hard to handle. The OpenVZ VM's are usually stable in nature, but I've encountered situations where OpenVZ VM was showing file system error and was refusing to start. You can follow the below steps to recover the VM. 1. ~# vzctl stop ctid 2. # ploop mount /vz/private/ctid/root.hdd/ DiskDescriptor.xml 3. now do a df -h and you will get the ploop id for the VM. 4. fdisk -l /dev/ploop12345 5. e2fsck /dev/ploop12345p1 6. ploop umount -d /dev/ploop12345 7. vzctl start ctid

Change IO Scheduler for SSD and HDD server with Software RAID

Hi Guys, It has been a while... I faced a problem with IO on one of the RAID 1 Linux servers and the IO schedulers were the default cfq in the server. It doesn't matter if the drive is in software RAID or not. If you are using SSD drives, then there are no mechanical parts in the HDD, so you need to change the IO schedulers to noop. If the SSD drive is mounted as /hda, then below command will do it. ---------------------------------------------------------- echo noop > /sys/block/hda/queue/scheduler ---------------------------------------------------------- For the HDD drive, the scheduler that is recommended is `deadline` in case of poor performance problems. ---------------------------------------------------------- echo deadline > /sys/block/hdb/queue/scheduler ---------------------------------------------------------- In case of Software RAID, just make sure that the drive you are changing the scheduler is inside the RAID. You can see the drives ins...

Check the number of connections using nf_conntrack or using tcpdump

tcpdump -tnn -ieth0 -c 20000 | awk -F ">" '{print $1}' | awk -F " " '{print $2}' | awk -F "." '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -nr | awk ' $1 > 100 ' Note : If the network interface card in use on the server is eth0 cat /proc/net/nf_conntrack > contoutput && cat  contoutput  | awk '{ print $7,$8 }' | sort | uniq -c | sort -rn | head

RAID - Collection of Information that I'm trying to learn.

=_= RAID =_= 1. How to confirm if your linux server has Hardware RAID installed in it? Execute the command provided below. /sbin/lspci -vv | grep -i raid If it results an empty screen, you can confirm that there is no RAID hardware controller installed in the server. 2. How to confirm if there is Software RAID installed in the server? Execute the command provided below. cat /proc/mdstat If it returns the below message, you can confirm that there is no software RAID installed in the server. Personalities : unused devices: <none>

General File System check for linux

You can either use fsck -fy  /dev/sda1 f - force y - yes /dev/sda1 - partition that I need to run file system check. This can be an LVM or a direct drive also. e2fsck also does the same but not found in all linux distributions because e2fsck stands for file system check in ext family, while fsck is the general one which can be used on other file system types.

To show the storage engines used by each tables in a database

1. Login to Mysql as root user. 2. Use db_name; 3. Execute the query "show table status ; ". It'll show the engine used by each tables in the database you have selected.

Apache is not running in linux due to semaphore error.

Fix ============= ipcs -s | grep apache | perl -e 'while (<STDIN>) { @a=split(/\s+/); print `ipcrm sem $a[1]`}'

Enable TUN/TAP and PPD in VM on open VZ

How to enable TUN/TAP and PPD in VM on open VZ = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = Please use the below steps to Enable TUN/TAP and PPD in VM on open VZ.    1.Check the Node is enabled with tun/tap by entering the same from solusVM.    >> Go to list node >> edit node >> check "Allow clients to set TUN/TAP" and "Allow clients to set PPP".        2.How to enable TUN/TAP in OpenVZ? ========================================================================================     +++++++++++++++++++++++++++++++++++++++++++++ STEP 1: Login to Node via SSH STEP 2: Run the below pasted command to find out tun module is already loaded or not [root@Node]# lsmod | grep tun [root@Node]#      If the output of the above commands returns a blank value means the tun module is not loaded in your Node. Run the below command to load tum modu...

MySql database backup shell script

#!/bin/sh # Setting date to name the backupfile with todays date date=`/bin/date "+%Y%m%d"` # creating backup folder /bin/mkdir -p /mysqlbkp/dbbackup_$date DBS="$(mysql -Bse 'show databases')" for db in $DBS do /usr/bin/mysqldump --routines $db > /mysqlbkp/dbbackup_$date/ $db.sql done The backups will be generated in the location /mysqlbkp under the folder dbbackup_todays date.

Simple Shell Script for Disk Space Notification

1. Create a file /home/diskspace.sh 2. Enter the following code in the script ============================= #!/bin/sh #Threshold is set to 70 here. THRESHOLD=70 MAILTO=”youremailaddress” TEMPFILE=/tmp/diskspace.temp HOSTNAME=`hostname` rm -f $TEMPFILE #Calculate the Current Disk Usage with the below command. CDU=$(df -h | tail -1 | awk ‘{print $5}’ | sed ‘s/%//’) #Compare the current value with the threshold one. if [ $(expr $CDU ">=" $THRESHOLD) -ne 0 ] then echo “Warning!!! Disk Space usage on server $HOSTNAME is ${CDU}%” >> $TEMPFILE fi #Send an email if /tmp/diskspace.temp is present. if [ -e $TEMPFILE ] then mail -s “Disk Space Notification” $MAILTO < $TEMPFILE fi rm -f $TEMPFILE ============================== 3. Save the file and execute using crontab crontab -e 0 1 * * 0  /bin/sh  /home/diskspace.sh

Process uptime - Linux

How to find the uptime of a Linux Process ----------------------------------------------------- 1. Login to the server as root. 2. Run the below provided command ps -eo pid,comm,lstart,etime,time,args | grep process-name Acronyms PID - Process ID .ie. the ID of the process that we are searching. comm - Display the process name. lstart - The time at which the process is started. etime - The time elapsed once the process is started. args - command with options and arguments. process-name - The process for which we need to check uptime. For eg : :~$ ps -eo pid,comm,lstart,etime,time,args | grep cron   937 cron            Fri Jan 30 09:42:17 2015       37:36 00:00:00 cron The cron deamon is running in the server from 9:42 AM and is alive for 37 minutes till now. Check the etime field. (37:36) This is how we can detect the uptime of a process in Linux. Added on Jan 30 10.26 AM   

cphulk blacklisted ip- please follow to remove IP

Remove IP from cphulk blacklist ------------------------------- Login in the shell as a root user and run the commands. # mysql    >USE cphulkd    >SHOW tables;    >BACKUP TABLE  brutes TO ‘/root/cphulk.log’;    >SELECT * FROM brutes WHERE  IP=’[IP Address]‘;    >DELETE  FROM  brutes WHERE  IP=’IP Address’;    >quit;  Replace “IP Address” as per your local machine ip address to resolve the issue. ============================================================ To delete all the brute force entries of blocked IPs, mysql> delete from brutes; mysql> delete from logins; ============================================================ Note:- To disable cPHulk through backend. ie, via terminal # /usr/local/cpanel/bin/cphulk_pam_ctl --disable