Linux Admin Blog

System Administration, Linux, Solaris, Backup, Networking, Security, Mysql, Script, Tips & Tricks

Archive for the ‘Mysql’ Category

Import data to mysql database from .txt files using “Sed”

Posted by sanjaydalal4u on May 8, 2009

Review the post :http://sanjaybdalal.wordpress.com/2009/05/08/export-mysql-tables-in-txt-files/  . In this post we have export the mysql database data into the .txt files. 

Now we are import the data from .txt files to mysql database. For that we have to reformate the .txt files data which will support in the sql format.

#cat example.txt

1,SANJAY,AHMEDABAD,SYSTEM ADMIN

1,PRIYA,PUNE,PERL DEVELOPER

1,AKSHAY,GONDAL,PERL DEVELOPER

1,MIHIR,MUMBAI,PERL DEVELOPER

Now using “sed” command we will format this file to support sql.

 

sed -e ’s/,/”,”/g’ -e ’s/^/insert into example values(“/g’ -e ’s/$/”);/g’ example.txt

#sed -e ’s/,/”,”/g’ -e ’s/^/insert into example values(“/g’ -e ’s/$/”);/g’ example.txt > example.txt.new

where ^ represent start to the line and $ represent end of the line.

Output :

#cat example.txt.new

 insert into example values(“1″,”SANJAY”,”AHMEDABAD”,”SYSTEM ADMIN”);

insert into example values(“1″,”PRIYA”,”PUNE”,”PERL DEVELOPER”);

insert into example values(“1″,”AKSHAY”,”GONDAL”,”PERL DEVELOPER”);

insert into example values(“1″,”MIHIR”,”MUMBAI”,”PERL DEVELOPER”);

Posted in HowTo, Mysql, Sed, Tips & Tricks | Leave a Comment »

Export mysql tables in .txt files

Posted by sanjaydalal4u on May 8, 2009

mysql> use test;

mysql> SELECT *FROM example;

+——+——–+———–+—————-+

| ID   | NAME   | ADDRESS   | REMARKS        |

+——+——–+———–+—————-+

|    1 | SANJAY | AHMEDABAD | SYSTEM ADMIN   |

|    1 | PRIYA  | PUNE      | PERL DEVELOPER |

|    1 | AKSHAY | GONDAL    | PERL DEVELOPER |

|    1 | MIHIR  | MUMBAI    | PERL DEVELOPER |

+——+——–+———–+—————-+

4 rows in set (0.00 sec)

#mysqldump -u root -p –fields-terminated-by=, –tab=/location –table test example 

Above command will create the file example.txt which contains the export data in /location directory

Output: example.txt

#cat /location/example.txt

1,SANJAY,AHMEDABAD,SYSTEM ADMIN

1,PRIYA,PUNE,PERL DEVELOPER

1,AKSHAY,GONDAL,PERL DEVELOPER

1,MIHIR,MUMBAI,PERL DEVELOPER

Posted in HowTo, Mysql, Tips & Tricks | 1 Comment »

Mysql database Backup Script

Posted by sanjaydalal4u on May 6, 2009

#!/bin/bash

MYSQL=`which mysql`             #mysql version

MYSQLDUMP=`which mysqldump`     #mysqldump file

HOSTNAME=`hostname`             #hostname of backup server

USERNAME=”username”                 #mysql server user

PASSWORD=”password”            #mysql user password

LOCALSERVER=”localhost”         #mysql local server name

BACKUPDIRECTORY=/root/mysqldbbackup     #backup directory where backup is stored

NOW=”$(date +”%d-%m-%Y”)”               #Backup file format

DATATBASELIST=”"                        #Mysql database list

IGNOREDATABASE=”exampledb”                  #Ignore database not to be include in backup

GZIP=`which gzip`                       #Gzip to compress the backup file

FILENAME=”"                             #Database backup file names

#store the database list in DATABASELIST

DATABASELIST=`$MYSQL -u $USERNAME -h $LOCALSERVER -p$PASSWORD -Bse ’show databases’`

#Fetch database on by one and take a backup

for db in $DATABASELIST

do

        skipdatabase=-1

        if [ "$IGNOREDATABASE" != "" ]; then

                for i in $IGNOREDATABASE        #check the ignore database list

                do

                        if [ "$db" = "$i" ]; then

                                skipdatabase=1        #it will skip this database and move to second database list

                        fi

                done

        fi

        if [ "$skipdatabase" = "-1" ]; then

                FILE=”$BACKUPDIRECTORY/$db.$HOSTNAME.$NOW.gz” #Backup file format & Backup directory

                #ALL in one command to take backup & compress the backup file

 $MYSQLDUMP -u $USERNAME -h $LOCALSERVER -p$PASSWORD $db | $GZIP -9 > $FILE

 

Posted in Backup, HowTo, Mysql, Scripts | Leave a Comment »

Mysql Replication status notification

Posted by sanjaydalal4u on May 2, 2009

Using the script you can get the alert message from the replication server if replication is down or not working.

——————————————————————————————–

#!/bin/bash

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

#script checking the replication is running or not.

#If replication is down then sent the alert mail.

slave_server_hostname=192.168.10.1

###check if already notified###

cd /root

if [ -f slave_problem.txt ]; then

rm -rf /root/slave_problem.txt

exit 1;

fi

 

###Check if slave running###

(

echo “show slave status \G;”

) | mysql -u username -h $slave_server_hostname -ppassword 2>&1 | grep “Slave_IO_Running: No”

if [ "$?" -ne "1" ]; then

echo “Replication Failed”

echo “Replication failed” > /root/slave_problem.txt

fi

 

###Send notification if replication down###

cd /root

if [ -f slave_problem.txt ]; then

#mail -s “Replication problem” mail_id@domainname.com< /root/slave_problem.txt

echo “Problem in replicaition”

fi

———————————————————————————————-

Setup this script in conrtab of Master server

#chmod +x /path/to/Checkreplication.sh

#crontab -e

* * * * * /path/to/Checkreplication.sh

Posted in HowTo, Mysql | Leave a Comment »

MySQL Reset Root Password

Posted by sanjaydalal4u on April 29, 2009

We can reset root Password using two techniques. First one is using the mysqladmin command and second one is using the mysql safemode technique. I have describe both the technique below. enjoy !!!!!!!!

Technique 1. Using Mysqladmin command

#mysqladmin -u root -p password ‘new-password’

Technique 2. Using Mysql safe mode 

# /etc/init.d/mysql stop

# mysqld_safe –skip-grant-tables  &

# mysql -u root

mysql> use mysql;

mysql> update user set  password=PASSWORD(“newpassword”) where User=’root’;

mysql> flush  privileges;

mysql> quit

# /etc/init.d/mysql stop

# /etc/init.d/mysql  start

Posted in Mysql, Tips & Tricks | 1 Comment »

Find Out Which Table is Consuming Resources using Mytop

Posted by sanjaydalal4u on April 29, 2009

Step 1 : Install required perl modules

cpan -i Term::ReadKey
cpan -i Term::ANSIColor
cpan -i Time::HiRes

Step 2 :  Install mytop in your system

#wget http://jeremy.zawodny.com/mysql/mytop/mytop-1.6.tar.gz

# tar -zxvf mytop-1.6.tar.gz

#cd mytop-1.6

#perl Makefile.PL

#make

#make test

#make install

Step 3 : To view information of database example run below command.

# mytop -u username -p ‘password’ -h Hostname -d example

Posted in HowTo, Mysql | Leave a Comment »

Shell script to backup MySql database

Posted by sanjaydalal4u on April 29, 2009

#!/bin/bash

# Shell script to backup MySql database

 

USERNAME=”mysql user name”    

PASSWORD=”PASSWORD”

HOSTNAME=”localhost” 

 

MYSQL=”$(which mysql)”

MYSQLDUMP=”$(which mysqldump)”

CHOWN=”$(which chown)”

CHMOD=”$(which chmod)”

GZIP=”$(which gzip)”

 

DESTINATION=”/directory1″    # Directoy to take backup

 

# Main directory where backup will be stored

BACKUPSTORE=”$DEST/mysql”

HOST=”$(hostname)”                                  # Get hostname

NOW=”$(date +”%d-%m-%Y”)”                  # Get data in dd-mm-yyyy format

FILE=”"                                                           # File to store current backup file

DATABASELIST=”"                                     # Store list of databases

IGNOREDATABASE=”temp”                   # DO NOT BACKUP THESE DATABASES

 

[ ! -d $BACKUPSTORE ] && mkdir -p $ BACKUPSTORE || :

 

$CHOWN 0.0 -R $DESTINATION

$CHMOD 0600 $DESTINATION

 

# Get all database list

DATABSELIST=”$($MYSQL -u $USERNAME -h $HOSTNAME -p$PASSWORD -Bse ’show databases’)”

 

for db in $DATABASELIST

do

    skipdb=-1

    if [ "$IGNOREDATABASE" != "" ];

    then

            for i in $IGNOREDATABASE

            do

                [ "$db" == "$i" ] && skipdb=1 || :

            done

    fi

 

    if [ "$skipdb" == "-1" ] ; then

            FILE=”$BACKUPSTORE/$db.$HOST.$NOW.gz”

            $MYSQLDUMP -u $USERNAME -h $HOSTNAME -p$PASSWORD $db | $GZIP -9 > $FILE

    fi

done

Posted in Mysql, Scripts | Leave a Comment »

Set Up Database Replication In MySQL

Posted by sanjaydalal4u on April 29, 2009

Configure the MySQL Master Server

Step 1 : edit /etc/mysql/my.cnf file. 

#skip-networking

#bind-address            = 127.0.0.1

(add below line in /etc/mysql/my.cnf file)

server-id               = 1

log_bin                 = /var/log/mysql/mysql-bin.log

binlog_do_db            = replicationdb

step 2 :  Restart Mysql server

#/etc/init.d/mysql restart

Step 3 : create a user with replication privileges:

#mysql -u root -p

mysql> GRANT REPLICATION SLAVE ON *.* TO ’replicationuser’@’%’ IDENTIFIED BY ‘<some_password>’; 

mysql>FLUSH PRIVILEGES;

 mysql>USE replicationdb;

mysql>FLUSH TABLES WITH READ LOCK;

mysql>SHOW MASTER STATUS;

Result of above command:

+—————+———-+————–+——————+

| File          | Position | Binlog_do_db | Binlog_ignore_db |

+—————+———-+————–+——————+

| mysql-bin.005 | 180      | replicationdb    |                  |

+—————+———-+————–+——————+

1 row in set (0.00 sec)

 Please remeber above information will need in slave server configuration 

  mysql>quit;

Step 4 : Dump replicationdb from the Master server 

#mysqldump -u root -p<password> replicationdb > replicationdb.sql 

Above command create  dump of replicationdb in the file replication.sql. Transfer this file to your slave server!

Step  5 : Unlock the tables

#mysql -u root -p

mysql>UNLOCK TABLES;

mysql>quit;

 

Configure The Slave Server

Step 1 : Create the database replicationdb

#mysql -u root -p
mysql>CREATE DATABASE replicationdb;
mysql>quit;

Step 2 : Extract the dump database in replicationdb database

#mysql -u root -p<password> replicationdb < /path/to/replicationdb.sql 

Step 3 : Add the below lines into /etc/mysql/my.cnf

server-id=2

master-host=192.168.10.175

master-user=replicationuser

master-password=secret

replicate-do-db=replicationdb

Step 3 : Restart MySQL:

# /etc/init.d/mysql restart

Step 4 :  Allow slave user to connet to Remote Master server
# mysql -u root -p

mysql>SLAVE STOP;

mysql>CHANGE MASTER TO MASTER_HOST=’192.168.10.175′, MASTER_USER=’replicationuser’, MASTER_PASSWORD=’<some_password>’, MASTER_LOG_FILE=’mysql-bin.005′, MASTER_LOG_POS=180;

mysql>START SLAVE;

mysql>quit;

Configuration is Complete now!  So whenever replicationdb is updated on the master server, all changes will be replicated to replicationdb on the slave server.

Enjoy !!!!!

Posted in HowTo, Mysql | Leave a Comment »

Setup Mysql Replication Between Linux(master) & Windows XP(Slave)

Posted by sanjaydalal4u on April 29, 2009

Step 1 – Configure the Master Server

First we have to edit /etc/mysql/my.cnf. We have to enable networking for MySQL, and MySQL should listen on all IP addresses, therefore we comment out these lines (if existant):

#skip-networking

#bind-address            = 127.0.0.1

 Furthermore we have to tell MySQL for which database it should write logs (these logs are used by the slave to see what has changed on the master), 

which log file it should use, and we have to specify that this MySQL server is the master. We want to replicate the database exampledb, so 

we put the following lines into/etc/mysql/my.cnf:

server-id               = 1

log_bin                 = /var/log/mysql/mysql-bin.log

binlog_do_db            = exampledb

 

 Then we restart MySQL:

/etc/init.d/mysql restart

Then we log into the MySQL database as root and create a user with replication privileges:

 

mysql -u root -p

Enter password:

 

Now we are on the MySQL shell.

mysql>GRANT REPLICATION SLAVE ON *.* TO ’slave_user’@’%’ IDENTIFIED BY ‘<some_password>’; (Replace<some_password> with a real password!) 

mysql>FLUSH PRIVILEGES;

 

Next (still on the MySQL shell) do this:

 mysql>USE exampledb;

mysql>FLUSH TABLES WITH READ LOCK;

mysql>SHOW MASTER STATUS;

 

The last command will show something like this:

 +—————+———-+————–+——————+

| File          | Position | Binlog_do_db | Binlog_ignore_db |

+—————+———-+————–+——————+

| mysql-bin.006 | 183      | exampledb    |                  |

+—————+———-+————–+——————+

1 row in set (0.00 sec)

 

Write down this information, we will need it later on the slave!

Then leave the MySQL shell:

 

mysql>quit;

 

Step 2 – Configure the Slave Server(Windows XP)


Edit the c:\program files\mysql\mysql server 5.0\my.ini 

server-id=2

master-host=db01.yourdomain.net (or IP address)
master-port=3306
master-user=slave_user
master-password=password

Step 3 – Restart Mysql Service 

goto> Control Panel>Administrative Tools>Services>Mysql 

Restart Service

mysql > Stop slave;

mysql>CHANGE MASTER TO MASTER_HOST=’192.168.10.175′, MASTER_USER=’slave_user’, MASTER_PASSWORD=’password’,MASTER_LOG_FILE=’mysql-bin.000008′,MASTER_LOG_POS=98;

mysql > Start slave;

Posted in HowTo, Mysql | Leave a Comment »

Mysql Slave Server(Replication) Status Check Script

Posted by sanjaydalal4u on April 29, 2009

Below script is returning the Mysql Slave server Status.

If Replication  is running then it will not show any message but if replication is stop then it will send the email notification for status.

—————————

#!/usr/bin/env bash

repeat_alert_interval=15 # minutes

lock_file=/tmp/slave_alert.lck

active=yes

 

## Check if alert is already sent ## 

 

function check_alert_lock () {

    if [ -f $lock_file ] ; then

        current_file=`find $lock_file -cmin -$repeat_alert_interval`

        if [ -n "$current_file" ] ; then

            # echo “Current lock file found”

            return 1

        else

            # echo “Expired lock file found”

            return 2 

        fi

    else

    return 0

    fi

}

 

## Find the location of the mysql.sock file ##

 

function check_for_socket () {

        if [ -z $socket ] ; then

                if [ -S /var/lib/mysql/mysql.sock ] ; then

                        socket=/var/lib/mysql/mysql.sock

                elif [ -S /tmp/mysql.sock ] ; then

                        socket=/tmp/mysql.sock

                else

                        ps_socket=`netstat -ln | egrep “mysql(d)?\.sock” | awk ‘{ print $9 }’`

                        if [ "$ps_socket" ] ; then

                        socket=$ps_socket

                        fi

                fi

        fi

        if [ -S "$socket" ] ; then

                echo UP > /dev/null

        else

                echo “No valid socket file “$socket” found!”

                echo “mysqld is not running or it is installed in a custom location”

                echo “Please set the $socket variable at the top of this script.”

                exit 1

        fi

}

 

 

check_for_socket

 

Slave_IO_Running=`mysql -u username -p’password’ -h Slavehostip -Bse “show slave status\G” | grep Slave_IO_Running | awk ‘{ print $2 }’`

Slave_SQL_Running=`mysql -u username -p’password’ -h Slavehostip -Bse “show slave status\G” | grep Slave_SQL_Running | awk ‘{ print $2 }’`

Last_error=`mysql -u username -p’password’ -h Slavehostip -Bse “show slave status\G” | grep Last_error | awk -F \: ‘{ print $2 }’`

 

 

if [ -z $Slave_IO_Running -o -z $Slave_SQL_Running ] ; then

        echo “Replication is not configured or you do not have the required access to MySQL” | mail -s “Mysql Replication Status ” username@domain.com

        exit

fi

 

if [ $Slave_IO_Running == 'Yes' ] && [ $Slave_SQL_Running == 'Yes' ] ; then 

    if [ -f $lock_file ] ; then

        rm $lock_file

        echo “Replication slave is running”

        echo “Removed Alert Lock”

    fi

    exit 0

elif [ $Slave_SQL_Running == 'No' ] ; then

    if [ $active == 'yes' ] ; then

        check_alert_lock

        if [ $? = 1 ] ; then

            ## Current Lock ##

            echo “up” > /dev/null

        else

            ## Stale/No Lock ##

             touch $lock_file

            echo “SQL thread not running on server `hostname -s`!” | mail -s “This is subject” mail_ID@somedomain.com

            echo “Last Error:” $Last_error

        fi

    fi

    exit 1

elif [ $Slave_IO_Running == 'No' ] ; then

        if [ $active == 'yes' ] ; then

                check_alert_lock

                if [ $? = 1 ] ; then

                        ## Current Lock ##

            echo “up” > /dev/null

                else

                        ## Stale/No Lock ##

                        touch $lock_file

                        echo “LOG IO thread not running on server `hostname -s`!”

                        echo “Last Error:” $Last_error

                fi

    fi

    exit 1

else 

        if [ $active == 'yes' ] ; then

                check_alert_lock

                if [ $? = 1 ] ; then

                        ## Current Lock ##

            echo “up” > /dev/null

                else

                        ## Stale/No Lock ##

                        touch $lock_file

            echo “Unexpected Error!”

            echo “Check Your permissions!”

                fi

        fi

    exit 2

fi

Posted in Mysql, Scripts | 2 Comments »