Useful MySQL Stuff
dbmon.pl A simple script to monitor MySQL queries and log them into a file for future processing. Optionally will kill long-running queries. There are a couple of supporting scripts:
  1. listQueries.py If you want to process the logs for long-running queries.
  2. getServerIndexes.pl This will go through the SQL logs from dbmon.pl, do EXPLAINs on the queries it finds, then determine a list of tables and indexes used by the set of queries.
  3. Python version of dbmon that isn't functional yet.

mysqlHammerMe.pl A simple script that inserts a bunch of rows, then randomly deletes many of them. Mostly tests index read speed during the deletes. Supposedly Brian Aker's "mysqlslap" utility does this sort of thing.

diskTest.pl Brad Fitzpatrick of LiveJournal described the concept of a script that tests storage devices to make sure that when they claim some data is on disk, it actually is. After some web searching, I could not find the script, but only a note from Brad that they hadn't released it due to its being incomplete, so I wrote my own. If your storage device is hooked up to a Linux box, you can use this script to make sure your hardware isn't lying to you. We used it at Digg to make sure our disks weren't lying to us. Note: We decided to put many of our disks into a configuration where they WERE lying to us, because we get more performance that way. But now I'm just being pedantic.

Perl and Python MySQL DBI skeleton scripts. I keep writing these things from scratch. About time I stopped.

  1. mysqlDbiSkeleton.pl Perl version, with random useful concepts.
  2. mysqlDbiSkeleton-oneTable.pl Perl version that does work on some table, pausing between each portion.
  3. mysqlSkeleton.py Python version of the generic-with-random-useful bits.

tableSync.pl A fairly complex script to check two tables, and attempt to generate INSERT, UPDATE, and DELETE statements to bring them into sync. Note there appears to be a more complete version of this as part of the Maatkit.

keepStartingSlave.pl Sometimes a slave has replication problems after the master goes down, or otherwise sometimes points to a master log location in the past. You want to skip one transaction, keep applying until the slave dies again, then skip one transaction again. Rinse, repeat. This Perl script does that. Note there appears to be a more complete version of this as part of the Maatkit.

LiveJournalScalingPaper.pdf A paper about how LiveJournal scaled up their system. Not just a MySQL paper, but covers a lot of the details you need to know about doing a MySQL-backed high-volume web site.

mysql-internals.pdf Internals of MySQL. I read this once, and I remember it being interesting, but not a lot of it stuck. Probably better for reference.

teMySQLcacti-20060810.tar.gz A highly-modified version of MySQL monitoring templates for Cacti. You import a single host, which includes a bunch of graphs and graph types. Put a special PHP into your cacti/scripts directory, and then you get pretty graphs of MySQL. Make sure you read the README before importing this template into your Cacti! This now includes also the host template and polling script for a memcached server. (also: previous 20060517 release)

Screenshot of a single host graphed with teMySQL Cacti host template. Here I've gone to the preview mode and typed the name of the host into the search text entry box. Larger graphs with legends appear if you view the host in the tree view.

Screenshot of one statistic for several hosts. To do this, go to the preview view of your graphs, make sure "host" is set to "none". In the text entry box just to the right of that, type "teMySQL - InnoDB I/O" (for example). Then you'll see the InnoDB Row statistic for all hosts that have a graph of that sort.

Also, if you run memcached, here is a host type for graphing memcached statistics.

To the left is a screenshot of the memcached host type. This is a cluster of four memcached servers.


From Faemalia dot Net by Philo Vivero.