Tuning LAMP Server Performance

Having trouble with a crashing webserver? Is MySQL or Apache eating up your RAM and trashing your drive swapping? Don’t worry, there are good tools to help you get started in tuning your LAMP server to avoid crashes.

There are two scripts that I find invaluable in getting me a first and fast opinion on the current status of the server. The trick is often to get the settings right so that they do not risk eating up your RAM for breakfast. Here are two scripts to help you get your MySQL and Apache settings right for you.

MySQLtuner
This script will test your MySQL settings and suggest performance improvements. By using statistics from MySQL of current state of performance it will suggest modifications to your settings.

Apachebuddy
Inspired by MySQLtuner.pl Apachebuddy.pl does the same thing to Apache server settings. It checks your current settings and calculates average as well as maximum RAM usage and suggest improvements.

These script does of course not replace knowledge. Use them as a first opinion but then educate yourself about the settings before changing anything.

Both scripts have nifty URLs to download from:
mysqltuner.pl
apachebuddy.pl

Free software is amazing

Almost all software I use is or is built upon free software. I believe Microsoft Office is one of the few major exceptions to this (in my program library).

This makes me think of Tron…
“I fight for the user!”

Matt Cutts says it again: do not follow (nofollow)

Matt Cutts favourite html attribute rel=”nofollow” is once again major news. The object of the latest suggestions is that infographics and widgets on pages should have rel=”nofollow” on links. The reasoning behind this is that page owners might not be aware that they share their Pagerank to the linked page.

According to Google “no follow” should be used on links to untrusted content, on paid links and lastly as a crawl prioritization. Matt Cutts clarified that press releases should count as paid links and therefore should also be marked as “no follow”.

The origin of this mess is Googles main page algorithm, the original “backrub” PageRank. Every link is counted as a “vote” on the linked site by the original site. Google has since started using the “no follow” as an indication of when links are not votes.

Why could this not have been the other way around? Why not leave all links as-is and instead mark which links actually should count as votes? There are many values that a rel-attibute can have besides “nofollow”, however non that could reflect a “vote”.

Adding a new attribute value just for SEO might sound bad. However, the “nofollow” value was actually added by Google for the very same reason! If they instead would have gone with adding values like “citation”, “recommended” or simply “vote” things would have been a lot easier. Soon the reverse is in effect, every insignificant link on the web should have “nofollow” whereas the “valuable” links should remain as is.

As a side note, “no follow” also adds another benefit and that is speed. A “no follow” link will not be followed by a search engine and therefore improves the crawling speed. I do not see it as a useless addition to the rel-attribute, however as an SEO tool it’s working backwards in my opinion.

Rest i Peace Hotmail

hotmaillogoOne of the oldest most recognized web services is no more, Hotmail has been turned off. What Microsoft describes as a successful transfer of Hotmail.com to Outlook.com marks the end for the former Internet giant.

Hotmail was cool back in 1996 for many reasons. The brand name was smart, HoTMaiL contained all the letters of the HTML acronym. The service marked the beginning cloud services. As a bonus it was fun to see friends who confused the word “mail” with “male” when typing the address of the service.

Microsoft bought Hotmail in 1997 for estimated $400 million.. and it has been downhill from there. I remember the outages and confusion when Microsoft first tried to migrate the service to Windows Servers. I wonder if there will be more of the same now that they have switched the whole service for their new baby Outlook.com.

Similarly to Hotmails failure Yahoo have AltaVista in the same state of hibernation. Google acquired YouTube, another “first Internet giant in its field”. Google however chose not to rebrand it but rather keep it as a separate “cool” brand. Ever heard of Google Video? They even had a competing service when they bought YouTube, yet they decided to keep the brand YouTube. Google Video have now gone the way of Hotmail and Altavista, but YouTube lives on of course!

RIP Hotmail.

Attack on WordPress – is WordPress secure?

wordpressSecurity firms are reporting a very large attack on what seems to be all sites using WordPress. One WordPress host reported an increase of an average of 40k failed logins per month to 77k failed logins per day! The failed logins are coming from a large amount of different IP-numbers and is therefore hard to block. The advice from Matt Mulleweg, creator of WordPress, is: remove the old standard admin-account (if it still exists), use a strong password and as always, keep your WordPress installation up to date!

This brings up the question “is WordPress a secure platform?”. In my opinion the answer is a resounding YES! If the hackers have a bot network at their disposal and the means of attack is a brute force password attack then there really isn’t much you can do about it. Had WordPress had any known single fatle flaw the hackers would have used that instead. Apparently it doesn’t!

Any platform large enough will be the target of hackers, much like Windows is under heavy attack as a operating system. There have been known bugs in WordPress, allthough the latest such vulnerability was acctually a bug in a popular templates subclass and not in WordPress in itself. The WordPress community quickly responded and fixed the bug.

I feel secure to continue to use WordPress as my main platform for my blogs, so should you!

Having fun with QR-codes

newseedqr

It all started when I saw an extremly complex QR-code for a simple URL. I wondred why they did not use a URL shortener for the QR code so that the code was much simpler and easier to scan.

I then wanted to create a service where you see the difference as you type. The result can be seen here; a QR-code generator that creates codes as you type.

This is a service that lets you create QR codes easy and at the same time clearly displays the complexity of the code. Clicking on the generated QR image changes the error correction level which also demonstrates the complexity.

Feel free to use this service to either generate your QR codes (but you need to download them to use them, they cannot be hotlinked) or simply to generate a code for a piece of data you want to transfer to your smartphone!

Creating a MySQL Master – Slave connection

I’ve setup several MySQL Master – Slave connections and like to share my procedure. During the trials there are several details I’ve come to learn how to handle and my own set of “best practices”.

The MySQL Master Slave connection works under the premise that “a statement executed on the master should create the exactly same result when executed on the slave given that their database is equal”. For this to work we need to start two servers that are identical and then make one follow the other.

We used MySQL Server 5.5.11 when creating the master slave connection in the guide below. Please consult the MySQL Documentation if you are using a different version.

Step 1: Setup servers
First of all you will need two MySQL-servers. The standard community edition works fine. They should be of the exact same version to avoid any problem that bugs in one or the other might introduce. If you introduce a slave into an existing MySQL server you will need to make plan for a downtime for the duration of running the “mysqldump” command.

TIP: Save the MySQL installation file if you want to add more servers later since you will need the exact same version.

Step 2: Configuration
Edit the my.ini-file off the future master and add the following settings:

# Unique Server ID
server-id=1
# Name of binary log
log-bin=mysql-bin

The Server ID can be any number as long as there are no two servers with the same number in the replication chain, i.e. in our case the slave must have a different number.

The log-bin setting tells the server to make a binary log of every statement executed on the server.

Edit the my.ini file off the future slave and add the following settings:

# Unique Server ID
server-id=2

TIP: Add the setting relay-log=relay-bin to name the relay log. Otherwise MySQL by default uses [hostname]-relay-bin. The problem with the default is that if the host ever change hostname the replication will break. It also breaks if you want to make a copy of the slave to a second slave (if you do not add the setting to the new slave as well).

As mentioned before, the Server ID of the slave needs to be different from the Server ID of the master. When these changes are done, restart the service on both MySQL machines to let the changes take effect. Use the following commands to restart the service:

Linux (requires super user access):

user@host:~$ service mysql restart

Windows (requires administrator privileges):

C:\net stop mysql
C:\net start mysql

After the changes you should see a binary log starting to grow in data data directory of your future master.

TIP: If you have made other modifications to the my.ini file these needs to be copied as well to the slave, otherwise the slave riscs behaving differently from the master.

Step 3: Create a user
The replication will be using a normal user with the replication privilege. I opted to create a new user for this using the following commands:

mysql> CREATE USER 'slave'@'%' IDENTIFIED BY 'mytrickypassword';
mysql> GRANT REPLICATION SLAVE ON *.* TO 'slav'@'%';

The user will be created on the master but if you replicate all databases (as this guide will) then the user will also be replicated to the slave.

TIP: You can use any password you like BUT the password will be visible in plain text on the slave server! In the file master.info that will be created later in this tutorial all the master information will be stored including username and password.

TIP: Make the slave user limited to a certain domain or IP so that security riscs will be minimized. In the above example the user slave can log in from any host.

Step 4: Copy database
Now the time critical portion of this tutorial begins, from here until the datadump is complete the master database will be unavailable for writing.

Execute the following command on the MySQL Master:

mysql> FLUSH TABLES WITH READ LOCK;

Now all tables will be locked so that no transactions can occur. This is required since we need to make a full database dump of the current state of MySQL Master. Next execute the following command:

mysql> SHOW MASTER STATUS;

Write down the reply of the following values: File and Position. An example would be:

File: mysql-bin.00001
Position: 1337

From the command line on the MySQL master issue the following command (change password etc as needed):

C:\mysqldump --user=root --password=rootpassword --all-databases --master-data --result-file=mydump.sql

TIP: Are you using non UTF-8 encoding? Add “–default-character-set=latin1” to the command line where latin1 is the encoding you are using. If you do not supply an encoding MySQL will assume UTF-8.

When the dump is complete and you have a file called mydump.sql you can unlock the tables. Issue the unlock command on the master:

mysql> UNLOCK TABLES;

The master server will now be on-line and working again.

Step 5: Create the slave
Copy the file mydump.sql to the slave server. When it is done execute the following command from the mysql command line (you might have to specify exact location of the mydump.sql file):

mysql> source mydump.sql

TIP: Do NOT use “mysql -u root -p < mydump.sql” from the normal command line since that can corrupt the encoding, again if you use non-standard encoding.

The database on the slave is now identical to what the master from a specific point in time. Now configure the slave to connect to the master and follow it from that point in time.

CHANGE MASTER TO
MASTER_HOST='master.example.com',
MASTER_USER='slave',
MASTER_PASSWORD='mytrickypassword',
MASTER_LOG_FILE='mysql-bin.00001',
MASTER_LOG_POS=1337;

Make sure that MASTER_HOST is the name or IP of the MySQL Master. MASTER_USER and MASTER_PASSWORD are the same as created in step 3 above. MASTER_LOG_FILE and MASTER_LOG_POS are the same as read from step 4 above.

TIP: Since we used the flag –master-data when creating mydump.sql the MASTER_LOG_FILE and MASTER_LOG_POS should allready be set. The remaining settings are however needed.

TIP: Unless you specifically need it I recommend to avoid using binary logging on the slave while it tries to “catch up” with the master. Also the “bin-log” command only triggers logging of commands executed directly on the server, not from replication. To make the slave write replication to it’s own binary log the following setting must be added: “log-slave-updates=1”.

Start the slave with the following command from MySQL command line:

mysql> START SLAVE;

Step 6: DONE
Congratulations, your slave server is now replicating everything on the master server. Depending on how long time it took between step 4 and step 5 the slave should most likely allready have caught up the the master. To check on the status run the following command on the slave server:

mysql> SHOW SLAVE STATUS;

Especially noteworthy fields in this reply are “Slave_IO_State” that informs us of what the slave is up to, most common reply here is “Waiting for master to send event”. “Seconds_Behind_Master” tells us how many seconds behind the slave server is at the moment. If the slave server has been done or restored to an old backup this value can be very high. Normally this value is zero indicating that the slave is up to speed.

TIP: Did you know you can “daisy chain” MySQL servers. Just setup the slave as master to a new slave! There are however some further considerations for doing that, maybe a future blog post!

TIP: The slave server is perfect to use as a “live backup” in case the master should fail. You can also temporarily stop/lock the slave to make a complete database backup without having to worry about downtime of the service. The slave will catch up with the master again once started.

TIP: As with every security meassure in information technology, try this out before trusting how it works! I give NO GUARANTEE OF ANYTHING WRITTEN IN THIS GUIDE, you have to try and verify it yourself. This works for me, it doesn’t necessarily work for you.

More tips, comments or questions? Please feel free to comment below!

PhoneGap / Cordova EXIF bug on iOS

I used to curse the rate of which new releases of PhoneGap / Cordova kept popping up. It often meant alot of hassle for me to upgrade projects to keep up. Sometimes these upgrades needed to be done over several versions of PhoneGap which of course made things harder.

Now I find myself at the complete opposite end, I just cant wait for new releases of PhoneGap! The reason for this is called CB-1285. A bug (or limitation) that results in images received from the PhoneGap API lacks the EXIF data that normally is present. One such example is the orientation meta data that is used to rotate the image. Other EXIF data includes camera type and sometimes also geocoordinates.

Trying to figure out why my images got rotated took some research. It wasnt made easier by the fact that PhoneGap is known under many different names. PhoneGap, Cordova, Callback and under either Adobe or Apache.

EDIT: The bug has been marked as resolved in 2.6.0 but I am still unable to get the result I want (ie an image with EXIF-data intact). I will update if I find out more.

ActiveState Perl DBI::ODBC Unicode Error

Using Perl on Windows I’m probably out begging for problems. Using 32-bit Perl on 64-bit Windows I probably deserve it. Reality however does have some needs that go above theoretical best practices.

Recently I ran into a problem with DBI::ODBC that I newly installed with Perl 5.16 on Windows Server 2008 (64 bit). Since Perl for ISAPI only works in 32 bit mode with IIS I naturally ran the 32 bit version of Perl.

All Perl-scripts where moved from our old 32-bit Perl 5.10 Windows Server 2003. They all worked great with one major exception: DBI::ODBC. We kept getting an unexpected encoding (unicode) from all our calls done by ODBC. Fortunately for us someone else had allready run into this problem.

It turns out that as of DBI::ODBC version 1.29 there was added support for unicode that “broke” expected response if one where not using unicode. After some digging into the problem with suggestions to rebuild DBI::ODBC (that I don’t even know how to do on Windows under ActiveState Perl) or encode/decode every variable I finally found what I thought was the easiest solution.

After each connection established with DBI::ODBC driver I added the flag “odbc_old_unicode” and set it to true. This makes all the subsequent calls act as they did before unicode support was added!

$dbh = DBI->connect('DBI:ODBC:MYSOURCE','USER','PASSWORD');
$dbh->{'odbc_old_unicode'} = 1;

Now I just need to find every occurence where a connection is established with ODBC in my Perl scripts… which reminds me that I am on a Windows platform and don’t have access to the “grep” command!

Note: after some digging (since I do not want to install any third party software on the servers) I found the FINDSTR command which was quite handy!

Browser statistics – time to wake up!

Usage_share_of_web_browsers_(Source_StatCounter).svg

Hello World! Time to wake up and smell the browser statistics of 2013! All of those sites and developers who still claim that you need Internet Explorer for viewing or using their site needs to wake up.

According to several sources Chrome is now the individually largest web browser at roughly 35% market share. Noteable exception from this statistic is tracking done by NetApplications that pegs Chrome at only 19% (and Internet Explorer at 54%). Still, 19% market share means almost one in five visitors use Chrome!

It’s time to let go of the past. Reverse your conditions and say that this site is best view in ANY web browser EXCEPT old Internet Explorer versions. Adhere to standards instead of individual browser versions. A website can be made to look and work almost identical in majority of web browsers (including the latest version of Internet Explorer) if one simply takes the effort to learn how to do that.

The only case when I can accept shortcuts with limiting testing to a single browser (and even version) is for intranet applications in a controled environment. The rest of the world should head for the future and develop for Chrome, Safari, Firefox, Opera and of course also Internet Explorer!