Wednesday 31 March 2010

My websites going down

This morning I posted to the Opera forums about my problem when Flash is absolutely positioned above text in Opera. After that I uploaded a photo to various photo sharing websites. I checked my email and the Web Squeeze, then started to correct some problems with images that I'd uploaded to my photo website, but were missing bits of metadata.

The first image had the GPS co-ordinates missing, so I tried copying the GPS tags from an XMP sidecar of one of the NEF files that the image was produced from. To do this I used exiftool with the following command (in Windows):
# Copy GPS details from file
exiftool.pl -tagsfromfile ^
^"J:\K&J-1.5TBWD\2009-10-03\360 outside the hotel\Original pics\_DSC6615.XMP^" ^
-GPS:GPSVersionID ^
-GPS:GPSLatitudeRef ^
-GPS:GPSLatitude ^
-GPS:GPSLongitudeRef ^
-GPS:GPSLongitude ^
-GPS:GPSAltitudeRef ^
-GPS:GPSAltitude ^
-GPS:GPSTimeStamp ^
-GPS:GPSSatellites ^
-GPS:GPSStatus ^
-GPS:GPSMeasureMode ^
-GPS:GPSDOP ^
-GPS:GPSSpeedRef ^
-GPS:GPSSpeed ^
-GPS:GPSTrackRef ^
-GPS:GPSTrack ^
-GPS:GPSImgDirectionRef ^
-GPS:GPSImgDirection ^
-GPS:GPSMapDatum ^
-GPS:GPSDestLatitudeRef ^
-GPS:GPSDestLatitude ^
-GPS:GPSDestLongitudeRef ^
-GPS:GPSDestLongitude ^
-GPS:GPSDestBearingRef ^
-GPS:GPSDestBearing ^
-GPS:GPSDestDistanceRef ^
-GPS:GPSDestDistance ^
-GPS:GPSProcessingMethod ^
-GPS:GPSAreaInformation ^
-GPS:GPSDateStamp ^
-GPS:GPSDifferential ^
^
-xmp-exif:GPSAltitude ^
-xmp-exif:GPSAltitudeRef ^
-xmp-exif:GPSAreaInformation ^
-xmp-exif:GPSDestBearing ^
-xmp-exif:GPSDestBearingRef ^
-xmp-exif:GPSDestDistance ^
-xmp-exif:GPSDestDistanceRef ^
-xmp-exif:GPSDestLatitude ^
-xmp-exif:GPSDestLongitude ^
-xmp-exif:GPSDifferential ^
-xmp-exif:GPSDOP ^
-xmp-exif:GPSImgDirection ^
-xmp-exif:GPSImgDirectionRef ^
-xmp-exif:GPSLatitude ^
-xmp-exif:GPSLongitude ^
-xmp-exif:GPSMapDatum ^
-xmp-exif:GPSMeasureMode ^
-xmp-exif:GPSProcessingMethod ^
-xmp-exif:GPSSatellites ^
-xmp-exif:GPSSpeed ^
-xmp-exif:GPSSpeedRef ^
-xmp-exif:GPSStatus ^
-xmp-exif:GPSDateTime ^
-xmp-exif:GPSTrack ^
-xmp-exif:GPSTrackRef ^
-xmp-exif:GPSVersionID ^
^
^"J:\K&J-1.5TBWD\2009-10-03\360 outside the hotel\Hotel Biz Myeong Dong and closed Print Shops.tif^"

But when I ran that command, I just got the message
No file specified
The system cannot find the path specified.
The system cannot find the path specified.
Both filepaths are correct, and enclosed in quote marks, so I don't know why it couldn't find the files.

Next I tried Robogeo, I put the TIFF file into Robogeo, and manually entered the altitude, latitude, and longitude. The latitude and longitude are stored in XMP in the DDD MM.MMMM format, while Robogeo (and most web applications) use decimal degrees (DDD.DDDD) format. There is a GPS co-ordinates converter that you can use to get the correct format though. After entering the data, I told Robogeo to update the file, which it said it did, but when I checked the file in Adobe Bridge, other than a GPSVersion XMP tag, there wasn't any GPS data.

Next I tried ExifTool GUI, and used the 'Import metadata from single file' option to import the GPS and XMP into the TIFF from the XMP sidecar. Now this finally worked, and Adobe Bridge now showed the GPS data when looking at the TIFF.

After updating all the files for my 2 images that were missing metadata (for each image there is a .psb, .tif, sRGB .tif, and .jpg), I checked my photo-edit script for photo website, since I would need to re-upload the image that was missing all exif info when I uploaded it previously.

I couldn't see anything wrong with the scripts, so I tried uploading the updated version of the image to my local copy of the photo website. But it didn't work. I found the problem and fixed it (at least I think). To try again I would need to reset the database since the last upload attempt would have messed it up. So I logged into the database on the webserver to make a copy of the non-messed up database, but actually I didn't because I was greeted by a 503 Service Unavailable message (or something along those line anyway).

I checked my pano website, which was open in another tab, and that had the same error. So I ssh'd into the webserver and found that supervisord and php weren't running. So I started supervisord (which starts php), then checked my messages in my webhost's control panel, and found my processes had been killed for breaching my memory limit. Both PHP processes were about 64MB at the time the processes were killed.

I had thought that the memory_limit setting in php.ini was the total memory that PHP could use, but apparently it is the total memory that a script is allowed to allocate, so I changed my memory_limit from 64M to 16M. However, this isn't a permanent solution as according to the PHP Manual
If memory limit is enabled by your configure script, memory_limit also affects file uploading. Generally speaking, memory_limit should be larger than post_max_size.
I need to be able to upload files larger than 16MB, and when you include XCache memory usage and the memory usage by the upload script, the memory needed will be even higher.

I checked the access logs and error logs that weren't empty for my sites that use PHP, but couldn't see anything that would cause the memory spike. So I replied to the message about my memory usage being too high to say I couldn't find the reason for the high memory usage by PHP, and I asked them if there was a way I could get decent mysql access (the mysql WebFaction provides themselves is next to useless, thus forcing you to install your own mysql instance, which then forces you to compile your own PHP and web server). I'd rather spend my time working on my own website or photos than messing about with web server stuff (though I do find it interesting).

I received a reply back from WebFaction to say that to use MySQL with access to triggers, stored procedures, etc. you do need to run your own webserver software and PHP instance.

Even though I'd set the memory_limit for PHP to 16M (and I checked it was set at this in a phpinfo() page), I found the the memory usage of a single PHP process was still going over 40MB. I asked WebFaction if they knew how my PHP process could be using more than the memory_limit, but they said they couldn't really advise me since it is a custom install, and not one they provided.

So I changed the number of child PHP processes from 2 to 1. This should hopefully mean that I won't go over my memory limit and get all my processes killed, but it also means that there is only one PHP process to handle requests, so requests will be queued. When I have an upload or batch process script that can take a few minutes to run, this would mean that all my websites would be unavailable until the script had finished running (not good). I think it may also cause problems if you request a page while the PHP process is being recycled.

So I had a look at Host Gator, and they have a convenient 'live chat' facility, where you can talk to one of their staff. So I checked with them if you can run triggers, stored procedures, etc. on their webhosting accounts. They said you can, but you must email the staff who add the procedures for you - you can't add them yourself. Not a problem really, since procedures and triggers tend to be a set once and forget (unless you write them wrong, like me).

I checked on WebFaction when my year's subscription is due to end, and it's in August, so quite a wait yet. I think though, I will look at moving to Host Gator shortly, and then see if I can get any money back from WebFaction if I cancel early. At least I can keep my sites live on Web Faction until I have them up and running on Host Gator, and if Host Gator is missing something I need for my sites to work correctly, then I won't have lost anything except the first month's payment to them.

In the evening I watched Lost and a documentary about adult fans of lego:

AFOL A Blocumentary from AFOL on Vimeo.



The weather was rain on and off all day.

Food
Breakfast: Tangerine Marmalade Toast Sandwich; Cup o' Tea.
Lunch: ½ French Brie Cheese Sandwich; ½ Sliced Pickled Beetroot Sandwich; Home-made Chocolate Pretzel; Cup o' Tea.
Dinner: Chicken Pie; Potatoes; Peas; Mashed Swede; Gravy; Ground Black Pepper. Pudding was Home-made Sponge Cake with Tinned Strawberries and Custard. Coffee.

No comments: