Feeding

This morning I was trying to work out if the timezone tables in MySQL needed to be filled, and after checking some more websites, it looks like they probably do need filling. I also checked the current timezone in MySQL, which still said 'SYSTEM'. So I checked the system time in a normal terminal by typing 'date', which printed the date and said UTC. So it looks like I should have MySQL working using UTC as the timezone now.

I spent the rest of the day working on my feed. So far I have the feed, but it doesn't validate, a major reason of which is because the urls aren't url encoded (I'm not a fan of ASCII only). I already had making URLs properly encoded according to standards on my 'to do' list, so looks like I'll have to do that next.

I also started work on the XSL to style the feed like the rest of the site. XSL is something I very rarely use, so I've just been doing lots of googling on it (and on xpath). A problem I had was that the feed was displaying without the XSL being applied. Apparently most browsers (well FF and IE at least) apply their own stylesheets to feeds and ignore any stylesheets specified in the feed. According to one of the commentors on the Firefox bug report for this behaviour, this is because in the large majority of cases a stylesheet is only specified to catch users who try to view the feed in a browser that doesn't support feeds:

Personally, I know of quite literally tens of millions of feeds which are styled for one and only one reason: to keep browsers from showing either raw XML or a helper app dialog to puzzled users

The fix is to apply 512 bytes of padding before the feed root node as the browsers only check the first 512 bytes to see if it is a feed.

For my site I am basing the feed on an example of a photo gallery feed in this IBM article An overview of the Atom 1.0 Syndication Format. When I view the feed without a stylesheet in a browser, it doesn't render any of the content (images), just the textual summary. So for a photo gallery, it does seem that using a stylesheet would be the best way to present a feed to users.

Not getting anything done

As so often seems to happen, events conspired against me to stop me getting any work done today. I checked one of my websites, and it wasn't working. So then I had to try and figure out why, and fix it. It turned out that although the php process was still running, it wasn't responding, so I had to restart PHP and then my websites came back to life.

There was also a config problem with one of the sites where I had managed to upload a copy of the config for that site that I use for local testing, instead of the live site config.

Then I had to try and find out why PHP had stopped responding. Unfortunately the PHP logs and Nginx logs weren't any help there. I found a thread on the Nginx forums where someone else seemed to be having the same problem: php-fpm randomly becomes unresponsive, segfaults and they suggest there that the problem may be due to MySQL.

As far as I'm aware this is the first time I have had php become unresponsive in about 6 months. And last time I tried installing the latest version of MySQL, it was a real pain. I think I would probably need to rebuild PHP as well to get it to use the latest MySQL driver. So although I will get round to upgrading them some day, what I really want is a method to email me when nginx starts generating 502s because php is unresponsive.

First I looked whether Nginx could send an email when it generated a 502. Although Nginx can act as a mailserver, I couldn't see any directives to do with making it send an email though.

I then spent quite a while trying to see if I could write a shell script that connects to the php socket, sends it some data, and then checks if there is a response. This could then be run on a cron job and send an email if there was no response. Unfortunately I couldn't get this to work, all the examples I found seemed to be about tcp sockets rather than unix sockets. Even if I could have made it connect to the socket, I'm not sure my idea would have worked anyway.

Next I looked at using a shell script on a cronjob to parse the last 10 lines of one of the website error log files. The script could then look for unresponsive upstream errors and send an email if there was one. If there were multiple errors, the script could restart php. Googling, it looked like awk and/or sed or maybe grep could be used to do this. I didn't try to write this script though, but sent my hosting provider a ticket to see if they had any better suggestions.

And they did, they suggested using a website monitoring service. So I signed up with mon.itor.us, who are free and looked quite good.

After getting that sorted, I did actually get some work done. I wanted to write a blog post for my photo website with some recent photos. But currently the website only lists photos in order of rating, not by date. So I thought I would write an RSS feed for it. I spent quite a while researching, and it seems that RSS doesn't support a paginated feed, so it only has the last X items. But there is another feed system called ATOM that seems virtually the same as RSS except it does support pagination.

While working on my feed, I found that I needed to format the timestamp stored in the database for use as the last updated date in the feed. The date format required by ATOM includes a timezone, so I wondered what timezone MySQL was using.

So I did some research on that, and how to set MySQL to use UTC. MySQL Server Time Zone Support gives a number of different ways of setting the timezone. It also states that you should use the mysql_tzinfo_to_sql program to fill MySQL's timezone tables. However, I was a bit confused at this as many other websites include info about how to set the timezone in MySQL, but don't mention mysql_tzinfo_to_sql or filling the timezone tables at all.

So I am not sure if all the other websites are giving info that won't work because the timezone tables won't be filled, or the MySQL manual is saying you need to fill the timezone tables when you don't actually need to?

Walking

This morning the weather was nice and sunny so I went out on a walk at 10am to see how autumny it was and get some photos. I followed footpath signs, but unfortunately there were a few places where the track split off in various directions and there wasn't any sign to say which way the footpath was, so I had to go down one route until it reached an end and see if there were any footpath signs at the end or on any points of entry e.g. gates along that route. Then if there weren't any footpath signs down that route I would have to go back and try the other route, doing the same thing. I had to do this a couple of times at different points.

I also spent quite a while in a field photographing a cow pat, part of the time lying down in the field to photograph it. If anyone else had came past they would have thought I was a bit strange. Eventually I got back home about 3.15pm. I was very thirsty as I hadn't taken any water with me. I didn't think I was going to be quite so long.

I copied all the photos to my PC, geo-coded them, and started checking my emails.

After dinner I went out to take a sunset panorama, but the sky was bare and the sunlight wasn't particularly orange, so it wasn't that great.

In the evening I watched Flash Gordon with Mauser and Bo. It was quite good, thanks mainly to Brian Blessed. After that I checked a few more emails, geo-coded this evening's photos, and wrote this post.

Trying to find and get software to work to do something I've already done

I spent most of today writing a photography article.

Then in the evening Billy said he had accidentally deleted all the Animal Crossing photos off the SD card, but he pulled the card out of the wii as soon as he realised so could I check if the card was corrupted or the photos still there. First my PC didn't seem to recognise the card at all, even after removing it from the card reader and removing the card reader from the PC and then re-inserting them again. So I checked Windows Device Manager, which showed the device as a USB Mass Storage Device, but said that the device had been marked for safe removal, but hadn't been removed yet. It said to use the device remove it and then re-insert it again (or something similar).

I didn't think that would work since that was exactly what we'd already done, but I did it and it worked. Hmm... strange. I checked the card and it worked okay, but all the Animal Crossing photos had been deleted, so obviously the Wii was quicker at deleting the photos than Billy thought.

I tried Handy Recovery, some old recovery software I have. This recovered some of the files, but the large majority that it recovered displayed no thumbnails in Windows Explorer, and said the file was corrupted or a type that:

Windows Photo Viewer can't open this picture because the file appears to be damaged, corrupted, or is too large

I thought that maybe specialist image recovery software might work better at recovering the the photos. After doing some googling I remembered that Sandisk Memory cards come with free photo recovery software. I found the website online, and downloaded a demo of the SanDisk RescuePro software. This seemed to show all the photos okay, indicating they could be recovered, but it cost about $60-70 for the full non-demo version that could actually recover the files. So rather than pay $70 for a piece of software that comes free with a memory card, I thought I might as well try and find one of my Sandisk memory card boxes that included the RescuePro software.

I got the 2 boxes of photo stuff boxes out of the garage, then Billy had a look and managed to find a 4GB Sandisk Compact Flash card box, which was the first CF card I bought with my Nikon D200, quite a few years ago. I checked the back of box, and was surprised to find that the RescuePro software was included with the card. But when I took the plastic packaging out of the box, there didn't seem to be any serial number or CD. Then after checking the pamphlet and stuff that was inside the plastic interior packaging, I gave up and decided to put it back in the box. After all, I still had two other newer Sandisk memory card boxes somewhere I could check.

But when I was putting the plastic interior packing back in the box, I found that actually the (mini) CD was in the box, rather than in the plastic packaging. So I installed the software, but when it came to start it, it wanted a serial number. I couldn't see one on the mini CD wallet, or the main box, or anywhere in any of the stuff that was in the box. I checked and checked, and tried a couple serial numbers that were on the bottom of the box, but they didn't work and I couldn't see the correct serial number anywhere. So I had to give up on that.

Doing some more searching through the big boxes of photo stuff, I found a SanDisk 4GB SD card box. This included the RescuePro software as well, and the serial number was printed on the back of a sticker on the mini CD wallet. Woohoo! First I tried entering the serial number in the prompt of the RescuePro software I'd installed from the Compact Flash card's CD. But this didn't work. So I uninstalled both the previous version (3) of RescuePro and the Demo version (3.5), and then installed the version that came with the SD card (3.3).

After entering the serial number, the program started up, but said it couldn't read the SD card!?!

I decided to check how much of the photos Handy Recovery had recovered, and thought that maybe it wasn't very good at recovering them all in one go but might do it if I only selected a few at a time. But when I checked, I found that actually it had recovered all the same photos that the Demo of the latest version of RescuePro had been able to show it could recover, So after spending all evening on this, Handy Recovery had actually worked fine. It was just that it had also 'recovered' hundreds of additional photos that couldn't be recovered fully and so were corrupted. These were probably photos that had been deleted from the card before, while the ones deleted this evening were able to be recovered.

Now I knew I had the files recovered okay, it was just a case of moving them to the correct directory for permanent storage. In hindsight I didn't need to do this, but at the time I thought that I needed to rename the files, as the recovered files all had filenames like _UU_00001.JPG while the existing files had filenames like RUU_00001.JPG. So I thought I needed to replace the starting underscore with an R to match the correct file naming system for Animal Crossing photos. Of course, once the photos are stored on your PC, it doesn't really matter what their filename is actually.

Anyway, I thought it did matter for some reason, so I tried to see how to batch rename the files. Unfortunately I couldn't find any info other than pay for software. If I was using Linux I'd probably try and write a shell script, but since I'm on Windows I thought I'd try and write a perl script instead. I'm probably equally useless at perl and bash scripting anyway. After some more googling I got the script written, but it didn't work. After doing some debugging I found the problem was that the filename held in the variable started with a './', which I wasn't expecting (though now I look at the script, it's obvious). So I adjusted my script and got it running okay.

#!/usr/bin/perl

use strict;

my @files = <./*.JPG>;
my $file;
foreach $file (@files) {
  if( rename($file, 'R'.substr($file, 3)) )
  {
    print "$file renamed\n";
  }
  else
  {
    print "Problem renaming $file\n";
  }
}

print "Operation Complete";

I just saved that as rename.pl, put it in the directory of photos, and then double clicked it to run it. With a bit of googling I'm sure it could be changed to use a regex replacement and do more complex batch renames.

And since I had to clear some photos off my memory card today, here is one from a few days ago showing some armatures in varied state of dress:

Setting up account with ping.fm

This morning and part of this afternoon I was signing up for social networking sites to use with ping.fm.
As part of this I wondered if I could change my Google Profile for the website I was doing this for to use the website name instead of my name. I tried this, and got a message that it didn't look like a valid name, along with a link to Google's policy on your name. This says:

Google+ does not support profiles for couples or groups of people. Additionally, you cannot create a profile for a non-person entity such as a pet or business.

So it seems that you cannot create a Google profile for a website or business, and yet it seems that somehow some people have, e.g. Fodder Network and TechCrunch. I'm guessing that probably Google used to allow businesses to register, but has since changed their rules, while still keeping previous business accounts active.

In the evening I watched Empire of the Sun with Mauser and Bo, which was alright, but a bit boring and cheesy.

Although I still had a bit of a runny nose and cough today, I'm feeling a lot better than yesterday.

Having a cold

The last few days I been brooding a cold, and the symptoms have been quite bad yesterday and today so I have spent most of my time sitting down watching films or in bed. Annoyingly, the symptoms also include a headache, so I am both mentally and physically incapacitated.

Article writing

Last few days I have just been writing photography related articles during the day and watching the Star Wars films with Ben, Mauser, and Clare in the evenings.

I was thinking of going out on a walk today, but then it was overcast all day and rained for quite a bit of it.

Various

This morning and part of the afternoon I was sorting out the formatting and adding photos to my how to photograph butterflies mini e-book.

In the afternoon I went out on a walk to see if there were any butterflies around down the old railway line, as I had found that I don't actually seem to have many good butterfly photos. I only saw a couple of speckled woods, and a few whites though, and none of them wanted to get into positions that would make for a good photo. It was a nice walk though, and I collected some seeds from some of the wild flowers, and took photos of other stuff.

When I got back home I spent quite a while in the back garden trying to get a good photo of a butterfly on the Buddleia, but again didn't have much luck.

In the evening I watched Star Wars Episode 2 Attack of The Clones with Mauser, Bo, and Clare. I thought it was quite good, about as good as the Phantom Menace. While it doesn't have much Jar-jar Binks, it does have incredibly cheesy dialogue between the characters of Anakin and Padme. I didn't notice Warrick Davies in there (not sure if he was or not), and Padme's house looked like Porco Rosso's (or maybe it was Porco Rosso's girlfriend's) house.

For the rest of the evening I plasticined up a body for Adam Smith, though I didn't finish it yet.

Article writing and Phantom Menace watching

Today I was mostly writing an article about butterfly photography. It was only meant to be a standard article size, but then it ended up quite long (2250 words), so I will make it into a mini e-book.

In the evening I watched Star Wars Episode 1 The Phantom Menace with Mauser, Bo, and Clare. It's quite good really, but the non-main characters all seemed to be played by rubbish actors, or were told to speak strangely so it sounds like they can't act.

Article writing

This morning I was mostly checking my emails and also went through a couple more SSSEO videos. Then the SSSEO website stopped working so I put into practice some of the stuff the videos I'd watched so far had said to do. I did that for the rest of the morning and most of the afternoon. I also cleaned the bird feeders in the afternoon.

In the evening I did some more armature making for plasticine models.

Various

This morning I was mainly finishing setting up the w3 total cache plugin for wordpress on my photo tips website. In the afternoon I was watching some of the stupidly simple SEO videos.

In the evening I watched Boukengers with Billy and then did some model painting and armature making. I painted Ronald's shoes red and made two adult armatures and one child armature. You can see them in the photo below along with two already done armatures, as well as Lenin and Karl Marx (who are lacking armatures), and Sir Fred Goodwin (who is the only finished model so far with an armature). I might re-make Lenin and Karl Marx's bodies with armatures.

Also, since it was on my camera's memory card, here is a photo from a few days ago. On the right are trees and on the left is a jack o' lantern and some electrics to make it light up.

Sarah's Wedding

Yesterday we all had to get up early and went off to Sarah's wedding in Sussex. We had to wait around for about an hour or two at the Torch Trust for the Blind, where we were going to be staying overnight. Then we went down to the church. The Wedding ceremony was really good, with a live band.

After the ceremony was more waiting around before we went off to the place where the reception was being held. Then more waiting around until the meal. During the meal there was a good Jazz band playing live.

After the meal we just sat down doing nothing for a few hours. Luckily there was a good live band who were playing for quite a bit of the time.

As you can see from the photo of Laddie above, we got rather bored. If I'd had an ipad or something so I could get some work done while I was there I wouldn't have minded waiting around so much. I had a pin held in a mini clothes peg on top of my head, but Clare thought there was someone at the wedding that wanted to kill me and was going to push the pin into my head?!?

I felt sorry for the Wedding photographer as he was taking photos all the time from the start of the ceremony at around 2pm, and was still at the reception when we left around 11pm. I expect he was also taking photos of Sarah getting ready, etc. before the ceremony started as well. And then after such a long day he would have hundreds, possibly thousands, of photos to sort through and process the best ones.

A couple of things I noticed about the photographer was that during the ceremony he was shooting with no flash, and seemed to be using some quite slow shutter speeds while shooting handheld (you can tell by the shutter sound). Then for group photos at the reception he was using two speedlights with an SU-800 on the camera, one on a light stand to the right, and one held by a human light stand to the left. The flashes were pointed directly at the people being photographed rather than bounced off the white ceiling of the marquee.

Considering there was a white wall right behind the people, I would have thought this method of direct lighting would be quite harsh with strong shadows. But I think the results will probably be quite good, he certainly seemed to know what he was doing.

On Sunday (today) we came back home. I updated my pog website and made a farmhouse fruitcake.

Wordpressing

This morning I was mainly doing some more website stuff. I had a problem with IE9 in that the developer tools wouldn't let me turn off javascript. Weirdly, I found out that you need to run IE9 as an administrator to be able to disable javascript!?!

After lunch I topped the pond up and read a series of articles on photographers who covered the world trade centre collapse.

In the rest of the afternoon and quite a bit of the evening I was doing more wordpress stuff, mostly trying to set up the mailpress plugin. It took me a while to get it configured, but it does look quite good.

In the evening I also made some more armatures for plasticine figures with L.

Wordpressing

Yesterday and today I have just been trying to get the w3 total cache plugin for wordpress set up on my live website. It seems to me that the site is actually slower with the plugin enabled though.

Once I have everything set up correctly I will maybe do some tests with the plugin activated and deactivated to see what difference it makes.

Wordpressing

I spent the large majority of today trying to get the w3 total cache plugin for wordpress working on my development site. The problem was that the configuration it wrote to the site's nginx .conf file didn't work properly. I found another configuration listed here: High-Performance WordPress with W3 Total Cache and Nginx, but that didn't work either.

Eventually I did get it working okay, but then found nginx wasn't gzipping static files at all, so looked into that, but couldn't get it working.

I also looked at setting up an email list for my photo tips website. Mail Chimp had been recommended, but they include your address on all emails, and I don't really want random people knowing my home address.

In the evening I watched the first half of the England vs. Wales game with Ben, Clare, and Mauser. But it was so boring we couldn't be bothered to watch the second half.

Wordpressing

Today I was working on my ebay wordpress plugin. I spent quite a while trying to find out why something wasn't working as I thought it should, only to find that it was because a value in the database was different to what I was expecting. I suspect I changed it when testing the admin panel for the plugin.

I also had some trouble with the images not appearing quite how I wanted when using the http://www.professorcloud.com/mainsite/carousel.htm">jquery carousel plugin and spent quite a bit of time playing with that before I got it how I wanted by editing the plugin js file.

When the carousel aspect of the ebay plugin (which is the only piece I am interested in making sure works properly immediately) seemed to be working okay, I activated the wp-minify plugin. This is meant to concatenate and minify all your js and css files. Unfortunately it didn't work. When I viewed the url of the file, it was all gobbledygook. It seems the problem was probably that wp-minify gzips the data and then it gets gzipped again or something.

Anyway, I couldn't see an option to turn gzip off but had a look at the PHP minify page, which is what wp-minify uses. Wp-minify allows you to specify additional parameters to PHP minify, so I wanted to check whether there was a turn off gzip option. However, I noticed on the PHP minify page it said that it was also used by the W3 Total Cache Wordpress plugin.

So I had a look at that, and the video on there mentioned that you use it with Amazon S3 as a CDN. On the video they said it only cost them about 85¢ a month for the Amazon service. So that got me wondering about it myself. Currently I use HostGator for archiving the full size photos from my photo website. At the moment its about 20GB, but it will probably be 30GB once I move some more photos across from my photo websites hosting account, which is with WebFaction.


AWS charges $0.093 per GB per month for Reduced Redundancy Storage, which 'only' gives 99.99% durability and 99.99% availability. I would guess this is probably at least as good as what HostGator offers. So for 30GB this would work out at $2.79/month.


For HostGator, I'm not exactly sure what I'm paying. It says my billing is tri-annually, and I paid $143.10. So this would work out to $3.975/month. Certainly more expensive than Amazon S3, but this also includes unlimited bandwidth and web hosting (S3 is only suitable for static files as far as I know). Anyway, since I seem to have already paid for 3 years of Hostgator (about 2 years left now), the point is moot really.

In the evening I watched an episode of Boukengers with Belly, then we tested out our latest armature for plasticine models. Unfortunately it didn't work that well as the model still kept bending and falling over.

The main problem is that the thicker wire is too strong to bend and thinner wire is too weak. This armature was made with twisted thinner wire, so next time we'll try twisted wire doubled up and twisted. I think that probably we won't bother with an animation, but armatures still make it easier to get more natural poses for photos.

Watching films

This morning I started preparing some more pog images for my pog website and went to Church.

In the afternoon I updated the pog website and watched "The Great White Silence", with Mauser and Bo. It was a documentary made on Scott's expedition to the South Pole.

In the evening I watched Ong Bak 2 with Mauser. I thought it was really good, kind of like a 70's kung-fu film but much better production values.

Stats checking

Today I was intending to check whether my ebay wordpress plugin was working, and if not, then fix it. But actually I ended up checking stats for my different websites when I remembered I hadn't done that for a while.

I don't really know how to use website stats to get info that allows me to improve my sites, I just use them to check I don't have any big error problems. I changed all the awstats .conf files on my Hostgator account to be much simpler and include a main .conf file for all the standard config, so I hope I didn't break them. I will find out when I remember to check the stats again, which might be quite a while.

Websiting

I spent most of today going back through older posts on my photo tips website. Some of the flickr photos weren't showing up, so I had to fix that by uploading the problem photos to my own hosting and then changing the images src to point there. One of the photos was still on flickr, but I noticed that the user had changed the description of the photo to say that you need to ask them if you want permission to use it.

While I can understand that, the reason I like using Flickr for finding CC licensed photos is that it saves quite a bit of time - you don't need to spend time messaging the photo's owner and waiting for their reply, and the owner doesn't have to spend time checking and replying to every single license request they get. What would be nice is if Flickr added another CC category - CC with reservations. This could be for photos where the author is willing to give a CC license but would like you to contact them first (for example so they can check you're not going to use the photo to promote something they don't agree with).

I think that putting your photos under a CC license and then adding restrictions in the photo description would also be problematic for services that use the Flickr API - the API would think that the photo was CC licensed and can't parse the description for any restrictions. As it currently stands I think it would be better for people who want to add restrictions to usage of their photos to just put them as All Rights Reserved, and then write in the description that can be CC licensed on request.

Anyway, rather than contacting the user in question to get permission to use a photo I'd already been using, I just swapped for a different CC licensed photo. Quicker and easier.

Many of the older posts on my website had a few amazon ads for relevant books etc. in them but used images that were licensed CC non-commercial to illustrate the article. Personally I wouldn't say this was commercial use, but the ads don't make any money anyway so I removed them unless they were illustrating part of the article. (e.g. For a section of an article about tripods it makes sense to have a photo of a tripod from amazon).

Also, for an article about stock photography I replaced most of the 'stock style' Flickr photos I had in there with photos from istock.

In the evening I did some research on a possible Scotland holiday, and did some armature making & plasticine modelling as well.