This morning I went on Animal Crossing, then went to Church.
After church I sorted some clothes.
After dinner I went on Animal Crossing and ran a virus check on my pictures folder in the hope that AVG would tell me if there was any corrupted files it couldn't scan. It didn't say it couldn't scan any files, so I'm hoping there aren't any corrupted files.
I quick formatted my old 1TB drive that I was storing my pictures on before, and then backed up my non picture files and folders to it. While that was working I checked deviant art, Andy Rouse's blog, Moose Peterson's blog, dpreview News page, Flickr, The Luminous Landscape, and DPReview Canon Lens forum.
In the evening, the backup was still going on, so I couldn't really do any work on my PC (except browsing the internet, since I don't backup my browser/temporary internet files), so I made a cake with L. Moccle was meant to make it with L, but he refused and went off to Mansfield or Worksop instead.
I checked Juza Nature Forums and panoguide, and then just watched my backup progress for ages as I wanted to start backing up to my other 1TB drive (since the backup takes ages, thought I might as well try and get it done overnight), and was too sleepy to do anything constructive.
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Dinner: Chilli con carne; rice; tortilla chips; grated cheese. Pudding was a fudge flavour Cadbury's ice cream cone. Coffee; piece of Fairtrade fudge.
Tea: Cheese on toast; couple of leaves of little gem lettuce; Fake Kettle Chips Sea Salt & Black Pepper flavour; Slice of toast with blackcurrant jam; 5 strawberries; 2 plums; Chocolate wafer biscuit; cup o' tea.
Sunday, 30 August 2009
Saturday, 29 August 2009
Getting awstats working/perl running as CGI in Nginx
This morning I was trying to get awstats working on the web server. I had extracted the awstats folder to a web-accessible location on the web server, and also set up the awstats.mysite.conf file.
When I ran the update command line
Then this morning I tried running the command to update again
So I updated the log format in the nginx configuration again, deleted all the current logs again, restarted nginx, and then visited the website to get a log entry. Now finally the update command
I tried accessing http://mysite.com/awstats/wwwroot/cgi-bin/awstats.pl?config=mysite, but just got prompted to download the file.
So I searched to find out how to run perl scripts as CGI under nginx, and found a tutorial Nginx - Perl FastCGI How To. There is also an Nginx Wiki article - Nginx Simple CGI.
Following the tutorial tutorial Nginx - Perl FastCGI How To, I downloaded the perl FCGI wrapper, which requires the FCGI Perl module. So next I needed to check if the FCGI perl module was already installed. I found out how to do that at Checking to see if a Perl module is in your @INC path:
I found it wasn't installed, so I had to download and install it. I followed the guide Installing Perl Modules into a Nonstandard Directory. All you do is untar the tarball, cd into the directory, then run
Next I needed to modify the perl FastCGI wrapper script so that it would look in the correct location for the FCGI module, since I had installed it in a non standard location. I found a tutorial on how to do this: What to do when Perl modules aren't in their normal locations. All I needed to do was before
Except this didn't actually work. When I tried running the script, it still couldn't find the FCGI perl module. What I actually had to add was
After setting that to the correct value, and of course setting the correct location for the socket file to be created, I could now start the perl FCGI wrapper
Next I needed to configure nginx, this was similar to what is specified in the Nginx - Perl FastCGI How To, except that my cgi-bin folder wasn't located at the root of my site, so I had to change
to
After restarting nginx I could finally access awstats, however I was currently accessing it via http://mysite.com/awstats/wwwroot/cgi-bin/awstats.pl?config=mysite, and wanted to try and get it so I could access it via http://mysite.com/awstats/awstats.pl?config=mysite, as is suggested in the guide to setting up awstats.
This meant implementing the equivalent of the
After lunch I went on Animal Crossing, then I carried on trying to get awstats working on the webserver.
After reading a bit of the NGinx wiki I tried:
So finally I came up with this, which seems to work:
I spent most of the afternoon playing on the Kendo style game on Wii Sports Resort. First I got the maximum rating you can get in the duel (2500 points) using L's technique of pressing block just as you hit the other person, which seems to make it so you can hit them even when they're blocking.
Then I tried the challenge mode (think that's what it's called), and I got to the last person on the reverse castle level, and they only had one heart, but then they beat me. Most of the time I got beaten quite near the start of the level though.
After dinner we (me, Moccle, L and McRad) went out on walk between Gartree and Foxton.
I finished copying my pictures to my 2 new 1.5TB drives (I had been doing this all day, as takes a few hours to copy to each drive), and then swapped the 1TB drive in my PC that currently had my pictures for one of the new 1.5TB drives.
After getting that working I tried to see if there were any programs I could use to make sure no files had been corrupted when I copied them across before I format the old 1TB drive. Unfortunately I didn't find anything other than a perl script for checking jpegs in a specific folder and moving them to a folder called 'bad' if they are corrupted. Unfortunately the site that the perl script was hosted on doesn't seem to exist any longer, so I couldn't actually download it.
I also found a page that said most antivirus programs will let you know if they come across a corrupted file. So I might try doing a virus check on the drive tomorrow. Synkron doesn't complain about any corrupted files, but I don't know whether this means there aren't any or not.
The weather was a mixture of sun and cloud in the morning, then mostly cloudy in the afternoon, then a mixture of sun and cloud again in the evening. It looked like there was probably a decent sunset. It was quite windy all day, though nowhere near as windy as the last few days.
Food
Breakfast: Blackcurrant jam toast sandwich; cup o' tea.
Lunch: Mature cheddar cheese with little gem lettuce and sliced raddish sandwich; Fake Kettle chips sea salt and black pepper flavour; 3x strawberries; 2x plums; Chocolate Wafer biscuit; cup o' tea; 4x pieces Sainsburys Truffle Chocoate.
Dinner: 2x delee Chilli sausages; baked beans; mashed potato. Pudding was a strawberry yoghurt.
After walk snack: Coffee; 2x Shortbread fingers.
When I ran the update command line
perl $HOME/path/to/awstats.pl -config=mysite -update, I got a message that my logs didn't match what I'd specified in the .conf file. So I checked and found that I had $http_x_forwarded_for on the end of each line of my log. So I removed that from the nginx log format, deleted all the logs, and restarted nginx last night.Then this morning I tried running the command to update again
perl $HOME/path/to/awstats.pl -config=mysite -update, but got the same message that the actual log format didn't match what I'd specified in the awstats site configuration file. Both what the log format should look like and what it did look like looked exactly the same to me, but after a bit of puzzling I realised that I had two spaces between the HTTP Status code and the bytes transferred value, when it should be just one space between them.So I updated the log format in the nginx configuration again, deleted all the current logs again, restarted nginx, and then visited the website to get a log entry. Now finally the update command
perl $HOME/path/to/awstats.pl -config=mysite -update would work.I tried accessing http://mysite.com/awstats/wwwroot/cgi-bin/awstats.pl?config=mysite, but just got prompted to download the file.
So I searched to find out how to run perl scripts as CGI under nginx, and found a tutorial Nginx - Perl FastCGI How To. There is also an Nginx Wiki article - Nginx Simple CGI.
Following the tutorial tutorial Nginx - Perl FastCGI How To, I downloaded the perl FCGI wrapper, which requires the FCGI Perl module. So next I needed to check if the FCGI perl module was already installed. I found out how to do that at Checking to see if a Perl module is in your @INC path:
perl -e 'use FCGI;' - if it prints nothing, the FCGI perl module is installed, if it says it can't locate the module, then it's not installed.I found it wasn't installed, so I had to download and install it. I followed the guide Installing Perl Modules into a Nonstandard Directory. All you do is untar the tarball, cd into the directory, then run
perl Makefile.PL PREFIX=$HOME/perlModules
make
make test
make installNext I needed to modify the perl FastCGI wrapper script so that it would look in the correct location for the FCGI module, since I had installed it in a non standard location. I found a tutorial on how to do this: What to do when Perl modules aren't in their normal locations. All I needed to do was before
use FCGI; I added use lib '/home/username/perlModules';Except this didn't actually work. When I tried running the script, it still couldn't find the FCGI perl module. What I actually had to add was
use lib '/home/username/perlModules/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi';, as this was where the FCGI.pm module had actually installed to.After setting that to the correct value, and of course setting the correct location for the socket file to be created, I could now start the perl FCGI wrapper
perl /path/to/fastcgi-wrapper.pl&Next I needed to configure nginx, this was similar to what is specified in the Nginx - Perl FastCGI How To, except that my cgi-bin folder wasn't located at the root of my site, so I had to change
location ~ ^/cgi-bin/.*\.cgi$ {to
location ~ /cgi-bin/.*\.cgi$ {After restarting nginx I could finally access awstats, however I was currently accessing it via http://mysite.com/awstats/wwwroot/cgi-bin/awstats.pl?config=mysite, and wanted to try and get it so I could access it via http://mysite.com/awstats/awstats.pl?config=mysite, as is suggested in the guide to setting up awstats.
This meant implementing the equivalent of the
Alias /awstatsclasses "/usr/local/awstats/wwwroot/classes/"
Alias /awstatscss "/usr/local/awstats/wwwroot/css/"
Alias /awstatsicons "/usr/local/awstats/wwwroot/icon/"
ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/" location /awstatsclasses {
root /home/username/webapps/htdocs/admin/awstats/wwwroot/classes;
}
location /awstatscss {
root /home/username/webapps/htdocs/admin/awstats/wwwroot/css;
}
location /awstatsicons {
root /home/username/webapps/htdocs/admin/awstats/wwwroot/icon;
}
location /awstats {
root /home/username/webapps/htdocs/admin/awstats/wwwroot/cgi-bin;
}After lunch I went on Animal Crossing, then I carried on trying to get awstats working on the webserver.
After reading a bit of the NGinx wiki I tried:
location /awstatsclasses/ {
alias /home/username/webapps/htdocs/admin/awstats/wwwroot/classes/;
}
location /awstatscss/ {
alias /home/username/webapps/htdocs/admin/awstats/wwwroot/css/;
}
location /awstatsicons/ {
alias /home/username/webapps/htdocs/admin/awstats/wwwroot/icon/;
}
location /awstats/ {
alias /home/username/webapps/htdocs/admin/awstats/wwwroot/cgi-bin/;
}So finally I came up with this, which seems to work:
#awstats config
location /awstatsclasses/ {
alias /home/username/webapps/htdocs/admin/awstats/wwwroot/classes/;
}
location /awstatscss/ {
alias /home/username/webapps/htdocs/admin/awstats/wwwroot/css/;
}
location /awstatsicons/ {
alias /home/username/webapps/htdocs/admin/awstats/wwwroot/icon/;
}
location /awstats/ {
alias /home/username/webapps/htdocs/admin/awstats/wwwroot/cgi-bin/;
if (-e $request_filename)
{rewrite ^/awstats/(.*)$ /awstats/wwwroot/cgi-bin/$1 last;}
}
I spent most of the afternoon playing on the Kendo style game on Wii Sports Resort. First I got the maximum rating you can get in the duel (2500 points) using L's technique of pressing block just as you hit the other person, which seems to make it so you can hit them even when they're blocking.
Then I tried the challenge mode (think that's what it's called), and I got to the last person on the reverse castle level, and they only had one heart, but then they beat me. Most of the time I got beaten quite near the start of the level though.
After dinner we (me, Moccle, L and McRad) went out on walk between Gartree and Foxton.
I finished copying my pictures to my 2 new 1.5TB drives (I had been doing this all day, as takes a few hours to copy to each drive), and then swapped the 1TB drive in my PC that currently had my pictures for one of the new 1.5TB drives.
After getting that working I tried to see if there were any programs I could use to make sure no files had been corrupted when I copied them across before I format the old 1TB drive. Unfortunately I didn't find anything other than a perl script for checking jpegs in a specific folder and moving them to a folder called 'bad' if they are corrupted. Unfortunately the site that the perl script was hosted on doesn't seem to exist any longer, so I couldn't actually download it.
I also found a page that said most antivirus programs will let you know if they come across a corrupted file. So I might try doing a virus check on the drive tomorrow. Synkron doesn't complain about any corrupted files, but I don't know whether this means there aren't any or not.
The weather was a mixture of sun and cloud in the morning, then mostly cloudy in the afternoon, then a mixture of sun and cloud again in the evening. It looked like there was probably a decent sunset. It was quite windy all day, though nowhere near as windy as the last few days.
Food
Breakfast: Blackcurrant jam toast sandwich; cup o' tea.
Lunch: Mature cheddar cheese with little gem lettuce and sliced raddish sandwich; Fake Kettle chips sea salt and black pepper flavour; 3x strawberries; 2x plums; Chocolate Wafer biscuit; cup o' tea; 4x pieces Sainsburys Truffle Chocoate.
Dinner: 2x delee Chilli sausages; baked beans; mashed potato. Pudding was a strawberry yoghurt.
After walk snack: Coffee; 2x Shortbread fingers.
Friday, 28 August 2009
Formatting hard drives
This morning my new hard drives from ebuyer finally arrived (they were supposed to have arrived by Wednesday).
On the drive label it had 3 jumper settings:
As far as I could work out, it seems that 1.5GB PHY means the drive will run in SATA 1 mode (1.5Gb/s) instead of SATA 2 (3Gb/s).
It seems that SSC is to do with electrical interference, and should not be turned on.
PUIS means the drive doesn't spin up as soon as power is required, but rather waits until it is needed before spinning up.
I put one of the drives in my SATA hotswap bay, and it's now formatting, which will take absolutely ages. Then I need to copy all my pictures to it, which again will take ages, then I need to do it all again with the other drive. After that I can swap the current 1TB drive in the PC with one of the 1.5TB drives (I think I'll use one of the Western Digital drives I got today since they are 'Green' instead of the Seagate drive I got a few months ago).
Then I'll need to format the 1TB drive that I've just removed, and backup my non-picture files to it, and then do the same to my current 1TB backup drive. Then I need to remove all the non picture files and folders from my current 1.5TB backup drive. And of course, for all this I'll need to create new Synkron backup files for each drive.
So it will take a long time, but I'll end up with: 1TB of storage for non-photos and 2 1TB backup drives; 1.5TB of storage for photos and 2 1.5TB backup drives. The benefit of having 2 backups of everything means that if your drive dies, and then your backup drive dies, you still have another backup.
This situation is very unlikely, but given the amount of time I've put into all the files stored on my PC, it's a relatively cheap insurance policy.
I checked my email, which took quite a while, then spent the rest of the morning working on a privacy policy for my pog website.
After lunch I watched part of a 'Me too!' episode, then I went on Animal Crossing. I checked my email and then carried on working on my privacy policy.
I finished getting most of the privacy policy done, and then saw that someone had replied to my question about showing a breakdown of requests returning 301 in awstats. About 16.20 the first of my two new hard drives had finally finished formatting, so I removed it and started formatting the other new drive.
For getting a breakdown of requests returning a 301 HTTP Status Code, it was suggested that I follow the awstats HTTP Error Drilldown Functionality guide. After following that I had a 301 section in my awstats report, but it was empty.
After some messing about I found that to get the section populated with data I had to delete the current awstats files for the domain/website in question, and then run an update for the site from the command line.
In the evening I mostly played on Wii Sports Resort, I'm still stuck on the Reverse Castle level of the Kendo Challenge (or whatever it's called). It's 22.15 now, and my 2nd new hard drive has just finished formatting. So I'll have to do the actual backups to the drives tomorrow.
The weather today started off with blue skies but quite quickly got cloudy, and rained a bit. Most of the day the sky was full of big dark clouds, but not actually raining, and often the sun would be shining through breaks in the clouds. It was very windy, which made it quite cold. There was a nice sunset.
Food
Breakfast: Toasted teacake with Flora buttery; cup o' tea.
Lunch: Mature cheddar cheese with sliced raddish and little gem lettuce sandwich; 2x Plums; 3x Strawberries; Chocolate wafer biscuit; cup o' tea.
Dinner: Battered fish portion; chips; peas; salt; ground black pepper; vinegar. Pudding was plum crumble with spleenvap. Coffee.
On the drive label it had 3 jumper settings:
- Jumpered pins 1 and 2 enables SSC (Speed Spectrum Clocking)
- Jumpered pins 3 and 4 enables PUIS (Power Up In Standby)
- Jumpered pins 5 and 6 enables 1.5GB PHY
As far as I could work out, it seems that 1.5GB PHY means the drive will run in SATA 1 mode (1.5Gb/s) instead of SATA 2 (3Gb/s).
It seems that SSC is to do with electrical interference, and should not be turned on.
PUIS means the drive doesn't spin up as soon as power is required, but rather waits until it is needed before spinning up.
I put one of the drives in my SATA hotswap bay, and it's now formatting, which will take absolutely ages. Then I need to copy all my pictures to it, which again will take ages, then I need to do it all again with the other drive. After that I can swap the current 1TB drive in the PC with one of the 1.5TB drives (I think I'll use one of the Western Digital drives I got today since they are 'Green' instead of the Seagate drive I got a few months ago).
Then I'll need to format the 1TB drive that I've just removed, and backup my non-picture files to it, and then do the same to my current 1TB backup drive. Then I need to remove all the non picture files and folders from my current 1.5TB backup drive. And of course, for all this I'll need to create new Synkron backup files for each drive.
So it will take a long time, but I'll end up with: 1TB of storage for non-photos and 2 1TB backup drives; 1.5TB of storage for photos and 2 1.5TB backup drives. The benefit of having 2 backups of everything means that if your drive dies, and then your backup drive dies, you still have another backup.
This situation is very unlikely, but given the amount of time I've put into all the files stored on my PC, it's a relatively cheap insurance policy.
I checked my email, which took quite a while, then spent the rest of the morning working on a privacy policy for my pog website.
After lunch I watched part of a 'Me too!' episode, then I went on Animal Crossing. I checked my email and then carried on working on my privacy policy.
I finished getting most of the privacy policy done, and then saw that someone had replied to my question about showing a breakdown of requests returning 301 in awstats. About 16.20 the first of my two new hard drives had finally finished formatting, so I removed it and started formatting the other new drive.
For getting a breakdown of requests returning a 301 HTTP Status Code, it was suggested that I follow the awstats HTTP Error Drilldown Functionality guide. After following that I had a 301 section in my awstats report, but it was empty.
After some messing about I found that to get the section populated with data I had to delete the current awstats files for the domain/website in question, and then run an update for the site from the command line.
In the evening I mostly played on Wii Sports Resort, I'm still stuck on the Reverse Castle level of the Kendo Challenge (or whatever it's called). It's 22.15 now, and my 2nd new hard drive has just finished formatting. So I'll have to do the actual backups to the drives tomorrow.
The weather today started off with blue skies but quite quickly got cloudy, and rained a bit. Most of the day the sky was full of big dark clouds, but not actually raining, and often the sun would be shining through breaks in the clouds. It was very windy, which made it quite cold. There was a nice sunset.
Food
Breakfast: Toasted teacake with Flora buttery; cup o' tea.
Lunch: Mature cheddar cheese with sliced raddish and little gem lettuce sandwich; 2x Plums; 3x Strawberries; Chocolate wafer biscuit; cup o' tea.
Dinner: Battered fish portion; chips; peas; salt; ground black pepper; vinegar. Pudding was plum crumble with spleenvap. Coffee.
Thursday, 27 August 2009
Trying site log analysers
This morning I tried setting up awstats on my Ubuntu Virtual Machine to see if I could get it working, and what sort of info it would give me.
Getting it installed wasn't too hard, just had to follow the instructions. The only problem I had was that first of all the icons/images weren't showing up. This was because the default value for the icons directory in the .conf file is a different value to the one the instructions say to add to your apache config. So updating the .conf file with the correct value for the icons directory (DirIcons="/awstatsicons" instead of DirIcons="/icon") fixed that.
The main thing I wanted to check was whether awstats could be used for multiple domains/websites. Before I installed it, I thought that I would need to install it in each directory that contained a website. But I was totally wrong, basically you create a different .conf file for each of your websites, and then in your .conf files you specify the log file for that website/domain (amongst many other options).
There seems to be two ways of accessing awstats - you can generate the reports at the command line if you have SSH/command line access, or otherwise you can place the awstats folder in a web accessible location, and then load up the perl script through your browser, specifying the .conf file for the site you want stats for in the url query string.
awstats gave me quite a lot of info, importantly including a list of pages/requests generating a 404 error, so you can see where you're getting 404s and fix it. It didn't give a list of what requests generated a 301 HTTP status code though, so I asked on the awstats forums if this information was possible to get.
After lunch I went on Animal Crossing for a bit, then I went out on a walk with Rad and Lad.
When we got back from the walk I tried installing Webalizer to see if that would give a list of pages/requests generating 404s and 301s. When trying to
So I googled for
So I gave up trying to build libgd-devel from source, and tried searching the synaptic package manager for libgd. There were loads of results, but I eventually found one called libgd2-xmp-dev which had the correct description (GD Graphics Library Version 2 Development Version), so I installed that.
After getting that installed I could finally install Webalizer. I ran Webalizer, and couldn't see any way to get a list of pages/requests returning 404, let alone a list of pages/requests returning 301. The other stats about page requests etc. I'm not too bothered about as I also plan on running Google Analytics, so it's mainly 301s and 404s I wanted the log analysers for.
After dinner I went on Animal Crossing for a bit, then I went on Wii Sports Resort, then went in the garden and took a few photos while topping up the pond with the hose.
The rest of the evening I was trying to get my photo website working properly. I was having trouble with an SQL query, so eventually I posted it to the sitepoint forums to try and get some help with it (it seems that quite often threads at the official mysql forums go unanswered).
Food
Breakfast: Toasted tea cake with Flora buttery; cup o' tea.
Lunch: Herby chicken slices with mayonnaise, sliced raddish and lettuce sandwich; 2x plums; slice of date cake with toffee & walnut sauce; Tesco fake caramel Rocky; cup o' tea.
Dinner: Spaghetti; Bacon sauce stuff; Ground black pepper. Pudding was ice cream and strawberries. Coffee; Piece of Sainsbury's Caramel Chocolate; Piece of Sainsbury's Turkish Delight Chocolate; Piece of Sainsbury's Truffle Chocolate.
Supper: Slice of date cake with toffee & walnut sauce; coffee.
Getting it installed wasn't too hard, just had to follow the instructions. The only problem I had was that first of all the icons/images weren't showing up. This was because the default value for the icons directory in the .conf file is a different value to the one the instructions say to add to your apache config. So updating the .conf file with the correct value for the icons directory (DirIcons="/awstatsicons" instead of DirIcons="/icon") fixed that.
The main thing I wanted to check was whether awstats could be used for multiple domains/websites. Before I installed it, I thought that I would need to install it in each directory that contained a website. But I was totally wrong, basically you create a different .conf file for each of your websites, and then in your .conf files you specify the log file for that website/domain (amongst many other options).
There seems to be two ways of accessing awstats - you can generate the reports at the command line if you have SSH/command line access, or otherwise you can place the awstats folder in a web accessible location, and then load up the perl script through your browser, specifying the .conf file for the site you want stats for in the url query string.
awstats gave me quite a lot of info, importantly including a list of pages/requests generating a 404 error, so you can see where you're getting 404s and fix it. It didn't give a list of what requests generated a 301 HTTP status code though, so I asked on the awstats forums if this information was possible to get.
After lunch I went on Animal Crossing for a bit, then I went out on a walk with Rad and Lad.
When we got back from the walk I tried installing Webalizer to see if that would give a list of pages/requests generating 404s and 301s. When trying to
./configure I got an error that libgd wasn't installed. I had read earlier that Webalizer required the development versions of various packages, so I tried sudo apt-get install libgd-devel, however this said that no such package existed.So I googled for
libgd-devel, and found the libgd website, which said you had to checkout the source from CVS and compile the devel version yourself. So I tried the command on their website to login to CVS, but I didn't have CVS installed. So I installed CVS, then tried the command again, but got a message that the user didn't exist. I couldn't see any list of users you can login with or any other way to download the source on their website.So I gave up trying to build libgd-devel from source, and tried searching the synaptic package manager for libgd. There were loads of results, but I eventually found one called libgd2-xmp-dev which had the correct description (GD Graphics Library Version 2 Development Version), so I installed that.
After getting that installed I could finally install Webalizer. I ran Webalizer, and couldn't see any way to get a list of pages/requests returning 404, let alone a list of pages/requests returning 301. The other stats about page requests etc. I'm not too bothered about as I also plan on running Google Analytics, so it's mainly 301s and 404s I wanted the log analysers for.
After dinner I went on Animal Crossing for a bit, then I went on Wii Sports Resort, then went in the garden and took a few photos while topping up the pond with the hose.
The rest of the evening I was trying to get my photo website working properly. I was having trouble with an SQL query, so eventually I posted it to the sitepoint forums to try and get some help with it (it seems that quite often threads at the official mysql forums go unanswered).
Food
Breakfast: Toasted tea cake with Flora buttery; cup o' tea.
Lunch: Herby chicken slices with mayonnaise, sliced raddish and lettuce sandwich; 2x plums; slice of date cake with toffee & walnut sauce; Tesco fake caramel Rocky; cup o' tea.
Dinner: Spaghetti; Bacon sauce stuff; Ground black pepper. Pudding was ice cream and strawberries. Coffee; Piece of Sainsbury's Caramel Chocolate; Piece of Sainsbury's Turkish Delight Chocolate; Piece of Sainsbury's Truffle Chocolate.
Supper: Slice of date cake with toffee & walnut sauce; coffee.
Wednesday, 26 August 2009
Taking more moth photos
Last night before I went to bed I caught a nice white moth, so this morning I took some photos of it. It was a good little chap and didn't fly around except after a bit, and even then it gave me some warning by beating its wings for a bit before flying.
I also looked at P3P privacy policies a bit more. All the P3P Generators I looked at either charged a fee (typically around $30-40), or no longer existed. I tried to see if I could find an example P3P xml file, thinking that I could maybe just modify it to suit my own site. However, when I found one (example P3P policy xml file), it looked quite complicated. I think you'd need to read and understand the P3P spec to modify a P3P xml file yourself.
I'm not sure how privacy policies / P3P policies should incorporate external sources that you incorporate into your site (e.g. Google adwords and Amazon ads), so I think I won't bother implementing a P3P policy for my site at the moment.
I had already downloaded a privacy policy template from Website-law.co.uk, but I needed to check what the Amazon and Google Adwords privacy policies were. The Google ad words privacy policy was pretty easy to find, but the Amazon ads privacy policy I couldn't find from the Amazon Associates website. Doing some googling though, I came across this website, which has a nice easy to read privacy policy, and links to the privacy policies of external sites they include content from, including Amazon.
I checked the websqueeze, and saw I had a reply to one of my questions about grouped CSS selectors. My question was in the context of using them along with javascript to zebra stripe tables. In order to reply to the reply I wanted to check that the zebra striping with javascript on my site didn't work correctly, but that it did work correctly with CSS.
So I loaded up one of my pages displaying the problem in Firefox, but the problem wasn't there. I remembered that I'd updated to FF3.5.2 yesterday, and I think that this must support the CSS pseudo selectors needed to style the table in CSS, so the javascript zebra striping wasn't being used, and this is why the table was now displaying correctly.
So next I tried Opera, but again the table displayed correctly. I guess Opera supports the pseudo selector as well. So I tried IE7, pretty sure that wouldn't support the CSS selector needed to style the table with CSS. But when I tried to load the page, I got a message that
Anyway, when I did get the site loaded in IE7, I could see that I was correct in the javascript zebra striping of tables not working correctly, while CSS Zebra striping (as used in the other browsers) did work okay.
After lunch I went on Animal Crossing. Then I tried adding the CDs I'd bought off Play.com recently to my computer. Normally I just download the CD rips of the CDs I've bought from isohunt or pirate bay as it's much easier than ripping the CD yourself, plus the downloads include things like the album artwork. However, for most of the CDs, I couldn't find them on isohunt or pirate bay, so I just had to rip them myself, and then I used tag&rename to try and get the album art from amazon.
In the evening I watched 'Seven Swords' with Mockel, which had good action bits, but I didn't think the story was particularly great, which is kind of weird since it said it was based on a book.
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Lunch: 1¾ cheese on toasts; clementine; plum; Tesco fake caramel Rocky; cup o' tea.
Dinner: Meatballs; Chinese sauce stuff; rice; courgette. Pudding was plum crumble with custard. Coffee.
I also looked at P3P privacy policies a bit more. All the P3P Generators I looked at either charged a fee (typically around $30-40), or no longer existed. I tried to see if I could find an example P3P xml file, thinking that I could maybe just modify it to suit my own site. However, when I found one (example P3P policy xml file), it looked quite complicated. I think you'd need to read and understand the P3P spec to modify a P3P xml file yourself.
I'm not sure how privacy policies / P3P policies should incorporate external sources that you incorporate into your site (e.g. Google adwords and Amazon ads), so I think I won't bother implementing a P3P policy for my site at the moment.
I had already downloaded a privacy policy template from Website-law.co.uk, but I needed to check what the Amazon and Google Adwords privacy policies were. The Google ad words privacy policy was pretty easy to find, but the Amazon ads privacy policy I couldn't find from the Amazon Associates website. Doing some googling though, I came across this website, which has a nice easy to read privacy policy, and links to the privacy policies of external sites they include content from, including Amazon.
I checked the websqueeze, and saw I had a reply to one of my questions about grouped CSS selectors. My question was in the context of using them along with javascript to zebra stripe tables. In order to reply to the reply I wanted to check that the zebra striping with javascript on my site didn't work correctly, but that it did work correctly with CSS.
So I loaded up one of my pages displaying the problem in Firefox, but the problem wasn't there. I remembered that I'd updated to FF3.5.2 yesterday, and I think that this must support the CSS pseudo selectors needed to style the table in CSS, so the javascript zebra striping wasn't being used, and this is why the table was now displaying correctly.
So next I tried Opera, but again the table displayed correctly. I guess Opera supports the pseudo selector as well. So I tried IE7, pretty sure that wouldn't support the CSS selector needed to style the table with CSS. But when I tried to load the page, I got a message that
After some messing around, it seemed that the problem was that IE didn't like the XSL stylesheet being on a different domain (well actually a different subdomain) to the main page. So changing the XSL file to be on the same domain as the webpage fixed it.<>The XML page cannot be displayed
Cannot view XML input using XSL style sheet. Please correct the error and then click the Refresh button, or try again later.
Access is denied. Error processing resource 'http://static1.photosite.com/xhtml.xsl'.
Anyway, when I did get the site loaded in IE7, I could see that I was correct in the javascript zebra striping of tables not working correctly, while CSS Zebra striping (as used in the other browsers) did work okay.
After lunch I went on Animal Crossing. Then I tried adding the CDs I'd bought off Play.com recently to my computer. Normally I just download the CD rips of the CDs I've bought from isohunt or pirate bay as it's much easier than ripping the CD yourself, plus the downloads include things like the album artwork. However, for most of the CDs, I couldn't find them on isohunt or pirate bay, so I just had to rip them myself, and then I used tag&rename to try and get the album art from amazon.
In the evening I watched 'Seven Swords' with Mockel, which had good action bits, but I didn't think the story was particularly great, which is kind of weird since it said it was based on a book.
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Lunch: 1¾ cheese on toasts; clementine; plum; Tesco fake caramel Rocky; cup o' tea.
Dinner: Meatballs; Chinese sauce stuff; rice; courgette. Pudding was plum crumble with custard. Coffee.
Tuesday, 25 August 2009
Taking moth photos
This morning I carried on processing pics from the 2nd day of our holiday in Shropshire back at the start of July.
After lunch I played on Animal Crossing, then I took some moth photos for the rest of the afternoon and evening. Quite a bit of time was spent trying to get the moths to go on a piece of paper, as they kept just flying around the room. The moths I was photographing were the ones that I caught in Moccle's room the other day. There was also a Silver Y and a small moth that I had caught the same day that were in Moccle and L's bathroom.
One of the moths was dead, and another moth was dead and stuck in honey, so I couldn't even take its photo. When I tried to get the Silver Y out of the insect container, it kept flying about like the maniac that it is, and I could feel my hand being fanned by its wings when it flew past my hand. Eventually it landed on the teaspoon that had honey on it, and then it beed a maniac of drinking the honey, so I could get a few photos of it.
Also in the evening I watched bits of 'The Quiet Man', and an episode of 'The Equalizer'.
I had a look at generating a P3P Privacy Policy for my websites. The W3C article How to Create and Publish Your Company's P3P Policy (in 6 Easy Steps) suggests a number of P3P generators that you can use, the first of which is the IBM P3P Policy Editor. However, when you come to download the IBM P3P Policy Editor and read the licence conditions, you can't actually use it:
So I tried the second P3P Generator listed - PrivacyBot.com. However, this wouldn't load the website. I just got
The weather today was a mixture sun, clouds and rain. There was a very nice sunset with lots of thin streamy clouds lit up yellow by the sun before it set, then later there was some Cumulonimbus on the horizon.
Food
Breakfast: Strawberry jam toast sandwich; cup o' tea.
Lunch: Beetroot sandwich; Clementine; plum; slice of date cake with toffee and walnut sauce; Caramel Rocky; cup o' tea.
Dinner: Breaded chicken burger with tomato ketchup and grated cheese in a cob; bowl of vegetable fake cup a soup; slice of toast. Pudding was a Muller fruit corner. Coffee; 2 pieces of Sainsbury's Turkish Delight Chocolate; piece of Sainsbury's Truffle Chocolate.
After lunch I played on Animal Crossing, then I took some moth photos for the rest of the afternoon and evening. Quite a bit of time was spent trying to get the moths to go on a piece of paper, as they kept just flying around the room. The moths I was photographing were the ones that I caught in Moccle's room the other day. There was also a Silver Y and a small moth that I had caught the same day that were in Moccle and L's bathroom.
One of the moths was dead, and another moth was dead and stuck in honey, so I couldn't even take its photo. When I tried to get the Silver Y out of the insect container, it kept flying about like the maniac that it is, and I could feel my hand being fanned by its wings when it flew past my hand. Eventually it landed on the teaspoon that had honey on it, and then it beed a maniac of drinking the honey, so I could get a few photos of it.
Also in the evening I watched bits of 'The Quiet Man', and an episode of 'The Equalizer'.
I had a look at generating a P3P Privacy Policy for my websites. The W3C article How to Create and Publish Your Company's P3P Policy (in 6 Easy Steps) suggests a number of P3P generators that you can use, the first of which is the IBM P3P Policy Editor. However, when you come to download the IBM P3P Policy Editor and read the licence conditions, you can't actually use it:
IBM grants You a limited, nonexclusive, nontransferable license to download, install, and use the Program during the evaluation period solely for internal testing and evaluation purposes and to provide feedback to IBM.
So I tried the second P3P Generator listed - PrivacyBot.com. However, this wouldn't load the website. I just got
Connection Interrupted
The document contains no data.
The network link was interrupted while negotiating a connection. Please try again.
The weather today was a mixture sun, clouds and rain. There was a very nice sunset with lots of thin streamy clouds lit up yellow by the sun before it set, then later there was some Cumulonimbus on the horizon.
Food
Breakfast: Strawberry jam toast sandwich; cup o' tea.
Lunch: Beetroot sandwich; Clementine; plum; slice of date cake with toffee and walnut sauce; Caramel Rocky; cup o' tea.
Dinner: Breaded chicken burger with tomato ketchup and grated cheese in a cob; bowl of vegetable fake cup a soup; slice of toast. Pudding was a Muller fruit corner. Coffee; 2 pieces of Sainsbury's Turkish Delight Chocolate; piece of Sainsbury's Truffle Chocolate.
Monday, 24 August 2009
Processing pics
This morning and this afternoon I was processing pics from the 2nd day of our holiday in Shropshire back at the start of July.
After dinner I watched 'The Good, The Bad, and The Weird' with Moccle, it was pretty good, and had great action scenes and special effects. I didn't think the story was as good as 'The Good, The Bad, and The Ugly' though, and of course it didn't have Clint Eastwood in it.
Food
Breakfast: Bowl of choco hoops; cup o' tea.
Lunch: Beef with mustard and salad sandwich; clementine; plum; Time out; cup o' tea.
Dinner: Minced beef with stuff inside a mostly hollowed out piece of Marrow with cheese on top; green beans; potato. Pudding was Date sponge stuff with toffee and walnut sauce that L and Clare made. Coffee; Piece of Fairtrade fudge.
After dinner I watched 'The Good, The Bad, and The Weird' with Moccle, it was pretty good, and had great action scenes and special effects. I didn't think the story was as good as 'The Good, The Bad, and The Ugly' though, and of course it didn't have Clint Eastwood in it.
Food
Breakfast: Bowl of choco hoops; cup o' tea.
Lunch: Beef with mustard and salad sandwich; clementine; plum; Time out; cup o' tea.
Dinner: Minced beef with stuff inside a mostly hollowed out piece of Marrow with cheese on top; green beans; potato. Pudding was Date sponge stuff with toffee and walnut sauce that L and Clare made. Coffee; Piece of Fairtrade fudge.
Sunday, 23 August 2009
Taking & processing photos and catching moths
This morning I went on Animal Crossing, then went to Church with Moccle and L.
After Church we had lunch, then I went on Animal Crossing again. I took a few photos in the back garden, then processed and sorted photos (mainly those from our short walk yesterday, though also a few macros) for the rest of the afternoon. When I was done I did a backup
Before dinner (we had dinner quite late), I went in the back garden and took a few more photos. Then after dinner I took some photos of a moth that had just been sitting on the draining board all day.
After that I processed and sorted the new macro photos I had taken and also checked The Luminous Landscape. Moccle had loads of moths in his room, and one of them landed on the 'Get on da mic' game, so it obviously wanted to get on the Mic! (Mic is another name for Moccle, though it's pronounced Mick rather than Mike).
Then downstairs there was a totally massive daddy long legs. I would say it's body and wings were bigger than a damselfly, its body was probably over 5mm diameter, but less than 10mm. I caught it in Ben's net and after a couple of tries managed to get it out the window. If Ben's container had been empty, I might have tried to put it in that, but the container already had a few moths in it, and they'd escape if I tried to put the daddy long legs in there.
Also, Shaz came home in the evening.
The rest of the evening I did another backup and also caught some more moths.
The weather today was pretty hot today, sunny with clouds, and a bit windy. I think there was probably a sunset, though it didn't look particularly great from what I could see of it (which isn't much).
Food
Breakfast: Bowl of choco hoops; cup o' tea.
Lunch: 2x Cheese on toasts; salad; clementine; home grown plum; Time out (single); cup o' tea.
Afternoon Snack: Strawberry Cornetto; cup o' tea.
Dinner: Slice of Pepperoni pizza; most of a jacket potato. Pudding was mandarin cheesecake; coffee.
After Church we had lunch, then I went on Animal Crossing again. I took a few photos in the back garden, then processed and sorted photos (mainly those from our short walk yesterday, though also a few macros) for the rest of the afternoon. When I was done I did a backup
Before dinner (we had dinner quite late), I went in the back garden and took a few more photos. Then after dinner I took some photos of a moth that had just been sitting on the draining board all day.
After that I processed and sorted the new macro photos I had taken and also checked The Luminous Landscape. Moccle had loads of moths in his room, and one of them landed on the 'Get on da mic' game, so it obviously wanted to get on the Mic! (Mic is another name for Moccle, though it's pronounced Mick rather than Mike).
Then downstairs there was a totally massive daddy long legs. I would say it's body and wings were bigger than a damselfly, its body was probably over 5mm diameter, but less than 10mm. I caught it in Ben's net and after a couple of tries managed to get it out the window. If Ben's container had been empty, I might have tried to put it in that, but the container already had a few moths in it, and they'd escape if I tried to put the daddy long legs in there.
Also, Shaz came home in the evening.
The rest of the evening I did another backup and also caught some more moths.
The weather today was pretty hot today, sunny with clouds, and a bit windy. I think there was probably a sunset, though it didn't look particularly great from what I could see of it (which isn't much).
Food
Breakfast: Bowl of choco hoops; cup o' tea.
Lunch: 2x Cheese on toasts; salad; clementine; home grown plum; Time out (single); cup o' tea.
Afternoon Snack: Strawberry Cornetto; cup o' tea.
Dinner: Slice of Pepperoni pizza; most of a jacket potato. Pudding was mandarin cheesecake; coffee.
Saturday, 22 August 2009
Taking photos and not doing much else
This morning I took some photos in the back garden and also looked for Stephen Fry videos/audio on the pinternet for making a song/video with.
In the afternoon I went on Animal Crossing, then did about 30 minutes work on my photo website. After that I went in the back garden again, and took some more photos. I processed and sorted the photos, and went on the dpreview canon lens forums while I was waiting for the photos to process. I did a backup, then it was dinner time.
After dinner I took a few more photos in the back garden, then went on an evening walk with Moccle, L, Clare and Brian.
When we got home I went to see KK on Animal, then Clare cut my hair. After that I had a shower, then I went on the dpreview canon lens forums again while I was waiting for the photos I had taken on the walk to copy across to the PC (took over half an hour I think for some cheesun, and there was only 1GB of pics).
The weather today was a mixture of cloud and sun, with the sun shining most of the day. It was a bit windy (nowhere near as much as previous days though), and in the evening there was hardly any wind (just enough to be annoying when taking 3:1 macro photos). There was probably a nice sunset, though I didn't see it.
Food
Breakfast: Bowl of choco hoops; cup o' tea.
Lunch: Mature cheddar cheese with iceberg lettuce sandwich made with fresh bread-maker-made bread; Strawberry jam on a crust of fresh bread-maker-made bread; A couple of plums from the garden; Time out (single); cup o' tea.
Dinner: Peppery sausage; barbeque flavour chicken wing; mashed potato; grated mature cheddar cheese; ground black pepper; baked beans. Pudding was a cream and jam filled doughnut (delee). Coffee.
In the afternoon I went on Animal Crossing, then did about 30 minutes work on my photo website. After that I went in the back garden again, and took some more photos. I processed and sorted the photos, and went on the dpreview canon lens forums while I was waiting for the photos to process. I did a backup, then it was dinner time.
After dinner I took a few more photos in the back garden, then went on an evening walk with Moccle, L, Clare and Brian.
When we got home I went to see KK on Animal, then Clare cut my hair. After that I had a shower, then I went on the dpreview canon lens forums again while I was waiting for the photos I had taken on the walk to copy across to the PC (took over half an hour I think for some cheesun, and there was only 1GB of pics).
The weather today was a mixture of cloud and sun, with the sun shining most of the day. It was a bit windy (nowhere near as much as previous days though), and in the evening there was hardly any wind (just enough to be annoying when taking 3:1 macro photos). There was probably a nice sunset, though I didn't see it.
Food
Breakfast: Bowl of choco hoops; cup o' tea.
Lunch: Mature cheddar cheese with iceberg lettuce sandwich made with fresh bread-maker-made bread; Strawberry jam on a crust of fresh bread-maker-made bread; A couple of plums from the garden; Time out (single); cup o' tea.
Dinner: Peppery sausage; barbeque flavour chicken wing; mashed potato; grated mature cheddar cheese; ground black pepper; baked beans. Pudding was a cream and jam filled doughnut (delee). Coffee.
Friday, 21 August 2009
Redirecting from a 404 in Nginx
This morning I was trying to get the new version of my pog website to redirect requests for old pages to the old website (which I have mounted under a v1 subdomain). So, I was doing the same thing that I was most of yesterday.
First of all I tried a simple test case with a 404 page:
I went to a non-existent address on the website, and got the same thing I was seeing yesterday - a blank page. Checking the headers I saw the same thing too,
Visting the 404.php page directly, the redirection would work. So I guess that with NginX when it can't find a page and loads the error page, it must add the HTTP Status code after receiving the page back from fcgi, thus overwriting any HTTP Status headers you send from the PHP page.
In Apache though, the redirect from a 404 page seems to work fine.
Next I tried to find out how to do the redirection in the NginX config. I imagine it should be much less resource intensive and faster to process the redirect in Nginx as opposed to in PHP anyway.
I spent quite a few hours yesterday looking at this, but the solution was actually quite obvious, and I got it working on my first try today:
If you used rewrite rules for your old site, you would just need to add these rules after you check if the file exists on the old site.
After getting this working, I realised that when going to http://domain.com/old_file.html, I would be re-directed to http://www.domain.com/old_file.html, and then be redirected to http://oldsite.domain.com/old_file.html. I did some googling to see if there was any problem with a double redirect. I didn't find much, and nothing saying they were a big no-no, but the consensus seemed to be that it's best to avoid double redirects if possible. Double Redirects May Take Google More Time To Pick Up On.
So to prevent double redirects, what I had to do was to copy all of my redirect rules from the config for www.domain.com into the config for domain.com. Then I had to edit all the rewrite rules for domain.com so that they would perform a 301 permanent redirect to www.domain.com. Obviously, I didn't edit the redirect rules for redirecting to the old site, and left them redirecting there.
Then after all the rewrite rules, I just needed one last rewrite rule to catch any urls that weren't matched by any of the other rewrite rules, to redirect to www.domain.com.
So it makes the server config quite a bit longer, and you would also need to make sure that when you add/change any rewrite rules for www.domain.com, you also add/change them for domain.com. But it does solve the double redirect issue.
After lunch I watched the end of an episode of 'Me too!' with L. Granny Murray asked a girl to give her a twiddle (I think she meant a twirl), and also told the girl to put her clothes in her fag (no idea what she was on about there). Raymond did some skill faces while presenting a fashion show as well. Raymond's wife (fat old Dame Edna style taxi driving woman) also had the ultimate accessory according to Raymond - a hat that you can fold up WOW!!!!!!
I went on Animal Crossing, then me and L watched the end of In The Night Garden. Upsy Daisy loves her bed, and that most definitely was a pip.
The me and L went in the garden and filled up the pond as the water level was getting a bit low again.
After this I checked my email, then re-enabled APC (I had disabled it in case it was causing the issue with not being able to redirect from a 404 page), and also set date.timezone in php.ini on the web server. I tried to get my photosite working again on my local PC, but then it was dinner time.
After dinner I carried on trying to get my local photo website working again. I was having a problem with the imagick.so extension not being loaded into PHP for some reason (at least I got an error about class imagick not being defined).
I abandoned working on my photo website for a while to play on Wii Sports Resort. After that I tried to rip 'Tom Brown's Schooldays', not because it's a good film (I guess it's okay), but so me or Moccle can make a video using clips of Stephen Fry.
I spent ages trying to figure out how to do this. I did have a good guide to ripping dvds with Gordian Knot, but I couldn't find it. I found some guides online on Doom9 (though not as good as the PDF guide I couldn't find), and following them I got to a point where the said click on 'Save and encode'.
I spent ages looking at all the different tabs and googling to try and find this 'save and encode' button, but there was no 'save and encode' button on Gordian Knot. Eventually I managed to find the PDF guide to ripping dvds with Gordian Knot, and this said that you need to click 'save and encode' on the video preview window.
It seems that when I had earlier minimized all windows, it had made the Gordian Knot video preview window disappear, so this is why I hadn't found the 'save and encode' button. And the online guides to using Gordian Knot didn't say that the 'save and encode' button was on the video preview window.
So now I had found the 'save and encode' button, I could actually start encoding the ripped DVD files, though the actual encoding process takes ages.
I downloaded and compiled the latest imagick extension, and then imagick seemed to work okay again. I guess the problem was that I had just copied the imagick extension and all the other files over from one user account to a different one. (The old user account name matched my username at Evohosting, the new user account name matches my username at WebFaction). Since when you compile imagick you need to specify the location of ImageMagick (and I had now deleted the old user account), I think this would be why it didn't work.
The weather today was a mixture of sun and cloud. It was very windy all day and also rained a bit. I think there was a good sunset.
Food
Breakfast: Strawberry jam toast sandwich; cup o' tea.
Lunch: Cheese on toast; Lidl or Aldi fake Kettle Chips; Nectarine; plum (from the garden); Time out (single); cup o' tea.
Dinner: Battered Fish portion; buttered peas; potatoes; ground black pepper. Pudding was a raspberry ripple mousse and a milk chocolate digestive. Coffee; piece of Fairtrade fudge.
First of all I tried a simple test case with a 404 page:
<?php
header('HTTP/1.1 301 Moved Permanently', true, 301);
header('Location: http://www.google.com');
exit();
?>
error_page 404 /404.php;
I went to a non-existent address on the website, and got the same thing I was seeing yesterday - a blank page. Checking the headers I saw the same thing too,
Location: http://www.google.com
404 Not Found
Visting the 404.php page directly, the redirection would work. So I guess that with NginX when it can't find a page and loads the error page, it must add the HTTP Status code after receiving the page back from fcgi, thus overwriting any HTTP Status headers you send from the PHP page.
In Apache though, the redirect from a 404 page seems to work fine.
Next I tried to find out how to do the redirection in the NginX config. I imagine it should be much less resource intensive and faster to process the redirect in Nginx as opposed to in PHP anyway.
I spent quite a few hours yesterday looking at this, but the solution was actually quite obvious, and I got it working on my first try today:
location / {
index index.html;
root /path/to/site;
#If the file exists don't do any processing
if (-e $request_filename)
{break;}
#A rewrite rule
if (-e $request_filename.php) {
#rewrite any other page to page.php
rewrite ^/([^\.]+)$ /$1.php last;
}
#If the file exists on the old site, redirect there
if (-e /path/to/old/site$document_uri)
{
rewrite ^.*$ http://oldsite.domain.co.uk$document_uri permanent;
}
}
error_page 404 /404.php;If you used rewrite rules for your old site, you would just need to add these rules after you check if the file exists on the old site.
After getting this working, I realised that when going to http://domain.com/old_file.html, I would be re-directed to http://www.domain.com/old_file.html, and then be redirected to http://oldsite.domain.com/old_file.html. I did some googling to see if there was any problem with a double redirect. I didn't find much, and nothing saying they were a big no-no, but the consensus seemed to be that it's best to avoid double redirects if possible. Double Redirects May Take Google More Time To Pick Up On.
So to prevent double redirects, what I had to do was to copy all of my redirect rules from the config for www.domain.com into the config for domain.com. Then I had to edit all the rewrite rules for domain.com so that they would perform a 301 permanent redirect to www.domain.com. Obviously, I didn't edit the redirect rules for redirecting to the old site, and left them redirecting there.
Then after all the rewrite rules, I just needed one last rewrite rule to catch any urls that weren't matched by any of the other rewrite rules, to redirect to www.domain.com.
So it makes the server config quite a bit longer, and you would also need to make sure that when you add/change any rewrite rules for www.domain.com, you also add/change them for domain.com. But it does solve the double redirect issue.
After lunch I watched the end of an episode of 'Me too!' with L. Granny Murray asked a girl to give her a twiddle (I think she meant a twirl), and also told the girl to put her clothes in her fag (no idea what she was on about there). Raymond did some skill faces while presenting a fashion show as well. Raymond's wife (fat old Dame Edna style taxi driving woman) also had the ultimate accessory according to Raymond - a hat that you can fold up WOW!!!!!!
I went on Animal Crossing, then me and L watched the end of In The Night Garden. Upsy Daisy loves her bed, and that most definitely was a pip.
The me and L went in the garden and filled up the pond as the water level was getting a bit low again.
After this I checked my email, then re-enabled APC (I had disabled it in case it was causing the issue with not being able to redirect from a 404 page), and also set date.timezone in php.ini on the web server. I tried to get my photosite working again on my local PC, but then it was dinner time.
After dinner I carried on trying to get my local photo website working again. I was having a problem with the imagick.so extension not being loaded into PHP for some reason (at least I got an error about class imagick not being defined).
I abandoned working on my photo website for a while to play on Wii Sports Resort. After that I tried to rip 'Tom Brown's Schooldays', not because it's a good film (I guess it's okay), but so me or Moccle can make a video using clips of Stephen Fry.
I spent ages trying to figure out how to do this. I did have a good guide to ripping dvds with Gordian Knot, but I couldn't find it. I found some guides online on Doom9 (though not as good as the PDF guide I couldn't find), and following them I got to a point where the said click on 'Save and encode'.
I spent ages looking at all the different tabs and googling to try and find this 'save and encode' button, but there was no 'save and encode' button on Gordian Knot. Eventually I managed to find the PDF guide to ripping dvds with Gordian Knot, and this said that you need to click 'save and encode' on the video preview window.
It seems that when I had earlier minimized all windows, it had made the Gordian Knot video preview window disappear, so this is why I hadn't found the 'save and encode' button. And the online guides to using Gordian Knot didn't say that the 'save and encode' button was on the video preview window.
So now I had found the 'save and encode' button, I could actually start encoding the ripped DVD files, though the actual encoding process takes ages.
I downloaded and compiled the latest imagick extension, and then imagick seemed to work okay again. I guess the problem was that I had just copied the imagick extension and all the other files over from one user account to a different one. (The old user account name matched my username at Evohosting, the new user account name matches my username at WebFaction). Since when you compile imagick you need to specify the location of ImageMagick (and I had now deleted the old user account), I think this would be why it didn't work.
The weather today was a mixture of sun and cloud. It was very windy all day and also rained a bit. I think there was a good sunset.
Food
Breakfast: Strawberry jam toast sandwich; cup o' tea.
Lunch: Cheese on toast; Lidl or Aldi fake Kettle Chips; Nectarine; plum (from the garden); Time out (single); cup o' tea.
Dinner: Battered Fish portion; buttered peas; potatoes; ground black pepper. Pudding was a raspberry ripple mousse and a milk chocolate digestive. Coffee; piece of Fairtrade fudge.
Thursday, 20 August 2009
Trying to figure out why php headers not working on 404 page
This morning I tried to change my site config for the domain I use to access phpMyAdmin so that it would restrict access based on ip address range, as well as basic auth asking for a username and password (and then of course phpMyAdmin also requires a username and password to log in).
However, NginX uses the CIDR notation for specifying an ip address range, and although I read the Wikipedia article on CIDR, I couldn't really get my head round it, so I asked on the web squeeze if someone could tell me the correct value to put on the end to cover the ip address range I needed.
After that, I stopped MySQL listening on a port by using the
Then I checked my email, which took quite a while.
Some useful webpages I've used recently:
Optimizing FastCGI performance
PHP on Fire: Five Opcode Caches compared
Benchmarking Drupal with PHP op-code caches: APC, eAccelerator and XCache compared
Alternative PHP Cache (APC)
The reason I decided to use APC as opposed to XCache or eAccelerator is that APC has some of the PHP developers working on it, so it seems that it will likely be more compatible with PHP and have less problems (though I have read reports of problems from users of all 3 opcode caches). It also seems that the performance difference between the 3 programs isn't very great.
After lunch I went on Animal Crossing, then I tried to get my photosite working again on my local PC. Since I had changed my file structure to match the web server, it needed a few bits changing to make it work again.
Then I tried the latest ImageMagick to see if it had fixed the problem of corrupting XMP. It seems it has been fixed, at least partially, though I still couldn't write an exif tag to the converted file using exiftool (not that I actually need to be able to do that).
After that I discovered that my 404 error page wasn't redirecting to the old website when a page from the old website was requested, but was just displaying a blank page instead. Try as I might, I couldn't get the 404 page to redirect to the correct page, and I couldn't work out how to write an equivalent rule in Nginx either.
The page works fine in my local environment, running on Apache, but not on the web server under Nginx. When I checked the headers being sent by the page, I could see the Location header had been sent (though the page wasn't actually re-directed), and the 301 redirect header didn't seem to be sent, just the 404 header (which I presume Nginx sends). I will try and set up a simple test on the server tomorrow to see if it works or not.
In the evening I also finished watching Thunderball with Moccle and L, and watched 'Tom Brown's School days', which has Stephen Fry in it, but weirdly he doesn't use an Apple Macbook to Tweet on Twitter in it at all!
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Lunch: BLT; Piece of (cold) Lemon Meringue pie; Nectarine; Time out (single); cup o' tea.
Dinner: Cottage pie; tomato ketchup; peas. Pudding was a Chocolate Chip Snackie bar. Coffee.
However, NginX uses the CIDR notation for specifying an ip address range, and although I read the Wikipedia article on CIDR, I couldn't really get my head round it, so I asked on the web squeeze if someone could tell me the correct value to put on the end to cover the ip address range I needed.
After that, I stopped MySQL listening on a port by using the
skip-networking option in my.cnf. Not listening on ports when all connections are local is 'Highly Recommended' by MySQL.Then I checked my email, which took quite a while.
Some useful webpages I've used recently:
Optimizing FastCGI performance
PHP on Fire: Five Opcode Caches compared
Benchmarking Drupal with PHP op-code caches: APC, eAccelerator and XCache compared
Alternative PHP Cache (APC)
The reason I decided to use APC as opposed to XCache or eAccelerator is that APC has some of the PHP developers working on it, so it seems that it will likely be more compatible with PHP and have less problems (though I have read reports of problems from users of all 3 opcode caches). It also seems that the performance difference between the 3 programs isn't very great.
After lunch I went on Animal Crossing, then I tried to get my photosite working again on my local PC. Since I had changed my file structure to match the web server, it needed a few bits changing to make it work again.
Then I tried the latest ImageMagick to see if it had fixed the problem of corrupting XMP. It seems it has been fixed, at least partially, though I still couldn't write an exif tag to the converted file using exiftool (not that I actually need to be able to do that).
After that I discovered that my 404 error page wasn't redirecting to the old website when a page from the old website was requested, but was just displaying a blank page instead. Try as I might, I couldn't get the 404 page to redirect to the correct page, and I couldn't work out how to write an equivalent rule in Nginx either.
The page works fine in my local environment, running on Apache, but not on the web server under Nginx. When I checked the headers being sent by the page, I could see the Location header had been sent (though the page wasn't actually re-directed), and the 301 redirect header didn't seem to be sent, just the 404 header (which I presume Nginx sends). I will try and set up a simple test on the server tomorrow to see if it works or not.
In the evening I also finished watching Thunderball with Moccle and L, and watched 'Tom Brown's School days', which has Stephen Fry in it, but weirdly he doesn't use an Apple Macbook to Tweet on Twitter in it at all!
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Lunch: BLT; Piece of (cold) Lemon Meringue pie; Nectarine; Time out (single); cup o' tea.
Dinner: Cottage pie; tomato ketchup; peas. Pudding was a Chocolate Chip Snackie bar. Coffee.
Wednesday, 19 August 2009
Trying to set php.ini location
This morning I vacuumed my room and cleaned up tea stains from the floor and door.
After that I checked my email, then purchased some more hard drives. I was trying to decide between Samsung EcoGreen F2 1.5TB Hard Drive SATAII 32MB Cache and Western Digital 1.5TB Hard Drive SATAII 7200rpm 32MB Cache. I couldn't find any tests comparing them, but thought I might as well go with the Western Digital one given its higher spindle speed, even though it costs slightly more.
Then I tried to find info regarding discounts with Google Checkout as I was sure I had read about them giving you £5 for free or something similar, but I couldn't find anything apart from an offer that had finished a long time ago. I also found that ebuyer weren't listed with quidco, mutualpoints, or pigsback, so I couldn't get any money/points back that way either.
After that, I tried to see if I could get PHP to load a php.ini from a different location using the different locations/methods mentioned on The configuration file PHP Manual Page.
The first thing I did was to change the user.ini file for my site so that it contained all the php.ini directives needed for the site to run correctly. This meant that if my trying to load a different php.ini file didn't work, the site would still work okay (as I renamed the php.ini file PHP was using by default to make sure it would load the php.ini in a different location rather than the standard php.ini).
I'm using the spawn-fcgi init script for CentOS from the lighttpd wiki. I was editing the top section, which looks like so:
I started by trying to pass the php.ini location to the php cgi program using the -c command line option:
Looking at this thread: spawn-fcgi and php.ini, I noticed that the correct form was to pass the -c parameter to php inside the -f parameter to spawn-fcgi. So I tried changing the
So I quickly scrapped that, and tried instead hard-coding the php location and -c parameter into the start function:
I had a look at the man page for spawn-fcgi and also running
Next I tried setting the php.ini location using the PHPRC environment variable, an example of how to do this is given at spawn-fcgi ignores php command line arguments like "-c /var/www/userxyz/php/php.ini". So I reverted my previous changes, and then just changed the top of the init script to add in a PHPRC variable:
What I had to do was to also change the line
After lunch I went on Animal Crossing for a bit, then I went in the garden and found a nice Poplar hawkmoth caterpillar in the green bin as Clare had trimmed back the Salix Caprea bush yesterday and put all the bits in the green bin. I put the caterpillar back on a low branch of the Salix Caprea bush, then went and got my 100mm macro lens, as I wanted to try and get a full body shot of it, and it was much too large to get a full body shot with the MP-E.
But by the time I'd swapped my lenses (maybe 30 seconds), I couldn't find it anywhere. Either it was really fast and had run deep inside the bush where I couldn't see, or run quite a bit away from the bush, or it had just disappeared. It was pretty hot outside (a proper summer day, weird!), so I came back inside.
At the moment I have my computer on a desk in front of the window. This is good as it means I can open the window and let the breeze blow into the back of the computer to help it keep cool. However, it also means that the computer gets direct sunlight on it, which obviously heats it up.
I'm guessing that the sunshine heats it up more than the breeze cools it down, so I thought that it might be a good idea to try moving my desk and computer to a different part of the room where it won't get direct sunlight (or I can at least close the curtains to stop it getting direct sunlight). I can, of course, close the curtains with the computer where it is now, but this means that the hot air expelled out of the back of the computer hits the curtains, and has nowhere to go, so not good for keeping it cool either.
I won't move my computer right now, but when I do, I think I might take it apart and give it a good clean inside, and re-seat the CPU fan as I probably used far too much thermal paste when I installed it.
I carried on looking at the different php.ini locations, and tried to get php.ini to load from the current directory of the php script/page running, but couldn't get it to do this.
Next I tried a php.ini file in Nginx's conf directory, however that didn't load either. I tried a php.ini file in the Nginx directory, and this did load.
Then I tried a php.ini file in the php directory. This didn't load. I tried a php.ini file in the bin folder inside the php directory, and this did load.
So now I think that I know all the different locations that php looks for php.ini in and how to set the location that php looks for php.ini in, but I'm not quite sure how the actual locations where my installation of php looks for php.ini map to the locations where PHP is supposed to look for php.ini files, so I posted a reply to a thread on the WebFaction forums I had started yesterday, asking about this.
In the evening I installed APC and spent quite a long time trying to get NGinx to deny dot files. Whatever I tried didn't seem to work, then I tried clearing my cache, and suddenly I couldn't access the dot files any more. So it had probably actually been working okay with most of the different ways of trying to block dot files that I had tried, just my browser was loading the dot file from cache.
Also in the evening, I watched an episode of Simpsons with Moccle and L, and an hour of the James Bond film 'Thunderball' with them, until Ben had to go to bed.
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Lunch: 1½ cheese on toasts; Nectarine; plum; Slice of sultana and cherry cake; cup o' tea.
Afternoon snack: Ice cream with chocolate sauce; cup o' tea.
Dinner: Beef burger with pus; baked beans; chips. Pudding was (hot) Lemon Meringue. Coffee.
After that I checked my email, then purchased some more hard drives. I was trying to decide between Samsung EcoGreen F2 1.5TB Hard Drive SATAII 32MB Cache and Western Digital 1.5TB Hard Drive SATAII 7200rpm 32MB Cache. I couldn't find any tests comparing them, but thought I might as well go with the Western Digital one given its higher spindle speed, even though it costs slightly more.
Then I tried to find info regarding discounts with Google Checkout as I was sure I had read about them giving you £5 for free or something similar, but I couldn't find anything apart from an offer that had finished a long time ago. I also found that ebuyer weren't listed with quidco, mutualpoints, or pigsback, so I couldn't get any money/points back that way either.
After that, I tried to see if I could get PHP to load a php.ini from a different location using the different locations/methods mentioned on The configuration file PHP Manual Page.
The first thing I did was to change the user.ini file for my site so that it contained all the php.ini directives needed for the site to run correctly. This meant that if my trying to load a different php.ini file didn't work, the site would still work okay (as I renamed the php.ini file PHP was using by default to make sure it would load the php.ini in a different location rather than the standard php.ini).
I'm using the spawn-fcgi init script for CentOS from the lighttpd wiki. I was editing the top section, which looks like so:
FCGI_DAEMON="/usr/local/bin/spawn-fcgi"
FCGI_PROGRAM="/usr/local/bin/php-cgi"
FCGI_SOCKET="/tmp/php-fastcgi.sock"
FCGI_PIDFILE="/var/run/spawn-fcgi.pid"
PHP_FCGI_CHILDREN=4
PHP_FCGI_MAX_REQUESTS=1000
prog="fcgi"
export PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS
RETVAL=0
start() {
echo -n $"Starting $prog: "
daemon $FCGI_DAEMON -f $FCGI_PROGRAM -s $FCGI_SOCKET -C $PHP_FCGI_CHILDREN -P $FCGI_PIDFILE
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
return $RETVAL
}
I started by trying to pass the php.ini location to the php cgi program using the -c command line option:
start() {
echo -n $"Starting $prog: "
daemon $FCGI_DAEMON -f $FCGI_PROGRAM -s $FCGI_SOCKET -c /path/to/my/php.ini -C $PHP_FCGI_CHILDREN -P $FCGI_PIDFILE
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
return $RETVAL
}Looking at this thread: spawn-fcgi and php.ini, I noticed that the correct form was to pass the -c parameter to php inside the -f parameter to spawn-fcgi. So I tried changing the
FCGI_PROGRAM="/usr/local/bin/php-cgi" line to FCGI_PROGRAM="/usr/local/bin/php-cgi -c /path/to/my/php.ini". However, now I couldn't stop the spawned processes when I tried php-fcgictl restart (I named my init script php-fcgictl).So I quickly scrapped that, and tried instead hard-coding the php location and -c parameter into the start function:
start() {
echo -n $"Starting $prog: "
daemon $FCGI_DAEMON -f "/usr/local/bin/php-cgi -c /path/to/my/php.ini" -s $FCGI_SOCKET -C $PHP_FCGI_CHILDREN -P $FCGI_PIDFILE
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
return $RETVAL
}I had a look at the man page for spawn-fcgi and also running
spawn-fcgi -h to see the help. The manual said: -fSo after reading the manual and the help option, I tried changing the start function to:
Filename of the FastCGI application to spawn. This option is deprecated and it is recommend to always specify the application (absolute path) and its parameters after "--";
the fcgiapp parameter is directly used for the exec() call, while for starting the binary given with -f /bin/sh is needed (which may not be available in a chroot).
start() {
echo -n $"Starting $prog: "
daemon $FCGI_DAEMON -s $FCGI_SOCKET -C $PHP_FCGI_CHILDREN -P $FCGI_PIDFILE -- $FCGI_PROGRAM -c /path/to/my/php.ini
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/$prog
return $RETVAL
}phpinfo(), I could see that the php.ini file was now being loaded, as 'Loaded Configuration File' was set to /path/to/my/php.ini. However, 'Configuration File (php.ini) Path' was still set to /path/to/php/lib. According to the PHP Manual page the first place that PHP looks for php.ini is the location specified with the -c command line parameter, so I don't understand why 'Configuration File (php.ini) Path' was still set to /path/to/php/lib.Next I tried setting the php.ini location using the PHPRC environment variable, an example of how to do this is given at spawn-fcgi ignores php command line arguments like "-c /var/www/userxyz/php/php.ini". So I reverted my previous changes, and then just changed the top of the init script to add in a PHPRC variable:
FCGI_DAEMON="/usr/local/bin/spawn-fcgi"
FCGI_PROGRAM="/usr/local/bin/php-cgi"
FCGI_SOCKET="/tmp/php-fastcgi.sock"
FCGI_PIDFILE="/var/run/spawn-fcgi.pid"
PHP_FCGI_CHILDREN=4
PHP_FCGI_MAX_REQUESTS=1000
PHPRC="/path/to/my/"
prog="fcgi"
phpinfo() it was still looking for php.ini in /path/to/php/lib, and no configuration file was being loaded.What I had to do was to also change the line
export PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS to export PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS PHPRC, and when I ran phpinfo() it was still looking for php.ini in /path/to/php/lib, but my php.ini file at /path/to/my/php.ini was now being loaded.After lunch I went on Animal Crossing for a bit, then I went in the garden and found a nice Poplar hawkmoth caterpillar in the green bin as Clare had trimmed back the Salix Caprea bush yesterday and put all the bits in the green bin. I put the caterpillar back on a low branch of the Salix Caprea bush, then went and got my 100mm macro lens, as I wanted to try and get a full body shot of it, and it was much too large to get a full body shot with the MP-E.
But by the time I'd swapped my lenses (maybe 30 seconds), I couldn't find it anywhere. Either it was really fast and had run deep inside the bush where I couldn't see, or run quite a bit away from the bush, or it had just disappeared. It was pretty hot outside (a proper summer day, weird!), so I came back inside.
At the moment I have my computer on a desk in front of the window. This is good as it means I can open the window and let the breeze blow into the back of the computer to help it keep cool. However, it also means that the computer gets direct sunlight on it, which obviously heats it up.
I'm guessing that the sunshine heats it up more than the breeze cools it down, so I thought that it might be a good idea to try moving my desk and computer to a different part of the room where it won't get direct sunlight (or I can at least close the curtains to stop it getting direct sunlight). I can, of course, close the curtains with the computer where it is now, but this means that the hot air expelled out of the back of the computer hits the curtains, and has nowhere to go, so not good for keeping it cool either.
I won't move my computer right now, but when I do, I think I might take it apart and give it a good clean inside, and re-seat the CPU fan as I probably used far too much thermal paste when I installed it.
I carried on looking at the different php.ini locations, and tried to get php.ini to load from the current directory of the php script/page running, but couldn't get it to do this.
Next I tried a php.ini file in Nginx's conf directory, however that didn't load either. I tried a php.ini file in the Nginx directory, and this did load.
Then I tried a php.ini file in the php directory. This didn't load. I tried a php.ini file in the bin folder inside the php directory, and this did load.
So now I think that I know all the different locations that php looks for php.ini in and how to set the location that php looks for php.ini in, but I'm not quite sure how the actual locations where my installation of php looks for php.ini map to the locations where PHP is supposed to look for php.ini files, so I posted a reply to a thread on the WebFaction forums I had started yesterday, asking about this.
In the evening I installed APC and spent quite a long time trying to get NGinx to deny dot files. Whatever I tried didn't seem to work, then I tried clearing my cache, and suddenly I couldn't access the dot files any more. So it had probably actually been working okay with most of the different ways of trying to block dot files that I had tried, just my browser was loading the dot file from cache.
Also in the evening, I watched an episode of Simpsons with Moccle and L, and an hour of the James Bond film 'Thunderball' with them, until Ben had to go to bed.
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Lunch: 1½ cheese on toasts; Nectarine; plum; Slice of sultana and cherry cake; cup o' tea.
Afternoon snack: Ice cream with chocolate sauce; cup o' tea.
Dinner: Beef burger with pus; baked beans; chips. Pudding was (hot) Lemon Meringue. Coffee.
Tuesday, 18 August 2009
Setting Per site php.ini
Sometimes when I start my PC, the Desktop resolution has been adjusted to 1600x1200, which is quite random, and annoying as I have to change the resolution back to 1280x960. So I decided to have a quick Google and see if I could find a solution to this.
I found a thread about Dell laptops reverting back to a lower resolution on hibernation/restart: Screen resolution changes on restart/hibernation, so a slightly different problem to mine, but along the same lines in that the resolution changes.
There was an answer in the thread, and everyone who tried it said it worked:
So I tried disabling that. Whether it will work or not, I'm not sure, since my problem is slightly different to one posted there.
After that I tried to get one of my websites to redirect from domain.com to www.domain.com. When the site was running on Apache, I had done this by adding the following to a .htaccess file located in the site root:
In NginX, you can't use .htaccess files, so the rewrite rules must go in the site config:
Or, probably a better way to do it would be to separate the www. sub domain and base domain into separate configs, then the rewrite rule won't have to be checked when a user is browsing the www. subdomain:
Next I was trying to see how I could set different php.ini values for each of my sites. With PHP running under Apache, you can use
When running PHP through suPHP or php_suexec, you can use a php.ini file (which you must place in each directory you want it to take effect).
With running PHP through spawn-fcgi, I have seen some threads suggesting running a different PHP process for each site you want a different php.ini file for, passing the -c option to spawn-fcgi with the location of the php.ini file you want loaded. Each process with a different php.ini file would need to have it's own port/socket as well. Then in the site config, you fcgi_pass to the correct php process. This thread shows how to do it: Alternative php.ini for a subdomain on Nginx.
However, the problem with this is that you need to run more php processes, and if a site suddenly gets heavy traffic, it will only be able to use that php processes that you've set the site to use. e.g. If I have 5 sites, and run 5 php processes, if one site gets heavy traffic, then normally that traffic would be shared between the 5 processes. However, if I'm running a different php.ini for each process (and so running each process on a different port/socket), and run 1 process for each site (so 5 processes in total again), then if one site gets some heavy traffic, your one php process won't be able to cope. You could run 5 processes per site (so 25 processes total), but you'd need lots of memory for that.
Luckily php5.3 has
Then I noticed in the
So now I can set a different include_path php.ini directive for each of my sites.
In the afternoon I did a bit more website stuff, played on Animal Crossing, and looked into some computer stuff someone had emailed me about.
After dinner I finished looking into the computer issues, and also watched a video by the next famous youtube person, who is a kid who mimes along to songs in the New York Apple store - quite funny seeing what all the people in the background do, and the kid doesn't get put off by them at all - nicholifavs: viking remix dat new new part 4.
Also, when looking into the computer stuff, the person who emailed me wanted to view a flash video (.swf format, not .flv). On my PC, I can just double click a .swf file, and it will open in Adobe Flash Player. However, I couldn't find a download for a standalone player on Adobe's website, just the browser plugins. Googling for Standalone Flash player, I found this blog post: Stand alone flash player, which explains how to download a stand alone Flash player.
However, it's a standalone executable, so you don't even need to install it. This means that it doesn't associate the .swf Shockwave Flash filetype with the Flash Player in Windows, so if you double click on a .swf file, you'll still get the 'Windows cannot open this file' dialog.
I loaded the .swf file into the stand alone Flash player, and noticed an option under the File Menu - 'Create Projection'. I chose this option, and it seems to convert the .swf file into a standalone Flash executable. So you can just double click the file that it creates, and it opens automatically in Flash player, presumably with no need for you to even have flash installed.
After that I went in the garden for a bit, got bitten by some mosquitoes (seems to be a lot around at the moment), and also took a few photos of a moth and a small caterpillar. Then I copied the pics to my comp and processed them, but for some chees-un, they all came out as .dngs instead of .jpgs. So I deleted all the .dngs, but managed to delete all the raw .cr2 files as well, and didn't realise until after I'd emptied the recycle bin. Doh!
So I started up Handy Recovery to try and recover the files, then after I'd started recovering the recycle bin I remembered that all the RAW files were still on the memory card. So I cancelled the recovery, then copied across the files from the memory card again, and processed them again (but making sure the output was set to JPG instead of DNG).
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Lunch: Prosciutto cotto ham with sliced cherry plum tomatoes and iceberg lettuce sandwich; a few Aldi/Lidl kettle chips; Slice of cherry and sultana cake; cup o' tea.
Dinner: Chicken pie; green beans; potatoes; gravy. For pudding I had a Strawberry and white chocolate crumble bits or sumat Muller Corner (was very nice). Coffee.
Supper: Dark chocolate digestive; hob-nob; cup o' tea.
I found a thread about Dell laptops reverting back to a lower resolution on hibernation/restart: Screen resolution changes on restart/hibernation, so a slightly different problem to mine, but along the same lines in that the resolution changes.
There was an answer in the thread, and everyone who tried it said it worked:
Open Task Scheduler ( All Programs/Accessories/System Tools/Task Scheduler)
Then go to "Task Scheduler Library/Microsoft/Windows/MobilePC" .
Then disabled "TMM"
So I tried disabling that. Whether it will work or not, I'm not sure, since my problem is slightly different to one posted there.
After that I tried to get one of my websites to redirect from domain.com to www.domain.com. When the site was running on Apache, I had done this by adding the following to a .htaccess file located in the site root:
RewriteEngine On
RewriteCond %{HTTP_HOST} ^milkcapmania\.com$ [NC]
RewriteRule ^(.*)$ http://www.milkcapmania.com/$1 [R=301,QSA,L]
In NginX, you can't use .htaccess files, so the rewrite rules must go in the site config:
server {
listen 80;
server_name www.domain.com domain.com;
location / {
index index.html;
root /my/site/root;
#rewrite to www.
if ($host ~ ^domain.com$) {
rewrite ^(.*)$ http://www.domain.com$1 permanent;
}
}
#Other site config stuff here
}Or, probably a better way to do it would be to separate the www. sub domain and base domain into separate configs, then the rewrite rule won't have to be checked when a user is browsing the www. subdomain:
server {
listen 80;
server_name domain.com;
#rewrite to www.
rewrite ^(.*)$ http://www.domain.com$1 permanent;
}
server {
listen 80;
server_name www.domain.com;
location / {
index index.html;
root /my/site/root;
}
#Other site config stuff here
}Next I was trying to see how I could set different php.ini values for each of my sites. With PHP running under Apache, you can use
php_value php.ini_property = value
php_flag php.ini_property = value
When running PHP through suPHP or php_suexec, you can use a php.ini file (which you must place in each directory you want it to take effect).
With running PHP through spawn-fcgi, I have seen some threads suggesting running a different PHP process for each site you want a different php.ini file for, passing the -c option to spawn-fcgi with the location of the php.ini file you want loaded. Each process with a different php.ini file would need to have it's own port/socket as well. Then in the site config, you fcgi_pass to the correct php process. This thread shows how to do it: Alternative php.ini for a subdomain on Nginx.
However, the problem with this is that you need to run more php processes, and if a site suddenly gets heavy traffic, it will only be able to use that php processes that you've set the site to use. e.g. If I have 5 sites, and run 5 php processes, if one site gets heavy traffic, then normally that traffic would be shared between the 5 processes. However, if I'm running a different php.ini for each process (and so running each process on a different port/socket), and run 1 process for each site (so 5 processes in total again), then if one site gets some heavy traffic, your one php process won't be able to cope. You could run 5 processes per site (so 25 processes total), but you'd need lots of memory for that.
Luckily php5.3 has
".htaccess" style user-defined php.ini files support for CGI/FastCGII found a page that described this change in a bit more detail: What is new in PHP 5.3 - part 4: __callStatic, OpenID support, user.ini, XSLT profiling and more. However, try as I might, I couldn't get the options I added in my user.ini file to show that they were in effect in a
phpinfo() page.Then I noticed in the
phpinfo() page, the user_ini.filename value was .user.ini (note the dot before the filename). Actually, this is pretty obvious on the PHP Manual page for .user.ini files, which I had read, though somehow I hadn't noticed the dot before the filename.So now I can set a different include_path php.ini directive for each of my sites.
In the afternoon I did a bit more website stuff, played on Animal Crossing, and looked into some computer stuff someone had emailed me about.
After dinner I finished looking into the computer issues, and also watched a video by the next famous youtube person, who is a kid who mimes along to songs in the New York Apple store - quite funny seeing what all the people in the background do, and the kid doesn't get put off by them at all - nicholifavs: viking remix dat new new part 4.
Also, when looking into the computer stuff, the person who emailed me wanted to view a flash video (.swf format, not .flv). On my PC, I can just double click a .swf file, and it will open in Adobe Flash Player. However, I couldn't find a download for a standalone player on Adobe's website, just the browser plugins. Googling for Standalone Flash player, I found this blog post: Stand alone flash player, which explains how to download a stand alone Flash player.
However, it's a standalone executable, so you don't even need to install it. This means that it doesn't associate the .swf Shockwave Flash filetype with the Flash Player in Windows, so if you double click on a .swf file, you'll still get the 'Windows cannot open this file' dialog.
I loaded the .swf file into the stand alone Flash player, and noticed an option under the File Menu - 'Create Projection'. I chose this option, and it seems to convert the .swf file into a standalone Flash executable. So you can just double click the file that it creates, and it opens automatically in Flash player, presumably with no need for you to even have flash installed.
After that I went in the garden for a bit, got bitten by some mosquitoes (seems to be a lot around at the moment), and also took a few photos of a moth and a small caterpillar. Then I copied the pics to my comp and processed them, but for some chees-un, they all came out as .dngs instead of .jpgs. So I deleted all the .dngs, but managed to delete all the raw .cr2 files as well, and didn't realise until after I'd emptied the recycle bin. Doh!
So I started up Handy Recovery to try and recover the files, then after I'd started recovering the recycle bin I remembered that all the RAW files were still on the memory card. So I cancelled the recovery, then copied across the files from the memory card again, and processed them again (but making sure the output was set to JPG instead of DNG).
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Lunch: Prosciutto cotto ham with sliced cherry plum tomatoes and iceberg lettuce sandwich; a few Aldi/Lidl kettle chips; Slice of cherry and sultana cake; cup o' tea.
Dinner: Chicken pie; green beans; potatoes; gravy. For pudding I had a Strawberry and white chocolate crumble bits or sumat Muller Corner (was very nice). Coffee.
Supper: Dark chocolate digestive; hob-nob; cup o' tea.
Monday, 17 August 2009
Having a headache
This morning I finished off getting my pog website working on the server. Then I checked my running processes on the server, and discovered that there was an apache worker process for each of my websites, so about 5 or 6 processes, and each one was taking about 20MBs of memory each, which was using up all of my memory allowance (even though I've upgraded to the hosting package that's one above the most basic one recently).
I didn't realise that this was the case, and thought all my static/cgi/php apps would run through a global instance of apache, and so not affect my memory usage (which is why I ran my sites as static/cgi/php apps rather than going through my custom app which points to Nginx). So it seems I'll now need to change the site again to get it running through Nginx rather than apache (this will also involve moving the site files to a different directory, so some of the filepaths are going to get messed up).
After lunch I watched an episode of Me, too! with Lad, in which the school had their Sports day in a public park!?! Also Mickey John couldn't jump, then fell over and looked weird. 'Twas funny.
Then played on Animal for a bit, then went to bed as I had a bad headache. Later in the afternoon I got up as my headache wasn't getting any better and I couldn't get to sleep, and went in the garden. Weirdly there didn't seem to be any insects around, though when I had been in the garden at about 11am in the morning, there had been loads around.
After that I tried installing ImageMagick, and PHP5.3 (with imagick). I also checked that I had the latest version of MySQL 5.4, and I did. When installing PHP5.3, I got a couple of error messages when doing the ./configure, that it didn't understand the options --enable-fastcgi --enable-force-cgi-redirect and --with-ming. It seems that PHP5.3 is built with --enable-fastcgi --enable-force-cgi-redirect by default now, and I guess that ming is an extension that used to be included with PHP, but isn't any more. I looked it up on the PECL website, and it seems that it's used for creating Flash, so I don't actually need it, so I just ran ./configure again without those options.
Installing those took the rest of the afternoon, as I made a few mistakes trying to configure and make ImageMagick, so I had to re-do it a few times.
While I was waiting for stuff to configure, make, and make install, I checked Andy Rouse's blog, and Moose Peterson's blog.
In the evening I went on Wii Sports Resort for a bit, and also tried to configure Nginx so it could handle my sites instead of Apache. I'll need to do some more work on this tomorrow.
The weather today was a mixture of sun and clouds. I thought that the sunset was going to be rubbish as it looked like the sun was going to set behind a bank of cloud, as it often does. However, the cloud must not have been going all the way beyond the horizon, so the sun could actually shine up from underneath it after it had set.
Food
Breakfast: 2x slices of Malt loaf with butter; cup o' tea.
Lunch: Ham with mustard, sliced cherry plum tomatoes, sliced peppers, and iceberg lettuce sandwich; Banana; Fairtrade chocolate wafer biscuit; cup o' tea; Piece of Sainsbury's Turkish delight chocolate; Piece of Sainsbury's Truffle chocolate.
Dinner: Spaghetti bolognese; pieces of bacon; ground black pepper; parmesan (or similar) cheese. Pudding was a slice of Genoa fruitcake.
Supper: Butter ring biscuit (cannae remember what the correct name for them is); Dark chocolate digestive; cup o' tea.
I didn't realise that this was the case, and thought all my static/cgi/php apps would run through a global instance of apache, and so not affect my memory usage (which is why I ran my sites as static/cgi/php apps rather than going through my custom app which points to Nginx). So it seems I'll now need to change the site again to get it running through Nginx rather than apache (this will also involve moving the site files to a different directory, so some of the filepaths are going to get messed up).
After lunch I watched an episode of Me, too! with Lad, in which the school had their Sports day in a public park!?! Also Mickey John couldn't jump, then fell over and looked weird. 'Twas funny.
Then played on Animal for a bit, then went to bed as I had a bad headache. Later in the afternoon I got up as my headache wasn't getting any better and I couldn't get to sleep, and went in the garden. Weirdly there didn't seem to be any insects around, though when I had been in the garden at about 11am in the morning, there had been loads around.
After that I tried installing ImageMagick, and PHP5.3 (with imagick). I also checked that I had the latest version of MySQL 5.4, and I did. When installing PHP5.3, I got a couple of error messages when doing the ./configure, that it didn't understand the options --enable-fastcgi --enable-force-cgi-redirect and --with-ming. It seems that PHP5.3 is built with --enable-fastcgi --enable-force-cgi-redirect by default now, and I guess that ming is an extension that used to be included with PHP, but isn't any more. I looked it up on the PECL website, and it seems that it's used for creating Flash, so I don't actually need it, so I just ran ./configure again without those options.
Installing those took the rest of the afternoon, as I made a few mistakes trying to configure and make ImageMagick, so I had to re-do it a few times.
While I was waiting for stuff to configure, make, and make install, I checked Andy Rouse's blog, and Moose Peterson's blog.
In the evening I went on Wii Sports Resort for a bit, and also tried to configure Nginx so it could handle my sites instead of Apache. I'll need to do some more work on this tomorrow.
The weather today was a mixture of sun and clouds. I thought that the sunset was going to be rubbish as it looked like the sun was going to set behind a bank of cloud, as it often does. However, the cloud must not have been going all the way beyond the horizon, so the sun could actually shine up from underneath it after it had set.
Food
Breakfast: 2x slices of Malt loaf with butter; cup o' tea.
Lunch: Ham with mustard, sliced cherry plum tomatoes, sliced peppers, and iceberg lettuce sandwich; Banana; Fairtrade chocolate wafer biscuit; cup o' tea; Piece of Sainsbury's Turkish delight chocolate; Piece of Sainsbury's Truffle chocolate.
Dinner: Spaghetti bolognese; pieces of bacon; ground black pepper; parmesan (or similar) cheese. Pudding was a slice of Genoa fruitcake.
Supper: Butter ring biscuit (cannae remember what the correct name for them is); Dark chocolate digestive; cup o' tea.
Sunday, 16 August 2009
Having a headache
We didn't go to Church today as Moccle might have Swine Flu, so we didn't want to risk giving it to anyone else.
So in the morning I played on Animal, then I went in the garden to see if there were any insects around to try and take photos of.
I did some more website stuff, then after dinner I had a bad headache so I went to bed.
After a few hours I felt better so I got up, and went in the garden for a bit after eating tea.
In the evening I did some more website stuff, played on Wii Sports Resort a bit, and spent quite a while in the back garden trying to find any insects (don't know where they've all gone?)
Mainly I was trying to minify my CSS and javascript. I didn't want a solution that sat on the server, but rather to minify my CSS and javascript files before I upload them to the server. I had found JSMin, which I compiled using the instructions here: Building C programs on Linux, but it didn't seem to work properly on CSS files (not surprising). Doing some googling I found a website called CSS Compressor, so I just used that to compress the CSS, which seemed to do a decent job.
The weather today was a mixture of cloud and sun, and was quite windy. There wasn't a sunset as the sun set behind a bank of cloud.
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Dinner: Chicken chow mein; noodles; soy sauce. Pudding was home-made mandarin flan with cream. Cup o' tea.
Tea: Scone with butter and Strawberry jam; Packet of Prawn Cocktail crisps; Clementine.
Supper: Dark chocolate digestive; Hob-nob; cup o' tea.
So in the morning I played on Animal, then I went in the garden to see if there were any insects around to try and take photos of.
I did some more website stuff, then after dinner I had a bad headache so I went to bed.
After a few hours I felt better so I got up, and went in the garden for a bit after eating tea.
In the evening I did some more website stuff, played on Wii Sports Resort a bit, and spent quite a while in the back garden trying to find any insects (don't know where they've all gone?)
Mainly I was trying to minify my CSS and javascript. I didn't want a solution that sat on the server, but rather to minify my CSS and javascript files before I upload them to the server. I had found JSMin, which I compiled using the instructions here: Building C programs on Linux, but it didn't seem to work properly on CSS files (not surprising). Doing some googling I found a website called CSS Compressor, so I just used that to compress the CSS, which seemed to do a decent job.
The weather today was a mixture of cloud and sun, and was quite windy. There wasn't a sunset as the sun set behind a bank of cloud.
Food
Breakfast: Lemon marmalade toast sandwich; cup o' tea.
Dinner: Chicken chow mein; noodles; soy sauce. Pudding was home-made mandarin flan with cream. Cup o' tea.
Tea: Scone with butter and Strawberry jam; Packet of Prawn Cocktail crisps; Clementine.
Supper: Dark chocolate digestive; Hob-nob; cup o' tea.
Saturday, 15 August 2009
Reading how everyone doesn't know how to write javascript properly
Today I just did some more work on my pog website. In the evening I spent quite a bit of time reading a newsgroup discussion on why jQuery is rubbish (linked from Asking for opinions about scathing jQuery critique. Unfortunately, while David Mark often suggests that people should learn how to write javascript correctly, he doesn't say where you can learn. I read quite a bit of the Newsgroup thread, though not all of it, and the only things I saw suggested was to read the comp.lang.javascript messages (seems like a rather dis-jointed way to learn) and Peter Michaux's blog, probably easier to read than a load of newsgroup messages, but I would prefer a step-by-step book really.
Actually, I just had a look and saw the comp.lang.javascript FAQ, which suggests 2 books, but it seems that there's still of mistakes in them. Reading the thread that the "JavaScript: The Definitive Guide," 5th Edition, by David Flanagan was linked to, it seems the suggestion by the guy there who was pointing out some of the mistakes in the book, was to read the ECMAScript Specification. Not at the moment thanks!
I guess the FAQ probably has some useful info though, and I don't really have the time to read a 900 page on javascript at the moment anyway.
Food
Breakfast: Strawberry jam toast sandwich; cup o' tea.
Lunch: ½ Mature cheddar cheese with iceberg lettuce sandwich made with fresh bread-maker-made bread; Cherry plum tomato; Crust of fresh bread-maker-made bread with Strawberry jam; Fairtrade chocolate wafer biscuit; cup o' tea.
Dinner: Mashed potato; baked beans; Cumberland sausages. Pudding was an Apricot yoghurt. Coffee; Piece of Sainsbury's Turkish Delight Chocolate; 2 Pieces of Sainsbury's Truffle Chocolate.
Supper: Cup o' tea; Dark Chocolate Digestive; Shortbread Finger.
Actually, I just had a look and saw the comp.lang.javascript FAQ, which suggests 2 books, but it seems that there's still of mistakes in them. Reading the thread that the "JavaScript: The Definitive Guide," 5th Edition, by David Flanagan was linked to, it seems the suggestion by the guy there who was pointing out some of the mistakes in the book, was to read the ECMAScript Specification. Not at the moment thanks!
I guess the FAQ probably has some useful info though, and I don't really have the time to read a 900 page on javascript at the moment anyway.
Food
Breakfast: Strawberry jam toast sandwich; cup o' tea.
Lunch: ½ Mature cheddar cheese with iceberg lettuce sandwich made with fresh bread-maker-made bread; Cherry plum tomato; Crust of fresh bread-maker-made bread with Strawberry jam; Fairtrade chocolate wafer biscuit; cup o' tea.
Dinner: Mashed potato; baked beans; Cumberland sausages. Pudding was an Apricot yoghurt. Coffee; Piece of Sainsbury's Turkish Delight Chocolate; 2 Pieces of Sainsbury's Truffle Chocolate.
Supper: Cup o' tea; Dark Chocolate Digestive; Shortbread Finger.
Friday, 14 August 2009
More URL encoding
So today I was still looking at URL encoding. This time I decided to see what the browser was sending to the server when a link (mostly) isn't URL encoded. I used the following link href again:
Safari
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2
Chrome
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2
Internet Explorer 7 (IE7)
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2
Opera 9.64
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-%60/Hawaiian-Milkcaps/2
Firefox
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2
*The above address bar value is probably different from the one I posted for the same url in Firefox yesterday - if you copy the address bar value from Firefox, then some characters are url encoded when you paste it, even though they don't actually appear url encoded in the address bar. The above value is what appears in the address bar (what it looks like to the user).
Request: GET /$-&-+-,-/-:-;-=-%3F-@-%27-%22-%3C-%3E-%23-%25-%7B-%7D-|-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-%27-%22-%3C-%3E-%23-%25-%7B-%7D-|-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2
Internet Explorer 6 (IE6)
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-%27-%22-%3C-%3E-%23-%25-%7B-%7D-|-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2
Next I tried checking to see if I could compare on the server the URL sent by the browser with what the URL should be. I was doing this test with the links NOT url encoded, other than the %, #, ? and \ characters url encoded, and all whitespace converted into hyphens -.
So I wrote some PHP to construct what the URL of the page should be, and then compared this to the urldecoded $_SERVER['REQUEST_URI']. However, any plus signs + in the url were being decoded to a space. If the whole URL was url encoded, then this wouldn't happen. To fix it without having url encoded links, all I had to do was to first replace any plus + signs in $_SERVER['REQUEST_URI'] before url decoding it
I tested this method with the same url/link I was using for the tests above, and also one with Chinese characters, and viewing the pages in all browsers (Chrome 2, FF3, IE6, IE7, Opera9.6, Safari4), I could see that the requested URL and what the URL should be matched okay.
One thing I noticed in my testing was that Google Chrome displays a URL encoded backslash %5C in the address bar as a plain unencoded backslash \. If you copy the url from the address bar in Chrome, and then paste it into any browser except Firefox (so IE6, IE7, Safari 4, Opera 9.6, and even Chrome itself), the backslash will be converted into a forward slash / (and sent to the server as a forward slash).
For my rewrite rule for the page in question, I only allow 5 forward slashes /, so if a url already has 5 forward slashes /, and also contains a back slash \ in it, then if that back slash is converted into a forward slash, there will be too many forward slashes for the rewrite rule to match, and the user will get a 404.
In the afternoon and evening today I watched some WC Fields shorts and Goldfinger. I also went on Animal Crossing and Wii Sports Resort. I went on MC Neat's website, and it had some tracks from his 'new' album 'Neat Situations', due out early Spring 2007 (no, not a typo). Apparently he has 'a totally new image and new sound to blow you away', though personally I didn't think much of the 'new' tracks.
His website also has a gallery where you can 'Check out Neat in his various natural poses'. It only has two photos, the first looks like his head photoshopped (though it's probably real) inside a car window, and in the second photo he's sitting on a sofa holding his head like he's got a headache.
The weather today was pretty windy and quite sunny.
Food
Breakfast: Blood Orange marmalade toast sandwich; cup o' tea.
Lunch: Ham with mustard, sliced cherry plum tomatoes (3 fruits in one), and iceberg lettuce sandwich; Clementine; Fairtrade Wafer biscuit; cup o' tea.
Dinner: Beef burger in a bun with mature cheddar cheese, tomato ketchup and iceberg lettuce; Fried vegetables with mexican flavouring stuff that didn't actually have much flavour. Pudding was a Cadbury's Magnum style ice cream (delee). Piece of Sainsbury's Turkish delight chocolate; Piece of Sainsbury's Caramel chocolate; coffee.
/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2Safari
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2
Chrome
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2
Internet Explorer 7 (IE7)
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2
Opera 9.64
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-%60/Hawaiian-Milkcaps/2
Firefox
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2
*The above address bar value is probably different from the one I posted for the same url in Firefox yesterday - if you copy the address bar value from Firefox, then some characters are url encoded when you paste it, even though they don't actually appear url encoded in the address bar. The above value is what appears in the address bar (what it looks like to the user).
Request: GET /$-&-+-,-/-:-;-=-%3F-@-%27-%22-%3C-%3E-%23-%25-%7B-%7D-|-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-%27-%22-%3C-%3E-%23-%25-%7B-%7D-|-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2
Internet Explorer 6 (IE6)
Address bar: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2
Request: GET /$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2 HTTP/1.1
Referer: http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-%27-%22-%3C-%3E-%23-%25-%7B-%7D-|-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2
Next I tried checking to see if I could compare on the server the URL sent by the browser with what the URL should be. I was doing this test with the links NOT url encoded, other than the %, #, ? and \ characters url encoded, and all whitespace converted into hyphens -.
So I wrote some PHP to construct what the URL of the page should be, and then compared this to the urldecoded $_SERVER['REQUEST_URI']. However, any plus signs + in the url were being decoded to a space. If the whole URL was url encoded, then this wouldn't happen. To fix it without having url encoded links, all I had to do was to first replace any plus + signs in $_SERVER['REQUEST_URI'] before url decoding it
$currentURL = urldecode(str_replace('+', '%2B', $_SERVER["REQUEST_URI"]));I tested this method with the same url/link I was using for the tests above, and also one with Chinese characters, and viewing the pages in all browsers (Chrome 2, FF3, IE6, IE7, Opera9.6, Safari4), I could see that the requested URL and what the URL should be matched okay.
One thing I noticed in my testing was that Google Chrome displays a URL encoded backslash %5C in the address bar as a plain unencoded backslash \. If you copy the url from the address bar in Chrome, and then paste it into any browser except Firefox (so IE6, IE7, Safari 4, Opera 9.6, and even Chrome itself), the backslash will be converted into a forward slash / (and sent to the server as a forward slash).
For my rewrite rule for the page in question, I only allow 5 forward slashes /, so if a url already has 5 forward slashes /, and also contains a back slash \ in it, then if that back slash is converted into a forward slash, there will be too many forward slashes for the rewrite rule to match, and the user will get a 404.
In the afternoon and evening today I watched some WC Fields shorts and Goldfinger. I also went on Animal Crossing and Wii Sports Resort. I went on MC Neat's website, and it had some tracks from his 'new' album 'Neat Situations', due out early Spring 2007 (no, not a typo). Apparently he has 'a totally new image and new sound to blow you away', though personally I didn't think much of the 'new' tracks.
His website also has a gallery where you can 'Check out Neat in his various natural poses'. It only has two photos, the first looks like his head photoshopped (though it's probably real) inside a car window, and in the second photo he's sitting on a sofa holding his head like he's got a headache.
The weather today was pretty windy and quite sunny.
Food
Breakfast: Blood Orange marmalade toast sandwich; cup o' tea.
Lunch: Ham with mustard, sliced cherry plum tomatoes (3 fruits in one), and iceberg lettuce sandwich; Clementine; Fairtrade Wafer biscuit; cup o' tea.
Dinner: Beef burger in a bun with mature cheddar cheese, tomato ketchup and iceberg lettuce; Fried vegetables with mexican flavouring stuff that didn't actually have much flavour. Pudding was a Cadbury's Magnum style ice cream (delee). Piece of Sainsbury's Turkish delight chocolate; Piece of Sainsbury's Caramel chocolate; coffee.
Thursday, 13 August 2009
Url encoding
This morning I was trying to work out how to get my tree menu to work properly in IE6. My menu looked like this:
As part of this process, I would enclose the category titles in an anchor element, and attach to this a class name, so that when you rolled over the category titles they would become underlined to show they could be clicked on, e.g. I was adding the className 'title', which had the following CSS properties:
The only problem I was having was that in IE6, the hover effect wasn't working. I found that I needed to set the href property of the anchor element for the hover effect to work. So I set the href property to a blank string. I now got the rollover/hover effect in IE6, but had a new problem.
With my menu, I wrote my js so that after converting the menu to a tree menu, it would then loop the links in the menu until it found the link that matched the current page URL. When it did, it would then 'click' on the parent category titles of that link, so they would be expanded and the menu would show the page that you were currently on:
But when I had added an empty href attribute to the category titles, the parents of the current page link weren't being expanded. After doing some testing, I realised what the problem was - an empty href links to the same page you are currently on, so the first empty href (on the first category title) was matching against the current page url, and then the loop would break.
Setting the href of the category titles to '#' instead of a blank string fixed this problem. But I now found another problem, in IE6, the category title of the current page still wasn't being 'clicked on'. Comparing the href of the link and the window.location.href in IE6 versus FF, I found that IE6 decodes the href, so while the actual link href had the value of
which matched the window.location.href value, IE6 reckoned that the value of the link's href was
which doesn't match the window.location.href value.
The solution was to re-encode those specific characters in IE6, so the urls would match correctly.
However, after doing that I found that the page url and link url didn't match in Webkit browsers (Chrome 2 and Safari 4) either, the link href was being evaluated as:
While the window.location.href was being evaluated as:
So some characters in the url were being url encoded, whilst the same characters in the link href weren't being encoded.
Eventually I came to the solution of just url decoding (unescape) both the window.location.href and the link href, and this seemed to work okay in all browsers.
After that I was trying to see if I should actually urlencode all characters or not. I found a wikipedia article on Nanjing, which had the characters in Chinese. However, I found that a link to the page using the Chinese characters (http://zh.wikipedia.org/wiki/南京), or the url encoded chinese characters (http://zh.wikipedia.org/wiki/%E5%8D%97%E4%BA%AC), both worked. Using Fiddler2, it looked like Firefox was url encoding the chinese characters before sending the GET request, so they were both sending the same page request.
Viewing source on the Firefox page, I could see that all the links in the page used URL encoded strings rather than the Chinese Characters. However, in testing with my own page, I found that for certain characters, if they are url encoded, then FF will display the hex code in the address bar, rather than the actual character that the hex code represents.
For example, the url
When the & < > " are converted to html entities, and the % ? # \ characters are url encoded, this gives:
And displays in the address bar as
The same url, but url encoded, and with forward slashes converted from hex back to forward slashes), gives:
And is displayed in the address bar as:
After that I tried a url with Chinese characters in it:
When it was url encoded, the chinese characters were displayed in the address bar in Firefox, Chrome, Safari, and Opera, but the hex code was displayed in the address bar in IE6 and IE7.
When it wasn't url encoded, the Chinese Characters were displayed in the address bar for all browsers except IE6, because I run IE6 in a virtual machine that doesn't have the Asian language packs installed, so it just displayed some blocks instead.
I checked the Apache access logs to see what was actually being requested, and it seems that all the browsers actually url encoded the requests anyway:
Interestingly, IE6 (on a virtual machine without the Asian language packs installed), still send the GET request urlencoded correctly, but sends the referrer header with question marks in place of the Chinese characters. IE7 sends the referrer in some method of writing hex code I guess.
When doing these tests with the Chinese characters, I noticed that the tree menu wasn't expanding correctly in Safari, though seemed to be working okay in all the other browsers. When I did some testing on this, I found that the the link href containing chinese characters was being unescaped as -金錢符è
In Safari the unescaped window.location.href was being converted as the chinese characters, but in all other browsers both the link href and the window.location.href were being unescaped as -金錢符è
I found a proper javascript function to encode and decode urls, and so tried using this instead of unescape. However, in Safari 4, using this function meant the window.location.href (which Safari had already decoded) was decoded again, and so still didn't match the link's href.
In Safari:
In Firefox
Interestingly, the same thing is logged in Firefox whether the original url was encoded or not, so it seems that all Browsers apart from Safari urlencode any unencoded characters in window.location.href, whilst Safari always urldecodes any url encoded characters in window.location.href. And all browsers including Safari seem to url encode any unencoded characters in the hrefs of anchors.
The weather today was a mixture of cloud and sun. I think there was probably a good sunset.
Food
Breakfast: Blood Orange marmalade toast sandwich; cup o' tea.
Lunch: 2x Cheese on toasts; cherry plum tomatoes; iceberg lettuce; clementine; Fairtrade chocolate wafer bar; cup o' tea.
Dinner: Shepherd's pie; grated cheese; tomato ketchup; carrots; peas. Pudding was a Caramel White Chocolate biscuit thing slice. Coffee.
<ul>
<li>$ & + , / : ; = ? @ ' " < > # % { } | \ ^ ~ [ ] `<ul>
<li><a href="/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2" title="Hawaiian Milkcaps">Hawaiian Milkcaps</a></li>
</ul></li>
<li><a href="/American-Games-Caps/4" title="American Games Caps">American Games Caps</a></li>
<li><a href="/Junior-Caps/7" title="Junior Caps">Junior Caps</a></li>
<li>Pogs<ul>
<li><a href="/Pogs/Series-1/1" title="Series 1">Series 1</a></li>
<li><a href="/Pogs/Series-2/3" title="Series 2">Series 2</a></li>
<li><a href="/Pogs/The-World-Tour/15" title="The World Tour">The World Tour</a></li>
<li>Series 1<ul>
<li><a href="/Pogs/Series-1/Chex/5" title="Chex">Chex</a></li>
</ul></li>
</ul></li>
<li><a href="/UK-Games-Caps/16" title="UK Games Caps">UK Games Caps</a></li>
<li><a href="/Zigs/6" title="Zigs">Zigs</a></li>
</ul>
As part of this process, I would enclose the category titles in an anchor element, and attach to this a class name, so that when you rolled over the category titles they would become underlined to show they could be clicked on, e.g. I was adding the className 'title', which had the following CSS properties:
#nav .title{
color: #FFF;
text-decoration: none;
}
#nav .title:hover{
text-decoration: underline;
cursor: pointer;
}
The only problem I was having was that in IE6, the hover effect wasn't working. I found that I needed to set the href property of the anchor element for the hover effect to work. So I set the href property to a blank string. I now got the rollover/hover effect in IE6, but had a new problem.
With my menu, I wrote my js so that after converting the menu to a tree menu, it would then loop the links in the menu until it found the link that matched the current page URL. When it did, it would then 'click' on the parent category titles of that link, so they would be expanded and the menu would show the page that you were currently on:
//Highlight current page and expand parents
var links = document.getElementById('nav').getElementsByTagName('A');
for(var i=0, link; link=links[i]; i++)
{
//alert(link.href +' != '+ window.location.href);
if(link.href == window.location.href)
{
//Highlight the link
var span = document.createElement('span');
span.className = 'LucidaSans';
span.style.color = '#C9C9F3';
span.appendChild(document.createTextNode('\u25C9 '));
span.style.marginLeft = '-1em';
link.parentNode.insertBefore(span, link);
//Assign a var for the parent ul - loop won't break if we just try and change the array val for links[i]
var parent = link.parentNode.parentNode;
while(parent.tagName == 'UL' && parent.parentNode.id != 'nav')
{
alert('gh');
//Click on the parent to expand it
if(document.createEvent) //DOM2 compatible browsers
{
var clicky = document.createEvent('MouseEvents');
clicky.initEvent("click", true, true);
parent.previousSibling.dispatchEvent(clicky);
}
else //IE
{
if(parent.previousSibling)
{parent.previousSibling.fireEvent('onclick');}
}
//parent.previousSibling.click();
parent = parent.parentNode.parentNode;
}
break;
}
}
But when I had added an empty href attribute to the category titles, the parents of the current page link weren't being expanded. After doing some testing, I realised what the problem was - an empty href links to the same page you are currently on, so the first empty href (on the first category title) was matching against the current page url, and then the loop would break.
Setting the href of the category titles to '#' instead of a blank string fixed this problem. But I now found another problem, in IE6, the category title of the current page still wasn't being 'clicked on'. Comparing the href of the link and the window.location.href in IE6 versus FF, I found that IE6 decodes the href, so while the actual link href had the value of
/$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2which matched the window.location.href value, IE6 reckoned that the value of the link's href was
/$-&-+-,-/-:-;-=-?-@-'-"-<->-#-%-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2which doesn't match the window.location.href value.
The solution was to re-encode those specific characters in IE6, so the urls would match correctly.
However, after doing that I found that the page url and link url didn't match in Webkit browsers (Chrome 2 and Safari 4) either, the link href was being evaluated as:
http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2While the window.location.href was being evaluated as:
http://www.milkcapmania.com/$-&-+-,-/-:-;-=-%3F-@-%27-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2So some characters in the url were being url encoded, whilst the same characters in the link href weren't being encoded.
Eventually I came to the solution of just url decoding (unescape) both the window.location.href and the link href, and this seemed to work okay in all browsers.
After that I was trying to see if I should actually urlencode all characters or not. I found a wikipedia article on Nanjing, which had the characters in Chinese. However, I found that a link to the page using the Chinese characters (http://zh.wikipedia.org/wiki/南京), or the url encoded chinese characters (http://zh.wikipedia.org/wiki/%E5%8D%97%E4%BA%AC), both worked. Using Fiddler2, it looked like Firefox was url encoding the chinese characters before sending the GET request, so they were both sending the same page request.
Viewing source on the Firefox page, I could see that all the links in the page used URL encoded strings rather than the Chinese Characters. However, in testing with my own page, I found that for certain characters, if they are url encoded, then FF will display the hex code in the address bar, rather than the actual character that the hex code represents.
For example, the url
$-&-+-,-/-:-;-=-?-@-'-"-<->-#-%-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2When the & < > " are converted to html entities, and the % ? # \ characters are url encoded, this gives:
$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2And displays in the address bar as
$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2 in Firefox,$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2 in Opera,$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-[-]-%60/Hawaiian-Milkcaps/2 in Internet Explorer 7,$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2 in Google Chrome,$-&-+-,-/-:-;-=-%3F-@-'-%22-%3C-%3E-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2 in Safari 4, and$-&-+-,-/-:-;-=-%3F-@-'-"-<->-%23-%25-{-}-|-%5C-^-~-[-]-`/Hawaiian-Milkcaps/2 in IE6The same url, but url encoded, and with forward slashes converted from hex back to forward slashes), gives:
/%24-%26-%2B-%2C-/-%3A-%3B-%3D-%3F-%40-%27-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-%7E-%5B-%5D-%60/Hawaiian-Milkcaps/2And is displayed in the address bar as:
%24-%26-%2B-%2C-/-%3A-%3B-%3D-%3F-%40-'-"-<->-%23-%25-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2 in Firefox,$-%26-%2B-,-/-:-;-%3D-%3F-@-'-"-<->-%23-%25-{-}-|-\-^-~-[-]-`/Hawaiian-Milkcaps/2 in Google Chrome,%24-%26-%2B-%2C-/-%3A-%3B-%3D-%3F-%40-%27-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-%7E-%5B-%5D-%60/Hawaiian-Milkcaps/2 in Safari 4,%24-%26-%2B-%2C-/-%3A-%3B-%3D-%3F-%40-%27-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-%5B-%5D-%60/Hawaiian-Milkcaps/2 in Internet Explorer 7,%24-%26-%2B-%2C-/-%3A-%3B-%3D-%3F-%40-'-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-~-%5B-%5D-`/Hawaiian-Milkcaps/2 in Opera 9.64, and%24-%26-%2B-%2C-/-%3A-%3B-%3D-%3F-%40-%27-%22-%3C-%3E-%23-%25-%7B-%7D-%7C-%5C-%5E-%7E-%5B-%5D-%60/Hawaiian-Milkcaps/2 in IE6.After that I tried a url with Chinese characters in it:
Pogs-金錢符號/Series-1/1When it was url encoded, the chinese characters were displayed in the address bar in Firefox, Chrome, Safari, and Opera, but the hex code was displayed in the address bar in IE6 and IE7.
When it wasn't url encoded, the Chinese Characters were displayed in the address bar for all browsers except IE6, because I run IE6 in a virtual machine that doesn't have the Asian language packs installed, so it just displayed some blocks instead.
I checked the Apache access logs to see what was actually being requested, and it seems that all the browsers actually url encoded the requests anyway:
192.168.0.4 - - [13/Aug/2009:20:43:52 +0100] "GET /Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1 HTTP/1.1" 200 2081 "http://www.milkcapmania.com/Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729)"
192.168.0.4 - - [13/Aug/2009:20:44:39 +0100] "GET /Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1 HTTP/1.1" 200 2081 "http://www.milkcapmania.com/Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1" "Opera/9.64 (Windows NT 6.0; U; en) Presto/2.1.1"
192.168.0.4 - - [13/Aug/2009:20:44:51 +0100] "GET /Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1 HTTP/1.1" 200 2081 "http://www.milkcapmania.com/Pogs-\xe9\x87\x91\xe9\x8c\xa2\xe7\xac\xa6\xe8\x99\x9f/Series-1/1" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; WOW64; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30618)"
192.168.0.4 - - [13/Aug/2009:20:44:57 +0100] "GET /Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1 HTTP/1.1" 200 2081 "http://www.milkcapmania.com/Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.5 (KHTML, like Gecko) Chrome/2.0.172.39 Safari/530.5"
192.168.0.4 - - [13/Aug/2009:20:45:13 +0100] "GET /Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1 HTTP/1.1" 200 2081 "http://www.milkcapmania.com/Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19.1"
192.168.0.4 - - [13/Aug/2009:20:45:51 +0100] "GET /Pogs-%E9%87%91%E9%8C%A2%E7%AC%A6%E8%99%9F/Series-1/1 HTTP/1.1" 200 2081 "http://www.milkcapmania.com/Pogs-????/Series-1/1" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
Interestingly, IE6 (on a virtual machine without the Asian language packs installed), still send the GET request urlencoded correctly, but sends the referrer header with question marks in place of the Chinese characters. IE7 sends the referrer in some method of writing hex code I guess.
When doing these tests with the Chinese characters, I noticed that the tree menu wasn't expanding correctly in Safari, though seemed to be working okay in all the other browsers. When I did some testing on this, I found that the the link href containing chinese characters was being unescaped as -金錢符è
In Safari the unescaped window.location.href was being converted as the chinese characters, but in all other browsers both the link href and the window.location.href were being unescaped as -金錢符è
I found a proper javascript function to encode and decode urls, and so tried using this instead of unescape. However, in Safari 4, using this function meant the window.location.href (which Safari had already decoded) was decoded again, and so still didn't match the link's href.
In Safari:
http://www.milkcapmania.com/Pogs-金錢符號/Series-1/1
!= http://www.milkcapmania.com/Pogs-ᢦﯓeries-1/1
== true
In Firefox
http://www.milkcapmania.com/Pogs-金錢符號/Series-1/1 != http://www.milkcapmania.com/Pogs-金錢符號/Series-1/1
== false
Interestingly, the same thing is logged in Firefox whether the original url was encoded or not, so it seems that all Browsers apart from Safari urlencode any unencoded characters in window.location.href, whilst Safari always urldecodes any url encoded characters in window.location.href. And all browsers including Safari seem to url encode any unencoded characters in the hrefs of anchors.
The weather today was a mixture of cloud and sun. I think there was probably a good sunset.
Food
Breakfast: Blood Orange marmalade toast sandwich; cup o' tea.
Lunch: 2x Cheese on toasts; cherry plum tomatoes; iceberg lettuce; clementine; Fairtrade chocolate wafer bar; cup o' tea.
Dinner: Shepherd's pie; grated cheese; tomato ketchup; carrots; peas. Pudding was a Caramel White Chocolate biscuit thing slice. Coffee.
Wednesday, 12 August 2009
Trying to get an .htaccess rewrite rule to work
Today I was trying to figure out how to get reasonably nice looking (and SEO friendly) urls. I found yesterday, that urls with %3F (url encoded question mark) in them weren't working, and neither were urls with %2F (url encoded forward slash).
I had posted a question about this on the WebFaction forums yesterday, and when I checked today, Japh had replied to it to say that the problem was that Apache's mod_rewrite does a url decode on the url before it checks it against the rules. So the bit of the url after the %3F would be considered a query string by apache I guess, and not part of the actual url address.
The solution he suggested was to double encode the url, so %3F would become %253F. If possible, I wanted to keep my urls as simple as possible, so I did some googling to see if there was any way to get round this. A url containing %3F actually works okay with the rewrite rules on my local system, just not on the webserver, so I did some testing using %2F (url encoded forward slash) in the url instead, as this doesn't work on the webserver or my local system.
I did a lot of testing, but just couldn't get any rewrite rules to work when %2F was in the url. The last post in this thread: mod rewrite fails with %2f character explains it -
Before lunch me, Moccle and L started watching 'The Lost World' (1925), after lunch I went on Animal, then we finished watching The Lost World.
After that, I carried on looking at what characters are acceptable in URLs, I found a useful guide - URL Encoding, so went through the characters there to see which characters were okay, and which weren't. Obviously, you could just url encode all non alpha-numeric characters, but this leads to messy looking urls, and I'm not sure how SEO friendly URL encoded characters are versus normal characters.
Anyway, going through those characters, I found only % ? and # need url encoding. (In my case anyway, as I'm discarding all the url except the end part which contains the number of the database record my page needs to look up). I also found that IE6 a backslash would be converted into a forward slash, so I also decided to url encode that.
In the evening I did some more looking at url rewriting urls with %3F in them, and eventually found that by putting the rewrite rule as a rewrite condition, and checking the rule against the REQUEST_URI, then I could check it okay even though %3F was being unencoded to a question mark. Here's a link to the thread at the WebFaction forums: URL re-writing not working when %3F or %2F in url
Also in the evening, I watched Bulletproof (Gary Busey film) with Moccle. Typical Gary Busey/80's action flick skillness. It also had the ultimate baddies of A-rab Russian Mexican South American Communists. Moccle also did a good job of spotting Danny Trejo in the opening gambit of the film.
It also had a dude in a car near the end who was fat like Hurley from Lost, and sounded totally like him as well. He had a beard though, and was about the same age as Hurley is now, and wasn't played by the same actor.
Food
Breakfast: Blood Orange marmalade toast sandwich; cup o' tea.
Lunch: Ham with mustard and iceberg lettuce sandwich; satsuma; piece of Chocolate cereal cake stuff; cup o' tea.
Dinner: Big german sausage in a sub with iceberg lettuce and tomato ketchup; vegetable fake cup a soup. Pudding was an Aero Mousse and also a caramel and white chocolate biscuit slice thing. Coffee.
Supper: Hob-nob; Shortbread finger.
I had posted a question about this on the WebFaction forums yesterday, and when I checked today, Japh had replied to it to say that the problem was that Apache's mod_rewrite does a url decode on the url before it checks it against the rules. So the bit of the url after the %3F would be considered a query string by apache I guess, and not part of the actual url address.
The solution he suggested was to double encode the url, so %3F would become %253F. If possible, I wanted to keep my urls as simple as possible, so I did some googling to see if there was any way to get round this. A url containing %3F actually works okay with the rewrite rules on my local system, just not on the webserver, so I did some testing using %2F (url encoded forward slash) in the url instead, as this doesn't work on the webserver or my local system.
I did a lot of testing, but just couldn't get any rewrite rules to work when %2F was in the url. The last post in this thread: mod rewrite fails with %2f character explains it -
The naked "%2f" is allowed in a query string. but not in a URL. In order to be valid, it would have to be encoded as %252f, which I think you will find to work as you expect.
Because the URL is itself invalid, the server is rejecting it before any Apache modules are invoked. On my server, not even the custom 404 error page is applied -- The server simply rejects the request out-of-hand.
Before lunch me, Moccle and L started watching 'The Lost World' (1925), after lunch I went on Animal, then we finished watching The Lost World.
After that, I carried on looking at what characters are acceptable in URLs, I found a useful guide - URL Encoding, so went through the characters there to see which characters were okay, and which weren't. Obviously, you could just url encode all non alpha-numeric characters, but this leads to messy looking urls, and I'm not sure how SEO friendly URL encoded characters are versus normal characters.
Anyway, going through those characters, I found only % ? and # need url encoding. (In my case anyway, as I'm discarding all the url except the end part which contains the number of the database record my page needs to look up). I also found that IE6 a backslash would be converted into a forward slash, so I also decided to url encode that.
In the evening I did some more looking at url rewriting urls with %3F in them, and eventually found that by putting the rewrite rule as a rewrite condition, and checking the rule against the REQUEST_URI, then I could check it okay even though %3F was being unencoded to a question mark. Here's a link to the thread at the WebFaction forums: URL re-writing not working when %3F or %2F in url
Also in the evening, I watched Bulletproof (Gary Busey film) with Moccle. Typical Gary Busey/80's action flick skillness. It also had the ultimate baddies of A-rab Russian Mexican South American Communists. Moccle also did a good job of spotting Danny Trejo in the opening gambit of the film.
It also had a dude in a car near the end who was fat like Hurley from Lost, and sounded totally like him as well. He had a beard though, and was about the same age as Hurley is now, and wasn't played by the same actor.
Food
Breakfast: Blood Orange marmalade toast sandwich; cup o' tea.
Lunch: Ham with mustard and iceberg lettuce sandwich; satsuma; piece of Chocolate cereal cake stuff; cup o' tea.
Dinner: Big german sausage in a sub with iceberg lettuce and tomato ketchup; vegetable fake cup a soup. Pudding was an Aero Mousse and also a caramel and white chocolate biscuit slice thing. Coffee.
Supper: Hob-nob; Shortbread finger.
Monday, 10 August 2009
Defishing fisheye images to rectilinear projection
This morning I vacuumed and dusted my room. After that, I installed DXO Optics Pro 5.3 to see how it work with de-fishing my Tokina 10-17mm/3.5-4.5 fisheye images. The problem I'm having with images taken with this lens is that if you tilt the lens up or down to try and get the horizon on a third line, the horizon will become quite bowed due to the fisheye effect.
Defishing the image, using a Cylindrical or Equirectangular projection, you will still get the same bowed Horizon line unless you place the image so as to correct for converging verticals. If the original image was shot with the horizon not through the center of the image, then this will mean you will need to crop off a lot of the bottom or top of the image (depending on if you placed the horizon through the top or bottom third of the image), and you'll end up with something similar to what you would have got if you had shot with the Horizon through the center, except you'll have less pixels due to the cropping.
Defishing the image using rectilinear projection, the edges (maybe about 25% of the image either side left and right, assuming a 10mm Landscape oriented shot), gets very distorted. I wanted to try DXO to see if I would still get the same problem. I did get the same problem, and also got some nasty CA that I don't see when converting the RAW in CaptureNX, and then defishing in PTGUI.
I did some googling to try and see if the Tokina fisheye image was much wider than the Sigma 10-20mm lens, and if this was why the defished Tokina 10-17mm/3.5-4.5 fisheye images were much more distorted than Sigma 10-20mm images. I found a thread on panoguide that discussed the two lenses Pentax 10-17mm fisheye vs Sigma 10-20mm, which has quite a bit of info in it.
According to that thread,
When I viewed the defished images that I processed through DXO, while I didn't like the heavy distortion at the left and right edges of the image, I did like the stretched effect that the clouds got. I think that probably I will need to try taking some images at 17mm, then defishing those to a rectilinear projection, and see how they look.
After that I tried defishing a 10mm to rectilinear projection in PTGUI, to see how much info I would loose by cropping off the distortions. I found that cropping off the distortions at the edges, the defished image would actually have more pixels than the original, however, the image was quite soft at the left and right edges. Cropping a bit tighter, to the same pixel width as the original image, the softness wasn't quite so bad at the edges (especially since the image hadn't been sharpened), but the image ratio was now square rather than landscape. Of course, you could also crop off the bottom and top of the image to restore the original image's ratio, and probably not loose any resolution, however you would loose even more of the original image's field of view.
I made a Flickr set of the different methods of defishing the image, with notes: Defishing Tokina 10-17mm/3.5-4.5 images set on Flickr
Really I would like to have a Sigma 10-20mm lens so that I could take the same shot with both lenses and compare them directly, unfortunately I don't have access to this lens, and it's too expensive for me to purchase at the moment.
When reading the panoguide thread about the Sigma 10-20mm rectilinear lens versus the Tokina 10-17mm fisheye lens earlier, I noticed Hans saying that the entrance pupil on a fisheye varies, and you should try and set your pano head to avoid parallax where the stitching seams will be. I didn't know this, and have probably calibrated my head to have no parallax where the edge of the images overlap, rather than in the middle of the overlap (where I assume the stitching program would create the seam). So I will probably need to try and re-calibrate my pano head sometime before I off on holiday in October.
In the afternoon I went on Animal Crossing, then spent the rest of the day working on my pog website trying to get it ready to upload to the webserver (as I've moved hosts there are various bits on it that need changing).
Food
Breakfast: Ludlow Food Centre Celebration Jam toast sandwich; cup o' tea.
Lunch: Slice of toast; vegetable fake cup a soup; Satsuma; Timeout (single); cup o' tea.
Dinner: Breaded fish portion; peas; potatoes; salt; ground black pepper. Pudding was Rhubarb and apple pie with a bit of cream. Piece of Sainsbury's caramel chocolate; Piece of Sainsbury's Truffle chocolate; Piece of Sainsbury's Turkish Delight chocolate; coffee.
Defishing the image, using a Cylindrical or Equirectangular projection, you will still get the same bowed Horizon line unless you place the image so as to correct for converging verticals. If the original image was shot with the horizon not through the center of the image, then this will mean you will need to crop off a lot of the bottom or top of the image (depending on if you placed the horizon through the top or bottom third of the image), and you'll end up with something similar to what you would have got if you had shot with the Horizon through the center, except you'll have less pixels due to the cropping.
Defishing the image using rectilinear projection, the edges (maybe about 25% of the image either side left and right, assuming a 10mm Landscape oriented shot), gets very distorted. I wanted to try DXO to see if I would still get the same problem. I did get the same problem, and also got some nasty CA that I don't see when converting the RAW in CaptureNX, and then defishing in PTGUI.
I did some googling to try and see if the Tokina fisheye image was much wider than the Sigma 10-20mm lens, and if this was why the defished Tokina 10-17mm/3.5-4.5 fisheye images were much more distorted than Sigma 10-20mm images. I found a thread on panoguide that discussed the two lenses Pentax 10-17mm fisheye vs Sigma 10-20mm, which has quite a bit of info in it.
According to that thread,
the Sigma 10-20 will give you a horisontal FOV of 76 and the Fisheye will give you around 90.So the Tokina 10-17mm fisheye has 18.4% wider HFOV than the Sigma 10-20mm Rectilinear lens, which doesn't seem like a lot (though adds up when doing 360° panos).
When I viewed the defished images that I processed through DXO, while I didn't like the heavy distortion at the left and right edges of the image, I did like the stretched effect that the clouds got. I think that probably I will need to try taking some images at 17mm, then defishing those to a rectilinear projection, and see how they look.
After that I tried defishing a 10mm to rectilinear projection in PTGUI, to see how much info I would loose by cropping off the distortions. I found that cropping off the distortions at the edges, the defished image would actually have more pixels than the original, however, the image was quite soft at the left and right edges. Cropping a bit tighter, to the same pixel width as the original image, the softness wasn't quite so bad at the edges (especially since the image hadn't been sharpened), but the image ratio was now square rather than landscape. Of course, you could also crop off the bottom and top of the image to restore the original image's ratio, and probably not loose any resolution, however you would loose even more of the original image's field of view.
I made a Flickr set of the different methods of defishing the image, with notes: Defishing Tokina 10-17mm/3.5-4.5 images set on Flickr
Really I would like to have a Sigma 10-20mm lens so that I could take the same shot with both lenses and compare them directly, unfortunately I don't have access to this lens, and it's too expensive for me to purchase at the moment.
When reading the panoguide thread about the Sigma 10-20mm rectilinear lens versus the Tokina 10-17mm fisheye lens earlier, I noticed Hans saying that the entrance pupil on a fisheye varies, and you should try and set your pano head to avoid parallax where the stitching seams will be. I didn't know this, and have probably calibrated my head to have no parallax where the edge of the images overlap, rather than in the middle of the overlap (where I assume the stitching program would create the seam). So I will probably need to try and re-calibrate my pano head sometime before I off on holiday in October.
In the afternoon I went on Animal Crossing, then spent the rest of the day working on my pog website trying to get it ready to upload to the webserver (as I've moved hosts there are various bits on it that need changing).
Food
Breakfast: Ludlow Food Centre Celebration Jam toast sandwich; cup o' tea.
Lunch: Slice of toast; vegetable fake cup a soup; Satsuma; Timeout (single); cup o' tea.
Dinner: Breaded fish portion; peas; potatoes; salt; ground black pepper. Pudding was Rhubarb and apple pie with a bit of cream. Piece of Sainsbury's caramel chocolate; Piece of Sainsbury's Truffle chocolate; Piece of Sainsbury's Turkish Delight chocolate; coffee.
Subscribe to:
Posts (Atom)