Saturday, 22 November 2014

Writing and moving words

The past week I've mainly been putting the church magazine together and working on a couple of articles for my photo tips website. This morning I made some Gingerbread Latte syrup. I had some in a Coffee with plenty of milk, but it didn't seem very tasty. Maybe you need 1½ teaspoons to taste it better.

Monday, 3 November 2014

Giving up on Vagrant

Well, I probably haven't got everything working with my Vagrant box yet, but I can at least access a local copy of my website through it.

The first issue I had today was that Nginx wasn't starting with the machine, and my @reboot cron job wasn't being run either. I found the reason and a solution here: If Nginx doesn’t restart on Vagrant up. The problem is because vagrant (virtualBox) only mounts the shared folders after init and cron have run. So, in my case, the nginx conf files were in my shared folder, and the script I wanted to run @reboot was too. The solution is to add an init conf file to /etc/init that will start nginx / run the script when the vagrant shared folders are mounted.

Another issue I had was that although I had imported my mysql database, the users / privileges didn't seem to have carried over. I found the solution to this here: How can I export the privileges from MySQL and then import to a new server?. Strangely there were quite a few privileges for tables that no longer existed, so I had to delete those to get the import to import everything successfully.

So far, there are three points that seriously hamper the usability of Vagrant for me. Actually, these aren't really related to Vagrant, but problems using shared folders with VirtualBox when running a linux guest OS on a Windows host OS:

  1. Permissions can't be set properly. You can set permissions for a whole folder through the mount_options when mounting a shared folder, but this applies to the whole folder contents, and is a bit of a kludge.
  2. You can't create symbolic links in the shared folder. If you do it from Windows, then the VM can't read the link.
  3. Shared folders aren't mounted early enough, causing issues with init scripts and @reboot cron jobs.

Despite these problems, I intend to persevere, at least for the moment. The issue with using a standard VM without shared folders is that even if you only change 1 byte of code in a single file, then you have the whole VM to backup. I guess you could always just backup the files from the VM, but that means having to start up the VM and use FTP whenever you want to backup.

Aggh! I just found another issue with the shared folders - it seems that the VM has the same restrictions as windows when working with long filepaths in the shared folders! There is a possible solution given here: VirtualBox bug tracker: long paths (>260chars) cannot be created using a host:win + guest:linux setup.

However, I couldn't get that to work in combination with Vagrant. I tried specifying the path in the VagrantFile, but that gave an error about //?/E not being valid when trying to vagrant up. (I was using an absolute path to a folder on my E: drive). I tried running the command, which added the folder, but VirtualBox would not auto mount it. And you couldn't set the Auto Mount option for it using the VirtualBox GUI (the OK button was greyed out).

I also tried setting that up, then using a standard path in my VagrantFile, but the path in the VagrantFile overwrote the path set up using VBoxManage.exe. Thus the \\?\ at the start of the path was lost.

So it seems to me that with all these issues, Vagrant using VirtualBox with Shared folders on a windows host OS is just not usable at the moment. I'm going to go back to my VMWare Ubuntu VM for now. It took me over a week to try and set up this VM, only to have to abandon it. But I did learn some things along the way. Most of them I'll probably forget, but hopefully I can take a look at replacing my current shared hosting account with a VPS at some point.

Sunday, 2 November 2014

Spyro Gyra got played on "HFM through the night" (no DJ)

Today I was still trying to get my Vagrant box set up. I'm getting closer, but still not quite there yet. I think if I've learned one thing from this process, it's that I really ought to be using VPS hosting so I don't have to compile everything from source. It would make installing, and just as importantly, keeping everything up to date, so much easier.

I also had another issue with Vagrant / VirtualBox, which is that as well as not being able to create symlinks in a shared folder, it can't read them either. Quite annoying, as I was thinking that creating the symlinks from windows might have been a solution in some instances.

I was quite happy with myself today as I managed to start writing an article in the afternoon, and had it ready to publish by 9pm. As usual finding and inserting the pics took a lot longer than the actual writing process, but I was able to find relevant pics a lot faster than I usually can.

Friday, 31 October 2014

Still Vagranting

This morning I got a bit further with setting up my Vagrant box - I can now access static HTML pages!

The first issue I had today was that I couldn't ssh in with my user. I found this is because I'm passing the password as a parameter when creating the user, and this requires a password encrypted using crypt(3), not a plain password as I was using. But when I tried to use crypt, I got a message that it wasn't installed and I needed to install mycrypt to use it. So how was passwd generating a password if crypt wasn't available?

I tried installing mcrypt, but that gave me crypt(1), not crypt(3). I'm not sure what the difference is, but man useradd does specifically say the password should be The encrypted password, as returned by crypt(3). I then found this resource: Generating Passwords Using crypt(3), which gives a number of options.

I tried the openssl method, and it worked. I don't understand how though, as each time I used it, it gave a different output. Presumably it uses a variable salt, but since the salt is not recorded I have no idea how it can decrypt the password or encrypt subsequent input of a password in the same way for comparison.

After fixing various problems with my nginx install script, I tried testing a webpage, but it didn't work. Unfortunately Fiddler2 didn't show any IP address information, so I couldn't tell whether it was even trying to connect to the Vagrant box.

After restarting Chrome though, Fiddler did give some information. Instead of the webpage I was trying to load, Fiddler output as the response that the connection was refused, and also included the IP address, showing that it was correct. Checking the running process on the Vagrant box ps -A, I found that the frontend Nginx wasn't running. A mistake on my part - I had service nginx reload when it should have been service nginx start.

With that up and running I could now connect OK. The next steps then, are installing my local copies of mysql and php.

In the afternoon I spent ages trying to get a directory protected under apache, but with access to certain files allowed by basic auth. I tried lots of different things, and couldn't understand why they weren't working. Then I realised - my FilesMatch block was never matching because the filename I was matching had brackets in it. After escaping the brackets I got it working how I wanted quite easily.

Installing mysql on my Vagrant box took ages - I would guess at least an hour for building the dependencies, and at least another hour for mysql itself. When it was installed I had some trouble getting it running, and this was due to permissions problems related to having my.cnf located in my shared folder. You can read more about this issue here: vagrant permissions and foldering.

Thursday, 30 October 2014

Vagranting still

After spending quite a bit of time this week in trying to get a Vagrant box set up I discovered an issue today that hampers its usability. Actually this issue is not with Vagrant, but rather using VirtualBox on Windows. The big benefit of this setup is being able to have all your work stored in Windows, and then having that folder available in the VM. But symlinks can't be created in the shared folder from the VM.

I think I'll probably be able to work around this issue, but it is a pain.

Wednesday, 29 October 2014

Still trying to set up Vagrant

Today I was still working on getting a Vagrant box set up for my web development environment. To get the box set up you have to create a shell script that will be run when starting up, and in this script you need to install and configure everything.

I wanted to test the script as I write it, to ensure that it works correctly. The simplest way to do this is to ssh into the box and then execute the script from there. However, when I tried to execute the script I got the error ": No such file or directory". I checked the permissions for the script, and execute was marked, so that wasn't the problem. After a bit of googling I found this thread, which had the answer: No such file or directory error when trying to execute startup script in Debian.

When I ran file ./Vagrant_Bootstrap.sh, I got the output:

./Vagrant_Bootstrap.sh: a bash\015 script, ASCII text executable, with CRLF line terminators

Note that it is not a bash script, but a bash\015 script, and also that it has CRLF line terminators. The script needs to use LF line terminators, which I thought that my editor Scite used. Obviously not. After converting to LF, the file command now gave:

./Vagrant_Bootstrap.sh: a bash script, ASCII text executable

With that done I could execute the script, though it didn't work. I found I needed to sudo the script to run it since it installs software. (Not that surprising). Presumably Vagrant sudo runs the script when running it as a bootstrap, since they don't use sudo in front of the commands in the script in their examples.

I found that I couldn't get apache2 to start as my configuration uses SuPHP_UserGroup, and this is not available in the libapache2-mod-suphp build from the Ubuntu repository. So you have to build it from source: How To Install suPHP On Various Linux Distributions For Use With ISPConfig (2.2.20 And Above). But I then had a load of issues getting that to install (no libtool or make installed by default, needing to libtoolize, etc). I need to try it again from fresh to check if all the steps I used in finally getting it to compile are needed or not.

After getting it compiled I still couldn't get apache2 to start. From the log, it looks like this is because the error log directories specified for the sites enabled didn't exist. So this essentially means getting most of my home directory copied from my current Linux VM over to the shared Vagrant folder in Windows.

The issue with doing this was that I have some filenames that aren't acceptable to Windows, such as filenames with '?' and ':' in them. As much as I dislike arbitrary restrictions like this, the only way I could think of dealing with this is to rename the problematic files to remove the problematic characters.

I have a lot of tabs open with setting up Vagrant related stuff, so I'm just going to list them here in case I want to come back to them:

Tuesday, 28 October 2014

Trying to setup a Vagrant box

This morning the weather was sunny, so I went out on a walk to take a few photos. I would have preferred a few clouds in the sky, but sun was the main thing I wanted. It was actually pretty hot, even in my T-shirt.

In the afternoon I geo-coded and processed a few of the morning's photos. Then I carried on trying to get a vagrant box set up, which I started yesterday.

The first issue I had was that there was no ssh on my system. So I had to install git and add its bin dir to the %PATH% environment variable.

But with that done I still couldn't get Vagrant working. I followed the tip here: Vagrant stuck connection timeout retrying to modify the VagrantFile so that the VM would be opened in a VirtualBox window. That showed that the VM booted up with no issues. When I tried Vagrant ssh nothing would happen and cmd would just go onto a new line.

I found someone with a similar issue here: Nothing happens when I type "vagrant ssh", though their vagrant up appears to work okay rather than getting an ssh connection timeout as I was. Reading more suggestions in the stack overflow thread, and this comment in particular, I found the solution - downgrade VirtualBox to 4.3.12. After doing this Vagrant up works with ssh connection issues and subsequently Vagrant ssh will ssh into the VM successfully.

Hopefully this will let me drop my VMWare Ubuntu machine and work from windows with the lighter weight Vagrant VM instead. I'll need some time to learn and practice before I can know whether this will actually be practical or not.