sphinx search config scripts

I just finished converting the bikesoup search to use the sphinx search engine instead of a simple doctrine fulltext search with filters.
One thing I found helpful are sphinx config scripts. Instead of hard-coding the configuration you can use any scripting language to produce the sphinx configuration. The just have to start with '#!' and sphinx will execute them and use the output as the configuration.
This is the simplified version of my main config file /etc/sphinx/sphinx.conf :

#!/bin/bash 
for i in /etc/sphinx.d/*.conf; do
 cat $i
done
for i in /etc/sphinx.d/*.conf.sh; do 
 $i
done

Then I have a /etc/sphinx.d/searchd.conf which just contains the "searchd {}" section.
And various symlinks like /etc/sphinx.d/bikesoup_dev.conf.sh into the project directories of the site.
Those scripts then create the correct configuration, they figure out the index names from the local configuration and start symfony tasks to create the xmlpipe2 output.
Nice and simple.

Symfony RPM for Fedora and RHEL

I am a big fan of Fedora and the PHP Symfony framework. Quite early I decided that it would be nice to have Symfony packaged as a Fedora package. What I didn't realize was how long it would take me.

I submitted my Package Review Request on the 12.8.2009. One of the first comments pointed to me to an older request, which was abandoned because the packager run into some difficulties with the Fedora guidelines. But

I am not a person who gives up easily, so I continued fixing the package until it got finally approved on the 19.1.2011. By now the packages for Fedora 14 and EPEL 6 are in the official repositories and you can install Symfony by installing the package php-symfony-symfony.

Why did it so long? The problem are the different philosophies between Fedora and Symfony. While Fedora has a strict no bundled libraries policy, Symfony comes with a number of bundled libraries to make it easy to install. Symfony also recommends to use a separate copy for each install, which also explains the bundling of the libraries.

I agree with both opinions, but I still wanted an RPM for Symfony. I think it can be useful for a few cases:

• if you want to create a quick project without having another copy of Symfony around
• if you have multiple websites on a machine which all run the same Symfony version and you want to save memory and manage Symfony by RPM
• if you want to build your own RPM, which includes your own Symfony application and you need some place to start.
• you start with the RPM version for a fast set-up and later convert it to a bundled symfony, which is rather easy

So, what libraries does Symfony bundle? These are in the vendor directories: doctrine, phing, propel, propel-generator, lime, swiftmailer. And then there is also sfYAML. I decided to do without propel support, because I didn't want to update the rather ancient propel RPM that already was available. I created new packages for doctrine and swiftmailer, while Remi already had sfYAML packaged for use with phpunit. So even if nobody likes having Symfony packaged, the doctrine and swiftmailer packages will be useful.

At the end of the review another problem was pointed out to me. There are too many versions of Symfony floating around. You got the one you get through the symfony pear channel (which I am using), the one from the website and the sandbox. The last ones as tgz and zip files. There are no checksums for any of these, which makes it impossible to find out if I provide the correct version or if someone replaced them on the site.

So, what can one learn from all of this?

• don't bundle libraries. The Symfony guys are very careful to make their classes well encapsulated and independent (even more so in Symfony2), the same holds through for library dependencies
• even if you don't manage to achieve you initial goal, other good things will come out of it
• whatever happens, you learn new things. By now I package a number of packages for Fedora.
• do not give up

Will I try to package Symfony2 ? Unlikely. Unless Symfony makes it easier to install the core and bundles separate and stops bundling libraries it will be another two year to package it. Though they seem to be on the right track with the Symfony components and the bundle system.