Other than that I extended the command-line parser, so you can now say “swec example.com -s /test.html” where you would previously have had to do “swec --baseurl example.com -s /test.html“. Other than that it’s mostly a bunch of cleanups, some refactoring and a few minor bugfixes, in addition to a new test suite so the thing can be properly sanity-checked before release.

If you need to sanity check dynamic websites, give SWEC a go.

mussort is a simple command-line music sorting program. It recursively processes a directory tree, and then sorts whatever music files it finds there, renaming the files and putting them in a nice directory tree.

0.2 added a load of features designed to make mussort faster. It introduced optional caching of file tags, which has a major impact on the performance on subsequent runs on a directory tree. I optimized away an insane amount of readdir() calls that it kept doing over and over, even though nothing had actually changed since the last readdir(). It also only supported id3info and ogginfo as sources for information, which is problematic because ogginfo is very very slow at times. So in 0.2 it can use the Audio::File perl module if it is available. It provides redundancy (should Audio::File fail for an ogg file, it falls back to ogginfo, should id3info fail for an mp3 file it tries Audio::File) and a large speed increase for ogg vorbis files. It can also use id3v2 if it is available. Because of the caching however, even without Audio::File, any subsequent runs on ogg vorbis files will be a lot faster.

When it comes to actual features, the largest one is support for detection of compilation albums. It will locate an album that contains a lot of different artists and then put those into a single directory named after the album, rather than put them into separate artist/album dirs. For those that don’t want that, it is important to note that the feature is optional and must be explicitly requested (like case-insensitive sorting).

Other than that there’s a bunch of code cleanups, along with minor additions, such as selectable verbosity (–verbose, –quiet) and the option to keep all duplicate files around (–keepdupes). mussort is also now hosted on github, so if you are interested, fork the repository and let’s see what cool stuff you can come up with! Remember to prod me with a pull request so that any nice things you do get included upstream.

The basic syntax will be something like this (any input is welcome):

URL http://...
GET
MATCH /regex/ or STRING
RUN_CHECKS

RUN_MAIN


Here any URL statement defines a new check, where all previous data is dropped. Each section can have a POST or GET statement, and then any number of MATCH statements, as well as a RUN_CHECKS statement. If any MATCH statement fails (ie. the regex doesn’t match, or the result isn’t equal to STRING) then it will skip the remaining tests and skip ahead to the next URL or RUN_MAIN statement.

MATCH is obvious, it runs a test on the entire content to see if it matches a regex, or equals a string.
RUN_CHECKS would start the standard (SDF-based) SWEC checks on the returned data.
RUN_MAIN would start the main SWEC mode
[SOMETHING] MATCH would create a named match, so the content within [ ] would be the returned error if it doesn’t match, instead of something generic like “failed to match /regex/”.

Other commands I’ve thought of that I might or might not want to do include one to clean the cookiejar, so that tests can be performed on how a page acts when cookies are missing, and a way to add custom skip filters based upon for instance the URL.

THese are just random ideas and plans that I’ve got at the moment, I haven’t begone coding it yet, but it’s definetely something I’m going to do at some point. I’ll welcome any input if you have any.

I wanted a simple way to sanity check a site, to ensure that my article changes didn’t suddenly break comments on images (lagacy apps are strange beasts). So I ended up writing SWEC, the simple web error checker. It’s a basic app that goes through all links in a site (or “webapp”) as long as those are present in the HTML (ie. it doesn’t run any JS, so its use in JS/AJAX/AJAJ-heavy webapps can be somewhat limited). It parses all pages it downloads, looking for known errors and then reports those. For instance, if you run it on a site based on Catalyst (perl) and catalyst crashes with its standard backtrace, SWEC will return which page it happened on, which page referenced it and a quick line about what happened. Ie. if it’s an exception it’ll say “Exception in Catalyst controller”.

It uses a very simple file format for writing tests (which is well documented in SWEC’s manpage). It has several different types of tests, but the most common one looks something like this:
[SWEC_CATALYST_CONTROLLER_EXCEPTION]
type = regexs
check = Caught exception in.*Controller.*Request.*Catalyst
error = Exception in Catalyst controller
sortindex = 11
What’s between the brackets [ ] is the name of the test. All tests that are shiped with SWEC are prefixed with SWEC_.
The type defines which “type” of test it is. This one is “regexs” which is a ‘smart’ regex, a standard perl regex that swec modifies during runtime to easier match HTML. The check is in this case a normal perl regex that is applied to the entire html document. As the type is regexs, swec will modify the regex to this during runtime: Caught(\\s+|&nbsp;|<[^>]+>)+exception(\\s+|&nbsp;|<[^>]+>)+in.*Request.*Catalyst

The error is the string that will be returned, and the sortindex is used for prioritizing tests, the lower the better (bundled tests will always be positive, so one only needs to give tests a negative index to ensure they will be run before bundled ones).

By default the bundled tests (default.sdf) and the user-specific rc file ~/.swecrc will be loaded. The user-specific one can disable bundled ones easily, and you can disable them on the command line on an individual basis.

SWEC supports sessions, where SWEC remembers previously checked URLs, and previous errors and can then either check pages that used to have errors before the others, or only report ‘new’ errors that did not exist before. This will also remember all settings that you set so you don’t have to type it every time (although it’ll allow you to do that as well). It has cookie support so it will run just fine as a logged-in user, though you probably don’t want to run it on a live database, but rather a test one, as it’ll click on any link it sees (with a few exceptions, it tries to avoid ‘logout’ and ‘delete’ links, additions to the exceptions list is welcome).

It’s GPLv3, so feel free to hack your own things into it. I’ll accept patches for the app itself, as well as new tests to be bundled. As long as they are either specific to a language, web server or framework, I’ll happily add more bundled checks (or fixes to existing ones), however I will try to avoid app-specific checks as that might just get a bit too much.

Happy hacking

The release focused primarily on fixes and cleanups. A bunch of minor bugs were fixed, and I rewrote most of the iCalendar back-end. It should be faster now and preserves files better than it used to.

This release includes a dummy maemo interface. It can currently fully display a Day Planner calendar, and has a UI quite similar to the desktop edition. However, it can not edit or add any events, and is as such not as useful as it might have been. The plan is for the maemo port to be ready for 0.11.

0.10 will (among other things) feature a new HTML exporting module (which has been in the works for quite a while), a cleaner, object-oriented version of the add/edit event windows (to simplify their use and maintainance. The current code that handles it is a bit ugly).

The major user-facing change will be the addition of a GUI for calendar subscriptions. 0.9 includes support for them (though it isn’t really announced because the feature isn’t ready), but one has to manually edit the config file.

In other related news, I’ve just released a development snapshot of Day Planner 0.9.
It is available at http://www.day-planner.org/index.php/download/devsnapshot. It contains all of the developed features for 0.9 (including the unfinished maemo port), but is not yet stable enough to be labelled stable, so If you’ve got time and feel like it, give it a spin and report any bugs you find. The whole iCalendar back-end has been almost completely re-written for this version, which is why I want some additional testing before I release a stable version.

Please do not package this release or submit it to news sites, it’s just a svn snapshot.

I’ll write a quick word on the various new projects, and note those that I have moved from older websites to random.

New stuff:

Audio Utils
This is just a collection of scripts I’ve had in use (and had uploaded for the world to download previoulsy, but never together nor on a proper website) that assists in various audio-related tasks. These are:
aac2ogg – a quick and dirty aac to ogg converter, using mplayer, faad and oggenc
wma2ogg – the same for wma, using mplayer and oggenc
reencode – a quick and dirty bash script that lets you re-encode an mp3 file to another bitrate. Useful for clearing up space on “mp3 players”, especially if you’re into audio-books, which can often have their bitrates drastically reduced and still be in a fairly decent quality.

GRandomWallpaper
This is a wallpaper randomizer for GNOME/Nautilus. It takes a list of wallpapers (a directory) and selects a random one. This can be done on a timer, or just once. It keeps a hitlist for how many times a wallpaper has been shown, and is more likely to pick those that have not been shown as often. It also lets you ban wallpapers, which will cause it to always ignore the wallpaper in question.

LatexB
This is a quick script that assists in building LaTeX files. It can call latex or pdflatex, automatically spawn xdvi(k) or evince after building, and detect bibtex. It’s merely a utility script so that I can do “latexb file.tex” to build the LaTeX file properly in one go (it calls latex multiple times to ensure that everything is referenced properly).

MonitorGrowth
This is a simple perl program that lets you monitor how fast a file is growing, displays information similar to that of download utilities like wget.

mussort
This program sorts your music collection. It puts them into nice directories (artist/album) and renames the files, so that everything is consistent. It can also let you delete dupes. It works on MP3, and OGG Vorbis-files.

SPGal
This is my first python project. It builds a static XHTML-gallery from a set of images. It can work as a drop-in replacement of iGal and jGal.

Old stuff, new website:

MagicPO
This is a program that helps you translate PO-files from one similar language to another. Right now it can do automatic translation of for instance Norwegian Bokmål to Norwegian Nynorsk (you only have to read through them afterwards).

PDFtoPNG
This is a quick program that lets you convert PDFs into a set of PNG files. It can also build HTML-files to go with the PNGs, for easy reading in a browser.

SLX-Dict
This is a simple command-line program that lets you look up words in the Norwegian computer translation dictionary. It’s useful if you don’t want to open firefox and search for words there all the time.

SSHMan
SSHMan is a simple ssh agent helper, along the lines of keychain. It does persistant management of ssh agents, will only ever prompt you for adding keys to the agent once, does not slow down logging into X and does not start when you’re logging in via ssh.

So I went to plan B, which was to reimplement a maemo-specific GUI in python that just talks to a perl back-end which takes care of all of the actual data processing. This is now well under way. A working prototype of the GUI in python is now in SVN, it can read and display calendar data, but has no edit capabilities yet. The back-end portion is just about finished, it is a mixture of code from the dayplanner perl client and the dayplanner-daemon, what’s missing there is more configuration file handling (which can’t be done yet, because I’m not quite sure what config options the maemo UI actually needs) and synchronization code.

This has helped make Day Planner even more modular. I split out some code that is useful elsewhere into a DP::CoreModules module. That module now has code that for instance handles the configuration files, parses date strings, creates config dirs, runtime module loading, summary string wrappers and localtime() wrappers. All code that can be shared (and doesn’t merit having their own module) will be put there.

I expect the maemo port to have initial editing capabilities within 1-2 weeks, depending on my workload.

The gtk2 perl bindings still don’t have a maemo port. I’m going to have a go at those first to see if I can get them running half-decently without too much work, if not I’ll have to look at other more dirty hacks. Though if I can get the bindings themselves running that would be much better and would make the port a lot easier to maintain. Here’s hoping!