fooninja.net

Drupal 7: Uncaught exception, MySQL server has gone away

2011-07-13T00:00:00+02:00

After upgrading to Drupal 7.4 I got the following error when trying to load /admin/modules:

Additional uncaught exception thrown while handling exception.

Original

PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away: SELECT s.lid, t.translation, s.version FROM {locales_source} s LEFT JOIN {locales_target} t ON s.lid = t.lid AND t.language = :language WHERE s.source = :source AND s.context = :context AND s.textgroup = 'default'; Array ( [:language] => sv [:source] => No tokens available. [:context] => ) in locale() (line 676 of /home/.../modules/locale/locale.module).

Additional

PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away: SELECT s.lid, t.translation, s.version FROM {locales_source} s LEFT JOIN {locales_target} t ON s.lid = t.lid AND t.language = :language WHERE s.source = :source AND s.context = :context AND s.textgroup = 'default'; Array ( [:language] => sv [:source] => %type: !message in %function (line %line of %file). [:context] => ) in locale() (line 676 of /home/.../modules/locale/locale.module).

Uncaught exception thrown in session handler.

PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away in _drupal_session_write() (line 203 of /home/.../includes/session.inc)

This problem - "MySQL server has gone away" - can happen if the query you send to the MySQL server is too large. Indeed, for me the problem went away when I raised MySQL's max_allowed_packet setting from 1M (default) to 16M in my my.cnf:

[mysqld]
...
max_allowed_packet=16M

fooninja.net goes mobile

2011-06-10T00:00:00+02:00

I bought an Android phone and decided to make fooninja.net a bit more mobile-friendly by creating a CSS for smaller screens. It basically just removes the ads and sets width to auto. This is what I added to the page header:

<meta name="viewport" content="width=device-width; initial-scale=1.0; maximum-scale=1.0;" />
...
<link rel="stylesheet" href="/css/handheld.css" type="text/css" media="only screen and (max-device-width: 800px)" />

It seems to work well, but I've only tested it on my Samsung Galaxy S2. Let me know if you find anything odd.

(I also added a print CSS. Forgot about that!)

Updating nodes programmatically in Drupal 7

2011-06-06T00:00:00+02:00

Just like creating nodes, updating/editing a node programmatically in Drupal 7 is very simple. See the node creation page for guidance on how to set/update various field types (text, date, taxonomy, etc.); I won't duplicate all that here. Instead I'll show you additional things like deleting field data, deleting attachments and managing revisions.

I highly recommend using print_r(node_load($nid)) (or drush) to check nodes before and after modification; it's a great way to learn about the data structures and see exactly what happens.

As always, don't forget to make a backup of your database before you begin. If you have any questions, corrections or suggestions, please feel free to leave a comment.

Basic example

Put the code below in e.g. foo_update.php in your Drupal root directory, and execute it either through the browser or from the command line. I prefer the latter, running the script from the Drupal directory (php foo_update.php). Also, just like when creating nodes, you have to make sure that the input is valid.

<?php
define('DRUPAL_ROOT', getcwd());
$_SERVER['REMOTE_ADDR'] = "localhost"; // Necessary if running from command line
require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$nid = 453; // The node to update

$node = node_load($nid); // ...where $nid is the node id

$node->title    = "Let's set a new title for this node";
$node->body[$node->language][0]['value']   = "And a new body text, too.";

if($node = node_submit($node)) { // Prepare node for saving
    node_save($node);
    echo "Node with nid " . $node->nid . " updated!\n";
}
?>

Deleting field contents

To remove field data, I suggest using unset() like this:

// Delete field with index 0
unset($node->field_textfoo[$node->language][0]);

// Delete third field from a multi-value field
unset($node->field_textfoo[$node->language][2]);

// Delete whole field (if field is multi-value, all values are deleted)
unset($node->field_textfoo[$node->language]);

So, to empty a field, just use unset(). Often, however, you might want to do it conditionally. Say you have a term reference (taxonomy) field for tagging and want to remove a certain tag from a node. You can do it like this:

// Remove tag with taxonomy id 8
foreach($node->field_tags[$node->language] as $index => $tag) {
    if($tag['tid'] == 8) {
        unset($node->field_tags[$node->language][$index]);
    }   
}

Deleting attached files

To delete an attached file from disk and delete its database record, use file_delete(): (Note that in D7, file_delete() takes a file object as an argument rather than a path; Drupal 6's file_delete() is now file_unmanaged_delete().)

// Load file object
$foofile = file_load($node->field_image[$node->language][0]['fid']);

// Delete file if it's not being used anywhere
file_delete($foofile);

// ...or force deletion
file_delete($foofile, 1);

To remove a file that's being used in the node, I'd do something like this:

$filefoo = file_load($node->field_image[$node->language][0]['fid']);
unset($node->field_image[$node->language][0]);
file_delete($filefoo);

If this doesn't seem to work, it's most likely due to a problem with permissions. Check directory permissions and make sure the user running the script is able to remove uploaded files.

If you want to delete an attached file but keep the actual file on the server, you might think you could just unset the field:

unset($node->field_image[$node->language][0]);

And you can. But the above will still keep the file's database record - it's removed from file_usage but will still be in file_managed. There doesn't seem to be any existing API call we can use straight away. Taking a hint from the file_delete() code, you can do this:

$filefoo = file_load($node->field_image[$node->language][0]['fid']);

unset($node->field_image[$node->language][0]);

module_invoke_all('file_delete', $filefoo);
module_invoke_all('entity_delete', $filefoo, 'file');

db_delete('file_managed')->condition('fid', $filefoo->fid)->execute();

...or simply hack core. Here is another way to do it.

If you want to remove a file without touching the database (perhaps because the file is not in the database to begin with), you can use file_unmanaged_delete():

file_unmanaged_delete("public://field/image/foo.jpg");

Revisions

I highly recommend using revisions when updating stuff programmatically. If something goes seriously wrong, it's much easier to roll back to a previous revision than fiddling with database backups.

To make a new revision, all you need to do is set $node->revision to 1, and optionally add a log message. Here's the basic example again, this time with revisions enabled:

<?php
define('DRUPAL_ROOT', getcwd());
$_SERVER['REMOTE_ADDR'] = "localhost"; // Necessary if running from command line
require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$nid = 453;

$node = node_load($nid); // ...where $nid is the node id

$node->title    = "Let's set a new title for this node";
$node->body[$node->language][0]['value']   = "And a new body text, too.";

$node->revision = 1; // Create new revision
$node->log = "Updated programmatically"; // Log message

if($node = node_submit($node)) { // Prepare node for saving
    node_save($node);
    echo "Node with nid " . $node->nid . " updated!\n";
}
?>

If you do print_r(node_load($nid)); you'll notice that the number of the latest revision is stored in $node->vid. The first time you create a node, $node->nid and $node->vid are the same. When you create a revision, $node->nid stays the same but $node->vid changes. Here's how you load a specific revision of a node:

$node = node_load($nid, $vid);

You can get a list of a node's revisions with node_revision_list($node). Note that this function takes a node object as its argument, so do something like this:

$node = node_load($nid);
$noderevisions = node_revision_list($node); // or node_revision_list(node_load($nid));
print_r($noderevisions);

From Wordpress to Jekyll and a new design

2011-06-04T00:00:00+02:00

Static site compilers have become quite popular in some circles in the past couple of years, especially since the rise of services like Disqus. I decided to make the switch from Wordpress to Jekyll, a "simple, blog aware, static site generator" written in Ruby, for the usual reasons: I got tired of keeping up with Wordpress security updates, and static files give me a warm, fuzzy feeling.

I can now host my site anywhere without ever having to worry about security or server load. Also, being a unix guy who pretty much lives in the terminal, having every post as a plain text file (with Markdown markup) and writing posts in Vim feels great. This also makes it easy to put the site in a Git repository or similar for version control.

Of course, with static files all the way, you need to use something like Disqus if you still want a comment system. Disqus is embedded with JavaScript. I'm not entirely happy with this solution. I prefer being in total control of my own site, and I don't like that JavaScript is required just to read the comments and I don't like the implications it has for archiving. Finding an alternative solution is on my todo list.

I initially played with nanoc, a much more sophisticated Ruby site compiler. However, being lazy, I found that Jekyll was easier to get started with and that I didn't really need the extra power of nanoc. (I will however use nanoc for other projects. It's a really neat piece of software.)

Getting started

Loads of people have their blogs on public GitHub repositories. There's also a long list of Jekyll sites on the Jekyll wiki; many of them have sources available. I started with Tom Preston-Werner's Jekyll site and modified it heavily.

Migrating posts from Wordpress

I imported all my Wordpress posts using the simple technique on the Jekyll blog migrations wiki page ("Using Jekyll + Mysql server connection"). I didn't care about tags, as I was going to redo them anyway. Here is a more advanced import script.

As for images, I just copied wp-content from my Wordpress installation to to the root directory of the new site. I then deleted in wp-content except wp-content/uploads.

To maintain the exact same post URLs I put this in _config.yml:

permalink: /:year/:month/:day/:title

Disqus

I think the easiest way to get all Wordpress comments into Disqus is to install the Disqus Wordpress plugin. (Be sure to sign up for Disqus first.) Then just import all existing comments to Disqus from the plugin's admin page. Voila. Finally, just grab Disqus' "universal code" and place it in _layouts/post.html (or whereever your posts layout file is).

Syntax highlighting

For syntax highlighting I installed Pygments and generated the CSS:

$ pygmentize -f html -S default > css/highlight.css

Pygment comes with a bunch of different styles; run pygmentize -L to list them. I settled on murphy.

One minor gotcha: PHP code didn't get highlighted unless it started with <?php. Turns out you have to add startinline to the highlight options to get around that, like so: {% highlight php startinline %} (end with {% endhighlight %}).

New design!

I decided to completely change the design while I was at it. I didn't realize until now how lacking the old one was. I'm still playing around with colors. If you see encounter any broken links or something that looks good, give me a shout.

Failed requests: Apache benchmarking (ab) gotcha

2011-05-24T08:15:50+02:00

When I used the Apache benchmarking tool ab for some simple tests I wondered why it returned a ridiculous number of failed requests for a simple page:

Complete requests:      10000
Failed requests:        8366
   (Connect: 0, Receive: 0, Length: 8366, Exceptions: 0

Wait a minute, length... I realized that ab expects the length (byte size) to be the same for each request, but I was requesting a dynamic page with some random elements, so it counted most of them as failed.

In other words, if you are benchmarking dynamic pages, ignore length errors.

You might also want to use httperf instead of ab.

Loading memtest from grub (Fedora)

2011-05-05T07:28:57+02:00

Programs started segfaulting for no good reason every now and then on my Thinkpad X61s running Fedora 14. So I turned to memtest86+, and it's easiest to just run it from Grub at boot (especially if you're lacking an optical drive or the ability to boot from a USB stick!).

memtest86+ might be installed by default; if not, run yum install memtest86+. Run /usr/sbin/memtest-setup to set up a grub entry. The following was added to my /boot/grub/grub.conf:

title Memtest86+ (4.10)
        root (hd0,4)
        kernel --type=netbsd /elf-memtest86+-4.10

All done. Reboot, start memtest and do something else for a while.

Guide to programmatic node creation in Drupal 7

2011-04-13T18:51:34+02:00

I had to import a few thousand items from a legacy database to a Drupal 7 site and found that it was quite easy to do so programmatically. Here I'll first show you the basic code for adding nodes and then I'll talk about different field types, including how to add images and term references (taxonomy). If you have any questions, just ask in the comments and I'll be happy to help!

Basic node creation - example

Put the code below in e.g. foo.php in your Drupal root directory, and execute it either through the browser or from the command line. I prefer the latter, running the script from the Drupal directory (php foo.php). Note that you have to make sure that the input is valid!

<?php
define('DRUPAL_ROOT', getcwd());
$_SERVER['REMOTE_ADDR'] = "localhost"; // Necessary if running from command line
require_once DRUPAL_ROOT . '/includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

$bodytext = "Foo m bar fnord?";

$node = new stdClass(); // Create a new node object
$node->type = "article"; // Or page, or whatever content type you like
node_object_prepare($node); // Set some default values
// If you update an existing node instead of creating a new one,
// comment out the three lines above and uncomment the following:
// $node = node_load($nid); // ...where $nid is the node id

$node->title    = "A new node sees the light of day";
$node->language = LANGUAGE_NONE; // Or e.g. 'en' if locale is enabled

$node->uid = 1; // UID of the author of the node; or use $node->name

$node->body[$node->language][0]['value']   = $bodytext;
$node->body[$node->language][0]['summary'] = text_summary($bodytext);
$node->body[$node->language][0]['format']  = 'filtered_html';

// I prefer using pathauto, which would override the below path
$path = 'node_created_on' . date('YmdHis');
$node->path = array('alias' => $path);

if($node = node_submit($node)) { // Prepare node for saving
    node_save($node);
    echo "Node with nid " . $node->nid . " saved!\n";
}
?>

$node->language is important! If you don't have the locale module enabled, the node will not be assigned any particular language. Or rather, the language code used then is LANGUAGE_NONE, which is a constant with the value und (undefined) in Drupal. If you have locale enabled nodes can exist in more than one language, and you should specify the language code. Go to Configuration -> Regional and language -> Languages to configure languages and see what code you should use. For English, it would be en; for Swedish, it would be sv; etc.

Other things you might want to set:

$node->status (1 or 0): published or not
$node->promote (1 or 0): promoted to front page
$node->sticky (1 or 0): sticky at top of lists
$node->comment: 2 = comments on, 1 = comments off

This all you need for basic node creation. I will continue with different field types. The code below should be placed before the node_submit line. All fields can be accessed with $node->field_fieldname, where fieldname is the name you find on Structure -> Content types -> Manage fields.

The best way to see how Drupal nodes are made up, and how to work with the various fields, is to simply look at the structure of an existing node. Just create a new PHP file with the first bunch of lines above (including drupal_bootstrap) and add print_r(node_load($nid)), where $nid is the node id. Or use drush.

Setting various field types

Text or integer field

Nothing special:

$node->field_fnordtext[$node->language][0]['value'] = "Fnord fnord fnord";

Multiple values (make sure the field is configured to allow this):

$node->field_author[$node->language][]['value'] = "Ford, Tom";
$node->field_author[$node->language][]['value'] = "Fnord, Dom";

Creation time

To set a node's creation time, you'd think you could just set $node->created. And you can - if you don't use node_submit() (or if you set it after node_submit()). Looking at the node_submit() function in node.module, we find this line:

$node->created = !empty($node->date) ? strtotime($node->date) : REQUEST_TIME;

Which means that node_submit() will set $node->created to the current time if $node->date doesn't exist. So, if you want to set the creation time, you have to do something like this:

$node->date = "2009-05-27";

This also means that if you are updating a node programmatically, don't forget to set $node->date; otherwise node_submit() will change the creation time. (I will write another post about updating nodes.)

Date field (datetime, date, datestamp)

If you use the date module, you get three new field types - date, datetime and datestamp. (See this page to read about the differences; also check out this discussion.)

If your field is named datetest, you could do:

// For datetime
$node->field_datetest[$node->language][0][value] = "2011-05-25 10:35:58";

// For date
$node->field_datetest[$node->language][0][value] = "2011-05-25T10:35:58";

// For datestamp
$node->field_datetest[$node->language][0][value] = strtotime("2011-05-25 10:35:58");

Note that you don't need to specify a complete date; for datetime and date you can just pad with zeros, e.g. "2011-05-00 00:00:00" (datetime), "2011-00-00T00:00:00" (date), etc. For datestamp you could just do e.g. strtotime("2011-05-25").

Important: Also note that while the exact value you specify will be stored in the database, the actual time displayed on the site might be different depending on timezone settings. When you create a new datetime/date/datestamp field, you get to choose between five different timezone handling methods. The default one is "site's time zone":

When entering data into the field, the data entered is assumed to be in the site's time zone. When the data is saved to the database, it is converted to UTC.

However, if you set a date field programmatically like in the above example then no conversion takes place, so make sure you account for the field's timezone settings. Or in other words, if you use "site's time zone", make sure the time is in UTC.

Boolean field

Single on/off checkbox:

$node->field_bork[$node->language][0]['value'] = 1;

Term reference (taxonomy) field

Set the term reference field tags to taxonomy term id 25 (for more than one term, just repeat the line; and note that it doesn't matter whether the widget type is select list, check boxes/radio buttons or autocomplete):

$node->field_tags[$node->language][]['tid'] = 25;

(If you are trying this with the tags field in the default article content type, and have locale enabled, and have set $node->language to e.g. 'en', and it doesn't seem to work: I also encountered this oddity (bug?). For some reason tags are saved to $node->field_tags[und] instead of $node->field_tags[en], while e.g. body is saved to $node->body[en] as expected. If I create a new term reference field, it works as it should. So either do that, or change the above to $node->field_tags[und][]['tid'].)

As you can see, you need the know the taxonomy term's id. Fortunately, there’s a Drupal function to help us with this: taxonomy_get_term_by_name(). You supply the name ("Italy") and it returns an array of matching term objects, so you can do something like this:

if($foo = taxonomy_get_term_by_name('Italy')) {
    $foo_keys = array_keys($foo);
    $node->field_tags[$node->language][]['tid'] = $foo_keys[0];
}

Unfortunately, you can't specify a certain vocabulary with this function. This means that if you have the same term name in more than one vocabulary, the above code will just use whatever happens to come first. If you want to specify a certain vocabulary, say the one with id 9, you could do something like this:

$foo = taxonomy_get_term_by_name('Italy');
foreach($foo as $term) {
    if($term->vid == 9) {
         $node->field_tags[$node->language][]['tid'] = $term->tid;
    }
}

(One way of figuring out the vocabulary id is to run print_r(taxonomy_get_vocabularies());)

But what if you want to create new vocabulary terms for those that aren’t already in the database? You can use taxonomy_term_save(), like this:

$new_term = array(
    'vid' => 1,
    'name' => 'Fugazi',
    // You can optionally also set id of parent term, e.g. 'parent' => 25
);
$new_term = (object) $new_term;
taxonomy_term_save($new_term);

$new_term is very conveniently updated with the tid of our newly created term, which you can get with $new_term->tid.

Node references

If you use the references module for node/user references you can set a reference like this:

// 453 is the id of the referenced node
$node->field_author[$node->language][]['nid'] = 453;

This way you can also add multiple references in the same field - just repeat.

Image field

Attaching an image to any given image field is easy. Create a file object, copy the file and associate the file object with the image field:

$file_path = drupal_realpath('foo.jpg');
$file = (object) array(
            'uid' => 1,
            'uri' => $file_path,
            'filemime' => file_get_mimetype($file_path),
            'status' => 1,
);
// You can specify a subdirectory, e.g. public://foo/
$file = file_copy($file, 'public://');
$node->field_image[$node->language][0] = (array) $file;

How to fix wifi after suspend/resume in Fedora

2011-02-11T02:03:08+01:00

Problem: Just like in Ubuntu, the wireless on my Thinkpad X61s running Fedora 14 would sometimes stop working after suspending and resuming. The wifi light would be on but no networks would be detected. The wifi card is an "Intel PRO/Wireless 4965 AG or AGN", according to lspci.

Solution: I used the following quick and dirty script:

#!/bin/sh

case "$1" in
    thaw|resume)
        modprobe -r iwlagn
        modprobe iwlagn
        ;;
    *) exit $NA
        ;;
esac

Save it as 90resetwifi.sh, run chmod +x 90resetwifi.sh and put it in /etc/pm/sleep.d.

Google Earth in 64-bit Linux (Fedora, Ubuntu)

2011-01-25T07:08:50+01:00

On my Fedora 14 machine I installed the x86_64 version of the Google Earth rpm. It did however not work:

$ google-earth /usr/bin/google-earth: ./googleearth-bin: /lib/ld-lsb.so.3: bad ELF interpreter: No such file or directory

Turns out the provided binary is in fact not 64-bit:

$ file /opt/google/earth/free/googleearth-bin /opt/google/earth/free/googleearth-bin: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped

Installing the package redhat-lsb.i686 fixed it:

# yum install redhat-lsb.i686

If you're running Ubuntu, I believe the package to install is lsb-core.

Richard comments that if you're using the 64-bit fglrx drivers from RPM Fusion with an ATI card in Fedora and get an error about libGL.so.1 not found, try this:

# yum install xorg-x11-drv-catalyst-libs.i686

screen: "No more PTYs" error

2010-12-02T15:39:28+01:00

On a Linux server running CentOS 5.5 I came across the error No more PTYs, Sorry, could not find a PTY when trying to run screen as a normal user. (It worked fine as root.) I added my user to the group tty and verified the relevant permissions in /dev. Still no dice.

This fixed the problem:

mount -t devpts /dev/ptmx /dev/pts

I made it permanent by adding the following to /etc/fstab:

/dev/ptmx       /dev/pts        devpts  defaults        0       0

Swedish characters in Ubuntu (svenska tecken (åäö) i konsolen)

2010-11-30T15:08:33+01:00

When ssh'ing to an Ubuntu box running a default English installation, Swedish characters wouldn't work in the console. I fixed it by adding the following to /etc/profile:

export LANG="en_US.utf8"

That was enough. You can also play with with the various LC_ variables to modify behaviour. Run locale to see the current settings. If you want a Swedish locale but it's not available, try locale-gen sv_SE.utf8 and dpkg-reconfigure locales.

If you happen to be using PuTTY, you might also have to change the encoding sometimes. Right-click on the window's title bar, choose Change settings and go to Window -> Translation and change "Character set translation on received data" to UTF-8.

Drupal 7 and cron

2010-11-29T14:49:09+01:00

When I tried to access Drupal 7's cron.php from localhost I couldn't figure out why it gave me a 403 Forbidden error, despite everything looking correct in the configuration. Turns out that you now need to access cron.php with a key. Log in to your Drupal installation and go to Reports -> Status report and you will see the URL you need to use.

I have something like this is my crontab (one line):

0 * * * * wget -O - -q -t 1 http://foobar.baz/cron.php?cron_key=RTnpslsmQgfjAs8zT29MhZ-FsdfhjKASDh28fH8fd1

Skype in Fedora 64-bit

2010-10-25T23:36:28+02:00

Skype only provides a 32-bit package, so to run it in Fedora you'll need to install a load of 32-bit libraries. I did this in Fedora 13, but it should be pretty much the same in earlier (and later) versions.

Install the required libraries and their dependencies:

# yum install qt.i686 qt-x11.i686 libXv.i686 libXScrnSaver.i686 pulseaudio-libs.i686 pulseaudio-libs-glib2.i686 alsa-plugins-pulseaudio.i686

Download the latest Fedora RPM from Skype's Linux download page and install it:

# rpm -Uvh skype-2.1.0.81-fc10.i586.rpm

Done!

rxvt-unicode with 256 colors in Fedora/RHEL/CentOS

2010-10-25T18:16:46+02:00

NOTE: Since rxvt-unicode 9.09 (November 2010) support for 256 colors is officially included, which means that in newer releases (e.g., Fedora 14) you can skip all the stuff below and just install rxvt-unicode-256color or rxvt-unicode-256color-ml.

OLD INSTRUCTIONS: My favorite terminal, rxvt-unicode (urxvt), has 88 color support by default. However, my favorite vim color scheme, Zenburn, needs more than that. While I found a good post about doing it in Ubuntu I couldn't find anything about doing it in Fedora/RHEL/CentOS; so here goes. I did this on a machine running Fedora 13.

The rxvt-unicode source actually includes an unsupported patch for 256 colors, so we need to get the source rpm, modify the source and rebuild.

Patch and rebuild urxvt

If you don't have yum-builddep and yumdownloader, run yum install yum-utils.

Install necessary build dependencies:

# yum-builddep rxvt-unicode

Get the source:

$ yumdownloader --source rxvt-unicode

(You will also need the usual development stuff if you don't already have them, e.g., yum install gcc make kernel-devel rpm-build and possibly others.)

Install the source rpm:

$ rpm -i rxvt-unicode-9.07-1.fc13.src.rpm

This will install the source RPM to your user's ~/rpmbuild directory.

Extract the source and make a copy of it:

$ cd ~/rpmbuild/SOURCES
$ tar xvfj rxvt-unicode-9.07.tar.bz2
$ cp -r rxvt-unicode-9.07 rxvt-unicode-9.07a

Patch the copy with the 256 color patch:

$ cd rxvt-unicode-9.07a/
$ patch -p1 < doc/urxvt-8.2-256color.patch

Create a patch file for the RPM build:

$ cd ..
$ diff -uNr rxvt-unicode-9.07 rxvt-unicode-9.07a > rxvt-unicode-9.07-256colors.patch

Now we have to edit the spec file so that the patch is included in the build:

$ vi ../SPECS/rxvt-unicode.spec

Add Patch0: rxvt-unicode-9.07-256colors.patch to the top section of the file, e.g.:

Source1:        rxvt-unicode.desktop
Patch0:         rxvt-unicode-9.07-256colors.patch
BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)

Also add %patch0 -p1 on a line before %build further down, e.g.:

%setup -q
%patch0 -p1

%build

Also change Release in the top of the file (probably line 3) to something higher, otherwise rpm will think that the already installed version of rxvt-unicode is newer. E.g., from 1%{?dist} to 2%{?dist}.

Use rpmbuild to patch, configure, compile and build a binary RPM:

$ cd ..
$ rpmbuild -ba SPECS/rxvt-unicode.spec

You will find the RPM in the subdirectory RPMS/arch, e.g., RPMS/x86_64/rxvt-unicode-9.07-1.fc13.x86_64.rpm. Install it as root:

# rpm -Uvh RPMS/x86_64/rxvt-unicode-9.07-2.fc13.x86_64.rpm

Fix terminfo

Now we just need to make terminfo aware of rxvt-unicode's new capabilities. Thanks again Garry: (execute the following as your normal user)

$ cd ~
$ infocmp -L rxvt-unicode > rxvt-unicode.terminfo
$ vi rxvt-unicode.terminfo

Change (on line 8 in my file)

lines_of_memory#0, max_colors#88, max_pairs#256,

lines_of_memory#0, max_colors#256, max_pairs#32767,

Final steps:

$ install -d .terminfo
$ tic -o .terminfo/ rxvt-unicode.terminfo

Start a new urxvt and try running the script 256colors2.pl. It's included in the xterm source, but you can also download it from here.

$ perl 256colors2.pl

It should look like this:

Protect urxvt from being updated

If you don't want yum to replace your patched urxvt when a newer version is available, edit /etc/yum.conf and add the following line to section [main]:

exclude=rxvt-unicode

Speed up encryption in Linux

2010-10-20T20:05:09+02:00

You might not have realized it but as it is now, dm-crypt - which most people use for encryption in Linux (it's also used by TrueCrypt) - is not SMP-aware; only one processor core is used for each device. This is of course not optimal considering even many consumer systems now have quad-core CPUs (or more!). Recently, however, Andi Kleen of Intel posted a patch to change this:

DM-CRYPT: Scale to multiple CPUs (May 2010)
DM-CRYPT: Scale to multiple CPUs v2 (June 2010)
DM-CRYPT: Scale to multiple CPUs v3 (October 2010; for 2.6.36-rc7)

This has however not yet made it into the mainline kernel, so you'll have to patch the kernel source and recompile yourself. Just get the Linux source for your distribution, apply the right patch and recompile. Exactly how you do this depends on your distribution.

Below is just a quick and dirty list of the exacts steps I took from start to finish on a standard Ubuntu 10.10 Maverick install running Linux 2.6.35; for actual explanations, there is documentation elsewhere. Ubuntu's documentation (which I followed) is a good starting point.

Do the following at your own risk! It worked for me, and Andi Kleen writes "the 2.6.35 and .32 patches were widely used by lots of users with good results", but if you don't feel comfortable recompiling the kernel I suggest you wait for these improvements to make it into the mainline kernel. If you do try this, I'd love to hear your comments.

Install the necessary tools:

sudo apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge sudo apt-get build-dep linux sudo apt-get install git-core libncurses5 libncurses5-dev libelf-dev asciidoc binutils-dev

Get the source (this will take a while) from Ubuntu's git repository:

git clone git://kernel.ubuntu.com/ubuntu/ubuntu-maverick.git

Build the control files:

cd ubuntu-maverick
fakeroot debian/rules clean

Get the dm-crypt 2.6.35 patch in mbox format and apply it:

wget https://patchwork.kernel.org/patch/103404/mbox/ -O dm-crypt-scale.mbox
git am dm-crypt-scale.mbox

It should say something like "Applying: DM-CRYPT: Scale to multiple CPUs v2".

Regenerate the config files (probably not necessary, but doesn't hurt):

debian/rules updateconfigs

Create the independent packages:

skipabi=true skipmodule=true fakeroot debian/rules binary-indep

Create the tools package:

skipabi=true skipmodule=true fakeroot debian/rules binary-perarch

Build the kernel and do something else for a while:

fakeroot debian/rules clean
AUTOBUILD=1 NOEXTRAS=1 fakeroot debian/rules binary-generic

If you have more than one CPU, you can make the process faster by running concurrent compile commands. Just prepend e.g. DEB_BUILD_OPTIONS=parallel=2 if you have two CPUs:

DEB_BUILD_OPTIONS=parallel=2 NOEXTRAS=1 AUTOBUILD=1 fakeroot debian/rules binary-generic

When this is finally done, you'll have a few new packages:

linux-headers-2.6.35-23-generic_2.6.35-23.35_amd64.deb
linux-image-2.6.35-23-generic_2.6.35-23.35_amd64.deb
linux-tools-2.6.35-23_2.6.35-23.35_amd64.deb

Install the packages:

cd .. sudo dpkg -i linux-headers-2.6.35-23-generic_2.6.35-23.35_amd64.deb linux-headers-2.6.35-23_2.6.35-23.35_all.deb linux-image-2.6.35-23-generic_2.6.35-23.35_amd64.deb

Grub should be updated automatically, otherwise check /boot/grub/brub.cfg. Reboot and hope for the best.

Get rid of ubuntuone-syncdaemon

2010-10-20T15:25:02+02:00

Even though I haven't enabled the Ubuntu One client anywhere on my 10.10 (Maverick) system, I see that ubuntuone-syncdaemon (might show up as ubuntuone-syncd in top) is running in the background and eating a bunch of memory. Sigh. If you don't use Ubuntu One, Canonical's cloud service, just remove it:

sudo apt-get remove ubuntuone-client

(This will also remove a bunch of dependencies.)

Notes on Omeka

2010-10-12T16:42:11+02:00

I've decided on using Omeka for a project after trying out several alternatives (Kete (Ruby+MySQL), CollectiveAccess (PHP+MySQL), ResourceSpace (PHP+MySQL). It's free software - PHP+MySQL - for creating digital archives and exhibitions. Fairly simple and very easy to install, yet flexible with Dublin Core metadata elements by default and support for OAI-PMH. There are also a whole bunch of plugins; I found Simple Vocab to be essential for my project. Plus, it looks good by default.

I will update this page with notes as the project progresses.

Problem uploading files with non-ASCII characters

When trying to upload a file with a non-ASCII character in the filename (åäö, ë, é etc.), Omeka (version 1.2.1) threw an error: "Something went wrong with image creation. Please notify an administrator." I looked in Apache's error log and noticed that convert (ImageMagick) was given the wrong filename - non-ASCII characters were stripped. I pinpointed the problem to application/libraries/Omeka/File/Derivative/Image.php, line 131:

$old_path = escapeshellarg( $old_path );

The real problem was however that LC_CTYPE wasn't set and thus incorrect. I fixed it by adding the following to the top of admin/index.php:

setlocale(LC_CTYPE, "en_US.UTF-8");

Link to original image

At first Omeka's naming conventions confused me a bit. Fullsize does not refer to the largest version of an image; it's actually an intermediate size, the one that's something like 600 pixels wide by default, the one shown on web pages. Omeka stores four different versions of every image: fullsize, thumbnail, square_thumbnail and archive. The last one is the original file. By default, the item/view pages show the fullsize image and link to the same fullsize image. But I wanted to show the fullsize and link to the original, big image.

In my theme's item/show.php (I used rhythm), I changed this (around line 11, after <div class="item-img">):

$file = get_current_file();
if ($file->hasThumbnail()):
    echo display_file($file,array('imageSize'=>'fullsize'));
endif;

to this:

$file = get_current_file();
if ($file->hasThumbnail()):
    $archiveurl = $file->getWebPath(archive);
    $fullsizeurl = $file->getWebPath(fullsize);
    echo " ";
endif;

Surely not the nicest way to do it, but it works for me.

Fixed: "PHP Warning: preg_match_all(): Compilation failed: unrecognized character"

2010-10-09T22:38:55+02:00

When testing a site with Amazon's Linux AMI on EC2 - which is actually a customized version of RHEL/CentOS - I got this message in a PHP script:

PHP Warning: preg_match_all(): Compilation failed: unrecognized character after (?< at offset 3 in ...

The solution was upgrading the pcre package. The installed version was 6.6-2.6, while the latest official version is 8.10.

If you're running CentOS/RHEL/Amazon Linux AMI, you can use Jason Litka's Utter Ramblings yum repository which has 8.x. Follow the instructions on his site and then do yum install pcre. Or, if you set enabled=0 like I did, run yum --enablerepo=utterramblings install pcre.

(For Amazon's Linux AMI, I also had to change "EL$releasever" to "EL5", otherwise I got a 404 error.)

Ubuntu: Apache/PHP/MySQL in 5 minutes

2010-10-08T21:37:46+02:00

I use Ubuntu on my laptop and thus need a working environment for web development. This means the usual suspects: Apache, PHP and MySQL with some common extras.

Here's all you need to do to get your Ubuntu system ready for Drupal, Wordpress, etc. I used Ubuntu 10.10 (Maverick), but everything should be about the same in any recent version.

First, install Apache, PHP and MySQL. This will also install a bunch of dependencies: $ sudo apt-get install apache2 php5 php5-cli libapache2-mod-php5 mysql-server php5-mysql Apache and MySQL are started automatically, and the PHP5 module is enabled in Apache's configuration (if it's not, do sudo a2enmod php5). Since the latter happened after Apache was started, you need to restart it:

$ sudo /etc/init.d/apache2 restart

You now have a working environment up and running with the web root in /var/www, but read on for the extras I normally install and how to add a quick virtual host.

Some useful extras

Many programs use ImageMagick or the fork GraphicsMagick for various image operations (resizing, cropping, etc.). GraphicsMagick has better performance. They can be installed alongside each other. GD is also popular, so let's install that too.

# sudo apt-get install graphicsmagick imagemagick php5-gd

Set up a local virtual host

Most of my sites have dedicated domains or subdomains, so I want the same when developing.

In the directory /etc/apache2/sites-enabled, create a new file, myhost.net, with the following: (replace myhost.net and the paths with whatever you want)

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        ServerName myhost.net
        DocumentRoot /home/fooninja/projects/myhost.net
        <Directory /home/fooninja/projects/myhost.net/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
        </Directory>
        ErrorLog ${APACHE_LOG_DIR}/myhost.net_error.log
        CustomLog ${APACHE_LOG_DIR}/myhost.net_access.log combined
</VirtualHost>

The ErrorLog and CustomLog paths have been changed in recent versions of Ubuntu. See default for reference. AllowOverride All might not be optimal for security, but this is just a development box.

Next, activate the configuration and reload Apache:

$ sudo a2ensite myhost.net
$ sudo /etc/init.d/apache2 reload

The first line just creates a symlink from /dev/apache2/sites-available/myhost.net to /etc/apache2/sites-enabled/myhost.net, which is the directory Apache is configured to look in.

If the hostname for your virtual host doesn't exist, or if it exists but points to somewhere other than your local computer, edit /etc/hosts and add 127.0.0.1 myhost.net:

echo '127.0.0.1 myhost.net' >> /etc/hosts

PHP configuration

In /etc/php5/apache2/php.ini I usually change at least some values, like display_errors (set to On to see error messages directly, otherwise you'll have to look in the log file), post_max_size and upload_max_filesize.

Get rid of wrong password delay in Linux

2010-10-07T18:18:05+02:00

The password delay when typing the wrong password (su, sudo, ..) seems to be hard-coded to 2 seconds - see pam_unix(8). You can, however, easily disable this often just annoying feature.

Disable delay in Ubuntu and Debian

Edit /etc/pam.d/common-auth and add nodelay to the line with auth ... pam_unix.so. E.g.:

auth    [success=2 default=ignore]      pam_unix.so nullok_secure

change to

auth    [success=2 default=ignore]      pam_unix.so nullok_secure nodelay

Disable delay in CentOS, RHEL and Fedora

Edit /etc/pam.d/system-auth and add nodelay to the line with auth ... pam_unix.so, e.g.:

auth        sufficient    pam_unix.so nullok try_first_pass

change to

auth        sufficient    pam_unix.so nullok try_first_pass nodelay

If brute-forcing scares you

You can have users locked out for a specified amount of time (or until you manually let them in again) if they type the wrong password a specified number of times. Look up pam_tally(8), but beware - many configuration examples that you find scattered on message boards just won't work, not least because the order of the PAM directives is important. (Trust me, I spent too much time googling this.)

In Ubuntu and Debian, the following will lock out users from SSH (but not from local logins) for 120 seconds after typing the wrong password 3 times:

Open /etc/pam.d/sshd in a text editor.
Right before @include common-auth, add the following on its own line: auth required pam_tally.so onerr=fail deny=3 unlock_time=120
Right before @include common-account, add the following on its own line: account required pam_tally.so reset

Use pam_tally to check a user:

# pam_tally --user foobar
User foobar (1000)  has 4

You can also reset the counter manually:

# pam_tally --user foobar --reset

Linux/BSD network install using PXE and Windows

2010-10-07T14:46:39+02:00

If you need to install Linux or one of the BSD:s on a computer without an optical drive and over the network, it's simple. Here's how to do it with Windows as the PXE host. I'll show you how to install Ubuntu, CentOS and OpenBSD, but you can easily adapt the instructions for other systems.

Prerequisites:

A Windows machine running tftpd32, a free TFTP/DHCP server
A client machine that supports PXE boot
PXE and boot files for the operating system you want to install

Install tftpd32

Tftpd32 is a free DHCP/TFTP/DNS/SNTP/syslog server, and also a TFTP client, for Windows. We only need the DHCP/TFTP part.

Download the latest version (I used 3.50) and install/extract it to C:\tftpd.
Create the subdirectory C:\tftpd\root. You will put the install files here later.

Prepare the network

Depending on your network you might not have to do this, but to avoid any issues with my normal network, which already has a DHCP server that I don't want to bother changing, I temporarily put the host computer and the client (my laptop) on their own network. I configured the host to use 192.168.1.10, netmask 255.255.255.0. After the installation is successfully started I connect the client to the normal network again and do the usual net install over the Internet.

Note: if you get an error like "PXE-E11: ARP timeout" on the client, make sure nothing strange is going on with the network. Apparently this error can be caused by a number of things, including the hub/switch you use. If you don't have a different one available, consider getting a crossover cable - cheap and often useful.

Configure tftpd32

Start tftpd32.exe and click on Settings.
In the tab GLOBAL, make sure that at least TFTP Server and DHCP Server are checked.
In the tab TFTP, point Base Directory to C:\tftpd\root.
In the tab DHCP, fill in your desired network settings. Since I had the host and the client on a separate network, and was going to switch over to the normal network later anyway, I just set IP Pool starting address to 192.168.1.100, Size of pool to 10 and mask to 255.255.255.0. Note the field Boot File - you will probably have to change this later.

Install Ubuntu

Download the netboot files from one of the Ubuntu archive mirrors: dists/maverick/main/installer-amd64/current/images/netboot/netboot.tar.gz The above line is for the amd64 version of Maverick (10.10); replace with whatever architecture/release you need. For your convenience, here's a direct link to the amd64/maverick file on archive.ubuntu.com. It's about 13 MB.
Extract netboot.tar.gz to C:\tftpd\root.
Copy the file C:\tftpd\root\ubuntu-installer\amd64\pxelinux.cfg\default to C:\tftpd\root\pxelinux.cfg (change amd64 to i386 if necessary)
In Tftpd32 -> Settings -> DHCP, set Boot File to pxelinux.0

You should now be good to go. In the client computer's BIOS, make sure that network boot comes first. (On my Thinkpad X61s, it's "PCI LAN: ..." in Startup -> Boot.)

If you get an error like "TFTP prefix: Unable to locate configuration file", double-check that you copied the files correctly.

If you still have problems, also see the comment by Dmitri below.

Install CentOS

Download the contents of the isolinux directory from one of the CentOS mirrors: e.g. 5.5/os/x86_64/isolinux.
Put all the files right in C:\tftpd\root (not in a subdirectory).
Create the directory C:\tftpd\root\pxelinux.cfg
Copy C:\tftpd\root\isolinux.cfg to C:\tftpd\root\pxelinux.cfg\default. Note: no extension! In Explorer's Folder options, make sure "Hide extensions for known file types" is not checked.
Download syslinux and extract the file pxelinux.0 to C:\tftpd\root. It's in the core directory in the syslinux zip/tar.gz.
In Tftpd32 -> Settings -> DHCP, set Boot File to pxelinux.0

Done!

Install OpenBSD

Download pxeboot and bsd.rd from your favorite mirror. They are in e.g. 4.7/amd64, 4.7/i386.
Put the files in C:\tftpd\root.
In Tftpd32 -> Settings -> DHCP, set Boot File to pxeboot.

That's it. If you require other settings than the pxeboot defaults, look here.

If you get the error "panic: reverse arp not answered by rarpd(8) or dhcpd(8)", maybe you loaded bsd instead of bsd.rd?

OpenBSD 4.8 on a Thinkpad X61s

2010-10-07T02:10:35+02:00

OpenBSD 4.8 sports some exciting developments for laptop users, most notably support for ACPI suspend/resume, so I grabbed the snapshot of the upcoming 4.8 release.

I installed it on my Thinkpad X61s (pretty much the same as X61, which is not very different from X60 and X60s).

The installation process was very quick and easy yet again, especially since I dedicated the whole disk to OpenBSD and let it decide the partitioning automatically. Less than 10 minutes from start to finish.

Stuff that worked out of the box:

Sound
Volume keys (verified with ears and mixerctl)
Brightness keys
Battery status (apm)
X - no configuration needed, just startx
Suspend/resume when pressing Fn-F4. X, wifi etc all working after resume.

The wifi card was recognized alright as "Intel Wireless WiFi Link 4965" with driver iwn but didn't work right away:

iwn0: error, 2, could not read firmware iwn-4965
iwn0: could not read firmware

This is however Intel's fault. OpenBSD comes with the driver but not the firmware files because, as man iwn explains, "Intel refuses to grant distribution rights without contractual obligations". The same man page provides a URL to a package with the firmware files that you can install right away with pkg_add:

# pkg_add http://damien.bergamini.free.fr/packages/openbsd/iwn-firmware-5.5.tgz

(This address was current as of this writing, but please check the man page to verify.)

After installing the firmware package, you should be good to go. I used this to connect to my WPA-protected home network:

# ifconfig iwn0 nwid fooninja wpa wpapsk `wpa-psk fooninja secretpassword`
# dhclient iwn0

If everything seems to go well but still doesn't work, make sure the system is not trying to use the wired interface em0. (Try ifconfig em0 down and route flush.)

By default, closing the lid does not suspend the computer. To change this, set the sysctl entry machdep.lidsuspend to 1:

# sysctl machdep.lidsuspend=1

To make the change permanent, put it in /etc/sysctl.conf.

Needless to say, 4.8 is yet another great OpenBSD release. With the small exception of the wifi, which requires some very minor effort (due to Intel), everything I want just works. I will update this page if I stumble upon any issues. I'm also planning on writing a post about using OpenBSD on the desktop.

Xubuntu 10.10 woes

2010-10-05T08:51:16+02:00

After installing Xubuntu (sudo apt-get install xubuntu-desktop) in Ubuntu 10.10 (Maverick) I had some issues: when starting Xfce, both xfce4-power-manager and gnome-power-manager started, resulting in two applets in the panel and general weirdness with screen brightness; and both xscreensaver and gnome-screensaver started. Also, NetworkManager wouldn't remember/save any passwords.

To fix the dual power applet thing, I removed /usr/share/dbus-1/services/gnome-power-manager.service. But gnome-power-manager was still being started in the background. The real solution - which fixed both the power manager and the screensaver - was to disable GDM, as I describe here. I put exec xfce4-session in my ~/.xinitrc and just startx. I don't like login managers, but if you do, try XDM or SLiM.

To fix the issue with NetworkManager not remembering passwords, I replaced it with wicd - which I think is superior, anyway: sudo apt-get install wicd. Next time you start Xfce, it should be in the tray. To access the wicd GUI right away, run wicd-client -n. (Remember, wicd-client is just a front-end; the wicd daemon is running in the background. Once you've configured a network with a password to autoconnect, you can forget about wicd.)

Editing H.264 video in Linux

2010-10-04T17:55:22+02:00

I recently bought a Canon 5D Mark II, partly to shoot video. Since Linux is my system of choice and since I couldn't find much about 5Dmk2+Linux I did a quick test of the two most popular editors, OpenShot and KDEnlive.

First, to clear up some confusion: The 5Dmk2 does not use AVCHD, although it does use H.264. It stores video in a QuickTime MOV container with the H.264 codec for video and PCM for audio. You shoot in 1080p (1920x1080). Thanks to firmware updates it's possible to shoot in 24p (24 fps - the traditional frame rate for cinema).

My test system is a AMD Phenom II X6 1055T with 8 GB RAM and a Radeon HD 4350 running Ubuntu 10.10 (Maverick) 64-bit. I will update this post as I go along. Input appreciated!

OpenShot

The version of OpenShot available in the official Ubuntu repository for Lucid was old (1.1.x), so I instead installed it from OpenShot's PPA:

sudo add-apt-repository ppa:jonoomph/openshot-edge
sudo apt-get update
sudo apt-get install openshot openshot-doc

After starting OpenShot I imported three video files from the 5D. They were 2.8 GB, 4 GB and 1.4 GB.

I put one of them on the timeline and played it in the preview window. The picture had some strange artifacts around the corners (white dots and lines), playback was a little too slow, sound bad (crackling, popping, laggy). After a lot of trial and error, including transcoding to various formats, the problems went away after upgrading to Ubuntu 10.10. Also, after playing around with KDEnlive I didn't bother returning to OpenShot. While OpenShot is nice and promising, I felt KDEnlive was more mature.

KDEnlive

Just like with OpenShot, the version of KDEnlive that you find in the official Ubuntu repository is probably outdated. The KDEnlive project recommends that you get the latest from a PPA by running the following:

sudo add-apt-repository ppa:sunab/kdenlive-release
sudo apt-get update
sudo apt-get install kdenlive

And just like in OpenShot, when dragging a 5Dmk2 .mov to the timeline, the sound was bad: crackling, popping, lagging. Turned out this wasn't KDEnlive's fault - see below; after fixing this issue, I can edit the files just fine.

KDEnlive has the ability to transcode (convert) your video files to a more editing-friendly format by either right-clicking on the clip and choosing Transcode, or via File -> Transcode Clips.

DNxHD is often a good choice. The only trouble is that there's no transcode profile for 24p footage - only 25p, 30p etc. (The frame rate is actually 23.98 in my file.) No problem - under FFmpeg parameters, just change -r 25 (or whatever it is) to -r 23.98. Transcoding went fine. However, the sound problems persisted, which lead me to believe that PulseAudio might be the real culprit. Indeed it was! See below.

Fixing sound issues

To fix the issue with crackling/lagging sound on Ubuntu with PulseAudio (PulseAudio is the default sound server these days), I finally fixed it by upgrading Ubuntu from Ubuntu 10.04 (Lucid) to 10.10 (Maverick) in the end. I use normal PulseAudio, all default settings. The cause must have been one of the underlying libraries. Before that, I tried to following, which might still be helpful for someone:

Several people - and the KDEnlive manual - suggested removing PulseAudio. I did this:

sudo apt-get autoremove pulseaudio

After doing that, however, the sound no longer worked in KDEnlive (but it did work in other programs): "SDL failed to open audio: no available audio device". Thanks to this post, I installed libsdl1.2debian-alsa:

sudo apt-get install libsdl1.2debian-alsa

...which brought back the sound. (Make sure the settings are correct under Settings -> Configure KDEnlive -> Playback -> Audio driver.)

This fixed it for good - or so I thought! Turns out it's fine for a while when, for example, previewing a clip for the first time. But when I pause and then play again, the popping noises begin. Also, the first second of playback is often repeated once. This happens in both OpenShot and KDEnlive. Fixed it by upgrading to 10.10. Phew!

Transcoding

H.264 isn't particularly suited for editing, so typically an intermediate format should be used. But which one? Final Cut Pro users often recommend transcoding to ProRes 422. Many recommend DNxHD, which ProRes is derived from. Some like CineForm. Etc. Confusing? You bet. It seems like the ideal choice depends on platform and software, and whether or not you need to edit the footage on different platforms.

For me, DNxHD seems to be the best choice. I can use it in KDEnlive, Final Cut Pro, Premiere, etc.

You can use ffmpeg to convert to DNxHD. ffmpeg says this about one of the original files:

Seems stream 0 codec frame rate differs from container frame rate: 47.95 (48000/1001) -> 23.98 (24000/1001)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'MVI_0909.MOV':
  Duration: 00:08:17.74, start: 0.000000, bitrate: 47802 kb/s
    Stream #0.0(eng): Video: h264, yuv420p, 1920x1080, 23.98 tbr, 23.98 tbn, 47.95 tbc
    Stream #0.1(eng): Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s

To transcode/convert to DNxHD, try the following: Note that you can do this from within KDEnlive by right-clicking on a clip; see above.

ffmpeg -i sourcefile.mov -s 1920x1080 -r 23.98 -b 120000k -threads 2 -vcodec dnxhd -acodec copy foobar.mov

The converts the video to DNxHD (-vcodec dnxhd) while the audio remains unchanged (-acodec copy). Note that this is for a 1920x1080 24 fps video; if you shot with different settings, change the -r and -s options accordingly. Also try playing around with bit rate.

I haven't had time to check it out yet, but many people recommend using EKD for transcoding.

How to show Windows partitions in Thunar (Xfce)

2010-09-29T14:12:34+02:00

Unlike Gnome's Nautilus, the default file manager for Xfce, Thunar, won't automatically show you Windows / NTFS / FAT32 partitions. No problem. If you want to show for example an NTFS partition in the side pane (along with Desktop, Trash, File System, Documents, etc.), or on the Desktop, just do the following:

Make sure the NTFS partition is mounted (run mount and check). If it is, skip to step 2. If it's not, run sudo fdisk -l to find out the device name:

$ sudo fdisk -l
...
Disk /dev/sda: 320.1 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3fcb3fca

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1       30596   245760448+   7  HPFS/NTFS

Ok, in my case it's /dev/sda1. Create a directory for it and then mount it:

$ sudo mkdir /mnt/windows
$ sudo mount /dev/sda1 /mnt/windows

To mount the partition on boot, add mount /dev/sda1 /mnt/windows to /etc/rc.local (before exit 0) or add an entry in /etc/fstab.

In Thunar (the file manager), go to /mnt (or wherever you put the directory with the NTFS mount) and simply drag the Windows/NTFS directory to the left sidepane or the desktop. Voila. You can also right-click on the directory and choose Send To -> Side Pane.

Of course, this is just a shortcut to a directory rather than a representation of the disk; you can't mount/unmount from Thunar. If you want that functionality, a good alterative to Thunar is PCManFM.

Ubuntu 10.10 (Maverick) on a Thinkpad X61s

2010-09-28T16:57:13+02:00

I grabbed the beta of Ubuntu 10.10 (Maverick) for installation on my Thinkpad X61s laptop. The X61s is pretty much the same as the X61 (except for a low-voltage CPU and slightly less weight) and the X61, in turn, not much different from the X60.

My X61s has a Core2Duo L7500 1.6 GHz, CPU 4 GB RAM and a 320 GB disk. It has a partition with Windows 7 on it, so I would dual-boot.

Some quick impressions of the installer:

Slick. I very much like that fact you do some configuration (timezone, account, etc.) while the installer is formatting and copying in the background.

"Log in automatically" is the default option. Unlike in previous versions, I believe. More dumbing down?

Usability issue: On the "Allocate drive space" screen, where you choose between "Install them side by side", "Erase and use the entire disk" and "Specify partitions manually (advanced)", the buttons are "Quit", "Back" and "Install Now". "Install Now" should be "Next", as "Install Now" would be the button for when you, er, actually start installing stuff - not when you haven't even partitioned the disk.

In the window that first shows a Welcome text and then various info about features in Ubuntu, the text was too wide for the window, so I had to use the scrollbar to read it. I'd guess a lot of people wouldn't realize that. It certainly looks ugly, anyway.

On the whole, the installation was very smooth and took less than 10 minutes. Dual-booting with an existing Windows installation is, just like before, a no-brainer: the installer takes care of it.

Stuff that works out of the box

On a fresh standard install of the Ubuntu 10.10 beta (where I ticked the use non-free software box during install) most everything seems to work fine right away:

Sound works.
Wifi works.
Volume keys on the laptop work.
Brightness buttons work.
Fn-F4 to suspend works.
Suspend when closing the lid (default) works.
Resume works. Desktop password-protected by default. Wifi, sound still work after resume.
Battery info (time left or time left until full charge) is correctly shown.

My Windows 7 NTFS partition is shown in Nautilus. Overall, no problems at all yet. More - including use of HDAPS - to come.

Random notes

Logging in for the first time, my impression was "ugly background!". That mix of orange and purple doesn't do it for me. Ok, I'm nitpicking.

Irritation: I right-clicked the Mail applet to remove it. This also removed the volume applet. Wtf? Right-clicking and choosing "Add to panel", I can see neither Mail nor Volume. OK. Turns out they both hide under the moniker "Indicator Applet" and there's no obvious way to remove just one of them. Since I'm not going to waste CPU cycles on Gnome anyway, I won't waste a few minutes googling it. The battery info also disappeared when I removed the Ubuntu One applet and there's no sensible way to get it back. Perhaps "Notification Area", but that just crashed. Oh well.

Curious to try Unity, the new netbook interface, I installed it (sudo apt-get install unity) and logged out and in again ("Ubuntu Netbook Edition" automatically became the default option in GDM). Slick, but I realized it's not really useful for a 12.1" screen. If I had a smaller screen, I'd use Openbox or a tiling window manager.

I initially had some issues with xubuntu-desktop and wrote about it here.

How to transfer files from the Canon 5D Mark II in Linux

2010-09-22T19:24:28+02:00

For some strange reason, the Canon EOS 5D Mark II doesn't have USB mass storage support. Annoying, but no big deal. In a standard Ubuntu Linux 10.04 (Lucid) install running Gnome, connecting the camera to the computer automatically mounts it using the gphoto2 backend and brings up a window asking what you want to do. You can choose Open directory and browse the files in Nautilus.

But what if you want to access the files without the help of Gnome/Nautilus, perhaps from the console?

Transfer files using the gphoto2 command-line program

With gphoto2, the 5Dmk2 is basically just like any other supported camera.

First, Install gphoto2 (Ubuntu/Debian: sudo apt-get install gphoto2). Depending on how your system is configured, you might need to be root to execute some of the following.

After connecting the camera to the computer with the USB cord, let's see if we can detect it:

$ gphoto2 --auto-detect
Model                          Port                                            
----------------------------------------------------------
Canon EOS 5D Mark II           usb:

Yay! (Reminder: if things suddenly stop working, it might be because you've set the camera to auto-sleep after a minute. Wake it up by half-pressing the shutter release.)

You can also run gphoto2 --summary to get a whole lot of information, including firmware version and memory card usage:

$ gphoto2 --summary
Camera summary:                                                                
Manufacturer: Canon Inc.
Model: Canon EOS 5D Mark II
  Version: 3-2.0.7
  Serial Number: 6372eb34528d4277be90d0f3824df7ab
Vendor Extension ID: 0xb (2.0)
...etc etc...
Device Capabilities:
    File Download, File Deletion, File Upload
    No Image Capture, No Open Capture, Canon EOS Capture

Storage Devices Summary:
store_00010001:
    StorageDescription: CF
    VolumeLabel: None
    Storage Type: Removable RAM (memory card)
    Filesystemtype: Digital Camera Layout (DCIM)
    Access Capability: Read-Write
    Maximum Capability: 16009461760 (15267 MB)
    Free Space (Bytes): 13157040128 (12547 MB)
    Free Space (Images): -1

To see what's on the camera's memory card, use --list-files:

$ gphoto2 --list-files 
There is no file in folder '/'.                                                
There is no file in folder '/store_00010001'.                                  
There is no file in folder '/store_00010001/DCIM'.
There are 194 files in folder '/store_00010001/DCIM/100EOS5D'.
#1     MVI_0805.MOV               rd 145630 KB video/quicktime
#2     MVI_0806.MOV               rd 106402 KB video/quicktime
#3     _MG_0667.CR2               rd 25299 KB image/x-canon-cr2
#4     _MG_0667.JPG               rd   668 KB 2784x1856 image/jpeg
#5     _MG_0670.CR2               rd 31917 KB image/x-canon-cr2
#6     _MG_0670.JPG               rd  1163 KB 2784x1856 image/jpeg
...etc...

To download everything, both photos and videos, use --get-all-files:

$ gphoto2 --get-all-files

You can also use --get-file n to only download a specified file or range of files, e.g.:

$ gphoto2 --get-file 5
$ gphoto2 --get-file 10-17

Check back later for a post on how to edit video from the 5Dmk2 in Linux.

Unix find usage examples

2010-09-22T16:04:46+02:00

In any Unix system, find is a great tool for finding files matching certain criteria. Here are a number of handy examples on how to use find. They should work in both GNU find (most Linux distributions) and the BSD versions of find, unless otherwise noted. See man find on your system for the full manual. You can, of course, combine any number of options in find.

Patterns

Find files in the current directory and its subdirectories ending with .tar.gz:

$ find . -name '*.tar.gz'

Find files ending with .tar.gz or .zip:

$ find . \( -name '*.tar.gz' -o -name '*.zip' \)

Find files starting with 2008 and ending with .jpg or .cr2:

$ find . -name '2008*' \( -name '*.tar.gz' -o -name '*.zip' \)

Time

Find files with a last modification time older than 30 days:

$ find . -mtime +30

Find files with a last modification younger than 30 days:

$ find . -mtime -30

For minutes rather than 24-hour periods, use -mmin.

You can also use atime/amin, for last accessed; or ctime/cmin, for last status change.

Find files modified more recently than the file foo.c:

$ find . -newer foo.c

Size

In both GNU find and BSD find, you can search for files using n bytes of space with -size nc, e.g., -size 256c. To find files larger than the specified value, use a leading + before the number. To find files smaller than the specified value, use a leading - before the number.

GNU find and FreeBSD find support additional unit specifiers: k for kilobytes, M for megabytes and G for gigabytes. (FreeBSD find also has T for terabytes and P for petabytes.)

Find files larger than 500 MB in /home/foo and its subdirectories (search is recursive by default):

$ find /home/foo -size +500M

Find files smaller than 500 MB:

$ find /home/foo -size -500M

Find files that are exactly 64 KB:

$ find /home/foo -size 64k

Type

There are a number of different file types in a Unix system. You can search for a specific file type with -type c, where c is one of the following:

         b       block special
         c       character special
         d       directory
         f       regular file
         l       symbolic link
         p       FIFO
         s       socket

Search for files (but not directories) starting with foo:

$ find . -type f -name 'foo*'

Permissions

Find files that are world-writable:

$ find /home/foo/public_html -perm -o=w

Just like with chmod, -o means "other users" (i.e., not the owner or the group) and w means write.

Find files that are user-executable:

$ find . -type f -perm -u+x

Find files with the permission 644:

$ find . -perm -644

Ownership

Find files owned by user fooninja:

$ find . -user fooninja

With GNU find, you can search for a UID with -uid. With BSD find, you can use -user uid; if there is no such username, uid will be treated as a UID.

Search depth

You can use -maxdepth if you don't want to search recursively.

Find files in the current directory only - no subdirectories:

$ find . -maxdepth 1

Find files in the current directory and one level of subdirectories, but no more:

$ find -maxdepth 2

Conversely, you can use -mindepth if you don't want find to search at a level less than the specified number. The following would find /home/foo/bar/xyz but not /home/foo/bar or /home/foo:

$ find /home/foo -mindepth 2

Performing actions with find

You can use -exec to execute a specified command on the results. The following removes files starting with foo:

$ find . -name 'foo*' -exec rm -f {} \;

For each file that find finds, rm -f {} is executed, and {} is replaced by the file name. The semicolon ; terminates the expression and we protect it from shell expansion with the backslash.

Remove files ending with .tar.gz older than 150 days:

$ find /foo/backup -mtime +150 -exec rm -f {} \;

Careful with these - run them without the exec part first to make sure you are deleting the right files.

Encrypting and decrypting files with a password

2010-09-12T14:51:26+02:00

The OpenPGP tool gpg, which is part of GnuPG, is great for encryption and decryption of files. You can use it with a key or simply a passphrase. Here I will show you how do encrypt and decrypt with a passphrase.

Encrypt files interactively

Encrypt foo.tar.gz:

gpg -c foo.tar.gz

gpg will ask you for a passphrase. -c encrypts with a symmetric cipher; CAST5 (CAST128) is the default cipher. This generates foo.tar.gz.gpg in the same directory.

You can specify a filename for the encrypted file:

gpg -c -o bar.gpg foo.tar.gz

Encrypt files non-interactively

To encrypt non-interactively, for example in a shell script or crontab, you can specify the passphrase on the command line:

gpg --passphrase supersecret -c foo.tar.gz

Decrypt files interactively

To decrypt a file:

gpg foo.tar.gz.gpg

You can specify a filename for the decrypted file:

gpg -o filename foo.tar.gz.gpg

Decrypt files non-interactively

To decrypt non-interactively, for example in a shell script or crontab, you can specify the passphrase on the command line:

gpg --passphrase supersecret foo.tar.gz.gpg

Using a different cipher

To use a cipher other than the default one (CAST5), use the option --cipher-algo name. Run gpg --version to get the list of supported ciphers. My gpg says:

Cipher: 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH, CAMELLIA128, 
        CAMELLIA192, CAMELLIA256

FTP upload in a shell script

2010-09-12T13:36:39+02:00

If you want to upload files non-interactively using the standard FTP client with a shell script, here's a way to do it. The following logs in to the specified host, renames backup.tar.gz to old-backup.tar.gz, uploads backup.tar.gz and exits.

#!/bin/sh
HOST="ftp.foo.bar"
USER="foobar"
PASSWD="secret"
FILE="backup.tar.gz"

ftp -n $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
rename $FILE old-$FILE
put $FILE
quit
END_SCRIPT
exit 0

Of course, FTP in itself is insecure. If you really have to use it for sending sensitive files across an untrusted network, such as the Internet, at least encrypt the files.

Lighttpd mod_rewrite examples

2010-09-12T03:31:00+02:00

I like to learn by example, so here's an ever-growing list of how to du various URL rewrites with Lighttpd's mod_rewrite. More will be added every now and then. If you need help with your rewrites, use the comment form below.

(On other sites you might see url.rewrite-final and url.rewrite instead of url.rewrite-once. They are all the same.)

When used in example requests, xyz and abc are placeholder names for anything matching a specified pattern that will be backreferenced in the rewrite.

url.rewrite-once = (
# /music -> /index.php?page=music
"^/music$" => "/index.php?p=music",

# /music/xyz -> /index.php?page=music&song=xyz
"^/music/([a-zA-Z0-9_-]+)" => "/index.php?p=music&song=$1",

# /photos/2009/13241 -> /pix.php?y=2009&id=13241
"^/photos/([0-9]+)/([0-9]+)" => "/pix.php?y=$1&id=$2",

# /feeds/rss.xml or /feeds/atom.xml to /index.php?feed=rss-or-atom
"^/feeds/(rss|atom).xml$" => "/index.php?feed=$1",

# In the directory /foo, rewrite requests for .php files to index.html
# E.g.: /foo/blabla.php -> /foo/index.html
#       /foo/foobar.php -> /foo/index.html
"^/foo/([A-Za-z0-9]+).php" => "/foo/index.html"

# Pass on the whole query string with all the GET variables, e.g.
# /foo/index.php?xyz=abc&bar=blah&etc=342 -> /test.php?xyz=abc&bar=..
"^/foo/index.php(?:\?(.*))?" => "/test.php?$1"
)

Clean URLs in Wordpress 2.x and 3.x

url.rewrite-once = (
    # Exclude certain directories and .php files in the root directory
    "^/(wp-admin|wp-includes|wp-content)/(.*)" => "$0",
    "^/(.*\.php)" => "$0",

    # Everything else is handles by Wordpress
    "^/(.*)$" => "/index.php/$1"
)

Clean URLs in Drupal 6.x

url.rewrite-once = (
    "^/system/test/(.*)$" => "/index.php?q=system/test/$1",
    "^/([^.?]*)\?(.*)$" => "/index.php?q=$1&$2",
    "^/([^.?]*)$" => "/index.php?q=$1",
    "^/rss.xml" => "/index.php?q=rss.xml"
)

Shell script for importing photos

2010-09-11T02:22:59+02:00

I like to sort the photos and videos from my cameras into a simple folder structure like photos/YYYY/YYYY-MM-DD, e.g., photos/2010/2010-08-15. To do this from the command line, I use the ugly but simple little script below. Given a source directory as an argument, it moves all the files from the source directory to new ones in the destination directory, based on the last modified date of each file. New directories for the dates are created if they don't already exist.

#!/bin/sh
DESTDIR="/home/foo/photos" # where you want the photos to go
if [ -z "$1" ]; then
    echo "Usage: importpics.sh <directory>"
    exit 1
fi

cd $1

for file in *; do # moves EVERY file. You could also specify *.jpg etc
    if [ -f "$file" ]; then # we move files but not directories
    mtime=$(stat "$file" -c%y | cut -d" " -f1) # e.g. 2009-11-15
    myear=$(stat "$file" -c%y | cut -d" " -f1| cut -d"-" -f1) # e.g. 2009
        if [ ! -d "$DESTDIR/$myear/$mtime" ]; then # if dir doesn't exist,
                mkdir -p "$DESTDIR/$myear/$mtime"; # ..create it
        fi
        mv -i "$file" "$DESTDIR/$myear/$mtime" # move file to dir
    fi
done
# chown -R foo $DESTDIR/$myear # if run as root I want to set the right owner

The script above moves every file in the source directory (but not directories). If you want to move only, say, JPEGs and RAW files, you could try adding FOO="$(ls .jpg .rw2)" to line 9 and change line 10 to for i in "$FOO"; do. (Obviously your file extensions might differ.)

Notes from installing Kete

2010-09-09T15:07:56+02:00

I set out to install the Kete digital archive software (both 1.2 and 1.3) on a server running Ubuntu 10.04 (Lucid). I followed the installation guide step-by-step but ran into a couple errors at various stages. Here are some of them, along with other notes about Kete. I will add to this post as I go along.

Zebra problems (Kete 1.2)

On the last screen of the Kete install, it tries to initialize the Zebra databases. For me, nothing happened - it never got anywhere. I then looked at zebradb/README and tried the following to wipe the databases and start fresh:

$ rake zebra:stop # gives an error but does stop zebra. Otherwise killall zebrasrv
(in /home/kete/kete-ga)
Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
$ rake zebra:start
$ rake zebra:init # wipes the public db and spits out a bunch of logging info
$ rake zebra:init ZEBRA_DB=private
$ script/backgroundrb start # make sure the background worker is running

Next, I went to "Rebuild search databases" in the admin menu and had "clear and restart search engine first" checked. No go: "Rebuild failed. rebuild worker not running anymore? - undefined method `[]' for nil:NilClass"

I then looked in log/backgroundrb_debug_11006.log and saw the following:

#<Errno::ENOENT: No such file or directory - packet_worker_runner \
16:15:zoom_index_rebuild_worker:86:/home/kete/kete-ga/lib/workers:/home/kete/kete-ga/script/load_worker_env>
/var/lib/gems/1.8/gems/packet-0.1.15/lib/packet/packet_master.rb:117:in `exec'
/var/lib/gems/1.8/gems/packet-0.1.15/lib/packet/packet_master.rb:117:in `fork_and_load'
/var/lib/gems/1.8/gems/packet-0.1.15/lib/packet/packet_master.rb:80:in `start_worker'
/var/lib/gems/1.8/gems/packet-0.1.15/lib/packet/packet_connection.rb:80:in `start_worker'
...etc...

...which means that the script backgroundrb couldn't find packet_worker_runner. locate packet_worker_runner showed that I had the program - it just wasn't in my kete user's $PATH. Easily fixed:

$ locate packet_worker_runner
/var/lib/gems/1.8/bin/packet_worker_runner
$ export PATH=$PATH:/var/lib/gems/1.8/bin
$ script/backgroundrb stop
$ script/backgroundrb start

I tried rebuilding the search databases again and... success:

Started at Thu Sep 09 12:43:34 UTC 2010

Working on Discussion

16 records processed
Completed.
All records processed at Thu Sep 09 12:44:19 UTC 2010.

Passenger (mod_rails)

I got the error "extconf.rb:8:in `require': no such file to load -- mkmf (LoadError)" when I tried to install Passenger with gem. Turns out I just needed to do apt-get install ruby1.8-dev first.

Various Ruby issues

I tried to use Kete with Ruby 1.9.1 but failed; I realized Ruby 1.8.x is the way to go at the moment, and used Ruby Enterprise Edition as recommended by the Kete installation manual. I will however keep the solutions to some of the problems I encountered, as they might be useful in other situations; and here you will also find issues that I encountered with Kete 1.3+Ruby Enterprise Edition (1.8.x).

system_timer

When running rake manage_gems:required:install in Kete 1.3, the one gem that failed was system_timer. "warning: #warning rubysig.h is obsolete", "ârb_thread_criticalâ undeclared (first use in this function)", etc. I was running Ruby 1.9.x at this point and after some googling it turned out that system_timer is not needed in Ruby 1.9.x. So I just ignored it.

mysql error when bootstrapping

When running rake db:bootstrap I got this error:

Couldn't drop kete_ga_development : #

The MySQL library was missing. gem install mysql (the error message even tells you do to this) fixed it.

RubyGem / rack version error

When running rake db:bootstrap, I got another error:

rake aborted!
RubyGem version error: rack(1.2.1 not ~> 1.0.1)

Turns out I need to downgrade, for some reason, with gem install rack --version=1.0.1.

net/https missing:

Not done yet:

$ rake db:bootstrap
(in /home/kete/kete-ga)
rake aborted!
no such file to load -- net/https

Solved with apt-get install libopenssl-ruby1.9.1.

Ruby/Zoom (ruby-zoom) with Ruby 1.9.x

The latest version of Ruby/Zoom didn't work with my version of Ruby, 1.9.1. This seems to be a known problem. I fixed it.

# gem install zoom
Building native extensions.  This could take a while...
ERROR:  Error installing zoom:
        ERROR: Failed to build gem native extension.

/usr/bin/ruby1.9.1 extconf.rb
extconf.rb:5: Invalid break
extconf.rb:10: Invalid break
extconf.rb: compile error (SyntaxError)

In /var/lib/gems/1.9.1/gems/zoom-0.4.1 I commented out both unless ... end clauses. When running rake in that directory, I got some compile errors:

rbzoomoptions.c: In function âruby_hash_to_zoom_optionsâ:
rbzoomoptions.c:35: error: âstruct RArrayâ has no member named âlenâ
rbzoomoptions.c:36: error: âstruct RArrayâ has no member named âptrâ
rbzoomoptions.c:37: error: âstruct RArrayâ has no member named âptrâ
rbzoomoptions.c:38: error: âstruct RArrayâ has no member named âptrâ
make: *** [rbzoomoptions.o] Error 1

I found this page which had the solution for a similar error with another program. Using that as guidance, I opened up src/rbzoomoptions.c and starting on line 35 I replaced

    for (i = 0; i < RARRAY (ary)->len; i++) {
        pair = RARRAY (ary)->ptr[i];
        key = RARRAY (pair)->ptr[0];
        value = RARRAY (pair)->ptr[1];

with

   for (i = 0; i < RARRAY_LEN(ary); i++) {
        pair = RARRAY_PTR(ary)[i];
        key = RARRAY_PTR(pair)[0];
        value = RARRAY_PTR(pair)[1];

And then I ran rake again in /var/lib/gems/1.9.1/gems/zoom-0.4.1 and installed the Gem (gem install pkg/zoom-0.4.1.gem). Success! (This worked for me; but I don't know C and cannot vouch for the correctness of my solution, so use it at your own risk.)

invalid multibyte char

$ rake db:bootstrap
(in /home/kete/kete-ga)
rake aborted!
/home/kete/kete-ga/vendor/plugins/better_nested_set/lib/better_nested_set_helper.rb:121: invalid multibyte char (US-ASCII)
/home/kete/kete-ga/vendor/plugins/better_nested_set/lib/better_nested_set_helper.rb:121: invalid multibyte char (US-ASCII)
/home/kete/kete-ga/vendor/plugins/better_nested_set/lib/better_nested_set_helper.rb:121: syntax error, unexpected $end, expecting '}'
...       tree.map{|item| [ "#{'Â·Â·' * item.level}#{item[text_...
...                               ^

Ok, so I fixed that by adding # encoding: utf-8 to the top of vendor/plugins/better_nested_set/lib/better_nested_set_helper.rb. But there's more:

$ rake db:bootstrap
(in /home/kete/kete-ga)
rake aborted!
/home/kete/kete-ga/vendor/plugins/translate/lib/translate_controller.rb:109: syntax error, unexpected ':', expecting keyword_then or ',' or ';' or '\n'
/home/kete/kete-ga/vendor/plugins/translate/lib/translate_controller.rb:111: syntax error, unexpected keyword_when, expecting keyword_end
      when "contains":

At this point I realized trying to use Kete with Ruby 1.9.x is futile for me and went with Ruby Enterprise Edition.

wrong number of arguments (4 for 3)

When running rake db:bootstrap, these errors came up:

rake aborted!
wrong number of arguments (4 for 3)
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/i18n-0.4.1/lib/i18n/backend/base.rb:34:in `lookup'
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/i18n-0.4.1/lib/i18n/backend/base.rb:34:in `translate'
/opt/ruby-enterprise-1.8.7-2010.02/lib/ruby/gems/1.8/gems/i18n-0.4.1/lib/i18n.rb:152:in `t'

I read somewhere that the i18m gem is not needed by Rails 2.3.5, so I uninstalled it: gem uninstall i18n

Working with Kete

Translating

I wanted to translate Kete to Swedish. In config/locales.yml, I added sv: Swedish on a new line. In config/locales, I copied en.yml to sv.yml and started translating. (Don't forget to change the second line from en: to whatever language you want to translate to.) That seems to be all you need. There is a translation interface if you are running Kete in development mode, but I found it easier and quicker to edit the file directly.

After translating, I noticed that the "Content by type" sidebar on the main page had all the items (generated by zoom_class_plural_humanize in the translation file) capitalized - except one, ämnen (Topics). Kete didn't know how to capitalize the Swedish character ä. Didn't have time for a real fix so I just replaced t.base.zoom_types.topic...etc.. with Ämnen.

Slow? (Not really)

At first, I found Kete to be quite slow. These instructions for configuring Passenger (mod_rails) helped a little. When an instance wasn't already running, the first request took about 11 seconds, which is of course unacceptable. I set PassengerPoolIdleTime to 0, which means applications instances are not shutdown unless necessary. Subsequent requests to the default index page took about 3.9 seconds, which I found hard to swallow.

At this point I realized, somewhat embarrassingly, that I was running Kete in development mode. After doing rake db:bootstrap RAILS_ENV=production, commenting out RailsEnv development in my Apache configuration and restarting Apache, Kete became very snappy. The first request still took about 6 seconds, but with the aforementioned configuration change (PassengerPoolIdleTime 0 in the Apache configuration file) that's not a problem. Requests to the index page now take about 0.2 seconds.

How to fix Ruby mkmf (LoadError)

2010-09-08T15:04:14+02:00

When you try to install a Ruby gem package (like Passenger a.k.a. mod_rails) in Ubuntu or Debian you might run into the following error:

# gem install passenger
Building native extensions.  This could take a while...
ERROR:  Error installing passenger:
        ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb
extconf.rb:8:in `require': no such file to load -- mkmf (LoadError)
        from extconf.rb:8


Gem files will remain installed in /var/lib/gems/1.8/gems/fastthread-1.0.7 for inspection.
Results logged to /var/lib/gems/1.8/gems/fastthread-1.0.7/ext/fastthread/gem_make.out

A required file is missing. Fix it by running sudo apt-get install ruby1.8-dev (replace 1.8 with whatever version you're running - see ruby -v).

How to install fonts in Ubuntu

2010-09-08T03:05:56+02:00

Install Microsoft core fonts

In Ubuntu 10.04 (Lucid), run the following:

sudo apt-get install ttf-mscorefonts-installer

(In older versions of Ubuntu, the package name is msttcorefonts.)

The core fonts package installs the fonts to /usr/share/fonts/truetype/msttcorefonts and includes Arial, Comic Sans, Courier New, Georgia, Times New Roman, Verdana etc.

Install Red Hat's Liberation fonts

Red Hat created a set of fonts that serve as free drop-in replacements for three of the most popular Microsoft Windows fonts: Liberation Sans (Arial), Liberation Serif (Times New Roman) and Liberation Mono (Courier New). They are metric-compatible, not exact copies.

To install the Liberation fonts, run the following:

sudo apt-get install ttf-liberation

Install custom TTF fonts from the command line

To install a TrueType font (.ttf), create a .fonts directory in hour home directory if it doesn't already exist:

mkdir ~/.fonts

Next, you simply copy the .ttf file(s) to ~/.fonts:

cp foobar.ttf ~/.fonts

That's it. Note that it will only be available to your logged-in user.

To install a TrueType font system-wide so that all users have access to it, copy it to /usr/share/fonts/truetype. You might want to create a directory for your own fonts, for the sake of organization:

mkdir /usr/share/fonts/truetype/myfonts
cp foobar.ttf /usr/share/fonts/truetype/myfonts

Note that you will have to restart GIMP, OpenOffice, etc. to see the new font(s).

Most guides will tell you to run fc-cache -fv after installing a new font, but it appears this is no longer (?) necessary.

Install custom TTF fonts from the GUI

In the file manager of GNOME/XFCE/etc, simply double-click on the .ttf file. This should open it in the program Font Viewer. In Font Viewer, click "Install Font".

Secure remote backup with rsync+ssh

2010-09-06T02:17:27+02:00

For backup, rsync is a great tool: it synchronizes files and directories using delta encoding, i.e., it only transmits the differences between files. Combined with SSH, it's a simple and secure way to do remote backups.

All you need is SSH and rsync on both machines. The following will transfer the file blah.tar.gz to the home directory of user foo on server bar:

rsync -av -e ssh blah.tar.gz foo@bar:

-a is "archive mode" and, in my version, equals -rlptgoD (recursive, copy symlinks as symlinks, preserve permissions / modification times / group / owner / device files / special files).
-v increases verbosity (you see what's happening).
-e ssh specifices that ssh is the remote shell to be used

rsync has many options you can play around with to, among other things, decide what files to keep or not. See man rsync. For example, to copy the home directory of user foo to the directory backup on server bar and delete files in the remote backup that no longer exist on your local computer, you could use the following:

rsync -av --delete -e ssh /home/foo foo@bar:backup

You can also use --exclude=PATTERN to exclude certain files or directories. The following rsyncs all of /home/foo except /home/foo/test:

rsync -av --exclude="foo/test" -e ssh /home/foo foo@bar:backup

How to tunnel everything through SSH

2010-09-06T00:18:00+02:00

Problem: You don't trust your Internet connection (public wifi at a café, airport, ..), or perhaps it's being limited by a firewall (China, your workplace, ..). Sure, many sites offer SSL encryption these days, but many still don't - and even if they do, others can still see what sites you are contacting. You want to tunnel Firefox/Chrome and other programs through a secure tunnel.

Solution: Use your OpenSSH client to create a tunnel. All you need is a shell account on a remote server. (I SSH to my home server.) There are a few different ways of doing this. I will start with the easiest one.

Simple SOCKS proxy with OpenSSH

OpenSSH can act as a SOCKS server and tunnel traffic securely to a remote machine. All you need to do is the following:

ssh -D 2323 user@server

SSH will ask for your password and then a SOCKS proxy is started on port 2323 on your computer. Any traffic to port 2323 will be forwarded through the secure SSH connection. Note that your DNS traffic is not covered by this, so people can still sniff your DNS lookups (or, if they control the network, do even nastier things). See "Tunnel your DNS lookups" further down on this page.

For programs with native SOCKS support, all you need to do is configure the program(s) to use localhost 2323 as a SOCKS server. A few examples follow.

SOCKS proxy in Firefox

Go to Edit -> Preferences -> Advanced -> Network -> Settings. Check "Manual proxy configuration" and enter localhost as a SOCKS v5 host, with the port 2323.

SOCKS proxy in Pidgin

Go to Tools -> Preferences -> Proxy. Select SOCKS 5 as the Proxy type. Enter localhost as host and 2323 as port. Close the window and restart Pidgin.

SOCKS proxy for programs that don't support SOCKS

Not all programs support SOCKS proxying natively. A program called tsocks lets you use these programs transparently.

First, install it (Ubuntu/Debian: sudo apt-get install tsocks).

Next, edit the configuration file:

If you have root access, edit /etc/tsocks.conf and replace the contents (delete or comment out) with the following:
```
server = 127.0.0.1
server_port = 2323
```
If you don't have root access, put the two lines above in ~/.tsock.conf and then create a script with the following:
```
#!/bin/sh
TSOCKS_CONF_FILE=$HOME/.tsocks.conf
export TSOCKS_CONF_FILE
exec tsocks "$@"
```
Save it as for example runtsocks.sh and make it executable (chmod +x runtsocks.sh).

Now, just prefix any program you want to run with tsocks, e.g.:

tsocks elinks whatismyip.com
tsocks irssi

Or, if you don't have root access:

./runtsocks.sh elinks whatismyip.com
./runtsocks.sh elinks irssi

(Don't forget to start the SSH SOCKS proxy first, as outlined above, if you haven't already done so.)

Tunnel your DNS lookups

Even if you have an SSH proxy running as per the instructions above, your DNS traffic will still be unencrypted. This might be bad - a third party could see for example what web sites you are visiting; in some countries certain sites will be blocked; etc.

In Firefox, the solution is easy. Simply type about:config in the address bar and set network.proxy.socks_remote_dns to true. This will have the remote end (i.e., the machine you are SSH'ing to) handle the DNS lookups.

To verify that DNS lookups in Firefox are indeed handled by the remote machine, use tcpdump or similar to inspect the traffic, e.g. tcpdump port 53.

SSH chains and jumphosts

In the example above, we have A -> encrypted tunnel -> B, where A is your local computer and B is the remote machine. Sometimes you might however need to go through two or more servers to get out of a restrictive network (or into one). E.g.: A can access B. B can access C. C can only be reached by B - but you want to tunnel your traffic from A to C via B.

One way to do it

To tunnel traffic from A through C via B, first run the following on A:

ssh -NL 4343:C:22 B

Traffic to port 4343 on your local computer will be forwarded to B, where it's sent to port 22 on C. (This will tie up a terminal. If you don't need to enter a password for B, you could add an & to the end to have it run in the background.)

Next, we use the tunnel above to make a connection from A to C and create a SOCKS proxy on A. Run the following on A:

ssh -o UserKnownHostsFile=/dev/null \
-o StrictHostKeyChecking=no -ND 2323 -p 4343 localhost

Enter your password for C. You now have a SOCKS proxy running on port 2323 and any traffic to it will be sent to C.

If you need to go through more hosts, see this blog entry.

Upgrade any Ubuntu from the command line

2010-09-02T16:59:13+02:00

The procedure for updating any recent Ubuntu to a newer version from the command line is the same. Note that you can't jump from e.g. 9.04 to 10.10 in one go; you have to do one version at a time: 9.04 (Jaunty) -> 9.10 (Karmic), 9.10 -> 10.04 (Lucid), 10.04 -> 10.10 (Maverick).

To upgrade to the next release from the console, first install the package update-manager-core if it's not already installed - it contains the files for the release upgrader:

sudo apt-get install update-manager-core

Next, edit /etc/update-manager/release-upgrades and make sure it says Prompt=normal so that it will prompt if a new version of Ubuntu is available. Finally, run sudo do-release-upgrade to launch the upgrade process. You should see something like this:

$ sudo do-release-upgrade
Checking for a new ubuntu release
Done Upgrade tool signature
Done Upgrade tool
Done downloading
extracting 'lucid.tar.gz'
authenticate 'lucid.tar.gz' against 'lucid.tar.gz.gpg'
...etc...

After a while it will ask if you want to start the upgrade, tell you how much you need to download and give you a time estimate. Depending on your computer and your bandwidth, it can take quite a while. It will prompt you for input at various stages, mostly about whether or not to keep modified configuration files. Finally, it will ask you to reboot. Do so and cross your fingers (if you are doing it remotely).

How to fix wifi after suspend/resume in Ubuntu

2010-09-02T16:45:41+02:00

Problem: After suspending and then resuming my Thinkpad X61s running Ubuntu 10.04 (Lucid), the wireless would no longer work. Networks were detected but I couldn't connect - nothing happened. The wifi card is an "Intel PRO/Wireless 4965 AG or AGN", according to lspci. (I had the same problem with Ubuntu 9.10 (Karmic)).

Solution: I still don't know the cause, but restarting networking and unloading/reloading the module for the wireless interface fixed the problem.

#!/bin/sh
# Stop networking and network-manager
stop network-manager
service networking stop

# Remove and reload the module for the wifi card
# (replace iwlagn with your own if you have a different one)
modprobe -r -f iwlagn
modprobe iwlagn

# Start networking and network-manager again
service networking start
start network-manager

Save the file as fixwifi.sh, run chmod +x fixwifi.sh and run it as root. Voila! (Hopefully.)

To make things smoother, if you always have this problem after resuming, you can have the script run automatically after resume. In earlier versions of Ubuntu you would put the script in /etc/acpi/resume.d. This has changed with 10.04 Lucid. You now have to put the script in /etc/pm/sleep.d, but you will have to make some modifications. After looking at the scripts in /usr/lib/pm-utils/sleep.d for guidance, I came up with the following:

#!/bin/sh

. "${PM_FUNCTIONS}"

resume_wifi()
{
        # Stop networking and network-manager
        stop network-manager
        service networking stop

        # Remove and reload the module for the wifi card
        modprobe -r -f iwlagn
        modprobe iwlagn

        # Start networking and network-manager again
        service networking start
        start network-manager
}

case "$1" in
        thaw|resume)
                resume_wifi
                ;;
        *) exit $NA
                ;;
esac

Save the file as 99fixwifi.sh, put it in /etc/pm/sleep.d and make it executable (chmod +x 99fixwifi.sh). Crude? Yes, but it works for me. If you have a nicer solution, let me know!

How to get real full-screen in VirtualBox

2010-08-25T16:05:20+02:00

Problem: When running for example an Ubuntu Linux guest on a Windows host with VirtualBox, you want it to use the whole screen. Maximizing the window or going to full-screen changes the VirtualBox window size, but not the size of the guest.

Solution: While the guest OS is running, install "Guest Additions".

The following steps work for a standard Ubuntu 10.04 Lucid Lynx desktop install:

Start your guest OS and log in.
Select Device -> Install Guest Additions in the menu, or by using Host Key+D. (On my Windows VirtualBox installation, the default host key is right Ctrl.) This will make a VBOXADDITIONS image appear on your Ubuntu desktop. (If you are not running a graphical environment, type mount so see where it has been mounted.)
Build and install the Guest Additions modules by doing one of the following:
- If you are running the default Gnome desktop: Right-click the VBOXADDITIONS icon on the Desktop and choose "Open with autorun prompt". After confirming and typing in your password, it will build and install all the necessary modules.
- If you want to do it from a terminal: Go to the Guest Additions directory by typing cd /media/VBOXADDITIONS_3.2.8_64453 (note that the numbers might be different in your version). If you are running a 32-bit guest, start the installer with sudo sh VboxLinuxAdditions-x86.run. If you are running a 64-bit guest, start the installer with sudo sh VboxLinuxAdditions-amd64.run.
Reboot the guest. When X starts you should now have glorious full-screen.

OpenBSD 4.6 on a Thinkpad X61s

2010-07-30T15:51:08+02:00

These notes are old; look here for my experience with OpenBSD 4.8 on the same laptop.

Having previously only used BSD on servers, I was eager to try OpenBSD on my Thinkpad X61s (pretty much the same as an X61 - I believe the difference is a slightly lower voltage CPU and a different screen). I grabbed the iso of the latest amd64 snapshot of OpenBSD 4.6 (dated 2009-07-30) and started the new installer. Installation was a breeze – the whole process took less than ten minutes.

X and sound worked out of the box. So did the hotkeys for sound and brightness.

Wifi didn't work immediately. While the card ("Intel Wireless WiFi Link 4965", driver iwn) was recognized, nothing happened. dmesg revealed:

Aug  2 01:47:17 jagr /bsd: iwn0: error, 2, could not read firmware iwn-4965
Aug  2 01:47:17 jagr /bsd: iwn0: could not read firmware

Consulting man iwn it turns out that thanks to Intel, the necessary firmware file cannot be included in OpenBSD, so you have to download it yourself. The man page provides a link to a prepackaged version to be installed with pkg_add, so this took only a couple of seconds. After ifconfig iwn0 down; ifconfig iwn0 up the wifi light started blinking and all was well.

Suspend/resume is not yet working, but is being worked on.

Ubuntu Linux, ATI Radeon HD 4350 and Panasonic PX81

2010-07-30T15:41:45+02:00

I helped configure a HP DC7900 with a Radeon HD 4350 graphics card connected to a 42" Panasonic PX81 plasma tv (the full name is "Panasonic Viera TH-42PX81"; it's a PX80 with some minor additional features) using HDMI and encountered a few bumps along the way. Ubuntu 9.04, "Jaunty", was the distribution of choice.

lspci and aplay -l output, respectively:

01:00.0 VGA compatible controller: ATI Technologies Inc RV710 [Radeon HD 4350]
01:00.1 Audio device: ATI Technologies Inc R700 Audio Device [Radeon HD 4000 Series]

card 0: HDMI [HDA ATI HDMI], device 3: ATI HDMI [ATI HDMI]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

When starting X for the first time the correct resolution (1280x720) was set, but the edges were missing on all sides (i.e., I couldn't see most of the top and bottom panels). Sound didn't appear to work and videos played way too fast. Strange.

Quick searching didn't yield anything fruitful about the edges issue so, lacking time, I then switched from the open-source radeon driver to the proprietary fglrx driver. X started in 1920x1080, which was sluggish. I tried to change resolution with gnome-display-properties, but it just displayed a blank window and made X start eating CPU. This is apparently a bug in fglrx; a temporary solution is to switch to a terminal (say, Ctrl-Alt-F1) and then back.

Using amdcccle ("Catalyst Control Center"), I changed "desktop area" to 1280x720 and it seemed to work fine. The program wrote a new xorg.conf. It did not, however, remember this setting the next time, so I added Modes "1280x720" to the Screen section in /etc/X11/xorg.conf.

Sound

With fglrx sound worked fine, but I had to change settings in gnome-sound-properties first. I set sound playback to "HDA ATI HDMI ATI HDMI (ALSA)" and default mixer tracks device to "HDA ATI HDMI (Alsa mixer)". I had to set this manually in some programs, for example:

mplayer -ao alsa:device=hw=0.3

In gmplayer's Preferences -> Audio, I chose alsa and then entered hw=0.3 in the Device field under Configure driver. This made sound work, but for some reason a ton of error messages – "[AO_ALSA] Unable to find simple control 'PCM'" – were being generated during playback. "Enable Software Mixer", also in the Audio preferences, made them go away.

Video tearing

All was well... until I noticed video tearing. This is apparently normal for the fglrx driver – at least for newer cards – when using xv output. So I tried OpenGL output in mplayer (-vo gl), which resulted in flickering when windowed. OpenGL did however work fairly well in fullscreen, and disabling compiz solved the flickering issue: System -> Preferences -> Appearance -> Visual Effects - None (or: gnome-appearance-properties). The flickering is fixed in a later version of fglrx, but the the recommended way to get rid of tearing still seems to be:

Disabling compositing ("desktop effects")
Setting "Wait for vertical refresh" to "Always on" in amdcccle -> 3D -> More Settings.
Using OpenGL renderer (-vo gl in mplayer)

You can also use the aticonfig tool to experiment with various settings. See aticonfig -h for all the options (for example, --vs on for vsync).

There's a lot of discussion about video playback issues with ATI cards. If you're having problems, check out the AMD/ATI Linux forum on Phoronix, where AMD developers participate. And, of course, see if there's a new version of the driver out; AMD releases new versions of Catalyst frequently.

Happy Hacking Keyboard Professional 2

2010-07-30T11:24:44+02:00

Having used the Happy Hacking Keyboard Lite 2 for several years, I came across geekhack one day. It's a wonderful forum where keyboard aficionados talk about their recent purchases, post audio clips of keyboards and discuss the finer points of key switches.

I saw the light and realized that I needed a Real keyboard.

I find the Happy Hacking layout superior to anything else I've come across – it's just right – so the choice was easy: Happy Hacking Keyboard Professional 2. The Lite 2 is your typical rubber dome keyboard, albeit with a great layout; the Pro 2 has almost the same layout, but also real mechanical switches instead of membrane. It makes all the difference.

I got mine from beNippon for about 250–260 USD. "$250 for a keyboard?!!" Oh, yes. Considering that it's one of my most important tools and the fact that I will likely use it for 10 000+ hours, it was one of my most rational purchases in recent years. I'd pay $500 if I had to. Why are people content with crappy, mushy membrane keyboards? Probably because they are so used to them, or because that type of keyboard is so utterly dominant these days that most people just don't know anything else.

Most keyboards have 100+ keys; The HHKB Pro 2 has 60 (still full-sized). No F1–F12, arrows, print screen, pg up, etc; those are accessible with Fn+. Since one Fn key is right below Enter/Return, I can actually do many of these operations quicker than on a normal keyboard. No numpad. No caps lock! Ctrl is in its right place. It makes for a very small, elegant keyboard; below is a comparison with a Unicomp SpaceSaver, a buckling-spring keyboard that I use when I need to wake people up with a wall of sound.

I bought the blank version; it looks better. And it's a good conversation piece.

So, the layout is great, but here's why it costs much more than the Lite 2: the key switches. They are light, quiet, but still have excellent tactile response.

What else? There are six DIP switches on the back; with these you can change mode ("HHK", "Lite Ext.", "Macintosh") and the behaviour of some keys. See this image of the manual for what they mean. The USB cable is detachable; great if you want to move the board around. There also two USB ports on the back.

It comes in a nice box. On the back you'll find descriptions like "good feeling of oneness with cup rubber".

If you don't like the layout of the HHKB Pro 2, an interesting and equally expensive alternative is the Topre's line of Realforce keyboards.

If you want something less expensive, check out Filco's Majestouch keyboards.

If you long for the buckling springs of yore, consider getting a fresh keyboard from Unicomp.

For the latest on what keyboards to buy and where to get them, check out the aforementioned forum geekhack.

Swedish characters (svenska tecken!)

I'm a Swede and I need my å ä ö. I use the most common Swedish layout, but the small number of keys on the HHKB Pro 2 necessitates some modifications to have important things like <, > and | easily available.

Linux and other systems with X

I used xkeycaps to dump the current keyboard map and xev to get the keycodes. I then put the following in my xmodmap file:

keycode 100 = ISO_Level3_Shift
keycode 102 = Alt_L
keycode 0x34 =  z               Z               z               Z               less
keycode 0x35 =  x               X               x               X               greater
keycode 0x3B =  comma           semicolon       comma           semicolon       bar
keycode 0x3C =  period          colon           period          colon           bar

What it does:

Meta/Alt-z: <
Meta/Alt-x: >
Meta/Alt-, and .: |

On the back of the keyboard I have SW3 and SW5 set to ON. (Remember: If you change the DIP switches, you might have to unplugin the USB cable and plug it back in again.)

Mac OS X

On Mac I had to make more modifications since characters like \ { } are not in sensible places (at least not on the default Swedish layout). I found the very useful but not particularly intuitive keyboard layout editor Ukelele, and then came across this excellent blog post by Claus Dahl. I quote the most important part:

Here's the slightly insane part: The layout is organized by modifiers. The entire layout, that is, e.g. "here's how the keyboard looks when option key is down". You can't use a modifier combination that isn't already in the layout. (You can add new ones through an interface so lacking in intuitiveness that I didn't figure it out). I had little luck mapping cmd-single key to a new character as an example. Mapping stuff to alt-something however works fine, as there is already an alt- keyboard.

You can download my keymap. Put it in ~/Library/Keyboard Layouts and then enable it in System Preferences|International|Input Menu. You might have to log out and in again. It modifies "Swedish - Pro" to do the following:

Alt-1: §
Alt-7: {
Alt-8: [
Alt-9: ]
Alt-0: }
Alt-+: \
Alt-,: |
Alt-z: <
Alt-x: >

Note that on the back of the keyboard I have SW2 and SW3 set to ON, and the rest are OFF. (Again, if you change the DIP switches, you might have to disconnect and reconnect the keyboard for the changes to take effect.) If you need to change what key is Command, Option, etc., try DoubleCommand.

Windows

AutoHotkey is an excellent tool for remapping and binding keys. It can do some advanced stuff, but I only use the following, which should be self-explanatory (e.g., Right Alt + , will produce a |):

RALT & 1::Send {§}
RALT & .::Send {|}
RALT & ,::Send {|}
RALT & z::Send {<}
RALT & x::Send {>}

Install AutoHotkey, paste the above in a new file, name it blah.ahk and run it. Put it in your startup folder to have it executed automatically. AutoHotkey includes a compiler that you can use to make standalone executables; very convenient if you use the keyboard on computers on which you don't want to – or can't – install AutoHotkey. Here's mine.

Cleaning a Happy Hacking keyboard

2010-07-29T15:43:03+02:00

One day I spilt a glass of Coke on my Happy Hacking Keyboard Lite 2. It still worked, but the keys were all sticky (both above and underneath) and tended to get stuck. It was practically useless. So...

(NOTE: I have no idea whether this procedure is the best one, or even a good one, but it worked for me. Don't complain to me if something Bad happens if you try this yourself!)

Before

Underneath, after more than 2½ years of use with no previous cleaning

Close-up. Yummy!

Lots of soap

Shiny!

Drying for a couple of days

The keycaps are poured into a bowl...

...and cleaned with water and soap

Works like a charm again!

Chromium and Chrome in Ubuntu

2010-07-29T09:34:39+02:00

First: There's a difference between Chromium and Google Chrome. Chromium is the open source web browser that Chrome is based on. Google Chrome is basically Chromium + Google branding + a couple of extra features, e.g., integrated Adobe Flash player.

Installing Chrome

Simply download it from Google. Download the .deb and install it, either by double-clicking on it in Gnome and using GDebi (or whatever program you prefer), or in the console (replace file name with actual file name): sudo dpkg -i google-chrome-stable_current_amd64.deb

Installing Chromium

There are three different flavors of Chromium available: daily, beta and dev. As of this writing they have packages for Maverick (10.10), Lucid (10.04), Karmic (9.10), Jaunty (9.04) and Hardy (8.04). You need to add the repository for one of the three flavors:

Daily gives you the very latest version of the browser, but the builds are maintained automatically by a bot and are completely untested - so don't be surprised by crashes. sudo add-apt-repository ppa:chromium-daily/ppa
Beta is more stable and matched the Google Chrome Beta Channel. sudo add-apt-repository ppa:chromium-daily/beta
Dev is in between - less frequently updated than daily, more frequently updated than beta. sudo add-apt-repository ppa:chromium-daily/dev

Next, update the package list and install Chromium: sudo aptitude update sudo aptitude install chromium-browser

Done!

If you want to switch from daily to beta or dev, first remove the daily build: sudo aptitude remove chromium-browser Remove the repository: sudo rm /etc/apt/sources.list.d/chromium-daily-ppa-lucid.list ...and then follow the instructions above again.

Change icon size in Gnome

2010-07-29T02:35:30+02:00

To change the size of an individual icon on the Gnome desktop, right-click on it and select Stretch icon.

To change the default icon size in Gnome, open Nautilus (e.g., Places -> Home Folder or Alt-F2 and nautilus). Go to Edit -> Preferences and change Icon View Defaults and List View Defaults to whatever you like.

(I have tried this is Ubuntu 10.04 (Lucid Lynx.))

Move the Network Manager icon in Ubuntu (Lucid, Maverick, Natty)

2010-07-29T01:51:25+02:00

So you've installed Ubuntu 11.04 and started customizing your desktop. After moving around the icons in the upper corner (if you're using Gnome), you find that the Network Manager icon/applet won't move like the others. The others have options like Move and Lock to panel, but not the Network Manager one. What gives?

The trick is to right-click slightly to the left of the Network Manager icon. Voila...

(This also applies to 10.04 and 10.10.)

Disable notifications in Ubuntu 10.04 (Lucid) and 10.10 (Maverick)

2010-07-29T01:35:43+02:00

One thing that always annoyed me when starting a fresh install of any of the recent Ubuntu releases - 9.10 (Karmic), 10.04 (Lucid) and 10.10 (Maverick) - was the badly implemented notification thing that's supposed to look fancy: notify-osd. You know, the big black rectangular boxes saying "Connection established", giving you Pidgin notifications or whatever. Naturally, I wanted to get rid of this thing.

(If you want to disable just Pidgin notifications, go to Tool -> Plugins, scroll down to Libnotify Popups and select it, click Configure Plugin and uncheck all the boxes.)

Run the following to get rid of the notifications (one line):

$ sudo mv /usr/share/dbus-1/services/org.freedesktop.Notifications.service /usr/share/dbus-1/services/org.freedesktop.Notifications.service.disabled

This works in all recent Ubuntu releases. After a restart you'll be fine. If you want to get rid of notifications right away, run killall notify-osd.

Text boot and disabling GDM in Ubuntu 10.04 (Lucid), 10.10 (Maverick), 11.04 (Natty)

2010-07-29T01:18:56+02:00

The days of /etc/inittab seem to be gone in Ubuntu. But fear not - if you for some reason don't want graphical login (GDM, KDM, etc) when booting Ubuntu, here's how to do it. This also shows how to get rid of the splash image when booting, if you would rather have the good old black screen with the whole boot process shown.

This applies to Ubuntu 10.04 (Lucid), 10.10 (Maverick), 11.04 (Natty) and possibly some earlier versions. Some people will tell you to edit /etc/init/gdm.conf and change the "start on..." stuff to "start on runlevel []", but there's an easier way:

To disable GDM/KDM/etc, edit /etc/default/grub and change the line GRUB_CMDLINE_LINUX="" to GRUB_CMDLINE_LINUX="text"
To show boot messages and disable the splash image, remove quiet splash from GRUB_CMDLINE_LINUX_DEFAULT so that it reads GRUB_CMDLINE_LINUX_DEFAULT="". (If you want the splash image and some text, use splash.)
Run sudo update-grubDone!

This worked fine for me - but the sound was no longer working:

$ alsamixer
cannot open mixer: No such file or directory

I fixed this by adding my user to the group audio:

$ sudo usermod -a -G audio fooninja

(Replace fooninja with your username.)

I logged out and in again and voila - sound was working.

Oh, and if you want to start X automatically when logging in on the first console, put something like this in e.g. ~/.bashrc:

if [ `tty` = "/dev/tty1" ]; then
    startx
fi

fooninja.net

Drupal 7: Uncaught exception, MySQL server has gone away

fooninja.net goes mobile

Updating nodes programmatically in Drupal 7

Basic example

Deleting field contents

Deleting attached files

Revisions

From Wordpress to Jekyll and a new design

Getting started

Migrating posts from Wordpress

Disqus

Syntax highlighting

Tags

New design!

Failed requests: Apache benchmarking (ab) gotcha

Loading memtest from grub (Fedora)

Guide to programmatic node creation in Drupal 7

Basic node creation - example

Setting various field types

Text or integer field

Creation time

Date field (datetime, date, datestamp)

Boolean field

Term reference (taxonomy) field

Node references

Image field

How to fix wifi after suspend/resume in Fedora

Google Earth in 64-bit Linux (Fedora, Ubuntu)

screen: "No more PTYs" error

Swedish characters in Ubuntu (svenska tecken (åäö) i konsolen)

Drupal 7 and cron

Skype in Fedora 64-bit

rxvt-unicode with 256 colors in Fedora/RHEL/CentOS

Patch and rebuild urxvt

Fix terminfo

Protect urxvt from being updated

Speed up encryption in Linux

Get rid of ubuntuone-syncdaemon

Notes on Omeka

Problem uploading files with non-ASCII characters

Link to original image

Fixed: "PHP Warning: preg_match_all(): Compilation failed: unrecognized character"

Ubuntu: Apache/PHP/MySQL in 5 minutes

Some useful extras

Set up a local virtual host

PHP configuration

Get rid of wrong password delay in Linux

Disable delay in Ubuntu and Debian

Disable delay in CentOS, RHEL and Fedora

If brute-forcing scares you

Linux/BSD network install using PXE and Windows

Install tftpd32

Prepare the network

Configure tftpd32

Install Ubuntu

Install CentOS

Install OpenBSD

OpenBSD 4.8 on a Thinkpad X61s

Xubuntu 10.10 woes

Editing H.264 video in Linux

OpenShot

KDEnlive

Fixing sound issues

Transcoding

How to show Windows partitions in Thunar (Xfce)

Ubuntu 10.10 (Maverick) on a Thinkpad X61s

Stuff that works out of the box

Random notes

How to transfer files from the Canon 5D Mark II in Linux

Transfer files using the gphoto2 command-line program

Unix find usage examples

Patterns

Time

Size

Type

Permissions

Ownership

Search depth

Performing actions with find