Frequently, within these dumps, I’m face with a buffer that wraps. That is to say, each entry in the buffer is filled and, once the buffer is full, the next entry to be filled is the first, again (overwriting the old value).

At the point in time that the dump is taken, there is an index value that points to the ‘start’ of this buffer (i.e. the point in the buffer that is the oldest). Unfortunately, when Perl gets hold of the buffer, it represents it as an array, with the first entry in the array being the buffer entry with the lowest memory address.

So… given an array representing a buffer and an index to the logical start of the buffer, what’s the simplest way to rejig it, so that the array represents the logical order of the buffer instead of the physical order?

Voila:

splice @buffer, 0, 0, (splice @buffer, $start_index);

This raises the question, when you decide to move an object to a new position in the sort order, what is the simplest way to update the other objects to ensure that you maintain consecutive sort positions.

Imagine the following SQL table:

CREATE TABLE "projects" (
    "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    "name" VARCHAR(255),
    "sort_order" INTEGER UNSIGNED
);

The ’sort_order’ field contains a simple integer that indicates the position that the object should appear in. To get the ‘projects’ in the desired order, you’d execute:

SELECT * FROM projects ORDER BY sort_order DESC

To move an object from one position to another is a fairly simple operation. Each object has a unique (and immutable) “id”.  An appropriate method signature would be something like this:

def change_sort_location(object_id, new_location)

The following steps should be followed:

1. Identify the current location of the objects as old_location
2. If old_location < new_location
   1. Subtract 1 from sort_order for all objects where old_location < sort_order ≤ new_location
3. If new_location < old_location
   1. Add 1 to sort_order for all objects where new_location < sort_order ≤ old_location
4. Set sort_order for the provided id to new_location

The SQL for this is pretty straightforward:

UPDATE projects SET sort_order = sort_order-1
  WHERE sort_order > @old_location AND sort_order <= @new_location

or

UPDATE projects SET sort_order = sort_order+1
  WHERE sort_order > @new_location AND sort_order <= @old_location

followed by

UPDATE projects SET sort_order = @new_location
  WHERE id = @object_id

Clearly, in a full implementation, you’d probably not be using SQL variables, but the point stands.

One complaint might be that the object that is getting moved can get updated twice here, but that’s unlikely to be a major performance impact.

Currently, once object move requires 2 SQL queries. N object moves will require 2N SQL queries. I’m currently trying to figure out a method to reduce the number of SQL queries needed for multiple moves.

We all know, by now, that Twitter limits its tweets to 140 characters. We’ve all got pretty good at limiting ourselves to 140 characters, but many overlook a hidden limit. This post outlines what that is and how we can avoid it.

Many users of Twitters are hoping that their followers will retweet (RT) their tweets. Twitter recently made a change to how these work, but in general, the following pattern is followed:

   UserXYZ tweets: Hey... here's something that's fascinating
   UserABC tweets: RT @UserXYZ: Hey... here's something that's fascinating

User XYZ’s tweet was 42 characters. UserABC’s RT was 52 characters, i.e. 10 characters were added in order to RT.

Put another way, if UserXYZ creates a tweet that was longer than 130 characters, nobody would be able to RT it with modifying the original tweet. If you’re trying to get a specific message out to the world, you might not be happy with lots of people fiddling with it.

I’ve created a new Greasemonkey script which will help you with this. I’ve written about Greasemonkey plugins before and this is another Twitter helper. If you install the script, you will see the following change:

You can now see, next to the normal character countdown, a bracketed countdown. This is the number of characters that you have left, before a tweet can no longer be RTed without modification. In this example, you would be able to send the tweet (as you have 6 characters left), but Twitter users would have to remove 8 characters before they could RT your Tweet.

To use this, it’s simple:

1. If you haven’t already, install Greasemonkey
2. Install the ReTweetable Alert script

That’s it! As ever, your questions and comments are most welcome

• Google Analytics will not be able to track these external links (if, for instance, you’re using my Google Analytics for external links)
• Non-visual User Agents will not be able to access these links
• As a subset, Google will not be able to crawl these links and so associate your site with the sites those banners point to.

The way to resolve this is to use the method of graceful degradation or, perhaps more appropriately, progressive enhancement. By this, I mean, the HTML should be written such that the above issues are avoided, and then the JavaScript should be written to provide the Rotating Banners functionality.

Using the Rotating Banners script as a starting point, the obvious approach is to dispense with the JSON representation of our banners and replace it with an HTML representation of our banners. This will be hidden using CSS. Much of the functionality will otherwise remain the same. By representing the links in the HTML document, webcrawlers (such as Google’s indexing service) will be able to parse these links and use them accordingly. In addition, Google Analytics (if installed on your site) can track exits from your site, via these banners.

I have created a new object which extends the original Rotating Banners object, such that it doesn’t cause SEO problems. In order to use this object, you need to include the original object in your HTML, as well as this one.

The new object is shown below:

/**
* SEO Safe Rotating Banner
*
* Specialized version of Rotating Banner
* @author dancrumb
*/
 
/*global rotatingBanner,$ */ 
 
var seoSafeRotatingBanner = function (spec, secrets) {
 
	var prvItems = secrets || {};
 
	var that = rotatingBanner(spec, prvItems);
 
	spec.banners = [];
 
	prvItems.rotate = function () {
		// This should only be executed if initialization is complete
		if (!prvItems.intialized)	{
			return null;
		}
 
		var newBanner = prvItems.nextBanner();
 
		$('#' + spec.divId + ' a').css('display', 'none');
		$('#' + newBanner.id).empty();
		var imgBanner = $(document.createElement('img')).
			attr('src', newBanner.src).
			attr('height', spec.height).
			attr('width', spec.width);
 
		$('#' + newBanner.id).append(imgBanner);
		$('#' + newBanner.id).css('display', 'inline');
 
		prvItems.cacheNextImage();
 
		return newBanner;
	};
 
	prvItems.initialize = function () {
		var bannerDiv;
		bannerDiv = $('#' + spec.divId);
		if (bannerDiv.length === 0)	{
			alert("Please provide the ID of an element on the page");
			return null;
		}
 
		$('div#' + spec.divId + ' a').each(function (i) {
			var aElem = $(this);
			// Ensure that there is an ID defined, for future manipulation
 
			var a_id = aElem.attr('id');
			if (a_id === null) {
				a_id = 'rb_auto_id_' + i;
				aElem.attr('id', a_id);
			}
 
			// Ensure that there is a target defined, and set it to the defaultTarget
			// if not
			var tgt = aElem.attr('target');
			if (!tgt) {
				tgt = spec.defaultTarget;
				aElem.attr('target', tgt);
			}
 
			// Ensure that the banner is hidden
			aElem.css('display', 'none');
 
			// Add the banner to the rotater
			that.addBanner(
				{
					"src": aElem.text(),
					"id": a_id
				}
			);
		});
 
		that.randomizeBanners();
		prvItems.intialized = true;
		prvItems.rotate();
                $('div#' + spec.divId).css('display', '');
	};
 
	return that;
 
};

This file can be downloaded here.
All that is necessary is to override the ‘initialize’ and ‘rotate’ methods.

Using this object on your page is just as straightforward as before. As before, you need to include the jQuery library. You also need to include the original rotatingBanners script, as well as this one. The initialization code is slightly different:

var rB = seoSafeRotatingBanner(
  {
    "width":  600,
    "height": 400,
    "interval": 5000,
    "divId": "bannerHere",
    "defaultTarget": "blank"
   }
);
rB.startRotation();

The final ingredient is the list of links itself. This is provided in HTML:

<div id="bannerHere" style="display:none;">
  <a id="banner1" href="link1">/image_location/img_src_1.jpg</a>
  <a id="banner2" href="link1" target="blank">../img_src_2.png</a>
  <a id="banner3" href="link1" target="some_target">img_src3.gif</a></div>

Note that the URLs to the images are just plain text. They are not IMG tags.

You must ensure that the style of the enclosing DIV contains ‘display:none‘ (per line 1) or your list of links will be visible to the user.

At this point, you will have your rotating banners, without sacrificing your SEO performance.

The main problem with the script is that it was too random. It took an array, or list, of banners and, on each rotation, it would select a random position in this list. The problem with this becomes obvious when you look at some examples of random number selections. The output from the following (Perl) script:

my $idx = 20;
while ($idx)
{
print int(rand()*5);
$idx--;
print "," if $idx;
}

is

0,1,2,0,3,0,3,3,3,3,0,3,0,1,1,1,0,4,4,2,4

If each number represents a specific banner, you can see that there is a significant bias toward 3 in this set of 20 rotations. This is not indicating an inherent bias towards 3. Quite the opposite; this is truly random, but human perception detects a bias because our sample is relatively low.

Leaving the statistics behind, it is quite easy to ensure that the order of banner rotation is random, but every banner gets equal time. All you have to do is to randomize the order of your list of banners when the page loads, and then cycle through them. On each page load, the order is random, but once a page is loaded, the order remains static.

With that in mind, I wrote a JavaScript object to provide the necessary function. I’ve been writing JavaScript for some time, but I recently bought ‘JavaScript: The Good Parts’ by Douglas Crockford. Crockford has been writing JavaScript for a long time now and has some strong opinions on the language. I for one applaud this book, although I’m not wholly sold on some of his beliefs. That said, I decided to take a crack at creating an object based on his writings. I’m not overly convinced that the code I’ve written is superior to anything else I would have written, but I’m willing to take the blame for any failings in my code here:

/**
 * @author dancrumb
 */
 
/*global $ */
 
var rotatingBanner = function (spec, secrets)
{
 // Private
 var that = {},
 prvItems = secrets || {};
 
 prvItems.workingList = [];
 prvItems.cached = {};
 prvItems.timeoutId = null;
 prvItems.intialized = false;
 
 prvItems.nextBanner = function () {
 var banner = prvItems.workingList.shift();
 prvItems.workingList.push(banner);
 return banner;
 };
 
 prvItems.rotate = function () {
 // This should only be executed if initialization is complete
 if (!prvItems.intialized) return null;
 
 var newBanner = prvItems.nextBanner();
 
 $('#'+spec.divId+' a img').attr('src',newBanner.src);
 $('#'+spec.divId+' a').attr('href',newBanner.href || '');
 $('#'+spec.divId+' a').attr('target',newBanner.target || spec.defaultTarget || '');
 
 prvItems.cacheNextImage();
 
 return newBanner;
 };
 
 prvItems.initialize = function() {
 var bannerDiv,
 eImg,
 eA;
 bannerDiv = $('#'+spec.divId);
 if(bannerDiv.length == 0)
 {
 alert("Please provide the ID of an element on the page");
 return null;
 }
 
 bannerDiv.empty();
 eImg = $(document.createElement("img")).
 attr('height',spec.height).
 attr('width',spec.width).
 attr('id','bannerImage');
 
 eA = $(document.createElement('a')).
 attr('href','');
 
 eA.append(eImg);
 bannerDiv.append(eA);                
 
 that.randomizeBanners();
 prvItems.intialized = true;
 prvItems.rotate();
 };
 
 /*
 * Place the next Banner Image into the browser cache
 */
 prvItems.cacheNextImage = function () {
 var upcomingBanner = prvItems.workingList[0],
 cacheImage;
 if (!prvItems.cached[upcomingBanner.src])
 {
 cacheImage = new Image();
 cacheImage.src = upcomingBanner.src;
 prvItems.cached[upcomingBanner.src] = true;
 }
 };
 
 /*
 * Ensure we, at least, have an array of banners
 */
 if(!(
 spec.banners &&
 typeof spec.banners === 'object' &&
 typeof spec.banners.length === 'number' &&
 typeof spec.banners.splice === 'function' &&
 !(spec.banners.propertyIsEnumerable('length'))
 ))
 {
 spec.banners = [];
 }
 
 /*
 * Confirm that the interval is sensibly set
 */
 if (!((typeof spec.interval === 'number') &&
 (isFinite(spec.interval)) &&
 (spec.interval >= 500))) {
 spec.interval = 1000;
 }
 
 // Public
 that.randomizeBanners = function () {
 // We shouldn't fiddle with knownBanners directly as this
 // is our master list of banners
 var holdingCopy = spec.banners.slice(0),
 index;
 
 if (prvItems.timeoutId)
 {
 clearTimeout(prvItems.timeoutId);
 }
 prvItems.workingList = [];
 
 // Randomize the order of the holding copy of known banners and put it
 // into the working list
 while (holdingCopy.length > 0)
 {
 index = parseInt((Math.random()*holdingCopy.length),10);
 prvItems.workingList.push(holdingCopy[index]);
 holdingCopy.splice(index,1);
 }        
 
 // If we cancelled the rotation, we restart it here
 if (prvItems.timeoutId)
 {
 that.startRotation();
 }
 };
 
 that.startRotation = function () {
 prvItems.timeoutId = setInterval(function(){
 prvItems.rotate();
 },spec.interval);
 };
 
 that.addBanner = function (banner) {
 spec.banners.push(banner);
 this.randomizeBanners();
 };
 
 that.setInterval = function (interval) {
 if (prvItems.timeoutId)
 {
 clearTimeout(prvItems.timeoutId);
 }
 spec.interval = interval;
 if (prvItems.timeoutId)
 {
 that.startRotation()
 }
 }
 
 $(function(){ prvItems.initialize();});
 return that;
};

You can download this script here

The code above relies on jQuery, so naturally, you would need to include that library first in any HTML page that uses this object.

Invoking this code is straightforward. Your webpage will need a DIV with a defined ID attribute; in this case we use bannerHere.

var rB = rotatingBanner(
 {
 "width":  600,
 "height": 400,
 "banners": [
 {"src":'fake1', "href":'fake1'},
 {"src":'fake2', "href":'fake2'}
 ],
 "interval": 2000,
 "divId": "bannerHere",
 "defaultTarget": "blank"
 }
 );
 rB.startRotation();

width and height are the desired dimensions for the banners
interval is the time (in ms) between each rotation
defaultTarget is the desired value for the TARGET attribute of the banner link
banners is a JavaScript array of JavaScript objects with the following fields:

src is the SRC of the banner image
href is the HREF of the banner link
target is the optional target for the banner link

The code is based on Crockford’s belief that we should not pretend the JavaScript is a class-based language. It isn’t, it’s a prototype-based language. This means that new objects are created from a ‘template’ object or from a factory method, which generates new object. Again, I’ll repeat that my implementation is quite probably not the best example of this and I’m not sure that I prefer this code to a class-based approach, but it works and it’ll do for now.

There are some SEO concerns with this implementation, but I’m going to save them for an upcoming post. For now, the code about should be enough for you to implement rotating banners on your website

I’ve started learning Ruby on Rails and the Ruby part uses things called ‘Gems’ in a similar way to Perl using Modules. Where you read ‘gem’, thing ‘cpan’.

I wanted to install a new gem on my server to support some Paypal integration, but ‘gem’ kept segfaulting on me.

If you’re like me, you’ll see this behaviour:

root@server [~]# gem install rubygems-update
Bulk updating Gem source index for: http://gems.rubyforge.org
Terminated

It turns out that ‘Bulk updating…’ part gobbles up memory like it’s going spare and leads to a segfault.

The way to avoid this problem is to update RubyGems… but to do that, you need to use ‘gem’… and that leads to a segfault… and around we go again.

To break the cycle, you simply use:

gem update --system --no-update-sources

to prevent the updating of sources.

Once I’d done that, I found that I was still getting segmentation faults. Also, when I ran:

root@server [~]# gem install activemerchant --no-update-sources

I now got

ERROR:  could not find activemerchant locally or in a repository

So the problem still wasn’t resolved!

The only solution was to force my system to the latest version of RubyGems. Unfortunately, this was not in my local repository. However, a manual update was pretty simple:

cd /tmp
wget http://rubyforge.org/frs/download.php/60718/rubygems-1.3.5.tgz
tar -xvzf rubygems-1.3.5.tgz
cd rubygems-1.3
ruby setup.rb

Job done!

I’ve been coding in Perl for years and, generally, I think I’m a pretty competent coder. However, every now and again, I fall into a hole that takes me a loooong time to get out of. Invariably, these problems boil down to upload issues, not code issues. As an aide-mémoire for myself and, hopefully, to you, I thought I’d jot down a few problems that I’ve seen and what the solution was.

Incorrect shebang

The ‘shebang’ is the first line of code in a Perl script. It consists of a hash character ‘#’, followed by an exclamation mark ‘!’, also know as a ‘bang’ (hence the name). Then, the full directory of the Perl binary should be given. As often as not, this will be:

#!/usr/bin/perl

or

#!/usr/local/bin/perl

This is actually a generic line and is not unique to Perl. It’s a Unix device that tells the operating system that the program that is capable of interpreting the following script can be found here.

Now… if you get this wrong, it won’t work. The problem is, if you’re developing on one box with Perl in one location and you upload your scripts to another box, where Perl is located elsewhere, you’ll have the phenomenon of your script working perfectly on your development box, but won’t work on your production box. In addition, if you try to execute the script from the command line of your production box by invoking Perl directly and then provide the script name, that will also work, because the shebang is ignored in this case. It’s not until you try to execute your script via an HTTP request that it all falls apart. You will see the following in your error log:

[Fri Jul 03 16:57:14 2009] [error] [client xxx.xxx.xxx.xxx] suexec failure: could not open log file
[Fri Jul 03 16:57:14 2009] [error] [client xxx.xxx.xxx.xxx] fopen: Permission denied
[Fri Jul 03 16:57:14 2009] [error] [client xxx.xxx.xxx.xxx] Premature end of script headers: script.pl

This should be your indicator to check the shebang in the scripts on your production server and ensure that the point to the correct location for the Perl binary.

Incorrect file formats

Sometimes, your Perl script works fine on your Windows development box, but when you transfer it to your Unix production box, it stops working. As above, it works fine when you run it directly from the command line and you invoke ‘perl’ directly, but an HTTP request results in a 404 error, even though the file is actually in place.

The problem here is that DOS and Unix use different characters to indicate the end of a line and, if you have DOS endings in a Unix environment, the O/S is not able to parse the shebang correctly and so cannot locate the Perl binary. The way to fix this is to use the ‘dos2unix’ command. This will convery a DOS encoded file into a Unix encoded file and everything will be peachy.

This normally occurs if you’re using an FTP client that is uploading Perl files in BINary mode instead of ASCII mode. It also occurs if you are transferring files via SCP, since SCP does not convert files as it copies them.

As an owner of a brand new Palm Pre, I recently got exposed to the vagaries of trying to import my Facebook Calendar into my Pre. It’s not as easy as it should be and the fault lies with Facebook and with Palm. While we’re waiting to sort things out, I’ve written a little application, along with the post, to help myself and others out of this pickle

The start of the problem is with Facebook. When you opt to export your Events, Facebook provides you with a URL pointing to an iCalendar file. This contains all the details of your upcoming events. However, what it does not contain (and this is critical) is any timezone information. When you access Facebook normally, it knows what timezone you are in from you computer’s clock and shows events at the according time. This is known as using a ‘floating’ timezone and makes things a lot easier on the FB backend.

However, when you import this iCalendar file into another calendar application, it has no idea what timezone the file represents. Some applications will assume it is your local timezone. Others (especially web applications) may assume that the timezone is GMT. Unless you live in the UK, or other GMT countries, you’re going to see an offset for all of your events.

How does this play into the Pre? Well, I live in Austin, which is in the Central timezone. When I imported my Facebook events, they were off by two hours. After some investigation, I came across this theory, that I find the most compelling: My Pre is synchronised with my FB events via a server at Palm, in the Pacific timezone. Facebook ‘floats’ my events to match the PST timezone. As a result, when they appear in my Pre, they are off by two hours.

So, I though to myself, why not import my FB Events into my Google calendar and use that as my primary calendar. Well, Google has other ideas. It sees an iCalendar file without a time zone, assumes GMT and so shifts all of my events by 6 hours.

As a result, I’ve made this app. Just provide the URL to your Facebook Events Export, set your timezone and give the new calendar a name. A new URL will be generated, which you can then pass to Google Calendars. Within an hour or so, Google will import your events and keep them updated.

You can then sync your Pre (or other PDAs) with your Google Calendar and, hey presto, there are your events.

This process doesn’t add any functionality to the calendar; it just sets to timezone correctly. Any delays in synching will come from the delay of Google synching with Facebook and your Pre synching with Google. My filter adds no significant time overhead at all.

This functionality is provided to you all free of charge. All I ask is that you add comments and suggestions. I’m happy to tweak this and respond to error reports.

Back in February, Whurley, Gio and I heard about a new product called the Palm Pre.
I was excited at the promise of a new, open mobile development platform and we decided to recreate the success of iPhoneDevCamp by creating preDevCamp.

I never expected Palm to provide assistance, but I hoped they would. Apple was flooding the market with advertisements, not for the iPhone, but for the iPhone App Store and the abundant apps.
Clearly, a thriving supply of mobile applications was the way to sell a new mobile device.

I’d developed apps for Palm OS in the past and I knew that there was a devoted community of developers out there; however, they were rather neglected developers, since Palm hadn’t really been a major player in the mobile arena for a while. With the advent of the Pre, I thought things were changing.

Time went by and there were fleeting moments of contact with Palm. We spoke to them; they seemed interested but asked us to put a disclaimer that we were not affiliated with them, before they would enter into a relationship with them. This seemed a little backwards to me, but we complied. Not much transpired after that.

Then Mitch Allen gave his web presentation on developing WebOS apps and gave us a shout out. I was really excited about this; the CTO of Software was aware of what we were doing, but there wasn’t any follow up from Palm following that.

Finally, last week, Palm sent us some NDAs in preparation for a meeting this Wednesday. We signed them and prepared ourselves for an interesting update. Gio sent out a tweet simply stating that we had a meeting and it was under an NDA, as a result Palm then cancelled the meeting and cancelled any discussions covered by an NDA. At that point, my hopes for a useful relationship with Palm died.

As a corporation, I acknowledge that Palm’s only responsibility is to its shareholders. There’s nothing self serving or evil about that; it’s how things work in big business. However there are many keen and willing developers out there, who have been waiting for the arrival of WebOS. A development platform is only a success if it is broadly adopted. Instead of embracing the grassroots upswell of interest in WebOS that preDevCamp fostered, Palm seem to be, at best, oblivious and, at worst, disdainful of the enthusiasm and good will engendered by these folk. I think they are missing a real opportunity to be involved in and to help generate the growth of a vital community.

My fellow preDevCamp founders and I may have differing views on the impact of Palm’s interactions with us. Personally, I’m left disappointed at what I view as a lack of foresight on Palm’s behalf. Palm will live or die by the success of the WebOS platform. The preDevCamp community will be a large part of this. However, my excitement remains about the WebOS platform. I couldn’t really give two hoots about Palm at this point. I *do* want preDevCamp to be a success and I *know* it will be; we have dedicated organizers all the way across the globe. We have a release date, at last. We have a date for preDevCamp. It’s all systems go. I encourage you to stay focused on the product and on the exciting possibilities that WebOS brings. My only hope, now, is that Palm runs the course with their indifference to community. If they don’t want to help us, that’s fine. I just hope they don’t try and get in our way.

Twitter recently changed the ids and classes that they use for the DIVs their pages, so I had to make an update to the Twitter WhoAmi script.

Still find it useful as I now have at least 4 Twitter identities (one personal and three project IDs).