All Geekness Great and Small

Archive for the ‘Programming’ Category

I’m currently working on some Perl that processes memory dumps from a hardware appliance.

Frequently, within these dumps, I’m face with a buffer that wraps. That is to say, each entry in the buffer is filled and, once the buffer is full, the next entry to be filled is the first, again (overwriting the old value).

At the point in time that the dump is taken, there is an index value that points to the ‘start’ of this buffer (i.e. the point in the buffer that is the oldest). Unfortunately, when Perl gets hold of the buffer, it represents it as an array, with the first entry in the array being the buffer entry with the lowest memory address.

So… given an array representing a buffer and an index to the logical start of the buffer, what’s the simplest way to rejig it, so that the array represents the logical order of the buffer instead of the physical order?

Voila:

splice @buffer, 0, 0, (splice @buffer, $start_index);

There are times when your web application needs to retain an arbitrary sort order for object. For example, you may have a slideshow of photographs and you want to be able to arrange them in any order you like. The simplest way to do this is to assign an attribute to each object that explicitly represents its sort position.

This raises the question, when you decide to move an object to a new position in the sort order, what is the simplest way to update the other objects to ensure that you maintain consecutive sort positions.

Imagine the following SQL table:

CREATE TABLE "projects" (
    "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    "name" VARCHAR(255),
    "sort_order" INTEGER UNSIGNED
);

The ’sort_order’ field contains a simple integer that indicates the position that the object should appear in. To get the ‘projects’ in the desired order, you’d execute:

SELECT * FROM projects ORDER BY sort_order DESC

To move an object from one position to another is a fairly simple operation. Each object has a unique (and immutable) “id”.  An appropriate method signature would be something like this:

def change_sort_location(object_id, new_location)

The following steps should be followed:

1. Identify the current location of the objects as old_location
2. If old_location < new_location
   1. Subtract 1 from sort_order for all objects where old_location < sort_order ≤ new_location
3. If new_location < old_location
   1. Add 1 to sort_order for all objects where new_location < sort_order ≤ old_location
4. Set sort_order for the provided id to new_location

The SQL for this is pretty straightforward:

UPDATE projects SET sort_order = sort_order-1
  WHERE sort_order > @old_location AND sort_order <= @new_location

or

UPDATE projects SET sort_order = sort_order+1
  WHERE sort_order > @new_location AND sort_order <= @old_location

followed by

UPDATE projects SET sort_order = @new_location
  WHERE id = @object_id

Clearly, in a full implementation, you’d probably not be using SQL variables, but the point stands.

One complaint might be that the object that is getting moved can get updated twice here, but that’s unlikely to be a major performance impact.

Currently, once object move requires 2 SQL queries. N object moves will require 2N SQL queries. I’m currently trying to figure out a method to reduce the number of SQL queries needed for multiple moves.

We all know, by now, that Twitter limits its tweets to 140 characters. We’ve all got pretty good at limiting ourselves to 140 characters, but many overlook a hidden limit. This post outlines what that is and how we can avoid it.

Many users of Twitters are hoping that their followers will retweet (RT) their tweets. Twitter recently made a change to how these work, but in general, the following pattern is followed:

   UserXYZ tweets: Hey... here's something that's fascinating
   UserABC tweets: RT @UserXYZ: Hey... here's something that's fascinating

User XYZ’s tweet was 42 characters. UserABC’s RT was 52 characters, i.e. 10 characters were added in order to RT.

Put another way, if UserXYZ creates a tweet that was longer than 130 characters, nobody would be able to RT it with modifying the original tweet. If you’re trying to get a specific message out to the world, you might not be happy with lots of people fiddling with it.

I’ve created a new Greasemonkey script which will help you with this. I’ve written about Greasemonkey plugins before and this is another Twitter helper. If you install the script, you will see the following change:

You can now see, next to the normal character countdown, a bracketed countdown. This is the number of characters that you have left, before a tweet can no longer be RTed without modification. In this example, you would be able to send the tweet (as you have 6 characters left), but Twitter users would have to remove 8 characters before they could RT your Tweet.

To use this, it’s simple:

1. If you haven’t already, install Greasemonkey
2. Install the ReTweetable Alert script

That’s it! As ever, your questions and comments are most welcome

I recently wrote about a script designed to generate Rotating Banners. The script works fine, but using JavaScript to present a user with links creates a few problems of its own:

• Google Analytics will not be able to track these external links (if, for instance, you’re using my Google Analytics for external links)
• Non-visual User Agents will not be able to access these links
• As a subset, Google will not be able to crawl these links and so associate your site with the sites those banners point to.

Read the rest of this entry »

I was recently sent a script designed to take a series of advertising banners and rotate them on a page. By ‘rotate’, I mean display on banner in a designated position and then, after a certain period of time, replace it with another, and then another, and so on. To be fair to those who paid for the banners, each banner was chosen at random so that each new visitor to the site would see a different banner first, second, third, etc. After taking a look at it, I spotted some problems and decided to fix them.
Read the rest of this entry »

All of my websites are currently running on a VPS server provided by HostIcan. I recently discovered a little quirk involving VPS servers and RubyGems.

I’ve started learning Ruby on Rails and the Ruby part uses things called ‘Gems’ in a similar way to Perl using Modules. Where you read ‘gem’, thing ‘cpan’.

I wanted to install a new gem on my server to support some Paypal integration, but ‘gem’ kept segfaulting on me.

If you’re like me, you’ll see this behaviour:

root@server [~]# gem install rubygems-update
Bulk updating Gem source index for: http://gems.rubyforge.org
Terminated

It turns out that ‘Bulk updating…’ part gobbles up memory like it’s going spare and leads to a segfault.

The way to avoid this problem is to update RubyGems… but to do that, you need to use ‘gem’… and that leads to a segfault… and around we go again.

To break the cycle, you simply use:

gem update --system --no-update-sources

to prevent the updating of sources.

Once I’d done that, I found that I was still getting segmentation faults. Also, when I ran:

root@server [~]# gem install activemerchant --no-update-sources

I now got

ERROR:  could not find activemerchant locally or in a repository

So the problem still wasn’t resolved!

The only solution was to force my system to the latest version of RubyGems. Unfortunately, this was not in my local repository. However, a manual update was pretty simple:

cd /tmp
wget http://rubyforge.org/frs/download.php/60718/rubygems-1.3.5.tgz
tar -xvzf rubygems-1.3.5.tgz
cd rubygems-1.3
ruby setup.rb

Job done!

I’ve been coding in Perl for years and, generally, I think I’m a pretty competent coder. However, every now and again, I fall into a hole that takes me a loooong time to get out of. Invariably, these problems boil down to upload issues, not code issues. As an aide-mémoire for myself and, hopefully, to you, I thought I’d jot down a few problems that I’ve seen and what the solution was.
Read the rest of this entry »

As an owner of a brand new Palm Pre, I recently got exposed to the vagaries of trying to import my Facebook Calendar into my Pre. It’s not as easy as it should be and the fault lies with Facebook and with Palm. While we’re waiting to sort things out, I’ve written a little application, along with the post, to help myself and others out of this pickle

Read the rest of this entry »

Back in February, Whurley, Gio and I heard about a new product called the Palm Pre.
I was excited at the promise of a new, open mobile development platform and we decided to recreate the success of iPhoneDevCamp by creating preDevCamp.

I never expected Palm to provide assistance, but I hoped they would. Apple was flooding the market with advertisements, not for the iPhone, but for the iPhone App Store and the abundant apps.
Clearly, a thriving supply of mobile applications was the way to sell a new mobile device.

I’d developed apps for Palm OS in the past and I knew that there was a devoted community of developers out there; however, they were rather neglected developers, since Palm hadn’t really been a major player in the mobile arena for a while. With the advent of the Pre, I thought things were changing.

Time went by and there were fleeting moments of contact with Palm. We spoke to them; they seemed interested but asked us to put a disclaimer that we were not affiliated with them, before they would enter into a relationship with them. This seemed a little backwards to me, but we complied. Not much transpired after that.

Then Mitch Allen gave his web presentation on developing WebOS apps and gave us a shout out. I was really excited about this; the CTO of Software was aware of what we were doing, but there wasn’t any follow up from Palm following that.

Finally, last week, Palm sent us some NDAs in preparation for a meeting this Wednesday. We signed them and prepared ourselves for an interesting update. Gio sent out a tweet simply stating that we had a meeting and it was under an NDA, as a result Palm then cancelled the meeting and cancelled any discussions covered by an NDA. At that point, my hopes for a useful relationship with Palm died.

As a corporation, I acknowledge that Palm’s only responsibility is to its shareholders. There’s nothing self serving or evil about that; it’s how things work in big business. However there are many keen and willing developers out there, who have been waiting for the arrival of WebOS. A development platform is only a success if it is broadly adopted. Instead of embracing the grassroots upswell of interest in WebOS that preDevCamp fostered, Palm seem to be, at best, oblivious and, at worst, disdainful of the enthusiasm and good will engendered by these folk. I think they are missing a real opportunity to be involved in and to help generate the growth of a vital community.

My fellow preDevCamp founders and I may have differing views on the impact of Palm’s interactions with us. Personally, I’m left disappointed at what I view as a lack of foresight on Palm’s behalf. Palm will live or die by the success of the WebOS platform. The preDevCamp community will be a large part of this. However, my excitement remains about the WebOS platform. I couldn’t really give two hoots about Palm at this point. I *do* want preDevCamp to be a success and I *know* it will be; we have dedicated organizers all the way across the globe. We have a release date, at last. We have a date for preDevCamp. It’s all systems go. I encourage you to stay focused on the product and on the exciting possibilities that WebOS brings. My only hope, now, is that Palm runs the course with their indifference to community. If they don’t want to help us, that’s fine. I just hope they don’t try and get in our way.

Twitter recently changed the ids and classes that they use for the DIVs their pages, so I had to make an update to the Twitter WhoAmi script.

Still find it useful as I now have at least 4 Twitter identities (one personal and three project IDs).