Steve's Ramblings

Its been quite some time since I started work developing Wordbooker after I forked it off Robert Tsai’s Wordbook plug-in.

It became obvious about 6 months ago that the current code base (version 1.x) was far from ideal and it was starting to confuse me, and when the only coder starts getting lost in his own code you really have to start getting worried.

So I decided that I’d start version 2. I’d take the basic process flow from 1.0 and I’d also re-use code where it made sense but I’d clean things up and change how I modularised the code. I also set up a Google Code page for the plug-in, not for sharing the code on but really to use it as a bug tracker where people could also request new features.

Wordbooker 1.x has two major failings :

• Comment Handling
• Resilience to Facebook failures

Plus several other less important issues like not being WPMU/WP Networked Blogs friendly (in that it creates multiple tables per blog) and of course forgetting everything when people de-activate it for WP upgrades or plug-in upgrades.

Comment handling was something that I sort of bolted on and its never really worked properly, and due to the fact that Facebook have imposed a maximum number of feed posts an application can do in a 24 hour window its not something I’ve ever really taken much interest in, but its something that a lot of the users of the application want and it didn’t work with pages so that was top of the “user wants” for version 2.

Being more resilient to Facebook failures is very important – these range from sessions expiring if the user changes their Facebook password, through users hitting the dreaded feed post limit, to the complete breakdown of the Facebook platform : something which is actually all too common.

I’d worked out in my head how I was going to do the first of the two and sitting in the car on the outskirts of Stroud armed with a pad of paper and a pen had given me time to actually turn those vague thoughts into something more concrete.

The second big issue was something that had got me really stumped – the Facebook API is far from stable, and far from reliable, and sometimes it is so slow everything just times out. I’d done some work in the status update code to mitigate against Facebook failing but I’d not really got any clear idea on how to extend the process out to the whole of the Wordbooker plug-in, but it had become obvious that code that expected Facebook to be there, and that had no fall back if it wasn’t, wasn’t really the way forward.

It was a chance request by someone who does a lot of posts at once but only wants them to “trickle” out to Facebook that got me thinking and then I realised that the way I was planning to handle comments could actually be extended to how I’d handle posts.

So I guess I need to explain how comment handling works in Version 1 and how its going to work in Version 2

In V1 comments are handled in two different ways: Comments coming from WordPress are pushed to Facebook immediately with no moderation (even if you’ve got WordPress set to moderate comments). Comments coming to WordPress are handled by a WP-Cron based job which runs once an hour – this basically goes through each of the WordPress posts that Wordbooker “knows” have been published to Facebook, finds the time stamp of the last recovered Facebook comment, then fetches (using Facebook’s own query language) any new comments from Facebook and posts them to the appropriate blog post. Of course there is a problem here in that Wordbooker has to be able to recognise if its pulling comments down from Facebook that its actually posted up in the first place.

Pushing comments from the blog to Facebook counts against the feed action limit so if you’ve done a blog post you are then limited to 25 other post transactions in the next 24 hours – so if you get 26 comments on your blog then that last comment will fail because the post limit has been reached.

So in version 2 when comments are posted on your blog they aren’t posted to Facebook. All Wordbooker does it make a record that a comment has been posted. Then in the background it will check if the comment has been approved and if it has it will push it up to Facebook. If the comment has been marked as spam or deleted then Wordbooker will remove it from its list of comments to be processed. When a comment is posted to Facebook it will record the unique Facebook ID for that comment

Comments coming from Facebook will be checked by Wordbooker to see if the FB ID exists in its comment table (i.e. it’s a comment that Wordbooker created) and if it is then it will quietly ignore it.

Using a process queue like this means that I can handle FB problems easily. If you max out your feed limit or FB isn’t responding properly. then Wordbooker will simply sit there in the background attempting to post and if it can’t then the comments will simply be processed at a later date.

So that’s how comments are going to be handled, well roughly how I plan to do it – no doubt I’ll refine it and change it when I really start working through it all. Now that I’ve explained comment handling I can now explain how post handling will be done.

At the moment when you publish a post to your blog Wordbooker posts it immediately to Facebook – so if you’ve made a glaring mistake in the first part of your post its now sitting there on your FB wall for all your friends to see. The work round for this is to leave the Wordbooker Default Post option turned off and then once you’re happy with the post go in and edit it and force its publication on Facebook.

In version 2 you’ll have the following options when it comes to publishing a post:

• Publish Immediately
• Batch for automatic publication
• Leave on queue

The first option “Publish Immediately” will behave in exactly the same way as publishing does in version 1 – and will be there for those people who are totally confident that their post is correct, and also for posts that have been scheduled for publication at a future date.

“Batch for automatic publication” will flag that the post should be pushed to Facebook at a later date – I’m still working on how the user will control when posts get pushed but it will allow users to basically change the order posts are pushed to FB.

“Leave on Queue” will do just that – the post will not get pushed to Facebook at all and you can edit/change the post to you hearts content and publish it later.

Of course this means that the options page will have to feature a “Queue Manager” which will allow users to manipulate their queued posts – so they can force a post to be published, or add it to, or remove it from, the batch list, or change the order in which posts should be sent to Facebook.

So by basically queueing all posts to Facebook (yes – even immediate post requests will be put onto the queue) it means that if the push to Facebook fails for any reason then the post isn’t lost, and Wordbooker will simply retry at a later date.

There will also be a “Queue process history” (but not with such a clumsy name) where you’ll be able to see the outcome of recent post activities – so rather than just getting a general “feed transaction limit reached” message you’ll be able to see which post failed and what Wordbooker has done since then.

I know that I’ve been telling people that the Options page was getting too complicated and scary, and this is just going to make it much worse. Well I’ve got a solution…the Wordbooker Options page will be divided into several sections. I did look at making it several tabs but it didn’t feel right – so instead I’m using collapsible sections like the ones on the WordPress admin menus.

Right – I think I’ve probably bored you all enough by now so I’ll stop here.

I know its been pretty quiet round here for a bit. Its just been one of those periods of time when I’d nothing really to write about, and even if I did I didn’t really have the time, or the motivation, or the energy.

When I’ve not been working I’ve been bashing my head against the wall over my Wordbooker plugin which I’ve been playing a continual catch up game with Facebook and WordPress and W3C compliance (some of which was nothing to do with me but actually Facebook’s decision to use non compliant tags and people having WordPress themes with not quite the right Doctypes and other things). I’ve been wanting for ages to start on Version 2 of the plugin – which takes a different approach to parts of the posting process and needs a complete re-write of the comment handling process too, but until I’d got the 1.x code base to the point that it was stable enough to leave alone so I could concentrate on 2.0 I couldn’t do that. I’d just got ready to start recoding when Facebook announced they were changing the method of authenticating users against applications… that’s right. Just a couple of months after they changed it totally they are proposing changing it all again. It didn’t help that they accidentally turned on the new method and broke everything!

But things seem to have settled down and so I’ve started coding it, but I also wanted to watch “Travelling Man” which Nick had bought me on DVD for my birthday after we’d both read Andrew Denny’s blog entry about it. It’s been quite enjoyable watching it and recognising a lot of the locations, even if they have changed a lot since it was shot : for example the derelict warehouses near Preston Brook have been turned in to rather up market apartments – but it was good to see that Claymore Navigation’s boatyard hasn’t changed much in the intervening years.

So today I’m having a day off work, and I could be coding but instead I’m listening to Bob Geldof’s solo albums and trying to rustle up enough motivation to do much more than slouch on the sofa all day. I know that some would say that NOT listening to them would be enough motivation to get off the sofa, and others might say that listening to them just sucks motivation away from you, but hey… it’s my day off and I’m waiting for the Dishwasher to go through a cleaning cycle!

Things move fast in the IT world:  Oracle spit out a new version of their database every couple of years, Microsoft keep…. well actually the less said about Microsoft the better to be honest, and as for Apple……

I’ve been coding away on my Wordbooker Plugin since the tail end of last year. I had great plans for version 1.8.  I was going to add internationalisation and completely rebuild comment handling from the ground up as its not really working properly.. Hey I coded it in a day and bolted it onto the existing code so what else would you expect…

So I’d got all these plans worked out in my head and then along came Facebook and their developer conference (F8) who shoved a huge great spanner into the works.

Now for those who have never tried programming against the Facebook API you wont know just how annoying it is. It is rather like trying to nail jelly to the ceiling : Facebook quite happily change the API on the live site without telling anyone which breaks things. Go on.. how many times have you seen the “Oops, something seems to have gone wrong” message? Well developers get that too – or rather they get calls that suddenly return no data, or incorrect and incomplete data, or error codes they’ve never seen before.

People file bug reports and Facebook go and change the documentation and then deny its an error. Or Facebook fix the bug and everything is fine for a few weeks then it re-occurs which suggests a lack of a proper test and release process. The worst one I’ve seen is a bug which apparently “fixed” itself with Facebook representatives saying they did not know what had caused the problem or why it was now working again. The PHP files they provide don’t even have version IDs in them so you can’t tell if you’ve got the latest version, or worse still : you can’t determine the version of the files that another WordPress plugin might have installed – so your code doesn’t work because the version of the Facebook files they have installed doesn’t have the function, or class, you need for your code to work.

So any way, along comes F8 and a whole new way of interfacing with Facebook : The Graph API, and along with that came a new data model and a new data permissions dialogue, and a new Data policy. Everything new and shiny and the statement that “On June 1, 2010, we’ll automatically transition all Facebook Platform applications and websites who have not yet migrated.”

Sounds good doesn’t it. Apart from the fact that no-one at Facebook has explained how you can use the new Data permissions dialogue with the “old” API, and everything I’ve tried simply comes back with meaningless errors, and from what I’ve read other developers have experienced the same problems.

So I guess I could just convert my application over to the new API couldn’t I, that would make things easier wouldn’t it? Ahhh…………..actually I can’t because Facebook haven’t actually written the PHP API libraries yet – well not to the point of supporting “offline” access, which is what my application relies on. Actually that last bit says a lot about Facebook : They announce a whole new API and a whole new data structure, but the only way to access it is hand crafting your own CURL statements and running a “live” Facebook session. Talk about the right hand not knowing what the left is doing….

Madness