Thursday, 9 October 2014

BBC News - Europe backs Hinkley nuclear plant

BBC News - Europe backs Hinkley nuclear plant:

Good job the economics of this have been worked out, eh, chaps? At least ten years to build (the US average is 14 years, but hey, let's be optimists!), may be a 30 year operating life (the global average is 25) and may be another 20 to decommission and 300 years to return the site to other usage. Not to mention we still have *no idea* what to do with High Level waste - the High Level waste from the Windscale Piles from 1950s doesn't even have a permanent home yet...

Wednesday, 8 October 2014

The Mystery of the iPod Touch Sync Problem

For nearly ten years I’ve used an iPod of some description as I’ve found it the best compromise between usability, reliability and quality. Sure there’s lots of other devices people will argue are better but for me it’s fine.

Being an old stuck in the mud I do nothing exotic, I use the latest firmware/iOS, do not root the device and use iTunes exclusively for sync and management. At the end of the day if you don’t do that Apple aren’t going to support you if you have problems.

And for most of the last ten years I’ve had no issues. Until last Friday.

I don’t sync very often - I have about 4000 tracks on a 64GB iPod Touch and tend to listen by album rather than playlist so once the music is on I don’t have much need to make changes. I have apps from Amazon, Audible, Google Play and Spotify for music from other sources. In the main I’ve been very happy with the experience.

Last Friday the sync seemed to hang and sat endlessly at “Preparing Items to Sync” and never finished. I tried all the restarts the device/iTunes/PC and nothing seemed to work. I wiped and reset the iPod Touch and tried reloading it. No dice. The Touch then started to show very strange symptoms, 4000 tracks were showing as using only 3GB when they should be 30GB. The “Other” category in iTunes showing storage on the Touch was filling up the entire device. Tracks would sync only the metadata, showing a listing for a particular track but no physical data. I engaged in more wipes and resyncs and nothing got me any further forward or made any sense. I made a point of not restoring any back up because I had no idea which if any might contain the issue.

In the end I gave up and went to bed. My 22 month old Touch was a goner and I decided the Flash Memory was broken.

The next day I headed into the Apple Store in Newcastle upon Tyne to tell them my tale of woe. The “Genius” tested the Touch thought that over all it seemed okay, restored iOS 8.0.2 to it, tested it again and said the device was working. Not unreasonably he said if the flash memory was bad it wouldn’t boot or it did would show fairly catastrophic signs of failure. His suggestion was to reset iTunes; by this moving out the iTunes Media folder, deleting the library and preferences file and then dragging the Media folder back into iTunes. At >120GB this is a fairly hefty operation but never the less I did. I got already to resync the Touch and? No dice. Again. Same issues. This was starting to confirm the issue was with the Touch and it had to be physical as the entire OS had been replaced at least twice. Made another Genius Bar appointment for Tuesday.

This time Apple replaced the unit, like for like with a new Touch. Very impressive as the unit is well out of warranty and I don’t have Apple Care. He did say one thing just before I left: if the issue happens again it could be a corrupt file I am trying to sync. Whoa.

No where have a I read this before, not on Apple’s support site, on forums or anywhere else. A corrupt files could balls the entire thing up and he said is very hard to diagnose.

As I have too much music to sync to the device I sync a selection of Smart Playlists so that everything by a particular artists will be synced. Data corruption is a serious and very hard to detect problem, it tends to happen silently and you’ll only know it when a file you’re trying to access fails. In 2007 CERN reckoned 1 in every 1500 files is corrupt ( which only a 16000 file music library is at least 10 corrupt files. Of course most applications can handle and detect corruption, they almost expect it. However the sync from iTunes to the Touch doesn’t seem to fail gracefully.

I had about 18 playlists to sync. To find the corrupt file I had to sync one list at a time until the next sync failed. This was extremely time consuming but I eventually narrowed it down to one of the files in amongst one of Howard Jones’ albums (sorry Howard!). I don’t know exactly which one as without syncing each track one at a time it’s very hard to detect and it may not even fail in this context.

Working through all the lists had for music back on the Touch sans Howard Jones. However I find this very worrying as a development. If a corrupt file can indeed poison the entire sync mechanism between iTunes and an iPod this is a very serious problem. With enormous disks and thousands of files corruption is going to be an increasingly common problem. Clearly the sync needs to mitigate this gracefully without chewing up the Music library on the Touch so badly a wipe is the only remedy. What I don’t know is when this problem was introduced. I find it implausible that in ten years I’ve never had a corrupt music file before. For about the first six years all my iPods were hard disk based and they were simpler;

The sync of the hard disk versions seemed to do an rsync/Robocopy type operation in one direction only to make device mirror library. The metadata also seemed simpler - there was not even any album artwork to sync which today iTunes seems to store separately from the music files even though you can embed this in MP3 (see and into the ID3 tag. I can only assume that one of the many reasons for this is to not unnecessarily bloat the music files with image data and that storing a single instance of the artwork is more efficient -- as well as the fact the first iPods had no support for any album art so in the original software specification this may not have been included at all).

As the years have passed and that all iPods have high quality colour screens, can show artwork and cover flow (etc) the sync process has become more elaborate. Instead of it being just a simple synchronise at a file level (we’re back to rsync) its really become a bidirectional database sync. The database containing metadata of the songs, playlists, preferences and even separate preferences for individual songs. To make this computationally realistic from the perspective of a portable device doing it the heavy lifting of this is done by iTunes and what the device gets is only the data it needs at anyone time. The music files themselves are just one part of a connected whole.

Because it is bidirectional makes this even more complex. In the past iPods were essentially read-only devices. Now you can create, edit and delete playlists and tracks. You can buy new music from iTunes on the device and it will reverse synchronise to iTunes on your PC/Mac. It’s extremely sophisticated stuff.

Naturally this all comes with overhead. Like Montgomery Scott once said “the more they overtake the plumbing the easier it is to stuff up the pipes”. If a corruption gets introduced at some point without error-checking built in then it’s likely the house of cards is going to come down. Writing something subtle enough to detect the kind of corruption I’ve experienced may be impossible if you want the software to be this sophisticated. It is no surprise that most music sync software is built along the same theme as rsync and then device builds a simple index of the data copied to its storage. This makes it robust but more basic. Introducing a corrupt file into this setup will possibly, at the worst, make that corrupt file not play or be skipped. The structured is atomised so a failure in one will not affect others.

It is very difficult to speculate on what was meant by a “corrupt” file - corrupt to iTunes, the device, the firmware, the storage? If it is a corrupt file by whatever definition that is then I wonder if iTunes sanity checks them - particularly as I reimported the entire contents back into the library. It couldn’t be the same physical file - it was a physically written new copy so the corruption had followed it. If this problem continues to manifest then Apple has no choice to either write better error correction/detection or provide some kind of tool that checks for the corruption they know will the sync and/or the device to fail. I am also left worried that in future the addition of a single file may cause this to happen again. Smart Playlists may have to be locked to prevent live updating - I am more of the opinion that I may need to start looking for a simpler kind of synchronisation.

Tuesday, 16 September 2014

Book Software

Collecting books, in my case those about certain aspects of the Cold War and nuclear energy, has eventually meant buying a database to keep a record of them, not least to prevent me from buying the same book twice. My needs were simple:

  • Recording the books themselves;
  • Using my phone as a bar code scanner to enter some of them;
  • Being able to export the data to agnostic formats like CSV;
  • Being able to look up Bibliographic data.

In the end, nothing too fancy. The first one I looked at was Book Collector from I wasn't terrible struck by the name “collectorz” which seemed a bit low-rent to me. Unfortunately it was the complexity of the whole deal that made it a non-starter: various bits of the product looked like endless paid-for extras:

  • Book Collector
  • “CLZ Barry” to scan bar codes
  • “CLZ Mobile App” to have the database on your phone
  • Cloud CLZ Account with a paid for option to edit entries on web

Not impressed – having to buy an expensive iOS App to have the database on my phone (around £10) was pretty bad, but even the appallingly named “CLZ Barry” just to scan barcodes for around £5 was out the question. I just felt like I was being shaken down for money. The CLZ Mobile App was hardly impressive for £10, it was a basic database version of the desktop one that really should be free/inexpensive. The names of the products are horrible, they don't look great and didn't represent a good deal to me.

I then looked at Readerware which seemed a much better deal ( This was much the same money as Book Collector but the Mobile app was part of the price. In some way it is less functional on the phone as you can't edit, but that was never a function of value to me. As long as it could scan and upload barcodes and produce a simple version of the database I was happy.

Like Book Collector, Readerware isn't the prettiest application. Also its ability to connect to iOS to upload barcodes/transfer the database could do with some serious refinement. However what it does do, it does well. All it needs to do is record the books I collect and use my phone as a means to scan their details. It does allow me to add library books in an “aim to read” list for later. Sometimes when I'm browsing a library shelf I'll scan in barcodes of books that look interesting that I intend to read later.

Part of the issue is that this is a specialised product and there really aren't many around so unfortunately you have to put up with what ever limitations they have. Ideally you'd be able import lists of books bought electronically from Amazon, Kobo, Google etc but unfortunately this isn't possible. Having a pre-configured “Library List” of books you've borrowed or would like to borrow would be very handy for book collectors and scholars alike.

Wednesday, 3 September 2014

Stolen Photos

There is currently great hullabaloo about stolen/hacked photos of Jennifer Lawrence and others in various states of undress supposedly stolen from their iCloud accounts (and possibly other as yet unconfirmed sources).

The key question is who is responsible and to what degree. Is it just the criminals who are responsible in a blatant act of theft, do technology companies have responsibility and do the actual owners of this material bear any kind of responsibility or is this "victim blaming"?

In the first instance there is no right of access to private property; no one has any right to obtain these materials by criminal or deceptive means which is obviously the first place to start. What has been quite surprising is that some of the loudest commentators on the right to private ownership of these materials are surprisingly much less vocal about the theft of other privately owned materials such as music, films and TV shows. There is no material difference in who owns what. No one is allowed to make copies of something you don't own whether you can or not or how easy it is, is irrelevant - is the victim actually to blame?

iCloud Terms and Conditions

One very obvious gotcha in all of this are Apple's Terms and Conditions of using iCloud. They specifically say:
You agree that you will NOT use the Service to:
a. upload, download, post, email, transmit, store or otherwise make available any Content that is...obscene, vulgar ... or otherwise objectionable;
On balance it's hard to make an argument to say photographs of a prurient nature don't fall foul of these terms; "otherwise objectionable" covers a multitude of sins and its extremely likely that nude photos sent to others or stored (and certainly rumoured sexually graphic video) will break these terms and conditions.

As a private service it's impossible to say how much objectionable content Apple removes and the Terms and Conditions make it extremely subjective how they may be applied. However it looks impossible for this kind of adult content not to be in breach of their Terms and Conditions so from the very beginning these individuals should never have been using the service in such a way.;

It's impossible to reconcile reasonable protection from prying eyes from something that shouldn't have been happening in the first place otherwise criminals the world over would be demanding their privacy. A reasonable expectation of privacy does not grant you the right to behave in breach of conditions of you've agreed to, let alone laws that define your behaviour.

I think the key point is this: had these individuals actually obeyed the rules of iCloud they would not be in such a panic about their stolen materials now; it does not excuse a crime perpetrated against them and they do not deserve any humiliation but they do carry some culpability.

Responsibility of Technology Providers

It's probably crucial at this point to understand the role of the technology providers like Apple. I don't think you'll find any technology provider who will warrant your data as being protected from all security threats. In fact most of them will explicitly deny any responsibility to the safe keeping of your data. Apple's Terms and Conditions make their responsibilities towards your data explicit (caps in original):
For any company storing your data it seems reasonable to me that their attempts to safeguard your data will not open them to unlimited liability. Most data is protected by fairly weak security: a username and password. There is no physical protection, there is no encryption and two-factor encryption is optional. If you can work out someone's username you're part of the way there to unlocking their password, through phishing, malware or even brute force attacks. Password recovery is also notoriously weak. Just look at the story of Mat Honan and his epic loss of personal data in 2012. There is literally a simple password (and people are terrible at choosing them) between your data and people not authorised to receive it.

Responsibility of Users

Clearly most users do not understand the "cloud"; they do not understand the limitations of technology, security and what their responsibilities towards their data is. To broadcast such intimate photographs of yourself across what is a public sphere protected only by a password is incredible if it wasn't true. This is not victim blaming; this is crime prevention. The public have been reminded for decades to lock their doors against burglars and thieves and this is no different yet the repercussions are far worse. Once data is stolen it can never be recovered; perfect first generation copies can be duplicated and circulated ad infinitum.

The cloud itself is relatively new; it is very convenient and very sophisticated but the security of it is well known to have been highly compromised through the Snowden files, the Heartbleed bug and other more advanced forms of criminal activity. There are people that interested to steal your data for entertainment, for the challenge or criminal gain and the higher profile the individual the more valuable they become.

Clearly there is some impact on users which they must learn to think about; we are an extremely long way from good security (I won't even say "perfect") and you have to act with this in mind. Outrageously explicit photos are not suitable to be kept or sent over the cloud and this applies to anyone where the distress and embarrassment of their release will be just as keenly felt whether you're a superstar or not. Data kept in the cloud should be secured at least with two-factor authentication as well as strong passwords. Automatically uploading/backing up photographs should be switched off unless you're confident that your photos wouldn't embarrass you if they appeared appear on the news. Know that just owning an Apple, Android or Windows Phone device already lets people know you have an account with that company so it's giving clues away about your usage.

Ultimately it's your data and yours to protect; no one is going to do this for you.

Monday, 28 July 2014

Regional TV Adverts of the 70s and 80s

Some hilarious comments about regional TV ads from TV Cream that I've resurrected - unfortunately they no longer appear on their own web site. It doesn't sound funny, I'll admit but wait till you read them...

Friday, 4 July 2014 - review

I've used Fastmail ( for nearly ten years as my main email provider and in that time my uses and needs have changed. They have probably changed more in the last couple of years and now Calendar and Contacts have become more important, particularly in syncing to other devices and storing them online rather than locally. For years I used Palm Desktop (showing my age) and things like Nokia Desktop that allowed local sync via Bluetooth between Outlook data and a S60 phone.

All that's changed and now things like ActiveSync and CardDav/CalDav have become the most popular ways of device sync. For sometime I've been using Google for Contacts/Calendar and Gmail (forwarding my Fastmail account to them) and generally its been a good experience. Unfortunately I am a little troubled by Google having so much personal data on me; I don't mind for a Blog like this one as all the data is effectively public by default. But private data is another issue.

This is where things get more difficult; trying to find a good Email/Calendar/Contacts provider that isn't Microsoft Exchange based is not that easy. My reasons for not wanting Exchange are quite complicated and really not worth going into. As much as I've also had excellent service from Fastmail (and I have no intention of leaving them) they do seem to be struggling with updating themselves to be more than just a provider of IMAP email. Currently they do have a calendar but this is only a few weeks old at this point (July 2014) having just been released.

Ideally I would like both ActiveSync and CardDav/CalDav as well as IMAP and a good web interface. Surprisingly few fit the bill. I finally decided to try as a provider for these services. The sign up was easy and I haven't read any negative comments about them. On the whole I was quite impressed. Once I was signed up, I did find some critical problems that were show-stoppers for me:

No CardDav/CalDav:

As far as I can see there is no CardDav/CalDav. This is pretty much a show-stopper there and then. I need CardDav/CalDav for local clients like eM Client and even Windows Mail on Windows 8.1. I also like it as a means to sync iOS and to sync with Memotoo, etc. They do offer ActiveSync that Windows 8.1 can also use for a local client (and possibly eM Client) but without the means of synchronising a local client this was a dead-stop.

Alias Support:

The support for aliasing email addresses requires a ticket to be created - in Fastmail (and just about every other provider) makes this just an option in account setup and although this is something you do irregularly, this was also a big problem for me. This should be a user-changed setting.

Poor Identity Support

At Fastmail you can almost set up any alias joe@fastmail.invalid and when mail is sent it will go as this identity with no hint as to the underlying login ID. I find exposing my login name to be a bad idea and sending this with an email as a email address to be just asking for trouble.

No Two Factor Authentication

Two factor authentication is becoming a way of life, especially for email accounts. Email accounts are frequently used as a means of resetting passwords on sites and establishing your identity so protecting them as securely as possible is just common sense.


With better documentation and more detailed pre-sales information I wouldn't have signed up with as a provider. This isn't actually a criticism of them, I just would have known up front that features I require weren't there and it would have saved everyone time and effort. In fairness to them, their general set up is excellent and the price very fair; with a few additional features noted above I'd be more than happy to go back to them as they are a very complete provider of email and calendar/contacts.

Thursday, 29 May 2014

"Poorest 40% spend more than they earn"

Another damning indictment of Britain as "UK's richest can save £18,680 a year as poorest 40% spend more than they earn." In truth it was ever thus; for decades the poorest have spent far more than their earnings; in years gone by it was the pawnbroker (the tales of Dad's suit going on Friday night for a loan and coming out again Monday morning are typical) through the rise of doorstep lenders in the 1980s and now for the payday lenders, cash for gold businesses and even selling their clothes by the kilo. Someone, somewhere has always been squeezing money from the poor for decades.

But it's clear the gap is getting wider as it's always the poor who carry higher hidden costs than anyone else; they'll pay more for pay as you go mobile phones, prepay credit cards and certainly prepay electricity and gas. The better off meanwhile are having a blast: their better credit ratings get effective discounts on contract mobile phones and cheaper rates for electricity and gas, no prepayment surcharge and even discounts for paying via direct debit. The better off are also more likely to buy on the Internet getting cheaper goods and services, they've got the money, the methods of payment and methods of access to better deals. It is expensive to be poor.