One of the more grandiose ideas which I have expounded upon is a personal repository, a store of my data in all its versions over the years. I still think there is some mileage in this, but it is not something which I will be continuing with. It's just far too much work.

I had an inkling of the huge mountains of time required for this when I wrote my last post on this subject - "It's taking much longer than I had hoped" is more or less code for "This is much harder than I had expected", although I rarely recognise this code myself up front. The real clincher is that as part of my effort to organise proper backups, I have been spending some time reconciling the different versions of my writing environments.

I had three slightly different versions on three machines, each with its own components which were more advanced: azathoth had a couple of scripts which were missing from the others, dagon had some fiction which somehow had not been backed up anywhere else, and the web site stuff was much further along on ithaqua. Just tidying this stuff up has taken a solid day for each comparison, and these are for environments which are fairly similar and haven't drifted much! The worst entropic force was the case corruption issue I mentioned when writing CDs from ithaqua, but still... there weren't that many variables, and it was hard work sifting and sorting even this relatively small amount of information.

Doing that comparison multiple times for progressing and mutating versions of my environment which were created on different filesystems and under different naming conventions does not fill me with excitement.

There is the possibility of writing some tools to aid in the process, of course, but that's non-trivial in itself - it is easy enough to determine if two files are the same using signatures and so on, and this can be used to figure out if the same data is in multiple places, but writing code to determine if two files are similar (ie if they contain different forms of the same or similar information) is much harder. An interesting problem, but not a simple hack by any means.

Of course, anyone with any experience at all of cataloguing and so on would probably have recognised the arrant optimism of the original idea. Maybe I'll come back to this, maybe I won't, although the chances of returning to this without having more sophisticated tools in hand is slight.

It's still fun rummaging through old files, though.

Posted by Dunx at November 5, 2003 03:13 PM