How to live with multiple computers

[A dump of a large number of small pieces of advice from years of shunting data. File under `boring but useful'.]

Problem statement: You work on multiple computers: home and away, or a desktop PC and a laptop, and you want to use both to work on the same projects, so you need some way to reliably shunt data back and forth. Me, I have three home laptops in various states of brokenness, a PC at (name of think tank) and a PC at (name of university), and a few accounts in the ether somewhere which I log in to from time to time, and all that has to keep reasonably synced up.

Buried into all of this are a few regimes for backing up data. Not to be mean or anything, but to those of you who aren't backing up their data regularly: what are you, fuck*ng stupid? If your work is worth doing once, it's worth making sure you don't have to do again. [Personal to Ms. JATMM of Mount Vernon, VA: my continuing condolences. I don't mean you.]

Step one: have a home dir: Unix geeks, you already have a home directory. But you Windows users probably have data spewed all over your hard drive. Many programs like to make their own little directories for storing data, so your MP3s are in c:\Program Files\Brand Name\Program Name\Data, while your great unfinished novel is in c:\Another Brand Name\Word Processor\user Xp458yjz\documents\. Get them all in one place, which I suggest you name c:\home. [If you have multiple partitions, use the one that the operating system isn't on, which will probably be d:\home. If you don't know what a disk partition is, don't worry about it.]

MacPeople, you also have a home directory, but OS X tries to hide it from you. You have a Unix command line (the Terminal, in the Utilities folder); get to know it. Since I don't have an iAnything at hand, maybe somebody else can offer more suggestions in the comment boxes below.

Put all your data in the home directory. Make yourself c:\home\novel, c:\home\music, et cetera, and stick with the plan, even when your programs try to distract you. Linking (aka shortcuts) can help with this: e.g., set a shortcut from the Desktop directory (wherever it is; Microsoft keeps moving it) to c:\home\desktop.

The idea here is that if your computer falls to pieces, all you have to do is reinstall the software and then copy back your home directory, and it will be as good as new. You probably have reinstall disks for the software somewhere; you're basically making yourself a reinstall disk for your work.

Further, the costs of hard drives will hopefully be in sync with the amount of crap that you increasingly collect over the course of your life---and so you'll never have to throw anything out, ever. Instead of putting your projects on floppies in the closet, you can keep everything you've ever done in your home directory, perhaps archiving from time to time into an archive subdirectory. Me, my /home/b/arch directory includes everything I've done on a computer since 1999. This comes in handy more often than you'd expect.

Also, dear Windows people, you may want to get a copy of Cygwin, which will let you execute the Unix commands below. You want this to be as easy as possible, and writing a batch file to run the command will, in the long run, be a lot easier than clicking lots of little boxes in WinZip every time, despite the initial setup. Also, children and animals will like you more.

option one: Directory replication We have two options here: the first is to find a place online, and the second is to carry around a physical object.

suboption one: Online: this web site is hosted by Spiderhosts.com. I pay fifteen dollars a year and get thirty MB of storage. [Tell `em I sent you and I'll get a discount on my renewal.] That's enough for a lot of the important stuff. There's also gmail.com, which will give you a frigging GB, and if you need more, I guess you can just get two accounts. Depending on the status of your connections, the internets may or may not always be available; this obviously won't work if you're not ubiquitously networked. [Personal to Mr. PH of Seattle, WA: now that Presidential decree has pluralized internets, I think it should be lower case.]

subsuboption one: with compression One line:
tar cvz --exclude /home/b/mp3s /home/b > my_home.tgz
Then email my_home.tgz to your gmail account, or what have you. When you get to the other computer, you can decompress by going just above your home directory (like c:\) and typing tar xvzf my_home.tgz
WinZip or StuffIt or what-have-you will work OK, but there's no easy way to tell it to exclude c:\home\mp3s. Maybe you can make it work.

subsuboption two: without compression Use rsync, which will only transfer the changed part of files which have changed, making for a very quick transfer:
rsync -avPe ssh --delete --exclude /home/b/MP3s /home/b/ b@fluff.info:home_bkup/
This is especially nice if your home or work PC is always on, in which case you can just transfer directly. Check your router to see your home IP address (which is not reliable because most DSL/cable companies will change on you to keep you from doing this sort of thing. If your ISP uses PPPoE instead of DHCP, complain.), then turn on sshd on your PC (easy if you're unixy; I'm told cygwin will also let you just type sshd at the command prompt but never tried it myself). All you need is one of your two PCs to be identifiable by an IP address and have sshd running and you're good to go.

The --delete option gives me a rush of fear and adrenaline every time I use it. But even without that, this is dangerous because if you reverse the from and to directories, then you'll overwrite your new stuff with old stuff, thus losing all the work you just did. Nothing sucks more. With compressed files, you can of course have the same problem.

More advice: think of one and only one drive (your home hard drive, work hard drive, your portable drive) as your primary home directory. In others, set up an xfer_me directory of things that need filing back to the primary drive. The assumption is that if any of the non-primary computers go up in flames, you won't care. Day to day, you just need to transfer the xfer_me directory back to the main from time to time, and then overwrite the non-primary home directories with fresh copies of the primary now and then.

Another useful tidbit: if your computer is a laptop, and you have a home network, then both of your computers are sharing a subnetwork. Your desktop probably has an address like 192.168.2.101 and your laptop something like 192.168.2.103. If one of these can run sshd, then you can directly rsync directories, and it'll be rather zippy.

suboption two: physical device: This would be something you carry around with you. Pretty much anything will work: those little USB keychains, your MP3 player, even your digital camera can probably store files for you. Portable storage is frigging everywhere. The obvious problem is that if you forget it at home, you're screwed, but it will of course work if you have no Net connection. Also, don't forget the cable, which I've done embarrassingly often. [Spare tip for buying a spare cable: A bit of poking around online should find you the formal name of your cable---more and more are A-mini B---and you'll find that you pay a lot less for an `A-whatever cable' than a `cable for brand name device'.]

Devices that your operating system recognizes as just another drive are the best; some devices (Kodak cameras, Apple iPods, Creative MP3 players) require additional software, which will require installation everywhere you want to use it---annoying. On such devices you may have to archive your home directory into one file (see subsuboption one, above) and then copy that single file over.

The world of devices falls into two categories: flash memory based and hard drive based. With a hard drive based device, you may as well make _that_ your home directory, since you should have enough space to do so. The only problem is that there's a slowdown with external hard drives; unless you've got really good equipment, it's noticeable.

[Me, I have an Archos Jukebox, which mounts normally and has the imprescindible feature of rubber bumpers. [That's Spanish for `can't-pass-up-able'.] I partitioned it into a vfat partition for the music (`cause the firmware demands it), and a reiserfs partition so I can have the pleasant features of real-live links, permissions, and a journal.]

Flash-based devices tend to give you much less storage---you'll have to leave the MP3s at home, and just back up the important stuff. That's OK with the above methods, since both give you options to exclude directories (which is why you want to do this from a batch file, where you can specify what to exclude once and for all). Not a problem, but don't forget to back up the data you're not carrying with you some other way.

Since you're not networked, you don't need the `e ssh' clause:
rsync -avP /home/b /mnt/cygdrive/e/home_bkup/

The biggest practical problem with both of these methods is that you have to remember to transfer every single time. If you work on the project at home, rush to work without transferring, and then work at the office, then reconciling the two versions will be a nightmare. You'll almost certainly lose data, which may make you cry.

option two: versioning systems The idea of a version system is that you have a repository somewhere which holds the project. When you want to work on it, you check it out, and then check it back in when you're done. Once checked in, your copy is entirely disposable.

Using a versioning system changes your mindset. You can screw with your copy of the project all you want, since it's just a copy; you don't have to save lots of revisions, since the system is doing that for you; you'll find your work will be more structured around work sessions which have a specific goal. It's fun.

Finally, it solves the problem of forgetting to bring to work changes you made at home, since it will do its best to merge two modified versions without losing anything. This sometimes requires human assistance (it'll tell you when), but you never lose work.

The standard revision control system is CVS, which (if you're using Cygwin or anything unixy) is on your hard drive now. CVS has been replaced by subversion, but subversion isn't yet common, in the sense of any given computer basically being guaranteed to have it. If you're reading this more than a year from now, try that first.

Once you set it up (RTFM), the only commands you'll ever need are
cvs get my_project
cvs ci (aka cvs commit)
cvs update
Oh, and when you add a new file to your project, don't forget
cvs add new_file ;
forgetting that is basically the only way to lose data with CVS. If you ever need an old copy, or to do weird things to your data, RTFM, but it's very easy. There are also graphical shells like tk_cvs, which I have on one or two machines but never really bother with, since 97% of the work is the above four commands.

Oh, and don't forget to back up your CVS directory from time to time. Mine crashed once. I managed to not cry but I was grinding my teeth in my sleep a lot after that. You'll also have to decide where to put the repository. If you're a student at a university or a not-Microsoft-dominated office, then you probably have a shell account where you can put it. It's surprisingly small, so a spiderhosts account will do just fine for the rest of you. Unless you're doing something screwy, your CVS repository will be under a few dozen MB, so a keychain drive will also work.

[Tip on keychain USB drives: avoid puffy ones. I saw a girl almost weep when the only copy of her presentation was on a USB drive which didn't work because the plastic casing got in the way on the laptop she was trying to plug in to. She was also silly for not having backup: when traveling to do the Big Presentation, leave a copy online, on your keychain, and on a burned CD; chances are that one of them will work, but you won't know which until you get there]

CVS will work great for that part of your life which is project based, but you'll have to go back to option one to handle your MP3s and family photos. I think you're getting the picture here: the project you're working on these days should probably be in CVS, and then you can carry around a hard drive with all of the not-so-frequently-changing stuff that makes your home directory a home.