Lightroom, database, files and backups

11 May 2018

Adobe Lightroom is a database. Some even call it a “Relationship database”. I have never found any contacts of the opposite sex in there, so I will keep the term as “Relational Database” Smile . Lightroom does not keep your images inside that database. The database itself is called a Catalogue, to make things more palatable for the general public. Nobody in his right mind would want to buy a “Photographic Relational Database”, however, most people would buy “Lightroom and Catalogue”. Simple question of marketing. That Adobe felt the need to create different versions of the same product with virtually the same name was not very wise, but like they say: “It is what it is”. Nothing I can do about that.

So, Lightroom is a database.

The database

What does that mean? It means that information ABOUT your pictures is stored there, but not the pictures themselves. If you wish, you can imagine the database as a set of interconnected (related) spreadsheets, each of them holding information about a specific detail of your images.

This is what that database(spreadsheet of spreadsheets) list looks like:

image

There is a lot more inside that database, but with this you get an idea of what it looks like.

One of those tables contains information about the files you have, as well as a lot of other info required to manage your pictures.

image

In this table you can recognise the file name of the pictures, the file type, but hardly anything else. The rest is for internal purposes, like filtering, development settings and much more.

All this information is packed together in a single .LRCAT file. But there is an issue.

Lightroom needs thumbnails to display in the grid. Those thumbnails will be taken from the raw files you import as well as from any other picture format. If the file does not have a suitable picture embedded in it, then a thumbnail will be created on the spot.

The thumbnail collection of all your pictures is stored in a separate location or folder, called Previews.LRDATA. In there, the thumbnails will receive a filename that is referenced in the central database or catalogue. At that point Lightroom knows what to display in the grid.

Lightroom is built on SQLite, the same database used for Skype and countless other applications that do not need the overhead of multiple concurrent users accessing the data. For that reason the database or catalogue cannot be stored on a network drive, as that would imply multi-user access.

The file structure

Now that you have had a glimpse of the innards of Lightroom, it’s time to forget about that. You will rarely see the insides of the Lightroom Catalogue. If you do, you have a problem and you’d better call in some professional help.

And that leaves us with YOUR pictures. They will be in the folder  that you designate. By default, Windows uses the “My Pictures” folder, a place where all windows pictures congregate. That includes any downloaded pictures from the internet, pictures from other apps and so on. Needless to say that I don’t need all that in my catalogue. No, my catalogue is reserved for my photos. To avoid Windows putting all my stuff together with the rest, I simply created a folder called \photos on my drive. I direct Lightroom to it whenever it “forgets” where to put the pictures. Usually it forgets when I have changed catalogues for some testing reasons. I only have to check the destination in the import panel to make sure the pictures go to the right place.

The catalogue and the previews have to stay together and again I don’t want them in the “My Documents” / “My Pictures” or “My [whatever]” folders. For the catalogue I have created a \lightroomData folder and Lightroom will only open that catalogue by default for me.

The backups

My complete organisation of Lightroom can be found here.

As you can see, the Lightroom Catalogue, the previews and your files are three separate entities for Lightroom. When you are presented with the Backup Dialog from Lightroom like this:

image

Lightroom ONLY proposes you to make a copy of the database or catalogue. That does not include the previews, as they can/will be regenerated if needed. It also does not include your photos. From the screenshot above you can see that I do a backup of my catalogue on DropBox. Those backup files are stored in a compressed format called ZIP. The total size of my 70k+ images’ catalogue is about 300MB. Needless to say that this size does not include previews or photos, right?

So how do I back up the rest of my files? There are tools for that. Like they say: “There’s an app for that”. Ok, there are dozens of apps for that. Personally, I use FreeFileSync from https://freefilesync.org/. As the name suggests it is free and it syncs or backs up your files.

FreeFileSync allows me to set up different backup strategies for different backup drives.

image

From this screenshot you can see that I back up to two different drives called S and T, Once a year I copy over everything from S to T as well. The S drive is also a “live” drive for the pictures I no longer keep on my internal drives. See here for all the reasons why. In short: space concerns. While backups are regular, if I make changes to any of the pictures on S, for example adding a TIFF copy made by a plugin, then those files will only exist on my S drive. Once a year I synchronise my two backup drives for those files.

When you click the “Compare” button, FFS will compare between your source and your backup drive and will quickly find any additional files.

image

At that point you only have to click the Synchronise” button on the top right and sit back. Unless you do this for the first time, you won’t have time to get a coffee while this is working. FFS is lightning fast.

The progress screen shows you how fast things are going on your machine:

image

Once you have run your backup on one or more drives, all you have to take care of is the physical location of your backup drives. One should be off-site, at a friend's house or in the bank. One can be close to your computer and your live data is on your machine at all times.

I know that should my house burn, I only need to grab my backup drive. The rest is insured and replaceable.

That’s my backup strategy for my pictures. Should anything happen (let’s not count on the burning house here) then I most likely have a backup of my pictures and a copy of my catalogue on DropBox. Copying all my pictures to DropBox or any other Cloud Storage would take way too much time and bandwidth to be feasible. It would take me weeks to copy all my files to or from the clouds. Physical hard drives are cheap nowadays and we should use them abundantly.