A RAW Workflow
in Olympus E-System , Thursday, August 03, 2006
Since I have a received a good few questions about my workflow, particularly from Olympus E-1 owners, I though it was maybe time to write something about it. Note, I do all of my digital photography on Macs, so I'm afraid this workflow is Mac specific, or at least the RAW conversion part is. I use three core tools on my workflow:
I also use Colorbtyte ImagePrint for printing, and FixerLabs SizeFixer for scaling up. For RAW processing, I sometimes use PhaseOne CaptureOne Pro and Olympus Studio. For Lumix LX1 RAW processing, I usually use Adobe Lightroom Beta. I only use Adobe Camera Raw in special cases, for example if I want to use Photoshop's HDR tool. In all cases, you can assume I'm using the latest versions of each application.
Stage 1: Ingesting
The first part of the workflow involves getting image files off the card and into the computer. I use MediaPro to import and rename the files, and to apply a basic personal metadata template. I don't really have a solid naming scheme, but I at least rename files to include a code indicating which camera the file comes from, and the date. I let MediaPro assign a serial number. Actually, the best renaming software I've seen is Olympus Studio, which allows you to use EXIF fields as components of the file name. The important thing is to have a unique name, and to keep it throughout the pipeline. In this way, it is possible to sync metadata in between different catalogues and different filetypes (so for example the captions from E1_20060603_0091.ORF and E1_20060603_0091.JPG can be synchronized. My filing system is quite simple: I have a master hard disk for RAW files, with folders for each month. Within these folders, I create subfolders whose structures depend a bit on the nature of the shoot. Usually, I create a folder for a single day, e.g "2006_06_03" (that's 3rd June…I'm European), but I might create a folder for a multiple day "shoot", e.g. "Iceland March 2006". No hard and fast rules, just whatever makes sense. If I end up using CaptureOne, it will require its own sub folder structure to be created inside this folder – at least, if you prefer to keep things simple it will. Other RAW developers will leave settings files inside these folders too, so it is important to ensure these remain safe. I generally back up each folder to DVD – one or more, as necessary, although this is not ideal, really, since the backup only store settings files created up to that point. I also keep an automated working backup using Retrospect. I maintain two "master" RAW catalogs for Olympus ORF and Lumix Raw (converted to Adobe DNG) respectively, and these automatically update. I plan in the future to manage archive backup creation from these catalogs (using a script to record the archive volume names in each image's metadata), but I haven't got around to that yet.
Stage 2: Preview
Once the files are organized, I create a local catalogue using MediaPro – by this I mean a catalogue of just the RAW files in the particular folder. I set MediaPro to produce the largest, highest quality previews it can create. I then use the comprehensive previewing and rating tools to decide which images I want to work further with, and how I want to categorise them (for example, I might well have "fun", "family" and, er, "art" shots from the same shoot. MediaPro 3's new Lightbox tool comes in extremely handy here, although it is very important to note that you are working with an 8-bit preview here, so the histogram for example is of the preview, not the 12bit RAW. However, it is easily good enough to show if an image is irrecoverably over- or under-exposed, and the method is fast and effective. At this point I will delete any files which are really trash, although if in any doubt, I keep them. Disk space is cheap. I end up with sorted, rated files, and I can even print contact sheets if I wish.
Stage 3: RAW Processing
For RAW Processing in the vast majority of cases I use Iridient RAW Developer (IRD). IRD, thankfully, has the good taste and common sense to be "just" a RAW developer, and has no pretensions to act as a full blown "workflow manager" (no, I'm not going to get sidetracked). IRD has drawn praise from a number of reputable sources, and is possibly the most full featured product in its category. Certainly it can't be beaten on sharpening options. Initially IRD seems very complex, but the complexity is only there if you need it, and it doesn't get in the way. It also often provides several ways to reach the same end, for example tone curves and tone sliders (I prefer curves, because Photoshop forced me to understand them, and Bruce Fraser's books explained them). IRD is especially highly rated by black & white aficionados, but, typically, I don't use it to B&W conversion. I tend to open RAW files in batches of related images, by selecting them in MediaPro, right-clicking and opening in IRD (this works as well for Olympus Studio or Camera RAW, but not very well for CaptureOne – ironically, since there is some sort of marketing alliance between CaptureOne and iView…well, there used to be, pre-Microsoft). Once the photos are open in IRD (note how I use "photo", "file" and "image" interchangeably), they can be selected from the open files drawer, and processed in turn. I won't go into detail on how I use IRD here, but my default settings for E-1 ORFs include using light "Difference of Gaussians" sharpening as "capture sharpening" – although I'm beginning to wonder if multi-stage sharpening is necessary considering the lack of artifacts IRD introduces – and using ProPhoto as the working colour space, and Joe Holmes' ExtaSpace as output space. My principal output files are 16 bit TIFFs, which I save in a temporary holding folder called "IRD Output". I also sometime process directly to JPG for print or web, but for my web galleries I use a set of Photoshop actions (coming later).
Stage 4: Post-processing
Currently I use Photoshop (.PSD) format as my final archive format, for fully processed images. There are several reasons for this, number one being layers, although even I'm not using any layers, I still save as PSD for consistency – if I see a PSD file, I know it is "finished", or at least it has been worked on. The second reason is that at least 25% of my output is still film-based, and my film workflow always culminates with Photoshop. I don't know of any compelling reason to change this practice. Therefore, I open all files from "IRD Output" in Photoshop, and at a minimum save them as PSDs on another disk volume, which is my "finished work" repository. Here, I simply maintain a folder for each month, and save the file into the current month's folder (I use MediaPro, not the filesystem, to catalog, so it doesn't matter if a photo I took in March 2005 ends up in the January 2006 "finished" folder). Depending on the photo, there are several things I might do in Photoshop. Generally, I will have sorted out tone in IRD, but I might run a local contrast enhancement action to see if it adds anything (I have evolved a variety, which act on different tonal ranges as necessary – usually I find excluding highlights is a good idea). Noise reduction is often required on E-1 files taken at 800 ISO or over, sometimes at 400 as well. For this I use the Noise Ninja plug-in. If I decide to convert to black and white, I use the Convert to BW Pro plug-in on a layer. I save files unflattened, although I might compact things a bit if they get out of hand. The "finished work" gets cataloged in my "reference" MediaPro catalog, where I add detail metadata, and construct various sub-catalogs and sets.
Stage 5: Output to Web and Print
Output requires sharpening and sometimes scaling. If I'm outputting to print, I take the PSD file, and set the output size as necessary. If the resolution drops a bit below 240dpi, I will scale up using Photoshop, but if it is well below, I use SizeFixer SLR. Once I get to the target size, I use Photokit Sharpener to apply output sharpening (note, for SizeFixer, it appears that sharpening before scaling is quite successful, but how Photokit's algorithms react to this, I'm not sure. Whatever – if it looks right, it is right). I then flatten the file, and save it as a copy to a temporary print folder, where ImagePrint picks it up. For my web galleries, I run an action which converts to 8bit, sRGB, then creates three different sizes of the file, appropriately sharpened, in an output folder hierarchy. The largest files are managed in a MediaPro catalog, and this is synchronized with my online mySQL database using a set of Applescripts, which glue MediaPro, MacSQL and Transmit FTP together. There are ways of using scripting additions to do the SQL and FTP parts, but they are complex, and not worth the trouble to me. The Applescript is very specific to my configuration, but I'm happy to send it to anybody who would be interested to see if they can adapt it.
So that's basically it. It takes longer to write about than to do it. The foundation stone is obviously MediaPro, which is a very powerful, but subtle application. The fact that Iridient RAW Developer constrains itself to doing one thing very well makes it very easy to introduce into a composite workflow. And Photoshop remains Photoshop... At some point I will follow up with my film-based workflow, but it isn't really so different.
- Microsoft iView Media Pro, for ingesting, preview, preliminary sorting, and cataloging
- Iridient RAW Developer, for RAW development and image processing
- Adobe Photoshop, for image post processing and output
I also use Colorbtyte ImagePrint for printing, and FixerLabs SizeFixer for scaling up. For RAW processing, I sometimes use PhaseOne CaptureOne Pro and Olympus Studio. For Lumix LX1 RAW processing, I usually use Adobe Lightroom Beta. I only use Adobe Camera Raw in special cases, for example if I want to use Photoshop's HDR tool. In all cases, you can assume I'm using the latest versions of each application.
Stage 1: Ingesting
The first part of the workflow involves getting image files off the card and into the computer. I use MediaPro to import and rename the files, and to apply a basic personal metadata template. I don't really have a solid naming scheme, but I at least rename files to include a code indicating which camera the file comes from, and the date. I let MediaPro assign a serial number. Actually, the best renaming software I've seen is Olympus Studio, which allows you to use EXIF fields as components of the file name. The important thing is to have a unique name, and to keep it throughout the pipeline. In this way, it is possible to sync metadata in between different catalogues and different filetypes (so for example the captions from E1_20060603_0091.ORF and E1_20060603_0091.JPG can be synchronized. My filing system is quite simple: I have a master hard disk for RAW files, with folders for each month. Within these folders, I create subfolders whose structures depend a bit on the nature of the shoot. Usually, I create a folder for a single day, e.g "2006_06_03" (that's 3rd June…I'm European), but I might create a folder for a multiple day "shoot", e.g. "Iceland March 2006". No hard and fast rules, just whatever makes sense. If I end up using CaptureOne, it will require its own sub folder structure to be created inside this folder – at least, if you prefer to keep things simple it will. Other RAW developers will leave settings files inside these folders too, so it is important to ensure these remain safe. I generally back up each folder to DVD – one or more, as necessary, although this is not ideal, really, since the backup only store settings files created up to that point. I also keep an automated working backup using Retrospect. I maintain two "master" RAW catalogs for Olympus ORF and Lumix Raw (converted to Adobe DNG) respectively, and these automatically update. I plan in the future to manage archive backup creation from these catalogs (using a script to record the archive volume names in each image's metadata), but I haven't got around to that yet.
Microsoft iView Media Pro
Stage 2: Preview
Once the files are organized, I create a local catalogue using MediaPro – by this I mean a catalogue of just the RAW files in the particular folder. I set MediaPro to produce the largest, highest quality previews it can create. I then use the comprehensive previewing and rating tools to decide which images I want to work further with, and how I want to categorise them (for example, I might well have "fun", "family" and, er, "art" shots from the same shoot. MediaPro 3's new Lightbox tool comes in extremely handy here, although it is very important to note that you are working with an 8-bit preview here, so the histogram for example is of the preview, not the 12bit RAW. However, it is easily good enough to show if an image is irrecoverably over- or under-exposed, and the method is fast and effective. At this point I will delete any files which are really trash, although if in any doubt, I keep them. Disk space is cheap. I end up with sorted, rated files, and I can even print contact sheets if I wish.
Stage 3: RAW Processing
For RAW Processing in the vast majority of cases I use Iridient RAW Developer (IRD). IRD, thankfully, has the good taste and common sense to be "just" a RAW developer, and has no pretensions to act as a full blown "workflow manager" (no, I'm not going to get sidetracked). IRD has drawn praise from a number of reputable sources, and is possibly the most full featured product in its category. Certainly it can't be beaten on sharpening options. Initially IRD seems very complex, but the complexity is only there if you need it, and it doesn't get in the way. It also often provides several ways to reach the same end, for example tone curves and tone sliders (I prefer curves, because Photoshop forced me to understand them, and Bruce Fraser's books explained them). IRD is especially highly rated by black & white aficionados, but, typically, I don't use it to B&W conversion. I tend to open RAW files in batches of related images, by selecting them in MediaPro, right-clicking and opening in IRD (this works as well for Olympus Studio or Camera RAW, but not very well for CaptureOne – ironically, since there is some sort of marketing alliance between CaptureOne and iView…well, there used to be, pre-Microsoft). Once the photos are open in IRD (note how I use "photo", "file" and "image" interchangeably), they can be selected from the open files drawer, and processed in turn. I won't go into detail on how I use IRD here, but my default settings for E-1 ORFs include using light "Difference of Gaussians" sharpening as "capture sharpening" – although I'm beginning to wonder if multi-stage sharpening is necessary considering the lack of artifacts IRD introduces – and using ProPhoto as the working colour space, and Joe Holmes' ExtaSpace as output space. My principal output files are 16 bit TIFFs, which I save in a temporary holding folder called "IRD Output". I also sometime process directly to JPG for print or web, but for my web galleries I use a set of Photoshop actions (coming later).
Iridient RAW developer
Stage 4: Post-processing
Currently I use Photoshop (.PSD) format as my final archive format, for fully processed images. There are several reasons for this, number one being layers, although even I'm not using any layers, I still save as PSD for consistency – if I see a PSD file, I know it is "finished", or at least it has been worked on. The second reason is that at least 25% of my output is still film-based, and my film workflow always culminates with Photoshop. I don't know of any compelling reason to change this practice. Therefore, I open all files from "IRD Output" in Photoshop, and at a minimum save them as PSDs on another disk volume, which is my "finished work" repository. Here, I simply maintain a folder for each month, and save the file into the current month's folder (I use MediaPro, not the filesystem, to catalog, so it doesn't matter if a photo I took in March 2005 ends up in the January 2006 "finished" folder). Depending on the photo, there are several things I might do in Photoshop. Generally, I will have sorted out tone in IRD, but I might run a local contrast enhancement action to see if it adds anything (I have evolved a variety, which act on different tonal ranges as necessary – usually I find excluding highlights is a good idea). Noise reduction is often required on E-1 files taken at 800 ISO or over, sometimes at 400 as well. For this I use the Noise Ninja plug-in. If I decide to convert to black and white, I use the Convert to BW Pro plug-in on a layer. I save files unflattened, although I might compact things a bit if they get out of hand. The "finished work" gets cataloged in my "reference" MediaPro catalog, where I add detail metadata, and construct various sub-catalogs and sets.
Stage 5: Output to Web and Print
Output requires sharpening and sometimes scaling. If I'm outputting to print, I take the PSD file, and set the output size as necessary. If the resolution drops a bit below 240dpi, I will scale up using Photoshop, but if it is well below, I use SizeFixer SLR. Once I get to the target size, I use Photokit Sharpener to apply output sharpening (note, for SizeFixer, it appears that sharpening before scaling is quite successful, but how Photokit's algorithms react to this, I'm not sure. Whatever – if it looks right, it is right). I then flatten the file, and save it as a copy to a temporary print folder, where ImagePrint picks it up. For my web galleries, I run an action which converts to 8bit, sRGB, then creates three different sizes of the file, appropriately sharpened, in an output folder hierarchy. The largest files are managed in a MediaPro catalog, and this is synchronized with my online mySQL database using a set of Applescripts, which glue MediaPro, MacSQL and Transmit FTP together. There are ways of using scripting additions to do the SQL and FTP parts, but they are complex, and not worth the trouble to me. The Applescript is very specific to my configuration, but I'm happy to send it to anybody who would be interested to see if they can adapt it.
So that's basically it. It takes longer to write about than to do it. The foundation stone is obviously MediaPro, which is a very powerful, but subtle application. The fact that Iridient RAW Developer constrains itself to doing one thing very well makes it very easy to introduce into a composite workflow. And Photoshop remains Photoshop... At some point I will follow up with my film-based workflow, but it isn't really so different.
