A RAW Workflow
in Olympus E-System , Thursday, August 03, 2006
Since I have a received a good few questions about my workflow, particularly from Olympus E-1 owners, I though it was maybe time to write something about it. Note, I do all of my digital photography on Macs, so I'm afraid this workflow is Mac specific, or at least the RAW conversion part is. I use three core tools on my workflow:
I also use Colorbtyte ImagePrint for printing, and FixerLabs SizeFixer for scaling up. For RAW processing, I sometimes use PhaseOne CaptureOne Pro and Olympus Studio. For Lumix LX1 RAW processing, I usually use Adobe Lightroom Beta. I only use Adobe Camera Raw in special cases, for example if I want to use Photoshop's HDR tool. In all cases, you can assume I'm using the latest versions of each application.
Stage 1: Ingesting
The first part of the workflow involves getting image files off the card and into the computer. I use MediaPro to import and rename the files, and to apply a basic personal metadata template. I don't really have a solid naming scheme, but I at least rename files to include a code indicating which camera the file comes from, and the date. I let MediaPro assign a serial number. Actually, the best renaming software I've seen is Olympus Studio, which allows you to use EXIF fields as components of the file name. The important thing is to have a unique name, and to keep it throughout the pipeline. In this way, it is possible to sync metadata in between different catalogues and different filetypes (so for example the captions from E1_20060603_0091.ORF and E1_20060603_0091.JPG can be synchronized. My filing system is quite simple: I have a master hard disk for RAW files, with folders for each month. Within these folders, I create subfolders whose structures depend a bit on the nature of the shoot. Usually, I create a folder for a single day, e.g "2006_06_03" (that's 3rd June…I'm European), but I might create a folder for a multiple day "shoot", e.g. "Iceland March 2006". No hard and fast rules, just whatever makes sense. If I end up using CaptureOne, it will require its own sub folder structure to be created inside this folder – at least, if you prefer to keep things simple it will. Other RAW developers will leave settings files inside these folders too, so it is important to ensure these remain safe. I generally back up each folder to DVD – one or more, as necessary, although this is not ideal, really, since the backup only store settings files created up to that point. I also keep an automated working backup using Retrospect. I maintain two "master" RAW catalogs for Olympus ORF and Lumix Raw (converted to Adobe DNG) respectively, and these automatically update. I plan in the future to manage archive backup creation from these catalogs (using a script to record the archive volume names in each image's metadata), but I haven't got around to that yet.
Stage 2: Preview
Once the files are organized, I create a local catalogue using MediaPro – by this I mean a catalogue of just the RAW files in the particular folder. I set MediaPro to produce the largest, highest quality previews it can create. I then use the comprehensive previewing and rating tools to decide which images I want to work further with, and how I want to categorise them (for example, I might well have "fun", "family" and, er, "art" shots from the same shoot. MediaPro 3's new Lightbox tool comes in extremely handy here, although it is very important to note that you are working with an 8-bit preview here, so the histogram for example is of the preview, not the 12bit RAW. However, it is easily good enough to show if an image is irrecoverably over- or under-exposed, and the method is fast and effective. At this point I will delete any files which are really trash, although if in any doubt, I keep them. Disk space is cheap. I end up with sorted, rated files, and I can even print contact sheets if I wish.
Stage 3: RAW Processing
For RAW Processing in the vast majority of cases I use Iridient RAW Developer (IRD). IRD, thankfully, has the good taste and common sense to be "just" a RAW developer, and has no pretensions to act as a full blown "workflow manager" (no, I'm not going to get sidetracked). IRD has drawn praise from a number of reputable sources, and is possibly the most full featured product in its category. Certainly it can't be beaten on sharpening options. Initially IRD seems very complex, but the complexity is only there if you need it, and it doesn't get in the way. It also often provides several ways to reach the same end, for example tone curves and tone sliders (I prefer curves, because Photoshop forced me to understand them, and Bruce Fraser's books explained them). IRD is especially highly rated by black & white aficionados, but, typically, I don't use it to B&W conversion. I tend to open RAW files in batches of related images, by selecting them in MediaPro, right-clicking and opening in IRD (this works as well for Olympus Studio or Camera RAW, but not very well for CaptureOne – ironically, since there is some sort of marketing alliance between CaptureOne and iView…well, there used to be, pre-Microsoft). Once the photos are open in IRD (note how I use "photo", "file" and "image" interchangeably), they can be selected from the open files drawer, and processed in turn. I won't go into detail on how I use IRD here, but my default settings for E-1 ORFs include using light "Difference of Gaussians" sharpening as "capture sharpening" – although I'm beginning to wonder if multi-stage sharpening is necessary considering the lack of artifacts IRD introduces – and using ProPhoto as the working colour space, and Joe Holmes' ExtaSpace as output space. My principal output files are 16 bit TIFFs, which I save in a temporary holding folder called "IRD Output". I also sometime process directly to JPG for print or web, but for my web galleries I use a set of Photoshop actions (coming later).
Stage 4: Post-processing
Currently I use Photoshop (.PSD) format as my final archive format, for fully processed images. There are several reasons for this, number one being layers, although even I'm not using any layers, I still save as PSD for consistency – if I see a PSD file, I know it is "finished", or at least it has been worked on. The second reason is that at least 25% of my output is still film-based, and my film workflow always culminates with Photoshop. I don't know of any compelling reason to change this practice. Therefore, I open all files from "IRD Output" in Photoshop, and at a minimum save them as PSDs on another disk volume, which is my "finished work" repository. Here, I simply maintain a folder for each month, and save the file into the current month's folder (I use MediaPro, not the filesystem, to catalog, so it doesn't matter if a photo I took in March 2005 ends up in the January 2006 "finished" folder). Depending on the photo, there are several things I might do in Photoshop. Generally, I will have sorted out tone in IRD, but I might run a local contrast enhancement action to see if it adds anything (I have evolved a variety, which act on different tonal ranges as necessary – usually I find excluding highlights is a good idea). Noise reduction is often required on E-1 files taken at 800 ISO or over, sometimes at 400 as well. For this I use the Noise Ninja plug-in. If I decide to convert to black and white, I use the Convert to BW Pro plug-in on a layer. I save files unflattened, although I might compact things a bit if they get out of hand. The "finished work" gets cataloged in my "reference" MediaPro catalog, where I add detail metadata, and construct various sub-catalogs and sets.
Stage 5: Output to Web and Print
Output requires sharpening and sometimes scaling. If I'm outputting to print, I take the PSD file, and set the output size as necessary. If the resolution drops a bit below 240dpi, I will scale up using Photoshop, but if it is well below, I use SizeFixer SLR. Once I get to the target size, I use Photokit Sharpener to apply output sharpening (note, for SizeFixer, it appears that sharpening before scaling is quite successful, but how Photokit's algorithms react to this, I'm not sure. Whatever – if it looks right, it is right). I then flatten the file, and save it as a copy to a temporary print folder, where ImagePrint picks it up. For my web galleries, I run an action which converts to 8bit, sRGB, then creates three different sizes of the file, appropriately sharpened, in an output folder hierarchy. The largest files are managed in a MediaPro catalog, and this is synchronized with my online mySQL database using a set of Applescripts, which glue MediaPro, MacSQL and Transmit FTP together. There are ways of using scripting additions to do the SQL and FTP parts, but they are complex, and not worth the trouble to me. The Applescript is very specific to my configuration, but I'm happy to send it to anybody who would be interested to see if they can adapt it.
So that's basically it. It takes longer to write about than to do it. The foundation stone is obviously MediaPro, which is a very powerful, but subtle application. The fact that Iridient RAW Developer constrains itself to doing one thing very well makes it very easy to introduce into a composite workflow. And Photoshop remains Photoshop... At some point I will follow up with my film-based workflow, but it isn't really so different.
- Microsoft iView Media Pro, for ingesting, preview, preliminary sorting, and cataloging
- Iridient RAW Developer, for RAW development and image processing
- Adobe Photoshop, for image post processing and output
I also use Colorbtyte ImagePrint for printing, and FixerLabs SizeFixer for scaling up. For RAW processing, I sometimes use PhaseOne CaptureOne Pro and Olympus Studio. For Lumix LX1 RAW processing, I usually use Adobe Lightroom Beta. I only use Adobe Camera Raw in special cases, for example if I want to use Photoshop's HDR tool. In all cases, you can assume I'm using the latest versions of each application.
Stage 1: Ingesting
The first part of the workflow involves getting image files off the card and into the computer. I use MediaPro to import and rename the files, and to apply a basic personal metadata template. I don't really have a solid naming scheme, but I at least rename files to include a code indicating which camera the file comes from, and the date. I let MediaPro assign a serial number. Actually, the best renaming software I've seen is Olympus Studio, which allows you to use EXIF fields as components of the file name. The important thing is to have a unique name, and to keep it throughout the pipeline. In this way, it is possible to sync metadata in between different catalogues and different filetypes (so for example the captions from E1_20060603_0091.ORF and E1_20060603_0091.JPG can be synchronized. My filing system is quite simple: I have a master hard disk for RAW files, with folders for each month. Within these folders, I create subfolders whose structures depend a bit on the nature of the shoot. Usually, I create a folder for a single day, e.g "2006_06_03" (that's 3rd June…I'm European), but I might create a folder for a multiple day "shoot", e.g. "Iceland March 2006". No hard and fast rules, just whatever makes sense. If I end up using CaptureOne, it will require its own sub folder structure to be created inside this folder – at least, if you prefer to keep things simple it will. Other RAW developers will leave settings files inside these folders too, so it is important to ensure these remain safe. I generally back up each folder to DVD – one or more, as necessary, although this is not ideal, really, since the backup only store settings files created up to that point. I also keep an automated working backup using Retrospect. I maintain two "master" RAW catalogs for Olympus ORF and Lumix Raw (converted to Adobe DNG) respectively, and these automatically update. I plan in the future to manage archive backup creation from these catalogs (using a script to record the archive volume names in each image's metadata), but I haven't got around to that yet.
Stage 2: Preview
Once the files are organized, I create a local catalogue using MediaPro – by this I mean a catalogue of just the RAW files in the particular folder. I set MediaPro to produce the largest, highest quality previews it can create. I then use the comprehensive previewing and rating tools to decide which images I want to work further with, and how I want to categorise them (for example, I might well have "fun", "family" and, er, "art" shots from the same shoot. MediaPro 3's new Lightbox tool comes in extremely handy here, although it is very important to note that you are working with an 8-bit preview here, so the histogram for example is of the preview, not the 12bit RAW. However, it is easily good enough to show if an image is irrecoverably over- or under-exposed, and the method is fast and effective. At this point I will delete any files which are really trash, although if in any doubt, I keep them. Disk space is cheap. I end up with sorted, rated files, and I can even print contact sheets if I wish.
Stage 3: RAW Processing
For RAW Processing in the vast majority of cases I use Iridient RAW Developer (IRD). IRD, thankfully, has the good taste and common sense to be "just" a RAW developer, and has no pretensions to act as a full blown "workflow manager" (no, I'm not going to get sidetracked). IRD has drawn praise from a number of reputable sources, and is possibly the most full featured product in its category. Certainly it can't be beaten on sharpening options. Initially IRD seems very complex, but the complexity is only there if you need it, and it doesn't get in the way. It also often provides several ways to reach the same end, for example tone curves and tone sliders (I prefer curves, because Photoshop forced me to understand them, and Bruce Fraser's books explained them). IRD is especially highly rated by black & white aficionados, but, typically, I don't use it to B&W conversion. I tend to open RAW files in batches of related images, by selecting them in MediaPro, right-clicking and opening in IRD (this works as well for Olympus Studio or Camera RAW, but not very well for CaptureOne – ironically, since there is some sort of marketing alliance between CaptureOne and iView…well, there used to be, pre-Microsoft). Once the photos are open in IRD (note how I use "photo", "file" and "image" interchangeably), they can be selected from the open files drawer, and processed in turn. I won't go into detail on how I use IRD here, but my default settings for E-1 ORFs include using light "Difference of Gaussians" sharpening as "capture sharpening" – although I'm beginning to wonder if multi-stage sharpening is necessary considering the lack of artifacts IRD introduces – and using ProPhoto as the working colour space, and Joe Holmes' ExtaSpace as output space. My principal output files are 16 bit TIFFs, which I save in a temporary holding folder called "IRD Output". I also sometime process directly to JPG for print or web, but for my web galleries I use a set of Photoshop actions (coming later).
Stage 4: Post-processing
Currently I use Photoshop (.PSD) format as my final archive format, for fully processed images. There are several reasons for this, number one being layers, although even I'm not using any layers, I still save as PSD for consistency – if I see a PSD file, I know it is "finished", or at least it has been worked on. The second reason is that at least 25% of my output is still film-based, and my film workflow always culminates with Photoshop. I don't know of any compelling reason to change this practice. Therefore, I open all files from "IRD Output" in Photoshop, and at a minimum save them as PSDs on another disk volume, which is my "finished work" repository. Here, I simply maintain a folder for each month, and save the file into the current month's folder (I use MediaPro, not the filesystem, to catalog, so it doesn't matter if a photo I took in March 2005 ends up in the January 2006 "finished" folder). Depending on the photo, there are several things I might do in Photoshop. Generally, I will have sorted out tone in IRD, but I might run a local contrast enhancement action to see if it adds anything (I have evolved a variety, which act on different tonal ranges as necessary – usually I find excluding highlights is a good idea). Noise reduction is often required on E-1 files taken at 800 ISO or over, sometimes at 400 as well. For this I use the Noise Ninja plug-in. If I decide to convert to black and white, I use the Convert to BW Pro plug-in on a layer. I save files unflattened, although I might compact things a bit if they get out of hand. The "finished work" gets cataloged in my "reference" MediaPro catalog, where I add detail metadata, and construct various sub-catalogs and sets.
Stage 5: Output to Web and Print
Output requires sharpening and sometimes scaling. If I'm outputting to print, I take the PSD file, and set the output size as necessary. If the resolution drops a bit below 240dpi, I will scale up using Photoshop, but if it is well below, I use SizeFixer SLR. Once I get to the target size, I use Photokit Sharpener to apply output sharpening (note, for SizeFixer, it appears that sharpening before scaling is quite successful, but how Photokit's algorithms react to this, I'm not sure. Whatever – if it looks right, it is right). I then flatten the file, and save it as a copy to a temporary print folder, where ImagePrint picks it up. For my web galleries, I run an action which converts to 8bit, sRGB, then creates three different sizes of the file, appropriately sharpened, in an output folder hierarchy. The largest files are managed in a MediaPro catalog, and this is synchronized with my online mySQL database using a set of Applescripts, which glue MediaPro, MacSQL and Transmit FTP together. There are ways of using scripting additions to do the SQL and FTP parts, but they are complex, and not worth the trouble to me. The Applescript is very specific to my configuration, but I'm happy to send it to anybody who would be interested to see if they can adapt it.
So that's basically it. It takes longer to write about than to do it. The foundation stone is obviously MediaPro, which is a very powerful, but subtle application. The fact that Iridient RAW Developer constrains itself to doing one thing very well makes it very easy to introduce into a composite workflow. And Photoshop remains Photoshop... At some point I will follow up with my film-based workflow, but it isn't really so different.
6 comments
Bernard Frangoulis August 12, 2006 - 10:42Step 1, ingesting
These days I am trying out PhotoMechanic, which I have come to like very much. It is very quick, and I like how it feels and works; plenty of nice usability touches everywhere. Of course using IVMP not only for cataloguing, but also for ingesting and rating the photos, makes one less piece of software to buy, learn and integrate into the workflow. But I am somehow in doubt about the future of iView with the MS acquisition. And Camera Bits (the makers of PM) say they are planning to add cataloguing capabilities to the program. We'll see... For now, since I still use IVMP for cataloguing, I am testing how metadata and ratings entered using PM are transmitted to IVMP.
Step 3: Raw Processing
- You are using DoG sharpening for capture sharpening; I must admit I have not explored it, I tend to use R-L Deconvolution; what advantages do you find in DoG?
- "although I'm beginning to wonder if multi-stage sharpening is necessary considering the lack of artifacts IRD introduces": same view here 😉
- You use Joe Holmes' EktaSpace as output space: no experience here, why this choice?
Bernard
6 comments
david mantripp August 14, 2006 - 3:03I don't know much about PhotoMechanic - it seems more geared to people taking a large number of photos, which I'm not. I don't have any issues with IVMP as far as ingestion goes. I suppose it could be better at renaming, but to be honest, I'm not sure it really matters what the files are called (I know I'm going against the flow here, but with IVMP I can work visually instead of using naming conventions, and that just seems obvious to me). As for the Microsoft issue, I don't think there is one. iView's business was founded on the Mac market, and therefore they must have had a strong Mac revenue stream. Microsoft is not a company to ignore a revenue stream. I'm not sure how much market the Windows version built up, but it certainly didn't reach the adoption levels seen on the Mac. I really doubt that there is much reason to fear for iView's future on the Mac. And if it happens, well I'll deal with it then.
I mainly use Joe Holme's Chrome 100 profiles when working with film, and when I decided to adopt a standard output space for archival, it made sense therefore to chose EktaSpace.
6 comments
Bernard Frangoulis August 14, 2006 - 4:08Thanks for your reply.
Joe Holmes profile 〈and its chroma variants〉 seems very interesting, I'll look them up. They seem to offer a wider gamut than Adobe RGB 1998, but not so extreme as ProPhoto RGB.
Any hint regarding DoG sharpening vs. R-L Deconvolution?
6 comments
david mantripp August 15, 2006 - 2:506 comments
scotth October 08, 2006 - 2:07I realize this is an older post, but I'e just recently found your site.
6 comments
David Mantripp October 09, 2006 - 6:43I'm just using iView to catalog, and I'm not writing anything back to the ORFs. If, one day, I wanted to migrate to a solution such as Aperture or Lightroom, this might help to ease the process - but I would certainly try a few experiments with some unimportant files first!