New Content Since Last Visit
No New Content Since Last Visit
This report presents the results of several weeks of experimentation with the [Originals] sub-folder looking for workarounds and opportunities to more effectively utilize Pro 3.0. Hopefully, some of what follows will help other Members as well.
INTRODUCTION
Pro 3.0's new non-destructive editing feature for Non-raw images uses a hidden sub-folder named [Originals]. Although it must be clearly acknowledged that the [Originals] sub-folder methodology does work quite effectively, it has also generated significant user objections, and has even caused some users to refuse to consider adopting Pro3.
The top complaints seem to be "I don't like my original images being moved" and "I don't want hidden sub-folders created throughout my image library". Objections from a more technical standpoint focus on sub-folders as a less than first-rate method of file management.
My main concern, and the reason for this report, is that the file handling approach with [Originals] is too rigid and inflexible. Like many Pro users, I have separate image libraries for Source Media, Work-In-Progress, and Finished Output. Unfortunately, techniques such as separating original images from Developed, or creating different output file types versus originals, are simply beyond the capabilities of the [Originals] design. Result - a lot of unnecessary copies of Developed images.
In spite of the above issues, I decided to go ahead with Pro3 in production for its undeniable improvements in image editing productivity. However, I also immediately began taking a hard look at the [Originals] sub-folders and experimenting with possibilities for changing or manipulating their operation. Following are the results from my tests which cover a number of relatively associated issues organized into a Question & Answer format.
Note: Although the [Originals] sub-folder operates for all Non-raw formats, there is strong emphasis in this report on JPEG, because it is a lossy format and special precautions are necessary to minimize image degradation.
EXPERIMENTS and TEST RESULTS
1. What determines Jpeg output QUALITY from Develop? There is an Options button which appears in the dialogue box during any Save As to Jpeg format. These same default Jpeg settings also control Jpeg output from Develop.
2. If a Jpeg image is Developed in several steps, is the entire process REALLY lossless? Yes. Each time a Jpeg is Developed, Pro3 starts with the original file, dynamically applies the previous settings, and then accepts additional changes. This can be repeated over and ever. Note that although the Develop "process" is lossless, the Jpeg output image is not lossless. When any Jpeg is saved, re-compression naturally occurs. To achieve 100% lossless output for a Developed Jpeg, users can either perform a Save As while in Develop Mode to a lossless format (e.g. Png or Tiff), OR, for lossless Jpeg batch conversion options, see #3 - #4 below.
3. Can BATCH FILE CONVERT losslessly create Png's or Tiff's from Developed Jpegs? No. This is a critical point for users who wish to losslessly Develop Jpegs, then convert to Png or Tiff. Batch File Convert does NOT dynamically recreate the images from their originals and XMP files as Develop does. Instead, it uses the Developed output files as it's source data. So, any Png or Tiff files created from Developed Jpegs using Batch File Convert, will not be lossless.
4. Is there ANY lossless method to batch convert Developed Jpegs, and avoid Save As? Yes. A lossless batch method for converting from Jpeg to Png or Tiff is available by using Batch Develop and its Export Option. This technique requires that users first create a Batch Develop setting called a "NUL Preset". A Nul preset doesn't actually do anything, but Batch Develop requires a preset to function. In other words, by running Batch Develop with a "do nothing" preset, Pro3 will dynamically recreate the Jpeg images and provide access to the Option to "Export files to another format". This technique will deliver true lossless conversion from Jpeg's to Png's or Tiff's in batch mode.
To create a NUL Preset, enter Develop, then on the blue bar labeled DEVELOP, click the gear-shaped button, then click "Save Preset". Enter a preset name e.g. "Nul", and click OK to finish.
5. Is there any way to "Restore to Developed" after using an External Editor e.g. Photoshop? Yes. It turns out that the very same Batch Develop with a Nul preset discussed in #4 above, also works perfectly as a "Batch Restore to Developed". For those using External Editors e.g. Photoshop, this could be a very handy tool to have.
6. When all Develop activity is complete, can the [Originals] sub-folder be deleted? Yes. The thinking here is that the [Originals] sub-folder can be viewed as a long-lasting "Undo/Redo" cache. This is the core concept of non-destructive editing which permits users to Develop images in multiple steps. By leaving the [Originals] sub-folder in place permanently, the ability to return to non-destructive editing remains open-ended.
However, there are numerous reasons why the [Originals] sub-folder and its Undo/Redo ability, might lose its value. Some users simply have no interest in any additional image editing after a project is complete. Other users may do additional editing work with External Editors (e.g. Photoshop) effectively locking-in prior Develop changes from Pro3. These and various similar situations reduce or eliminate the need for the [Originals] sub-folder and its Undo/Redo data.
7. How does Pro3 react when the [Originals] sub-folder is deleted? Can Pro3 become unstable? Although deleting [Originals] is not openly supported by Pro3, it became very clear in the course of my testing that the Pro3 Developers anticipated this eventuality. During my "crash tests", I literally tried every "dirty trick" imaginable and I could not get Pro3 to crash a single time. I deleted XMP files, but saved the originals. I deleted the originals, but saved the XMP files. I deleted the entire [Originals] sub-folder. With all of these variations of intentionally deleted files, I performed every available Pro3 function including: Manage, View, Process, Copy/Paste, Batch functions, you name it.....not one Pro3 crash.
I did get some "confused" thumbnails, and one humorous result. If you delete an original, but leave the XMP file, Pro3 will reapply the Develop parameters to the former output image, effectively doubling up the changes. The bottom-line here is that the Developers appear to have wisely anticipated that deletions in the [Originals] sub-folder might occur, either by accident or by user intervention. They took special steps to protect Pro3 from such occurrences and did an excellent job of it based on my test results.
8. Will the "D" indicator still be visible on the thumbnails if the [Originals] sub-folder is deleted? Yes, actually. The D indicator is apparently kept in the Pro3 database and not in the XMP file. After [Originals] data or the entire sub-folder are deleted, the "D" will remain on Developed images. The two ways I found to remove the "D's" are to run Rebuild Thumbnails, or to move the images using Cut & Paste. The "D's" will be preserved during file moves by using either Drag & Drop, or Copy & Paste followed by a manual delete of the source files.
9. Which software should be used to accomplish image moves and [Originals] file deletions? Pro3 and Windows Explorer should be used together, complimenting each other's functions. Pro3 is aware of the "three-file relationship" for each Developed image (original image, XMP file, output image). In Pro3 file operations, moves, copies, and deletions will always involve all three associated files. Windows Explorer is not aware of any such relationships and operates strictly on a file-by-file basis. Users can take advantage of these differences in file handling to achieve desired results.
10. Is there a danger of duplicate file names if "Working Copies" of original images are MOVED? Yes. And, this probably doesn't need to said, but just for the record...... It is common for users keep a master backup copy of original images. In this situation, users are effectively performing their Develop functions on "working copies" of the originals. The obvious danger here is that the Developed output images will have the identical filenames as the originals. Running Batch Rename to create some minor naming difference in the working copies will prevent filename duplication. This will also allow the final Developed images to be moved back into the folder with the real originals, if the user chooses to do so.
WORKFLOW EXAMPLES
The following examples demonstrate the basic [Originals] deletion techniques which came out of my testing. From these examples, interested users can adapt their own approaches which match their unique workflows and image management practices.
Workflow Example #1 Introduction: This simple workflow example begins with Pro3's non-destructive editing, then concludes as a classic "Edit In Place" destructive model. Objective: Develop selected images in place. When complete, "lock-in" all the newly Developed images, delete all the saved [Originals] data, retain the "D's" on the thumbnails, and leave both Developed and non-Developed images in the initial folder (e.g. no file moves).
Solution: Use Windows Explorer to delete the entire [Originals] sub-folder.
Comments: Note that deleting the [Originals] sub-folder only deletes the originals of the Developed images, but has no impact on any un-Developed images. The "D's" can optionally be removed by running Rebuild Thumbnails.
Workflow Example #2 Objective: Same as #1 except, add the requirement that a select group of the Developed images are to remain available for additional editing with their "D's" retained and their [Originals] data in tact. All others are to have their "D's" removed and their [Originals] data deleted. Undeveloped images are to be left untouched. There are two different solutions for this example.
Solution #1: Using Pro3, Select the Developed images for which the [Originals] data is to be deleted, then click Edit/Copy. Using Windows Explorer, select a TEMP folder, and click Edit/Paste. Next, return to Pro3 and Delete the selected images. Using Windows Explorer again, select all the TEMP images and click Edit/CUT. Finally, return to Pro3 in the initial folder, and click Edit/Paste.
Comments: By using Pro3 to select and copy Developed images, then using Windows Explorer to paste them into the TEMP folder, the 3-file linkage is broken and only the Developed images are actually copied. Conversely, by using Pro3 to perform the selected deletions, only the [Originals] files for the selected group are deleted, leaving all others untouched.
Solution #2: Using Pro3, select the Developed images for which the [Originals] data is to be saved, then click Edit/Copy. Still using Pro3, select the TEMP folder and click Edit/Paste. Return to the initial folder, then use Windows Explorer to delete the entire [Originals] sub-folder. Use Pro3 in the initial folder to run Rebuild Thumbnails to remove all the "D's". Finally, using Pro3 in the TEMP folder, select the saved images, click edit/cut, and use Pro3 to paste them back into the initial folder.
Comments: This second solution saves certain images, deletes [Originals] data for all the other images, then copies back the saved images and their [Originals] data. This may be a simpler solution if only a very few images are to retain their data for additional Develop activity.
Workflow Example #3 Introduction: This example introduces three additional factors: (1.) Creating working copies of originals in a Work-in-Progress (WIP) folder, (2.) performing Develop work in the WIP folder, and (3.) moving the final Developed images to some "other" final destination folder. The optional destination will be determined by the user's image library structure and file management practices.
Objective: Create a set of "working copies" of original images. Review images, select some to Develop, leave others undeveloped. When complete, move only the Developed images to the destination folder, remove the "D's", delete undeveloped working copies and all [Originals] data.
Solution: Copy a full set of the original images into the working WIP folder. Run Batch Rename using a Template (for example, *-1) to slightly alter the filenames. Review and select images, then Develop normally. When complete, use Pro3 to Select and Edit/Copy only the Developed images. (Tip: On the Pro3 File List Toolbar - click "Group", then click "Processed State"). Then, use Windows Explorer to Edit/Paste in the Destination folder. Lastly, delete all remaining data in the working WIP folder.
Comments: This example, and #4 below, work well for users who object to their originals being moved. They begin with non-destructive editing, then conclude as a classic "Save As" model. Example #3 concludes with a destructive final move to the destination folder, while #4 below does the opposite.
Workflow Example #4 Objective: Same as #3 except, preserve the "D's" and the [Originals] data for some, or possibly all, of the Developed images.
Solution: Same procedures as #3 until the final file move to the destination folder. Use Pro3 to select Developed images, and use Pro3 for the Cut & Paste to move them to their destination folder.
Comments: This variant of #3 retains all the [Originals] data to allow additional Develop activity at a later time. By mixing the final copy techniques of Examples #3 and #4, users can achieve a combination of destructive and non-destructive editing.
Example #5 Introduction: This example outlines a novel solution to a situation that affects only a very limited number of users. Unfortunately, I am one of them. This final example is only relevant if all three of the following circumstances exist: (1.) The source media consists of significant numbers of large Developed Png or Tiff files which are each 10mb+ in size. (2.) The equally large output images created by Develop are essentially not used. Instead, faster, smaller Jpeg copies are created for viewing in a separate Finished Output folder. (3.) Maintaining the option to potentially do additional Develop work is a requirement.
My Pro3 production experience revealed that what started out as just a bunch of large, but unused Png or Tiff files required by Pro3, grew rather quickly into a major multi-gigabyte problem. Therefore, this final experiment focused on the possibilities of somehow "compressing" these very large Developed output files for Png and Tiff images without disturbing Pro3's ability to return to Develop. There were several possible solutions, but the following was the most technically interesting and actually the most effective.
Solution: Select the target Png or Tiff output images. Run Batch File Convert to create lowest quality Jpeg images - send the output to a TEMP folder. Select all the new Jpegs in the Temp folder and run Batch Rename (Search & Replace) to change the file endings to Png or Tiff matching the source images. Using Windows Explorer, Cut & Paste the renamed Jpegs back to the initial folder replacing the Developed output files. It turns out that the "D's" will be preserved and the compressed Jpegs "masquerading" as Png's or Tiff's works quite well.
