Commentary on "DNG is not the answer"

(Juergen Specht was asked to check this page for errors. His reply didn't identify any errors).



About one hour after posting a link to this response in an Adobe forum, Thomas Knoll responded: "Barry's response ... to this rant on the OpenRAW mailing list is very good (and accurate)". (4:58am Apr 4, 06).


Who said "DNG is not the answer"?

The OpenRAW team have been cool towards DNG since they launched their web site at the end of April 2005. Just after they redesigned their web site at the start of April 2006, they prominently published a large article "DNG is not the answer" by Stuart Nixon. (Juergen Specht said "I will publish Stuart's mail regarding DNG etc. as article, he gave me the permission"). This article was edited and renamed "Notes on the future of Open RAW formats, and a look at DNG" a few days later.

It is not clear what question Stuart Nixon thought DNG is not the answer to! Based on OpenRAW's motto, ("Digital Image Preservation Through Open Documentation"), perhaps the question he was (rightly) concerned with is "what do we do to achieve digital image preservation?"

This page is a response to some of the misleading parts of the edited version of the article. The format is to use 2-column tables, with some of Stuart Nixon's statements in the left column, and a response to those in the right column. Except where there are other specific points to address, the responses are concerned with "what do we do to achieve digital image preservation?"


Response to the preamble

(These are in the first 15 sentences of the article).

Stuart Nixon's statement Commentary on that statement
DNG is not an open standard in that it does not document all the essential information contained in current RAW format files like NEF and CR2 (which also don't document this information).

DNG documents what it knows. It isn't the fault of DNG that camera manufacturers haven't published their private information! (So they are not the answer either). Adobe have already documented that they move NEF and CR2 (and some other) Makernotes to DNGPrivateData. If Nikon and Canon document that information of NEFs and CR2s, this will also provide the documentation of DNGPrivateData.

If cameras themselves output DNG, we need open documentation of all the "private" bits in the DNG files, and OpenRAW might help that. This is the ideal target for raw shooting. A few camera manufacturers provide their own DNG Converters, which should help ensure comprehensive conversion.

In many ways, DNG can be viewed as simply yet another RAW format with undocumented information - except that DNG has the added risk that information can be lost during conversion to/from DNG and other RAW formats.

DNG is vastly more than "yet another RAW format"! It contains a camera profile, describing colour and other characteristics of the camera, so that it is self-contained. It is the only raw file format designed to be suitable for archiving.

Does information loss occur? When we have openly published documentation for those other raw formats, we will know for sure! Adobe's DNG Converter is constrained by the best information Adobe has available to it. Where Adobe knows lots about a camera's raw file format, it can ensure that its DNG Converter preserves all of the data. Where Adobe doesn't know enough, yes, there is a risk that data will be inadvertently lost. Openly published documentation will enable Adobe, and others, to write DNG converters that will preserve all of the data, and the users of those converters will know what has been preserved.

From a software developer point of view, DNG is a step backwards.

That is bullshit! My qualifications for saying so are here:
An informal summary of my curriculum vitae

From a camera manufacture's perspective, DNG does not address the missing elements in EXIF. See the fuller response below. Also: Exif - equivalent XMP parameters.
From a photographers perspective, DNG is dangerous because people believe they are storing for the future with the format, when nothing could be further from the truth. The real danger is that, even where DNG can safely be used to help store for the future, many photographers are too suspicious of it to use it. There is a lot of unreasonable "fear, uncertainty, doubt" about DNG, and many photographers are probably making trouble for themselves in future. Open publication of the original raw file formats will enable photographers to make better-informed decisions about whether DNG is storing the data they care about.

Response to "Important Formats and Standards for Photographers"

Stuart Nixon's statement Commentary on that statement
DNG - Bad:
Takes the proprietary RAW format problem and makes it much worse, with MakerNotes et al now being moved about. See the fuller response below.
Offers the option to include the entire original raw file - but unless everyone actually uses this (thus doubling file size) just gives a false sense of security.

It is fascinating to see DNG being criticised for offering an option that some photographers use so that they can retrieve the original raw files later! No one would be better off, and some people would be worse off, if this option were removed. So how is this "bad"?

Makes no attempt to define a standard way to store all the data currently stored in undocumented ways in MakerNotes.

It is fascinating to see DNG being criticised for not solving a problem that many companies and organisations have demonstrably failed to solve over several years! At least DNG doesn't appear to have made things worse.

But potentially DNG has made things better. DNG offers the ability to hold that metadata in XMP format, with appropriate XMP-namespaces. This worked for IPTC metadata, so perhaps could it also assist with Exif metadata. We know that some Exif -type information can be represented in XMP form, so perhaps lots more can. This certainly can't be criticised as "no attempt"!

Takes MakerNotes and moves them into another format, both perpetuating the problem and making it worse (as decoders often rely on absolution location information in Makernotes).

If Makernotes rely on absolute location information, for example offsets, then Adobe can't solve that particular aspect of the preservation problem. DNG can't cater for bad raw file formats - we need to eliminate bad raw file formats. Such badly-behaved Makernotes make it hard to use the raw file as a repository for all the rights management and asset management metadata that is also needed for image preservation.

DNG provides more than enough options to solve this problem - the onus is upon the provider of the raw information. Adobe appear confident that the Makernotes in NEFs and CR2s and some others are safe to move in this way, so in those cases this is actually not an issue.

Makes no attempt to extend EXIF or TIFF/EP in a coherent fashion.

DNG is addressing aspects of the Exif problem, see the fuller response above.

In effect, DNG is TIFF/EP brought up to date and made fit for purpose.

Controlled by a single manufacturer - Adobe - who given don't have a good history of managing the TIFF standard or allowing software developer of Adobe controlled software profiles or standards.

The TIFF 6.0 specification has been owned by Adobe for more than a decade. TIFF 6.0 is used successfully hourly by lots of cameras and lots of photographers and lots of software products and lots of service and publishing organisations. Adobe fund forums to help all of those.

Response to "What does everyone want?"

Stuart Nixon's statement Commentary on that statement
Photographers:
Want a way to store photos that will be around in 50 years. This requires open publication of image formats plus critical mass of key formats. It is thwarted by any proliferation of image formats. Just as important is the ability to find them among the vast accumulation of images and other data. DNG supports asset management metadata. Tools exist that can catalogue and search on this metadata.
Want to be able to use any product to edit any photo. True, and lots more. They want to be able to have a workflow of their choice, and populate it with tools of their choice. Those tools may include ingestion products, raw converters, viewers, asset managers, photo-editors, etc.
Large software vendors:
(This section varies from one vendor to another. Stuart Nixon is discussing DNG, therefore is probably talking about Adobe, so the commentary below will do the same).
Want to have the standard under their control. Some of Adobe's specifications have been developed into ISO standards. ISO developed TIFF/EP from TIFF 6.0, and variants of PDF such as PDF/A from PDF. The latter includes XMP.
Want a legal hold over potential competitors.

Adobe have published a royalty-free license for anyone to develop and supply products using the DNG specification. Adobe have released a freely-available (optional) SDK, and a royalty-free license for anyone to use it.

(They own the copyright of the DNG specification, and the trademark of the DNG-icon, but so what?)

Want to limit use of photos on competitive OSs/products.
Want hidden "submarine" patent/legal/copyright control over standards.
Small software vendors:
Want a common and uniform standard. There is evidence against that generalisation. In a Raw Magick user forum, Iliah Borg said "It makes little sense to force a standard file format on companies, which will instead stifle their innovation". In a Bibble forum, Eric Hyman said "... but no that doesn't mean we will support DNG for cameras that don't nativly write it".
Want protection from legal actions from large software vendors. Adobe have published a royalty-free license for anyone to develop and supply products using DNG.
Enlightened camera companies:
Must be able to extend the standard as technology evolves. The standard needs to be extensible. But by whom? Stuart Nixon himself has said, "Standards groups are almost by definition several years behind the "state of the art" in any given industry". This suggests that standards in a fast-moving industry are likely to be de facto standards, not controlled by such a slow-moving standards group.
Don't want to have class actions against them in future years by people losing access to their photos. Surely that applies to all of the people and organisations and companies here? A raw file format that is openly-documented, freely-available, and has "critical mass", will offer economies of scale to make it likely that it will be supported in decades to come.
Other camera companies:
Want to make money out of their own software and compete with software vendors. That applies to Nikon. Does it apply to any of the others?
Additional people not mentioned by Stuart Nixon - Users of photographs:
A fully portable raw container to enable such a user to handle raw files from 100+ cameras. This needs a common raw format which is either used natively by cameras and digital backs, or is one to which other raw files can be converted. DNG is the only contender.
A self-contained raw file format that doesn't need such a user's tools to have awareness of the parameters of any particular camera model. DNG is designed in this way. It contains a camera profile, describing colour and other characteristics of the camera, for that reason.
A raw format that also supports management information such as rights management metadata and asset management metadata. DNG supports XMP, which can handle such metadata. Tools exist that support XMP within DNG for this purpose.