Article: A Format for Digital Preservation of Images: A Study on JPEG 2000 File Robustness

For those of you interested in JPEG2000, you may want to read this article. Abstract:

Digital preservation requires a strategy for the storage of large quantities of data, which increases dramatically when dealing with high resolution images. Typically, decision-makers must choose whether to keep terabytes of images in their original TIFF format or compress them. This can be a very difficult decision: to lose visual information though compression could be a waste of the money expended in the creation of the digital assets; however, by choosing to compress, the costs of storage will be reduced. Wavelet compression of JPEG 2000 produces a high quality image: it is an acceptable alternative to TIFF and a good strategy for the storage of large image assets.

Moreover, JPEG 2000 may be considered a format that can guarantee an efficient robustness to bit errors and offers a valid quality with transmission or physical errors: this point of view is confirmed by the case study results that we report in this article, concerning image quality after occurrence of random errors by a comparison among different file formats. Easy tools and freeware software can be used to improve format robustness by duplicating file headers inside or outside the image file format, enhancing the role of JPEG 2000 as a new archival format for high quality images.

---

Technorati tag: JPEG2000

Presentation: JPEG2000 for Archives and Libraries

The slides for this presentation - JPEG2000 for Archives and Libraries - are online. Both Dr. Robert Buckley and Justin Dávila delivered information in this session. It's easy to flip through the slides, so I'll not excerpt anything here. Be sure to look at the definitions of JPEG2000.

---

Technorati tag: JPEG2000

Slides from Archiving in Practice with JPEG2000 available online

Thanks to Peter Murray for posting this info to a discussion list.

The presentation slides are now online from a session hosted by the JPEG2000 in Archives and Libraries Interest Group of the LITA division of ALA. The session was Archiving in Practice with JPEG2000.

---

Technorati tag: JPEG2000

Press Release: JPEG 2000 a great step forward for the archival community

I've been keeping my eyes open for JPEG2000 info and this crossed my desk this morning.

---

JPEG 2000 a great step forward for the archival community

The Digital Preservation Coalition has examined JPEG 2000 in a report published today. The report concludes that JPEG 2000 represents a great stride forward for the archival community. The format now allows for greater compression rates and a recompression rate that is visually lossless.

The findings come as the Digital Preservation Coalition launch its latest Technology Watch Report written by Dr. Robert Buckley, a Research Fellow with Xerox, ‘JPEG 2000 – a practical digital preservation standard?’. The report looks in-depth at the new format and the challenges it has to cope with. JPEG 2000 is widely used to collect and distribute a variety of images from geospatial, medical imaging, digital cinema, and image repositories to networked images. Interest in JPEG 2000 is now growing in the archival and library sectors, as institutions look for more efficient formats to store the results of major digitisation programmes.

The report is aimed at organisations involved in the management and storage of digital information. The in-depth report will help archives, libraries and other institutions make informed decisions about JPEG 2000 format and their future storage needs.

JPEG 2000 can reduce storage requirements by an order of magnitude compared to an uncompressed TIFF file. Dr. Buckley says, “This new format has come at a time of heightened awareness about the access to digital documents. Any format that can assist archives and libraries to do this is welcome.”

The format will also enable users to open as much of the file as they need at that time. This means a viewer, for example, could open a gigapixel image almost instantly. This is achieved by retrieving a decompressed low–resolution display sized image from the JPEG 2000 codestream. Coupled with this, the users’ ability to zoom, pan and rotate an image have been enhanced.

Adrian Brown, head of digital preservation, The National Archives said: “This is a very timely addition to the DPC's Technology Watch Report series as many organisations are themselves reviewing the JPEG2000 format. This concise, comprehensive and clear guide will be of interest to practitioners across the digital preservation community.”

The report concludes that JPEG 2000 offers much more flexibility and features than JPEG, but at the cost of greater complexity. It is however a great stride forward, and of major significance for the information management community.

To download a pdf of the report please go to: www.dpconline.org/graphics/reports/index.html#twr0801

For further information please contact, Tim Matthews, tim.matthews@nationalarchives.gov.uk, or 020 8392 5277.

For further information on the DPC please contact, Frances Boyle, fb@dpconline.org or 01904 435320.

About The Digital Preservation Coalition (DPC)

The Digital Preservation Coalition (DPC) is a cross sectoral member organisation established in 2001 to foster joint action to address the urgent challenges of securing the preservation of digital resources in the UK and to work with others internationally.

---

Technorati tag: JPEG2000

Article: What is...JPEG?

The February 2008 issues of the Notices of AMS includes a three-page article by David Austin, professor of mathematics at Grand Valley State University, that explains how JPEG and JPEG2000 (J2K) compress image data. Austin also wrote an article in 2007 about JPEG entitled “Image Compression: Seeing What’s Not There.” Both article contain many formulas and are not for the mathematically challenged. Anyone, though, can glean useful information especially towards the end of each article, when he returns to looking at the big picture (no pun intended).

In talking about J2K, Austin notes that its "principal advantage lies in providing a much more flexible format for working with images in environments where the increased complexity [of the format] is not problematic." In other words, he is saying that one should not automatically choose J2K over JPEG. According to him, each have their advantages.

Thanks to Peter E. Murray for pointing out current article.

---

Technorati tag: JPEG2000

JPEG2000 (J2K) vs. TIFF

I've just been looking at a presentation Ronald J. Murray, from the Library of Congress, did at MCN 2005 on J2K. In it, Murray compares J2K with TIFF, using information that is familiar to some about photography (e.g., Halide-Based Imaging Systems).

In the presentation, he notes that TIFF is like taking a photograph, where you may make some adjustments to the image when it is taken, but then fix the image and make it permanent. Derivatives are made by making copies of the master.

He then talks about J2K as being a format where much data is captured, then

File management and delivery system meters out image quality (up to a maximum established or available) on demand

Notice that "creating" different quality access images with J2K doesn't require creating derivative images (files), but instead relies on the delivery systems for displaying on-demand what the user wants.

You can view Ron Murray's complete presentation here.

---

Technorati tag: JPEG2000

Comments on JPEG2000 blog post

For those who are interested in JPEG2000, the blog post I wrote on Tuesday has several very good comments. If that file format is of interest to you, please read the comments.

BTW I have been Downstate since Wednesday (i.e., New York City and Long Island). Yesterday I did an "Introduction to Second Life" workshop that had 57 participants. Today I attended the Regents Advisory Council on Libraries (RAC) meeting. RAC is a group that should be of interest to every librarian in NYS, so I'll blog about the RAC meeting later. (And yes, the topic of digitization did come up.)

---

Technorati tag: JPEG2000

Quick thought about JPEG2000

I promised myself in October that I would spend time reading and learning about JPEG2000, and that I would blog about the standard. The end of November is almost here and I've done none of that. Sigh.

Reading the technical details about the standard takes me back to my days in IT, when I would spend hours reading about something technical, then run a few "experiment" in hopes of mastering it (or more likely fix a problem that had been tossed my way). What I need are the "Clif Notes" or the quick whiteboard session with Scotty from Star Trek to cut through and tell me what I really need/want to know about JPEG2000. (Or maybe I need the "Easy Button" from Staples.)

Here's what I do know about JPEG2000:

• It should have been named something different because comparing it to the JPEG we know and love doesn't do it justice.
  
• It is more versatile that I suspect most people realize.
• It is should not be feared.
• We should be using it.
• The software for using/creating JPEG2000 needs to be more widely available/known.
• Library consortia (and others) should be offering mini-workshops on JPEG2000 to help people feel comfortable with it and use it.
  
• It needs someone to do something really, really cool with it in order for people to stand up and take notice (and adopt it).
  

BTW I'm off on another business trip tomorrow and I will again promise myself to read the documents I have on JPEG2000, although staring out the train window on my way to NYC and Long Island sounds good too.

---

Technorati tag: JPEG2000

Press Release: Library of Congress Collaborates with Xerox to Test Format for Digitally Preserving, Accessing Treasured Images

The Library of Congress and Xerox Corporation are:

studying the potential of using the JPEG 2000 format in large repositories of digital cultural heritage materials such as those contained in the Library and other federal agencies. The eventual outcome may be the creation of leaner, faster systems that institutions around the country can use to store their riches and to make their collections widely accessible.

Later the press release states:

The images to be used from the Library's collection are already digitized (primarily in TIFF format), but JPEG 2000, a newer format for representing and compressing images, could make them easier to store, transfer and display. According to Michael Stelmach, manager of Digital Conversion Services in the Library's Office of Strategic Initiatives, JPEG 2000 holds promise in the areas of visual presentation, simplified file management and decreased storage costs. It offers rich and flexible support for metadata, which can describe the image and provide information on the provenance, intellectual property and technical data relating to the image itself.

Xerox scientists will develop the parameters for converting existing TIFF files to JPEG 2000 and will build and test the system, then turn over the specifications and best practices to the Library of Congress. The specific outcome will be development of JPEG 2000 profiles, which describe how to most effectively use JPEG 2000 to represent photographic content as well as content digitized from maps. The Library plans to make the results available on a public Web site.

This is very good news since it will help members of the cultural heritage community (libraries, archives, etc.) understand the JPEG 2000 format.

By the way, it is a shame that the format was named "JPEG 2000", since it is different than the JPEG format we all now and the "2000" puts a date stamp on the format that we won't appreciate in a few years. It is abbreviated J2K or JP2, and maybe in time we'll just use one of those abbreviations so that the reputation of JPEG isn't attached to JPEG 2000.

---

Technorati tag: JPEG2000

Discussion of the JPEG 2000 standard

My post on Oct. 4 started a productive conversation in email and by phone about JPEG 2000. My plan is to post more here about JPEG 2000, so that a larger number of people understand the the JPEG 2000 standard better. Unfortunately, when one talks about standards, the conversation migrates to intense technical aspects, so I hope to be able to provide information on JPEG 2000 in a way that is more accessible to everyone. (This will take time to do, so please be patient.)

Tim Vitale and Ron Murray did follow-up posts yesterday to the IMAGELIB discussion list on this topic and the conversations that have occurred. Tim's summarizes what occurred over the last 1+ weeks and Ron's provides some additional information. Tim has reported that he is going to design some "exquisite bit of torture for J2K" lossless compression and report back on his work. It is clear that this testing will help to clear up some of the possible misinformation about the standard.

I want to thank Tim, Ron and several others who have participated in the email conversation on this topic. It was indeed a conversation that included lots of sharing, which was extremely beneficial to all. I know we'll continue to be in contact as we find ways of making more information on JPEG 2000 available to people (in a way that is easily understandable).

---

Technorati tags: Digital Preservation, jJPEG2000

JPEG2000 -- A follow-up post to "Articles on long-term file access..."

[10/5/2007, 1:45 p.m. please see addendum below as well as note the inserted new text.]

I received an very interesting email comment on the post I wrote entitled "Articles on long-term file access and digital preservation." It was pointed out the the Vitale article reportedly contains some inaccuracies as well as out-of-date information on JPEG 2000. I obviously don't know the details of JPEG 2000 as well as Ronald J. Murray (Digital Conversion Specialist at the Library of Congress) does, and so I appreciate him taking the time to give me his perspective. (I've since also heard from Tim Vitale, who wonders what the specific inaccuracies are.) Ron Murray pointed to two resources for learning accurate information about JPEG2000:

• JPEG 2000 in Archives and Libraries, http://j2karclib.info
• JPEG 2000 related blogs at Disruptive Library Technology Jester, http://dltj.org/ (Peter Murray, the Jester, also contributes to the first site.)

The Charles Olson Research Collection, on which Ron Murray consulted, used JPEG2000 and there is a case study on that web site about their use of JPEG 2000. The web site for the project notes that:

To our and his knowledge, the Delmas-supported CHARLES OLSON'S MELVILLE PROJECT is the first project in an academic library or archives to use this new image standard.

Ron Murray said that people in the library & archives communities generally don't have the engineering and mathematical literacy in order to understand JPEG 2000's capabilities. He, however, encourages us to read more about the format and become comfortable with it.

If you have had experience with JPEG 2000, please leave a comment and tell us about it. Please take a moment to teach us what you've learned. (Or if politics prohibit you from leaving a comment, email me at hurst [at] hurstassociates [dot] com)

ADDENDUM (10/5/2007, 1:45 p.m.): I want to publicly apologize to Tim Vitale for not checking with him first before publishing this post. That was bad etiquette. Evidently different communities are arguing over JPEG 2000 (JP2K) and whether or not it should be adopted. Here are my questions:

• Are there specific uses that people are disagreeing over such as JPEG 2000 for video?
  
• Is the problem/concern when JPEG 2000 is used instead of a TIFF, rather than just being a replacement for the JPEG file?
  
• Is the problem that the library community determined to use this format, no matter what?

At this point, I would enjoy hearing from others who have knowledge and experience with JPEG 2000. I have been given the name of someone who worked on the original committee and guess I better quickly contact him for his opinion!

---

Technorati tag: Digital Preservation

Blog post: JPEG2000 for digital preservation

Peter Murray has written in his blog, Disruptive Library Technology Jester, about the use of JPEG2000 for digital preservation. Murray sees JPEG2000 as a suitable replacement for TIFF and offers up five reasons why he believes this to be so. His fifth reason is:

The JPEG2000 is an open standard with defined and emerging protocols for guaranteeing compliance with the standard.

Later he says:

One of the concerns about JPEG2000 is some language from the JPEG2000 website about how “undeclared and obscure submarine patents may still present a hazard…” to open use of the standard. This seems like lawyer CYA to me as nothing has come up that I’m aware of in the seven years after the standard was ratified...And, if in the end it is found that a patent would cause an embargo ‘unlicensed’ versions of JPEG2000 codecs for some period of years, we can always run a batch conversion back to TIFF until the embargo period is up and/or something else better comes along.

More programs are using JPEG2000, including the Princeton University Library. On their web site, it states:

For the most part, we'll be deriving JPEG2000 images from the master TIFF files. JPEG2000 is a recently-developed imaging standard that is based on wavelet technology. Wavelets allow a great deal of end user functionality (like zooming, panning, etc.), while retaining small file sizes and little loss from a great deal of compression. JPEG2000 has its own security model, and can allow for metadata to be stored internally to the image, both of which may prove very valuable for retaining intellectual property rights over materials in the coming years.

JPEG2000s are still not viewable in most browsers, so we've acquired the Aware JPEG2000 server software for dynamically displaying JPEG2000s as JPEGs, while still retaining the same flexible end user interaction and tools.

In thinking about storing metadata in a JPEG2000 file, Murray says that he would store the authoritative version of the metadata there. This would provide a backup of the metadata with the actual images.

---

Technorati tag: JPEG2000