AIIM Standards Watch

How will you preserve electronic documents?

Most of our documents are digital born. Have you given any thought to how you are going to preserve those documents? AIIM research shows that over 50% of the documents that are scanned originally were born digital. This means that the digital born documents were printed then scanned. Why? Aren't we using enough paper for other things? Register for the AIIM and PDF/A Competence Center seminar on this coming Friday, March 26 at the Doubletree Washington Hotel to find out about a file format (PDF/Archive) to preserve your digital documents. Register today at http://bit.ly/ce71Iv .

PDF/Archive is an ISO standard (ISO 19005-1) that was adopted as an ANSI standard (ANSI/AIIM/CGATS/ISO 19005-1). This seminar will introduce you to the standard and explain its importance to your organization's digital preservation strategy. The Washington, DC PDF/A Seminar will feature the following presentations:

• Susan Sullivan present on Digital Preservation and PDF/A
• Mark McKinney present PDF/A for Scanned Documents: Paper Becomes Digital
• Mark Maragni present on PDF/A and Output Management
• Leonard Rosenthol present on Metadata and Digital Signatures
• Stephen Levenson present a PDF/A Case Study and Future of PDF/A

Attend this seminar to learn about PDF/A so that you and your organization will be ready. Register today at http://bit.ly/ce71Iv.

File Formats

Without meaning to mix apples and oranges, I managed to do just that in my blog post of January 22 by mixing image and document file formats. These are two distinct file formats. File formats such as TIFF, JPEG, etc are image file formats and PDF and XPS are document formats. It should also be noted that XPS is a formal ECMA standard (ECMA 388).

Image File Formats

In the last post, we dealt with compression and the TIFF file format. Now, let's talk about JPEG,PDF and XPS.

JPEG (pronounced jay-peg) is another imaging standard and stands for Joint Photographic Experts Group. JPEG is a lossy compression format for photographic images. It is designed for use with either full color or gray-scale images. JPEG is best when used with photographs rather than text. JPEG specifies how an image is transformed into a stream of bytes, but not how those bytes are ecapsulated in any particular storage medium. JFIF (JPEG File Interchange Format), created by the Independent JPEG Group, specifies how to produce a file suitable for computer storage and transmission over the Internet from a JPEG stream.

JPEG/JFIF is commonly used to store and transmit photographs over the Internet. It is not suitable for use with line drawings or text because its compression method does not perform well with these types of images. PNG and GIF are used in these instances. JPEG is best used with photographs and paintings of realistic scenes with smooth variations of tone and color. In many cases, JPEG will produce a much higher quality image than other common methods.

With the increasing use of multimedia technologies, image compression requires higher performance and new features. JPEG2000 is intended to advance standardized image coding systems to serve applications for years to come. JPEG2000 is a new image format based on state-of-the-art wavelet compression. It is applicable for a number of different applications in the digital imaging market including digital cameras, pre-press, medical imaging and others. JPEG2000, Part 1 (ISO 15444) offers both lossless and lossy compression and provides better image quality at smaller file sizes than JPEG. JPEG2000, Part 2 (ISO 15444/6) is used to compress scanned color documents containing both bitonal elements as well as images.

The development of JPEG2000 is the result of collaboration between the International Organization for Standardization (ISO), the International Telecommunications Union (ITU-T, formerly CCITT) and input from a multitude of industry experts.

Usually, when you think of PDF, the first thing that comes to mind is Adobe Acrobat. But did you know that there are over 2000 PDF product suppliers? PDF, Portable Document Format is a file format originally developed by Adobe Systems for representing documents in a manner that is independent of the original application software, hardware, and operating system used to create those documents. PDF is an open standard and anyone may write applications that can read or write PDFs royalty free. In 2008, the PDF specification was published as ISO 32000-1:2008. A PDF document is a self-contained cross-platform document. It is a file that will look the same on the screen and in print, regardless of what kind of computer or printer someone is using and regardless of what software package was originally used to create it. Although they contain the complete formatting of the original document, including fonts and images, PDF files are highly compressed, allowing complex information to be downloaded efficiently.

There is also another file format, XPS, XML Paper Specification. XML Paper Specification or XPS is a document storage and viewing specification developed by Microsoft that describes electronic paper in a way that it can be read by hardware, software and the human eye. XPS provides a page view of the way the document will print. It describes the appearance of fixed format documents by using an XML based format so that the layout won't change. XPS is viewed as a potential competitor to PDF (Portable Document Format) but will not replace PDF in the instances where dynamic content capabilities are required which XPS cannot handle. By dynamic content, we mean the type of content that may be contained in a drop-down on a form. Microsoft released XPS with a royalty-free patent license to encourage wide adoption by the industry. The XPS document format is included with Windows Vista and 2007 Microsoft Office System products. The XPS viewer is included with Windows Vista and is available for Windows XP and Windows Server 2003. The viewer allows users to open, read and apply digital signatures to XPS documents without needing the full XPS generating software. XPS allows electronic documents to print better, be shared easier, archived and better maintain the security of the information in the document.

XPS relies on two additional specifications - the Open Package Conventions which describe the method for creating packages of content, resources, and metadata in a compressed ZIP file and Open XML Markup Compatibility Specification that describes the mechanism to support versioning and extensibility of XML. Ecma International's TC46 chaired by Martin Bailey, Global Graphics, will continue the work on the XPS standard and facilitate its fast track processing at ISO to become an ISO standard. XPS is platform independent.

There are a multitude of image file formats to choose from. Whatever image file format and compression that your organization chooses, is dependent on the application you are using. It is important to take into consideration the type of documents you will be scanning, the graphical content contained in the documents and how they will be used. The file format you select should meet the inteded use and be capable of including the compression scheme you choose.

Image File Formats

There has been a lot of discussion lately around image file formats. So, I am repurposing an article that I wrote and was published in the January/February 2006, AIIM E-DOC Magazine.

In this post we will discuss image file formats in general and specifically hone in on TIFF. In a subsequent post, we will deal with other image file formats. Both of these posts will be from a standards basis.

Image file format standards have helped the widespread adoption of imaging technology take place. Image file formats provide a standardized method of organizing and storing image data. A scanned document or image consists of picture elements or pixels that represent the brightness and color of the information on the page. There are numerous graphic and image file formats such as TIFF, JPEG and PDF which are three of the standards used in document imaging. (JPEG and PDF will be dealt with in the next post.)

Let's start by discussing compression.ANSI/AIIM TR 33, Selecting an Appropriate Image Compression Method to Match User Requirements provides an explanation of compression algorithms and useful information in selecting the best compression algorithm for your application. It is important to note that one compression method is not applicable for all scanned documents and in choosing the best method, one must consider the type of document that will be scanned. Compression scheme is the method used to reduce the amount of data needed to store or transmit a representation of an image. Compression is lossless when the data is compressed by efficient coding of the information in the image and where the reconstructed image contains the same amount of information. Lossy compression is where images are compressed by selectively removing information from the image. This does not mean that words, phrases or sentences are removed. Through complex algorithms statistically redundant information as well as perceptually irrelevant or unimportant information is removed leaving only the useful information.

TIFF, Tagged Image File Format is a file format used mainly for storing raster images, including photographs and line art and is largely credited with founding the imaging industry. Aldus (acquired by Adobe Systems, Inc.) is credited with developing TIFF for use with PostScript printing. It is now widely used for images along with JPEG. TIFF's primary goal is to provide a rich environment within which applications can exchange image data. This richness is required to take advantage of the varying capabilities of scanners and other imaging devices.

TIFF uses tags to handle multiple images and data in a single file. These tags describe the size of the image or define how the image data is arranged and identifies the compression algorithm if any that is used. Images created using TIFF can be used for archiving purposes as TIFF is a lossless format which means that the file may be edited and saved without losing any compression.

In document management, TIFF is used in conjunction with CCITT Group IV compression typically used with facsimile technology. Typically, black and white documents are captured using TIFF; however, color may also be used. In large volume applications, documents are typically scanned in black and white, rather than color or grayscale to conserve on the file size. Because TIFF supports multiple pages, a multi-page document can be scanned to a single file rather than an individual file for each page scanned.

There are variations of the TIFF specification developed by Aldus and controlled by Adobe Systems Inc. TIFF is a viable file format. As you think about file formats, the question really becomes what file formats will be around in the years to come. Today's file formats are PDF, TIFF, XML,etc. Will they be here in 50 years? What new one will be 'The File Format' that everyone will be talking about.

The next blog post in this mini-series will deal with JPEG and PDF and the characteristics of sustainable file formats.

Adopting Standards

Adopting standards support the creation of your organization's information technology architecture. Therefore, it is important to establish a process for identifying standards to be adopted. With a process in place, anyone in the organization will be able to recommend a standard be adopted. Once a standard has been recommended for adoption, the IT department should conduct a preliminary review of the standard to determine if it conforms to the IT architecture strategy and business needs of the organization. Additional consideration should be given to determine the degree of scalability, extensibility, portability and interoperability that the standard will afford to the organization. After the preliminary evaluation has been conducted, IT should form a task team to participate in the evaluation and adoption of the standard. This may require the participants to seek the advice of others in the organization and bring the comments to the task team for discussion. Once the task team has reached consensus on adopting the standard, it is important to provide notification to the organization of the adoption and the perceived benefit it will bring.

There are three basic criteria that should be investigated when you consider whether to adopt a standard. The three criteria of maturity,adoption and endurance are what you should use to determine the appropriate standards for your organization to adopt or impement. You don't want to implement a draft standard as they are subject to change so you will want to go with a published version. If you want to be "first" to implement a standard, wait until the draft is almost final. Don't go with an alpha version. Look at who has adopted the standard. A standard that is widely adopted will endure. Endurance is the third characteristic to look at. Make sure the standard and the organization that developed the standard have been around for a while.

To see what standards AIIM is working on, visit http://www.aiim.org/standards. We are always looking for new standards projects. If you have an idea, send me an email at [email protected].

Standards - de jure, de facto - which to use?

Happy New Year! I hope you had a wonderful holiday season and are now back to being focused on work and keeping your resolutions for the new year. The new year provides a good feeling of being able to set the past behind and starting fresh. One of the many resolutions I made beyond all the good health ones is to be more faithful with this blog.

In this first blog post of 2010, I thought we could deal wth the general topic of standards. There are many different types of standards which makes the whole idea of standards quite confusing.

The two types of standards that we will deal with in this post are - de jure and de facto. De jure standards are standards developed within an accredited standards development organization or through a standards setting organization. The development is specified through a detailed set of procedures that are strictly adhered to. In the de jure standards process, concepts like consensus and due process are considered the hallmarks of the process. De jure standards are typically produced by organizations working with ISO, ANSI, ITU and others like them.

The other type of standards, de facto standards, are market driven standards that receive wide acceptance from many organizations in the industry. These standards may be developed following a process that may be as rigorous as the de jure process but may be lead by a few organizations representing the industry rather than a diverse group representing numerous interest groups inhe industry. De facto standards become a standard primarily because they are widely adopted and readily available.

Some examples of these standard may be:

• PDF, ISO 32000-1, is now a de jure standard as it was developed based on the ISO procedures. Prior to ISO 32000-1, PDF was also a de facto standard as it was a specification that was owned and maintained by Adobe Systems.
• PDF/A, ISO 19005-1, is also a de jure standard that was also developed based on the ISO procedures and adopted as an ANSI standard (ANSI/AIIM/CGATS/ISO 19005-1) using the AIIM's standards development procedures which were approved by ANSI.
• TIFF is an example of a de facto standard. It is a specification that is owned by Adobe Systems. The specification is freely available on the Adobe Systems web site but the specification is controlled by Adobe Systems.

In the next post, we will take a look at some of the factors you should consider when determining which type of standard to use in your organization.

I hope this information is helpful to you. Let me hear from you.

How are you using standards?

I guess before we talk about how we are using standards, we should first ask "Are you using standards?"

Today, more than ever, organizations are struggling to ensure that the technology and processes that they have in place will result in positive impact on the organization's bottom line. In other words, we need to be operating at maximum efficiency and effectiveness. If an organization is implementing new technology or a new process, it needs to know that what it is doing will be successful.

Standards are one way that we can increase the level of confidence in what we are implementing and using. So, if you are using standards, tell me how you are using them and if they are helpful to your organization. You can do this by sending an email to Betsy Fanning. I will share the responses via posts to this blog and on Twitter as well as on Information Zen (http://www.informationzen.org.

If you are using standards, you most likely have a lot of good ideas, thoughts and advice as to how to best use them. Why not share them? Let's do a standards focused '8 Things' exercise much like what my manager, John Mancini is doing for various ECM related topics. This is quite easy and will via this blog and the other AIIM blogs provide you additional visibility which is so critical today. While you are relaxing with family and friends over the holidays, start jotting down the 7 or 8 "things" about implementing standards in general or a specific standard that are important to share. In addition to the list, you might also want to include a brief explanatory paragraph for each. Send your list along with a brief bio about yourself and a photo to [email protected]. We will publish the lists we receive.

Have a terrific holiday! The new year promises to be filled with wonderful new opportunities. I was never good as a young person at keeping a diary. If I am lucky, my diaries have a week at most of entries and then pristine, clean pages! So, it is no wonder that my blog is so quiet. My focus for 2010 is to promote AIIM and its AIIM Standards program so you will be seeing more from me, hopefully.

StratML Standard

AIIM is pleased to announce the publication of a new standard, StratML. Part 1: Core elements. This standard specifies an Extensible Markup Language (XML) vocabulary and schema (XSD) for the elements that are common and considered to be part of the essential core of the strategic plans of all organizations worldwide. Included in the standard are two kinds of elements - core elements and administrative metadata.

StratML formalizes the strategic planning that all organizations go throught. This standard will enable information contained in strategic plans to be efficiently shared and analyzed in an organization. StratML will enable efficient discovery, sharing, referencing, reuse and analysis of strategic information.

StratML has a lot to offer organizations and their customers. It will allow more efficient and effective engagement of employees, partners and other stakeholders in the strategic planning process. From a customer or investor perspective, it will allow you to be able to review and provide feedback on the company strategy for those companies that are truly customer focused and open to sharing their plans with their customers and investors.

This standard is being issued in parts. Part 1 focuses on the core elements for the schema focused on strategic planning. The committee is working on Part 2 which will focus on the performance plans and reports that result from the strategic planning process. Part 2 is expected to be finalized in 2010.

For more information about StratML, visit http://www.aiim.org/stratml. If you would like to become involved in the work with this standard, send an email to Betsy Fanning.

How are you archiving your electronic documents?

Archiving electronic documents is more than making sure backup copies of the media on which the documents are stored are preserved in some fashion. It also involves making sure you are using the correct file format. Depending on your specific needs, you may want to preserve your electronic documents using their native file format, TIFF, XML or PDF/Archive (PDF/A).

As you decide which file format to use, you should consider the characteristics of a sustainable file format that NARA and the Library of Congress have identified, which include:

• Published documentation and open disclosure
• Widespread adoption and use
• Self-describing formats
• External Dependency
• Impact of Patents
• Technical Protection Mechanism

What is PDF/A? PDF/A is a file format for long-term preservation of electronic documents using PDF. It is one of several file formats that you should consider in your electronic document archiving plan. For more information on PDF/A, visit http://www.aiim.org/pdfa/

In September, AIIM with the PDF/A Competence Center produced a half day seminar in Chicago. The seminar was so well received that we are planning to do another seminar in March 2010 in the Washington, DC area. As details about this seminar are finalized, we will be posting the information at http://www.aiim.org/standards. Once again, we will have presentations on input, output, metadata, digital signatures, and the standard itself.

PDF/A Seminar in Chicago

How are you going to preserve your electronic information so that you can be sure that the documents will be able to be displayed and read in a reliable format? If by chance you do not know the answer to this question and you are going to be in the Chicago area on Tuesday, September 15, you may want to attend this seminar to find out if PDF/A is right for your organization.

PDF/Archive(PDF/A) is a sustainable file format that is device independent so you don't have to use the same computer, software or operating system to read a PDF/A file as was used to create the file. Each PDF/A file is also self-contained and self-documenting which means that it has all the fonts used to create the document and the metadata identified to describe the document. This means that your document will be displayed just as you intended for it to be displayed when you created it.

Want to know more about PDF/A and how your organization can use it to preserve its electronic documents? AIIM and the PDF/A Competence Center are providing a half day seminar on Tuesday, September 15, 2009 at McCormick Place in Chicago. Join us to learn more about how PDF/A can help you to preserve your electronic documents.

To register to attend this valuable half day seminar, visit http://www.aiim.org/standards/article.aspx?ID=37092 . If you have any questions, please feel free to contact, Betsy Fanning.