|
Articles / Simple metrics for
documentation
Good documentation will generally have certain basic
characteristics or attributes that ensures its effectiveness as a means of
communicating information to the reader. In this article we list various
criteria that you can use to appraise the effectiveness or quality of
documentation. While ultimately it is the degree to which a particular
document matches the requirements of its targeted audience that determines
its effectiveness, this set of criteria at least provides an objective means
of measurement.
You might use the criteria, for instance, to judge the
merits of an existing document or set for documents, or as a checklist
against documentation being written, or as a means of comparing two sets of
documentation.
For example, to compare one set of documents against
another (yours against a competitor, for example), a spreadsheet matrix
could be set up to include the comparison criteria, against which the two
sets of documentation can be judged. For each criterion, the documentation
can be marked on a sliding scale of 1 to 5, as follows:
Scores can then be tallied. The higher the score the
better.
The set of criteria is by no means exhaustive and will not
fit all forms of documentation (it is best applied to procedural type
documentation), but may provide a good starting point.
The criteria are separated into categories, as follows:
Deployment
This category reflects upon how a document is made
available to its intended readers.
Can the reader easily locate and open the relevant
documentation?
A fundamental aspect of good documentation management is
that intended readers can locate the relevant documentation quickly. Is the
location of the documentation evident? Is the document naming convention
used easy to understand? Is it clear which documents are current? The choice
of how the documentation is deployed will also affect their accessibility.
Is the reader using the same version of the application to read the document
as was used to create the document? Does the reader need any special tools,
such as a PDF reader or flash plugin, and if so are they available to them?
Can the reader easily view the documentation while
carrying out their tasks?
It makes sense that the choice of media for the
documentation takes into account the environment and circumstance in which
it will be used.
Is the environment in which the documentation is to be
accessed, appropriate for its targeted audience in terms of ease and speed
of access and use? Can the reader view the document on their screen while
using an application? Can they print the document if need be?
Is the document secure?
Are there appropriate read/write restrictions imposed on
the document. That is, only readers with the appropriate authorization
should be able to view or modify the document.
Content
This category concentrate on the usefulness and value of
the material being written.
Does the content reflect the current implementation of
the system or processes?
The documentation should reflect the current system or
processes in place. As the system or processes change then the documentation
should also be changed. Keeping documentation up-to-date is a crucial
component of good documentation management.
Is the documentation under version control
Version control is a way of keeping track of different
iterations of a document. It is a way of identifying which document is the
most recent version, or which of two versions is the most current. It is an
important element in ensuring that readers are using the most recent version
of any particular document. Version control can be as simple as indicating
the version on the front of, or in the footers of, the document, or it can
be controlled through sophisticated check-in/check-out type methodologies or
tools. Employing documentation online and restricting printing is one way of
ensuring the currency of the documentation used.
Is metadata provided for each procedure?
By metadata we mean any information about the procedure
itself. This is normally provided at the beginning of the procedure,
although it may be placed at the end. When a procedure is updated this
information should also be updated. This ensures that anyone reading the
procedure, or updating it, has all the necessary information about the
procedure to hand. Such metadata can alternatively be provided at the
document level, rather than for each procedure, or for non-procedural topics
such as policies.
The following shows the kind of information that can be
included for each procedure in a document:
|
Overview
|
The background to and a short overview of the
procedure.
|
|
Considerations
|
Any additional information of relevance to this
procedure.
|
|
Definitions
|
Any terms specific to this procedure.
|
|
Purpose
|
The reason for or end result of the procedure.
|
|
Scope
|
The areas covered by this procedure.
|
|
Related Material
|
Any other extant documents that may be of
relevance to this procedure.
|
|
Approval
|
The personnel responsible for approving or
modifying this procedure.
|
|
Responsibilities
|
The roles associated with this procedure and their
responsibilities.
|
|
Tracking
|
How is this procedure tracked to completion?
|
|
Records
|
The records that exist for the procedure and their
location.
|
Is the reader introduced to each procedure properly
(i.e. is it made clear when and why they should carry out the procedure)?
Before the reader is immersed in the details of the
procedure, they should be given an introduction: what is the procedure
about, why are they carrying it out, what will they achieve and so forth.
Is the reader provided with appropriate guidelines (if
any) relevant to a task?
Quite often a set of procedures must adhere to the
company’s existing polices or guidelines. These should be included as part
of the procedure or in a separate referenced document.
Is the content comprehensive in its coverage of all
applicable tasks and other background information?
A procedure must be complete enough for the reader to be
able to carry it out with the information at hand. The reader should have
all necessary information in order for them to carry out each specific task
competently.
For instance, each step in the procedure, however trivial,
should be documented. Each step should clearly describe what the reader
might need to know in order to carry out the step. If the reader has a
choice for actions, then each possible action should be described. If there
is a possibility that a step in the procedure cannot be done, then the
reader should be directed as to what to do then. Any peripheral tasks that
the reader needs to know in order to carry out the procedure should be
described or referred to. What should be avoided is a situation where what
the reader is doing and what the procedure says, do not match.
Is the content free of material superfluous to the
reader’s needs?
Procedures should be succinct and direct. Superfluous,
redundant, repeated, or vague directions should be avoided.
Is it made clear to the reader as to when they are
supposed to carry out a task?
If a task is time dependent (that is, it should be done at
a particular time or date) then that should be clearly indicated. For
instance, if the task is backing up a directory then it may be important to
say that it should be done at six o’clock each day, or Friday afternoon at
four, or every morning and so on.
Are notes peripheral to the task at hand provided and
clearly identifiable as notes?
Sometimes the reader requires some additional information
that may not be directly related to the task they are engaged in but may
however be useful to them. For instance, a procedure may describe how to
make conference calls. A note may be included to remind the reader about
allowing for differences in time zones.
Readability
This category explores the degree to which the content is
intelligible.
Is the text clear, precise, and concise?
Regardless of the content of the documentation, it should
be written in plain, simple, clear English, using proper sentences, standard
punctuation and so on. The reader should not have to struggle to understand
the writer’s intention. Long, complex, rambling, unintelligible sentences
usual means that the writer has not quite understood the object of their
discourse.
Is the text written at a level that is easy to
comprehend for this reader audience?
The complexity of the documentation will depend on the
audiences it is directed towards. Complex concepts should be explained where
it cannot be assumed that the reader will understand them. On the other
hand, sophisticated readers will not want things they already understand
explained to them again or unduly spelt out.
Is the text free of grammatical and punctuation errors
and ambiguities?
Before publication, a document should go through a final
proofreading, by someone other than the writer. Even the most thorough
writer will miss the odd typographical error (typo), misspelling and so on.
Spelling checkers are useful but may simply replace a misspelling with a
correctly spelt but incorrect word, making it harder to discover. Such typos
may not impact on the comprehension of what is written but it is surprising
how irked readers can become on discovering one.
Is the style of writing consistent throughout the
documentation?
This may be a problem where more than two writers are
involved. Even if this is the case, at least one writer should do a final
edit to ensure that the material reads the same throughout. If a document
reads like it has been written by a committee then it probably has!
Is jargon avoided, or where used, explained effectively
If the writer knows that the audience will understand a
piece of jargon then there is no need for them to define the word again.
However, if there may be ambiguity about what the jargon means, then an
explanation will avoid any misunderstanding. A comprehensive glossary of
terms is a good idea. Failing this, a definition should be given the first
instance a jargon word is used. This applies equally to acronyms or
abbreviations.
Navigation
This category highlights those aspects of a document that
facilitate a reader’s ability to find relevant information.
Is a logically grouped, easy-to-navigate table of
contents provided, for accessing particular information?
With large documents the reader should have some way of
browsing through the heading levels to locate their particular section of
interest. For paper documentation, a table of contents should be provided at
least. For online documents, a drop-down hierarchy of contents is
particularly useful.
Can the reader search for information (either by index
or through full-text search)?
In addition to browsing, the ability to search for a word
or phrase is useful. With paper documents an index should be provided. For
online documents, the two main search methods are by index (where key words
are highlighted), or through a full text search (where the reader can enter
any combination of words).
Is suitable header or footer information provided
In a paper document, footer information that includes the
page number and current topic will assist the reader as they browse through
the document. Other information can be included in the footer (and header)
if need be. With online documentation, where a header or footer do not
exist, the reader can still be given details of the current section of the
document, or document path, like this:
How to build a time machine:Getting started:Finding a
suitable clock
In any case, the reader should always know where they are
in the context of the document.
Is the reader provided with appropriate references to
other material in a suitable place?
Where the reader’s attention needs to be redirected
towards additional material, this reference should be included in the text
itself, or in a footnote, or perhaps in a separate referenced section. The
reference should identify the name and location of the referenced material.
Mentioning another source of material without given the location is
annoying. With online documentation such references can be in the form of
hypertext links, although they are of little use if the link is no longer
current.
Are procedures broken down into numbered steps?
Breaking the details of a task into a set of numbered
steps is a good way to simplify the task and instructions. Each step should
identify and describe a single discreet action, along with any additional
information pertinent to that action (such as the likely result of the
action). Such steps are more easily understood and followed than simple
paragraphs of text.
Does the flow of information reflect the reader’s
need for that information?
Documentation should be written from the perspective of
the reader rather than the process or application it describes. Information
should be structured so that it is task-orientated. The reader will want to
find out how to print a document rather than how does the Print button work.
Of course, a document may also contain reference material identifying all
the elements of an application but normally this would be in addition to the
task-type information.
The structure of the document should match as far as
possible the reader’s requirements for information. So, details they need
up front (such as a Getting Started section) will be at the start of the
document, whereas information they might require as an afterthought (such as
reference material or troubleshooting material) might appear much later in
the document.
This precept applies as much to a chapter or topic as it
does to the whole document. For instance, you don’t put vital warning
information right at the end of a procedure.
Do steps follow on in a logically contiguous manner?
The steps in any procedure should mirror the order in
which the reader will be carrying out the steps. They should not have to
jump from step three to step six and back to step four. This may seem quite
obvious but it is common for writers to specify a step and then in the next
sentence to indicate some preconditions relevant to this step. For instance:
1. Plug the computer in. Make sure the computer is
switched off before plugging it in.
This could better be written:
1. Ensure the computer is switched off.
2. Plug the computer in.
Are steps clearly defined and given an appropriate
lead-in?
A lead-in heading should identify the set of steps, like
this:
To start the Neutron Flux Detector:
Each lead-in should have a consistent structure. For
instance, start each one with the preposition ‘to’. Doing this with each
lead-in heading will make it easy for the reader see that what follows is a
set of steps.
Is it made clear to the reader that a particular task
is one that they can carry out according to their given role?
In some cases, a set of steps describing a task can be
broken down by role. These steps are then only applicable to a reader who
has that role. In this way the reader can quickly see what particular steps
are applicable to them, while also being able to see the steps carried out
before and after their ones (so as to view them in context). Once the role
has been identified then steps can be given using the third-person, like
this:
Administrator
1. Locate the main switch. You will find this switch
under the label marked ‘Main’.
2. Turn the Main switch off.
Receptionist
3. Check that all office lights are off.
4. Lock the office doors.
The ‘you’ or person being referred to in each step is
therefore the person identified in the role. Where a step or part of step is
applicable to a particular role it should be identified thus:
1. Give the report to the Administrator. The
Administrator should review this and return it to you.
The point being that there should not be any ambiguity
about who is supposed to carry out the task.
Is information that is relevant to more than one topic
documented separately and cross-referenced as needed?
As far as possible the information the reader requires
should be available in the document when they expect or require it. However,
there are occasions when a piece of information is relevant to more than one
section of the document. In this case it can be provided in one place, such
as an appendix or table, and then referred to each time it is required.
Layout
This category considers the look and feel of a document
and how these ultimately impact on its usefulness.
Is the information laid out in a coherent and
consistent manner?
A document’s layout and formatting may assist or hinder
the reader’s ability to find and read the material. Things that assist the
reader are:
-
Clearly identified headings (by being in bold, or
through colour).
-
Clearly labelled sections, such as notes.
-
The judicious use of space.
-
Appropriate fonts and font sizes.
The use of many colours, different fonts, superfluous
lines and so on can make it more difficult for the reader to find their way
around the document and understand its content, regardless of how aesthetic
it all may look. The different requirements for paper documents and online
documents should be taken into consideration.
Are the elements used within the document (fonts,
colours, graphic formats etc) appropriate to the implementation of the
documentation?
Whether a document is to be viewed online or printed
determines the most appropriate use of fonts, colours and other formatting
elements. Certain fonts, for instance, work best for printed documents,
while others work best for online viewing. The use of colour may be
appropriate for online documents but is lost when a document is printed to
paper (unless a colour printer is used). Documents that may be both viewed
online and printed out will have to accommodate this dual role in their
layout and formatting. For instance, headings could have a particular colour
to distinguish them online but also additional formatting, such as bolding
or italicization, to identify them when printed.
Is the layout of the different parts of the
documentation consistent?
It may take a few moments for the reader to understand the
meaning of the various pictorial elements and signs in the document. Finding
that these suddenly change halfway through the document can be annoying.
Are headings clearly and consistently labelled and
applied?
Most documents will have a hierarchical level of headings.
These should follow each other, like this: heading 1, heading 2, and heading
3, rather than jumping a heading, like this: heading 1, heading 3. They
should have a similar grammatical structure, such as "Printing a
report," "Saving your document" and so forth, rather than a
hodgepodge of styles: "Printing a report", "To save a
document", "How the application is started". The level of a
particular heading should be clearly identified by its formatting.
Are relevant diagrams and screen dumps provided in the
appropriate place and clearly referenced?
A good diagram can assist the reader in understanding the
text, while screen dumps can help the reader in orienting themselves
relative to an application. These should occur close to the text that they
are related to or, particularly if they are used frequently, clearly
referred to in the text (i.e. "see Diagram 5").
The use of diagrams and screen dumps should enhance or
clarify the reader’s understanding of a task or topic. Conversely,
diagrams and screens that provide no useful additional or clarifying
information should be avoided.
For instance, screen dumps should be used sparingly in
online documentation if it can be assumed that the reader can view the same
screen in the application itself. They are of most use when supplemented
with additional text-boxes.
Do screen dumps contain contextually correct sample
data relevant to that section of the documentation?
If screen dumps are used, and the reader can view the data
on the screens, then that data should be appropriate to that particularly
use of the screen.
Is good use made of standard structuring devices such
as bullets and tables?
Bullet points are an excellent way of untangling a
complicated paragraph, particularly if it contains some underlying list of
elements or ideas. Tables are a more sophisticated way of structuring
information in a coherent fashion. The use of these along with other visual
devices such as diagrams also helps to break up an otherwise dull sea of
paragraphs.
Does the content and layout of the document adhere to a
set of standards and guidelines?
If a company or organization has a particular house style
for its documentation, then the closer its documentation follows this house
style, the more quickly its readers will comprehend them (particularly if
they are to read more than one document from the same company). Consistency
in style also reinforces a company’s branding and marketing.
|