review NeXus manual draft

[Pete Jemian]

review the NeXus manual:

• PDF: http://www.nexusformat.org/images/f/fc/NeXusManual.pdf
• source: SVN repository: ​https://svn.nexusformat.org/definitions (look in trunk/definitions/manual)

[Pete Jemian]

add index to manual

[Pete Jemian]

posted new draft revision for review, same URL

Propose separating the chapter with the class definitions (for all known base classes, application definitions, and contributed definitions) into its own book. Makes the manual shorter. Also revisions to classes should not publish completely new version of manual.

[Pete Jemian]

(In [524]) emphasize unique group names, start to document data types for use in data files. Refs #86

[Pete Jemian]

Mark Koennecke said in today's conference call Introduction

This should be a slightly expanded version of what we do in a NeXus talk. This section should introduce the most important concepts of NeXus. Also benefits of NeXus. This is crucial! The introduction is the only thing most people will ever read! At the end of this, there is a section which gives a guide what is to be found in the various sections of the manual.

NeXus Design This section details the design rules for constructing NeXus files including coordinate systems

Applying NeXus # Make an application definition # How to construct a real NeXus file # Interacting with the NIAC

NAPI Reference

NXDL Reference

NeXus Base Class Reference

NeXus Application Definitions

NeXus History

The NIAC This section also should also include all the pointers and mailing list yada, yada...

NeXus File Format Mapping

[Pete Jemian]

comments posted by an independent reviewer of the NeXus documentation

[edit: finally, I arrived at the section "NeXus: the basics for the truly impatient" which was actually what I was looking for. I guess this makes many of my comments below irrelevant; I'll send them anyway as I have written them down already.]

• I did not get that the wiki version is actually outdated
• The manual is indeed very long. I have the feeling that much text could be spared when more examples were given. Personally, I like very much ​http://diveintopython.org/toc/index.html which contains basically only examples.
• More real-world examples would be useful. E.g. a file from a triple-axis spectrometer (one point detector scanning some angle) or FOCUS, a neutron time-of-flight spectrometer.
• When describing the classes, it seems to me that there is the same information twice: once as xml text and once as table. One of them should be enough.
• [minor] The xml descriptions have some typos, repeated words etc
• It becomes not clear which entries are required and which are optional.
• Make the "what is NeXus" more clear; I had always the impression that NeXus is a set of subroutines AND a set of design principles AND ... -- and that I would need to follow all of them. Make more clear that these items are completely independent of each other. That I can write a NeXus file without using the subroutines, for example.
• In fact, the verysimple.xml looks different from what Mark Koennecke has sent me as a FOCUS file. Is there no standard or is the example outdated? Or is Marks FOCUS file outdated?
• You say that several file formats are available, namely HDF4/5 and XML. That sounds again as if I could write NeXus only in these file formats. However, It is only /the subroutines/ which are limited to these file formats.
• Talk about old ways of doing something in footnotes only (if at all)
• How the NXDL relates to NeXus is still unclear to me

[Pete Jemian]

(In [543]) add more to the FAQ, refs #86

[Pete Jemian]

Mark Koennecke suggests restructuring the manual something like this:

        Reorganize using a form such as this:
        ======================================================================
        .....................................................
        Volume I
        - - - - - - - - - - - - -
        Introduction
           Derive from a typical NeXus talk
           Comment that NeXus is a standards community
        NeXus Rules
           Design rules for constructing NeXus files, including coordinate systems.
           Mark's talk for the May 2010 workshop would be a good basis or example ere.
        Applying NeXus
          1. Make an application definition
          2. How to construct a real NeXus file
          3. Interacting with the NIAC
        NeXus History
        NeXus Community
          NIAC
          Wiki
          Mailing Lists
          Subversion
          Issue Reporting
        NeXus File Format Mapping
           HDF4
           HDF5
           XML
        .....................................................
        Volume II
        - - - - - - - - - - - - -
        NAPI Reference
        NXDL Reference
        NeXus Base Class Reference
        NeXus Application Definitions

Changes yesterday ([571]-[580]) made progress toward this.

[Pete Jemian]

(In [581]) 2010CodeCamp: cleaned up this section, refs #86

[Pete Jemian]

(In [596]) Refs #86

[Pete Jemian]

(In [602]) Refs #86

[Pete Jemian]

items marked in the DocBook source as TODO

[Pete@como,3030,manual]$ grep TODO *.xml
GenericFeatures.xml:    <!-- TODO: write about generic features or leave this chapter out -->
NXDL.xml:            TODO: Edit content from Trac ticket #65
NXDL.xml:        <!-- TODO: make a better description of the NXDL and how to use it -->
NXDL.xml:        <!-- TODO: describe and make tables of the stuff in nxdlTypes.xsd -->
Verification.xml:            <!-- TODO: Will we describe how validation code can check to see if it is 
Verification.xml:            <!-- TODO: describe what is and is not being validated -->
Verification.xml:                to validate any NXDL file. Here's an example using <code>xmllint</code>: <!-- TODO: check this command -->
Verification.xml:                UNIX/Linux command line tools <!-- TODO: check this command -->
Verification.xml:                <!-- TODO: Care to give an example of validating an XSLT using xmllint or saxon? -->
design.xml:            <!-- TODO Is the previous line OK? -->
design.xml:                TODO Need a better writeup than this!
design.xml:                            TODO Comment about strings
design.xml:                            TODO Comment about binary data
design.xml:                            TODO Comment about images
impatient.xml:  TODO This could be rewritten from the ground up.
impatient.xml:      <!-- TODO and what about types? -->
introduction.xml:            <para><!-- TODO --></para> <!--also refer to later section-->
introduction.xml:                <!-- TODO: build example from LRMECS data.
preface.xml:        <!-- TODO -er-, will be --> 

[Pete Jemian]

(In [607]) 2010CodeCamp: Refs #86

[Pete Jemian]

(In [608]) 2010CodeCamp: refs #34, refs #86, refs #116

[Pete Jemian]

(In [609]) 2010CodeCamp: refs #34, refs #86

[Pete Jemian]

(In [610]) 2010CodeCamp: fixes #34, refs #86

[Pete Jemian]

2010CodeCamp: manual has been reviewed, deficient sections have been identified and assigned for revision