Developer doc tidy 3 rebased

developers/Clients/ImportLibrary.txt

@@ -18,7 +18,7 @@ The primary classes which make up the Import Library are:
 -  :source:`ImportCandidates.java <components/blitz/src/ome/formats/importer/ImportCandidates.java>`
    which takes file paths and determines the proper files to import
 -  :source:`ImportConfig.java <components/blitz/src/ome/formats/importer/ImportConfig.java>`,
-   an extensible mechanism for storing the
+   an extensible mechanism for storing the properties used during import
 -  :source:`ImportEvent.java <components/blitz/src/ome/formats/importer/ImportEvent.java>`,
    the various events raised during import to ``IObserver``\ and
    ``IObservable`` implementations
@@ -27,10 +27,10 @@ The primary classes which make up the Import Library are:
 -  :source:`OMEROWrapper.java <components/blitz/src/ome/formats/importer/OMEROWrapper.java>`,
    the OMERO adapter for the Bio-Formats ``ImageReaders`` class
 -  In OMERO.insight, the main entry point is the importImage method of
-   :source:`OMEROGateway.java <components/insight/SRC/org/openmicroscopy/shoola/env/data/OMEROGateway.java>`.
+   :source:`OMEROGateway.java <components/insight/SRC/org/openmicroscopy/shoola/env/data/OMEROGateway.java>`
 -  In the CLI, the main entry point is the
    :source:`CommandLineImporter <components/blitz/src/ome/formats/importer/cli/CommandLineImporter.java>`
-   class.
+   class
 
 Earlier Import Workflow
 -----------------------
@@ -53,7 +53,7 @@ Example
 -------
 
 The ``CommandLineImporter.java`` class shows a straight-forward import.
-An ``ErrorHandler`` instance is passed both to the ImportCandidates
+An ``ErrorHandler`` instance is passed both to the ``ImportCandidates``
 constructor (since errors can occur while parsing a directory) and to
 the ``ImportLibrary``. This and other handlers receive ``ImportEvents``
 which notify listeners of the state of the current import.

developers/Model.txt

@@ -33,7 +33,7 @@ libraries and can be used in both the server and the RMI clients.
 mapping <components/blitz/resources/templates>` to
 generate |OmeroJava|, |OmeroPy|, and |OmeroCpp| classes, which can be
 :source:`mapped <components/blitz/src/omero/util/IceMapper.java>`
-back and forth to the server object model. *This document only discusses the
+back and forth to the server object model. *This document discusses only the
 server object-model and how it is used internally.*
 
 Instances of the object model have no direct interaction with the
@@ -42,7 +42,7 @@ Hibernate_. That means, by and large,
 generated classes are data objects, composed only of getter and setter
 fields for fields representing columns in the database, and contain no
 business logic. However, to make working with the model easier, and
-perhaps more powerful, there are several features which we've built-in.
+perhaps more powerful, there are several features which we have built-in.
 
 .. note::
 
@@ -52,6 +52,86 @@ perhaps more powerful, there are several features which we've built-in.
     objects. This documentation will eventually be updated to reflect both
     hierarchies.**
 
+OMERO type language
+-------------------
+
+The |OmeroModel| has two general parts:
+first, the long studied and well-established core model and second, the
+user-specified portion. It is vital that there is a central definition
+of both parts of the object model. To allow users to easily define new
+types, we need a simple domain specific language (or little language)
+which can be mapped to Hibernate mapping files. See an example at:
+
+-  :source:`components/model/resources/mappings/acquisition.ome.xml`
+
+From this DSL, various artifacts can be generated: XML Schema, Java
+classes, SQL for generating tables, etc. The ultimate goal is to have no
+exceptions in the model.
+
+.. figure:: /images/model-generation.png
+   :align: left
+   :width: 60%
+   :alt: Code generation steps figure from http://dx.doi.org/10.1038/nmeth.1896
+
+
+Conceptually, the XSD files under the :sourcedir:`components/specification`
+source directory are the starting point for all code generation. Currently
+however, the files under :sourcedir:`components/model/resources/mappings`
+are hand-written based on the XSD files.
+
+The ant-task created from the :sourcedir:`components/dsl/src` Java files
+is then used to turn the mapping files into generated Java code under the
+file:`model/target/generated/src` directory. These classes are all within the
+ome.model package. A few hand-written Java classes can also be found in
+:sourcedir:`components/model/src/ome/model/internal`.
+
+The build-schema target takes the generate ome.model classes as input and
+generates the :sourcedir:`sql/psql` scripts which get used on :omerocmd:`db
+script` to generate a working OMERO database. Files named like
+"OMEROVERSION__PATCH.sql" are hand-written update scripts.
+
+The primary consumer of the ome.model classes at runtime is the
+:sourcedir:`components/server` component.
+
+The above classes are considered the internal server code, and are the only
+objects which can take part in Hibernate transactions.
+
+External to the server code is the "blitz"" layer. These classes are in the
+omero.model package. They are generated by another call to the DSL ant task
+in order to generate the Java, Python, C++, and Ice files under
+file:`components/blitz/generated`.
+
+The generated Ice files along with the hand-written Ice files from
+:sourcedir:`components/blitz/resources/omero` are then run through the
+``slice2cpp``, ``slice2java``, and ``slice2py`` command-line utilities in 
+order to generate source code in each of these languages. Clients pass in 
+instances of these omero.model (or in the case of C++, omero::model) objects. 
+These are transformed to ome.model objects, and then persisted to the 
+database.
+
+If we take a concrete example, a C++ client might create an Image via new
+omero::model::ImageI(). The "I" suffix represents an "implementation" in the
+Ice naming scheme and this subclasses from omero::model::Image. This can be
+remotely passed to the server which will be deserialized as an
+omero.model.ImageI object. This will then get converted to an
+ome.model.core.Image, which can finally be persisted to the database.
+
+Keywords
+~~~~~~~~
+
+Some words are not allowed as properties/fields of OMERO types. These
+include:
+
+-  id
+-  version
+-  details
+-  ... any SQL keyword
+
+
+Properties
+~~~~~~~~~~
+Mutable, annotated, global
+
 
 Improving generated data objects
 --------------------------------
@@ -115,7 +195,8 @@ Details works something like unix's ``stat``:
     Change: 2006-01-25 20:40:30.000000000 +0100
 
 though it can also store arbitrary other attributes (meta-metadata, so
-to speak) about our model instances. See :ref:`Model#dynamic` below for more information.
+to speak) about our model instances. See :ref:`Model#dynamic` below for more 
+information.
 
 The main methods on Details are:
 
@@ -146,7 +227,7 @@ This should hopefully save a good deal of re-coding if we move to true
 ACL rather than the current filesystem-like access control.
 
 Because it is a field on every type, Details is also on the list of
-Keywords for the :doc:`/developers/Model/TypeLanguage`.
+keywords in the type language (above).
 
 .. _Model#Interfaces:
 
@@ -187,8 +268,8 @@ their "implements" clause:
 For example, ``ome.model.meta.Experimenter`` is a "global" type,
 therefore it has no ``Details.owner`` field. In order to create this
 type of object, you will either need to have admin privileges, or in
-some cases, use the ``ome.api.IAdmin`` interface directly. (In the case
-of enums, you will need to use the ``ome.api.ITypes`` interface.)
+some cases, use the ``ome.api.IAdmin`` interface directly (in the case
+of enums, you will need to use the ``ome.api.ITypes`` interface).
 
 .. _Model#Inheritance:
 
@@ -197,18 +278,19 @@ Inheritance
 
 Inheritance is supported in the object model. The superclass
 relationships can be defined simply in the mapping files. One example is
-the annotation hierarchy in :source:`components/model/resources/mappings/annotations.ome.xml`.
+the annotation hierarchy in 
+:source:`components/model/resources/mappings/annotations.ome.xml`.
 Hibernate supports this polymorophism, and will search all subclasses
 when a super class is returned. *However*, due to Hibernate's use of
 bytecode-generated proxies, testing for class equality is not always
-straight-forward.
+straightforward.
 
 Hibernate uses CGLIB and Javassist and similar bytecode generation to
 perform much of its magic. For these bytecode generated objects, the
 getClass() method return something of the form
 "ome.model.core.Image\_$$\_javassist" which cannot be passed back into
 Hibernate. Instead, we must first parse that class String with
-:source:`Utils#trueClass() <components/model/src/ome/util/Utils.java>`).
+:source:`Utils#trueClass() <components/model/src/ome/util/Utils.java>`.
 
 Model report objects
 ~~~~~~~~~~~~~~~~~~~~
@@ -244,7 +326,7 @@ acceptFilter
 ^^^^^^^^^^^^
 
 An automation of calls to ``putAt / retrieve`` can be achieved by
-implementing an ome.util.Filter?. A Filter is a VisitorPatter?-like
+implementing an ome.util.Filter. A Filter is a VisitorPattern-like
 interface which not only visits every field of an object, but also has
 the chance to replace the field value with an arbitrary other value.
 Much of the internal functionality in OMERO is achieved through filters.
@@ -294,7 +376,7 @@ Each instance can be found in one of several states. Quickly, they are:
     exception. The backend detects these proxies and restores their
     value before operating on the graph. There are two related states
     for collections -- null which is completely unloaded and filtered in
-    which certain items have been removed (more on this below.)
+    which certain items have been removed (more on this below).
 
 .. figure:: /images/ObjectStates.png
     :align: center
@@ -310,14 +392,11 @@ table does not have a primary key field -- subclasses contain a foreign
 key link to their superclass. Therefore all objects without an id are
 assumed to be non-persistent (i.e. transient).
 
-Though the id cannot be the sole decider of equality (there are issues
-with the Java definition of ``equals()`` and ``hashCode()``. See the discussion
-at :ref:`TypeLanguage#ToBeDone`), 
-we often perform lookups based on
-the class and id of an instance. Here again caution must also be taken
-to not unintentionally use a possibly bytecode-generated subclass. See
-the discussion under :ref:`Model#Inheritance`
-above.
+Though the id cannot be the sole decider of equality since there are issues
+with the Java definition of equals() and hashCode(), we often perform lookups
+based on the class and id of an instance. Here again caution must be 
+taken not to unintentionally use a possibly bytecode-generated subclass. See 
+the discussion under :ref:`Model#Inheritance` above.
 
 Class/id-based lookup is in fact so useful that it is possible to take
 an model object and call ``obj.unload()`` to have a "reference" --
@@ -379,7 +458,7 @@ With these states in mind, it is possible to start looking at how to
 actually use model objects. From the point of view of the server,
 everything is either an assertion of an object graph (a "write") or a
 request for an object graph (a "read"), whether they are coming from an
-RMI client, an :doc:`/developers/server-blitz` client, or even
+RMI client, an :doc:`server-blitz` client, or even
 being generated internally.
 
 Writing
@@ -482,10 +561,12 @@ newProject as well. Instead, an ``OptimisticLockExceptions`` is thrown
 if a user tries to change an older reference to an entity. Similar
 problems also arise in multi-user settings, when 2 users try to access
 the same object, but it is not purely due to multi-users or even
-multi-threads, but simply to stale state. (Note: There is an issue in
-the multi-user setting in which a ``SecurityViolation`` is thrown
-instead of an ``OptimisticLockException``. See
-:ticket:`1649` for more information).
+multi-threads, but simply to stale state. 
+
+.. note:: 
+    There is an issue in the multi-user setting in which a 
+    ``SecurityViolation`` is thrown instead of an ``OptimisticLockException``. 
+    See :ticket:`1649` for more information).
 
 Various techniques can help to manage these duplications are:
 
@@ -563,8 +644,8 @@ A special form of links collection model the many-to-many
 relationship between two other objects. A Project can contain any number
 of Datasets, and a Dataset can be in any number of Projects. This is
 achieved by ``ProjectDatasetLinks``, which have a Project "parent" and a
-Dataset "child". (The parent/child terms are somewhat arbitrary but are
-intended to fit roughly with the users' expectations for those types.)
+Dataset "child" (the parent/child terms are somewhat arbitrary but are
+intended to fit roughly with the users' expectations for those types).
 
 It is possible to both add and remove a link directly:
 
@@ -622,7 +703,7 @@ necessary to do something along the following lines of:
 The unlink method in this case, removes the link from both the
 Project.datasetLinks collection as well as from the Dataset.projectLinks
 collection. Hibernate notices that both collections are in agreement,
-and deletes the ProjectDatasetLink? (this is achieved via the
+and deletes the ProjectDatasetLink (this is achieved via the
 "delete-orphan" annotation in Hibernate). If only one side of the
 collection has had its link removed, an exception will be thrown.
 
@@ -640,10 +721,10 @@ Future topics
 -  Validation: Since the accessor methods themselves are largely
    logic-less, the work of validating the objects has been offset to
    validation objects and the Hibernate system. For each given object, a
-   validation method can be specified which will check instance fields.
+   validation method can be specified which will check instance fields
    (TODO: the null-policy should be configurable based on whether or not
-   the object is currently in a session) Validation is intended to
-   verify the specification constraints which can not (easily and/or
+   the object is currently in a session). Validation is intended to
+   verify the specification constraints which cannot (easily and/or
    quickly) be verified by the database.
 -  Versioning/Locking?
 -  ObjectFactory? for wrapping model objects from :doc:`/developers/server-blitz`

developers/Server.txt

@@ -133,7 +133,6 @@ Topics
 -  :doc:`/developers/Modules/ExceptionHandling`
 -  |OmeroModel|
 -  :doc:`/sysadmins/server-security`
--  :doc:`/developers/Model/TypeLanguage`
 
 
 .. seealso:: |OmeroGrid|

developers/Server/Aop.txt

@@ -9,11 +9,11 @@ complicate an initial examination of the code. Therefore, it is important
 to be aware of what portions of the OMERO code base are "advised" and
 where to find the advisors (in the case of OMERO solely interceptors).
 
-In ` Spring <http://www.springsource.org/>`_, advisors are declared in
+In `Spring <http://www.springsource.org/>`_, advisors are declared in
 the bean definition files (under
-:sourcedir:`components/server/resources/ome/services` : :source:`services.xml <components/server/resources/ome/services/services.xml>`,
+:sourcedir:`components/server/resources/ome/services`, :source:`services.xml <components/server/resources/ome/services/services.xml>`,
 :source:`hibernate.xml <components/server/resources/ome/services/hibernate.xml>`,
-and others.`
+and others.
 
 In these configuration files, various Spring beans (shared objects) are
 defined with names like "proxyHandler", "eventHandler",
@@ -22,11 +22,11 @@ interceptor which is passed execution before the actual logic is
 reached. The interceptor can inspect or replace the return value, but
 can also stop the method execution from ever taking place.
 
-Unlike with ` AspectJ <http://eclipse.org/aspectj/>`_, the AOP
+Unlike with `AspectJ <http://eclipse.org/aspectj/>`_, the AOP
 implementation used by OMERO only allows for the advising of interfaces.
-Simply creating a new service implementation via "new QueryImpl?()" will
+Simply creating a new service implementation via "new QueryImpl()" will
 not produce an advised object, which in turn will not function
-properly, if at all. Instead, an advised objects can only be acquired
+properly, if at all. Instead, advised objects can only be acquired
 from the Spring :doc:`context </developers/Server/Context>`.
 
 By and large, only the :doc:`API service methods </developers/Modules/Api>` 

developers/Server/ExtendingOmero.txt

@@ -81,11 +81,12 @@ used in a fine-grain manner.
 Code generation
 ^^^^^^^^^^^^^^^
 
-Since it is prohibitive to model full objects with the SAs, one
-alternative is to add types directly to the code generated. By adding a
-file named ``*.ome.xml`` to :sourcedir:`components/model/resources/mappings`
-and running a full-build, it is possible to have new objects generated
-in all :doc:`/developers/server-blitz` languages. Supported fields include:
+Since it is prohibitive to model full objects with the SAs, one alternative is
+to add types directly to the :doc:`generated code </developers/Model>`. By
+adding a file named ``*.ome.xml`` to
+:sourcedir:`components/model/resources/mappings` and running a full-build, it
+is possible to have new objects generated in all
+:doc:`/developers/server-blitz` languages. Supported fields include:
 
 -  boolean
 -  string

developers/Server/Properties.txt

@@ -3,8 +3,9 @@ Properties
 
 As of milestone :milestone:`OMERO-Beta4`
 (:ticket:`800`), client usage of these properties has
-significantly changed. Please see the :doc:`sysadmin documentation </sysadmins/server-overview>`
-for how to configuration your server installation.
+significantly changed. Please see the 
+:doc:`sysadmin documentation </sysadmins/server-overview>`
+for how to configure your server installation.
 
 Under the ``etc/`` directory in both the source and the binary
 distributions, several files are provided which help to configure