This is a collection of ideas for a next-generation Epics-based data collection software It is a work in progress.

Some initial ideas

• Use Epics for maximum coverage of beamlines and hardware.
• A GUI is necessary for setting up and viewing simple EXAFS and other scans.
• The control system must be fully scriptable, so that "macros" can be run.
• There should be a well-defined low-level API so that anything from a macro to a new and different GUI could be written

• Remote access is very highly desirable. Working through firewalls should be considered up front.

• Tight integration of advanced espresso machine technology

One approach would be (more details here and some more fleshed-out ideas here):

1. a client/server model, with a single server listening for requests from clients, and queuing up commands to be run.

2. a database used to hold the pending "commands" (this would also serve as a log of all events).
3. local client (GUI program) simply sends request to the server, which will do the actual data collection
4. web-based access to the database would provide remote access.

Issues that are left un-addressed by this approach (but which could be important) are

1. remote access to data / data security. Like, how should a web-client fetch data and display it?
2. data format and storage. I'd suggest HDF.

Bruce's  comment on the database aspect of this:
I would like to see the user's interface with the system be
"sample-centric".  That is, you could imagine a sample being assigned a
barcode or an rfid tag.  Then the user could register a sample from home,
and do things like enter comment lines, set up scan parameters, or import
the CIF file you will later need to analyze the sample.  At the beamline,
then, the sample could be scanned and all that information is then
associated with the measurement.  Another interesting thing about having an
easily identified sample is that you could ask a question like "What
temperature was the mono crystal when I was measuring this sample?" and
easily get the right answer.

The Epics-only Question

Whether to target Epics alone, or to aim for supporting multiple "control system backends" is an important question. Epics is widely used at the APS and other synchrotrons, has an active community and has support for a really large number of hardware, especially (for these needs) detectors. However, Epics is not used exclusively. Notably, TANGO is used at many XAFS beamlines. Other systems exist as well.

Matt's opinion on the Epics-only Question (partly addressed in this conceptual design ):

In my opinion, targeting multiple backends is a fine idea in principle, but it may slow progress
First, it's not at all clear that the various systems are really compatible enough.  I also don't know that we can find people
who know multiple systems.  The idea of abstracting every control-system concept is likely to limit the end result,
Epics provides a rich and very simple mechanism to address every aspect of a device that Epics supports.  For example,
knowing the right address, one can move a motor, change a monochromator energy, make a complex move of a diffractometer, tell a detector to collect an XRD image, change the peaking time on a solid-state detector, etc.   A control program can then expose this interface to the beamline-scientist, so that setting up the system is simply a matter of supplying the needed addresses.  It also allows '''unplanned''' capabilities and customization for beamline-specific and new hardware.  I am reluctant to give up this power and flexibility.  Systems like Epics (TANGO, perhaps others) have similar designs and are probably possible to support.   Other well established network protocols, such as HTTP (say, for web-cameras), and possibily USB are probably tolerable, though it would seem that using an Epics interface should be preferred.
I see no reason to have direct support for other devices (serial, etc) or for proprietary interfaces.
On the other hand, the control system can and should put the Epics interface "very low". For an Epics interface, I believe that Channel Access put(), put_with_wait(), and get() are all necessary -- they may be sufficient.  Currently, for example, SPEC supports these (not sure about put_with_wait() actually).

Update 18-Sept-2009

OK, I'm viewing this as a blog on data collection software now...

I've been working on a more comprehensive draft document on data collection software, thinking about a couple issues:

1. data formatting for collection, storage, visualization. I was very pleased to see this:

   • http://www.esrf.eu/UsersAndScience/Experiments/TBS/BLISS/documentation/bliss-talks-1/NeXusDiscussionIntroduction.pdf as it closely matches my own impressions (pro-hdf5, lukewarm on NeXuS), so I'm happy to see such thoughts at ESRF (and CHESS??)

2. Non-Epics interfaces, including SSRL's control system. I talked at lenght with Sam Webb last week, and I think we can find common ground -- I need to make a draft doc and proto-type system.