SuperK Outer Detector DAQ Manual

Mei-Li Chen and Jordan Goodman

Preliminary Version March, 96

• Introduction
• Description of Hardware
• Description of Software
• Operation instructions
• Trouble Shooting

In this document we describe the current status of the DAQ system for the SuperK outer detector. We give a description of the hardware, software and operating instructions. We also give some techniques for diagnosing trouble. Below is a sketch of a single quadrant of the OD DAQ system. There are four of the FSCC's one in each counting house and four sets of a DC2 and a pair of dual ported memories all in a single VME crate in the central hut. This version is still a draft - suggestion and comments to either of us will most probably be appreciated!

The FSCC (Fastbus Smart Crate Controller) is a Fastbus module that reads data from the LeCroy 1877 TDC's and soon from the Struck latches. The data from the FSCC is sent out through an auxiliary card over a long twisted flat cable to a DC2/DM115 card on the back plane (VSB - Bus) of the VME crate in the central hut. This back plane is broken into four separate pieces (one for each quadrant). The DC2 speaks to a pair of dual ported memories (DPM) via the VSB bus. Each DC2 is isolated from the other on the VSB bus. The DPM's contain 32Mbytes each of memory and can be accessed from either the VSB side or VME side. (If there are simultaneous requests from VSB and VME the DPM arbitrates). The Bit 3 is a controller which communicates via the VME bus with each of the dual-ported memories and the Sun. Data is written to one DPM while the other is available to be read out. This system is called ping-ponging. In the remainder of this document we provide information about each component and operational information.

Description of Hardware

1. FSCC
   • Fastbus controller
     • Has a Motorola 68020 processor running VxWorks 5.1
     • It is operation in the processor controlled mode although it can run in a list mode which is slightly faster.
     • The user can either talk to it via a front panel RS232 interface (9600 baud) or via ethernet.
     • This module reads out the TDC's and latch modules and pushes data out in a stream
   • Mounting - it can go in any slot in the crate.
   • Useful information
     • All functions are performed via micro-code specifically written for this module
     • A "main" program written in c sets up and monitors these function calls.
     • The FSCC boots from the Sun and downloads all it operating software. There is a bootup script which loads all software and begins execution of the data-taking program.
     • The current version of the data taking program is called sktrig_lcsr_fbc.o
     • We use the default factory settings which should never be changed!
     • If there is a lemo cable on the front panel connecting PERM-IN and PERM-OUT, Do not remove it.
     • There is a reset button on the front panel. This will reboot the computer and reload the software.
       
     

2. Auxiliary Card
   • Purpose: to convert the ouput signals (TTL) from the FSCC to RS485.
   • Mounting - it mounts on the back plane of the Fastbus crate directly behind the FSCC.
   • Switch Settings-
     • All four switches are normally open. This sets up a "loose" data taking condition where the FSCC sends data even if the DC2 is not ready. This is desirable to keep from causing hang-ups and should never happen in normal (or even supernova) situations.
     • Switch 2 closed causes the FSCC to wait for the DC-2 OK.

3. DC2/DM115
   • DM115
     • This is a level adapter and latch it basically receives signals from the Aux card a latches them. It is mounted on the DC2 card
     • It has two twisted flat cables which run from the Aux card.
   • DC2
     • The DC2 is a smart DMA controller which contains a MC68340 processor. The "68340" contains the integrated DMA part. The DC2 passes data from the FSCC via the Aux card and Dm115 to the Dual-ported memories. This controls the ping-ponging of the data between DPM's so that the data flows without conflict.
     • The DC2 (and Dm115) mount on the back of the VME crate in the central hut. They attach to the segmented VSB bus. Some care should be taken to assure that the cards plug in fully as the support system on the back plane is not great.
     • The DC2 is programmed in micro-code an it is loaded on boot-up from on-board PROMS.
   • Useful Information
     • There are two jumpers on the DM115 plugged into the middle position of the jumper block.
     • There are two buttons located on the back of the DM115 the upper one is the rest button.
     • The VSB addresses for each pair of DPM's are the same. As a result the micro-code for all DC2's is identical, making them interchangeable.

     

4. Dual-Ported Memory
   • Description
     • Each DPM module contains 32Mb of memory.
     • The module can be addressed from either the VME or VSB bus. However if they are addressed simultaneously from both busses the module holds one until the other is done. Because of this maximum speed is obtained by writing one DPM while reading the other and then switching.
   • Mounting
     • There are four pairs of DPM's - one for each quadrant. Modules have specific addresses selected by switch settings which must be observed.
     • The addresses are as follows: The VME access addresses :
       • FB-crate 1 (DMP #1 and #2): MBX base addr.= 0x08000000
       • FB-crate 2 (DMP #3 and #4): MBX base addr.= 0x0e000000
       • FB-crate 3 (DMP #5 and #6): MBX base addr.= 0x14000000
       • FB-crate 4 (DMP #7 and #8): MBX base addr.= 0x1a000000
     • The Base address for the second module in a pair is 0x3000000 higher so the base address for DPM2 is 0x0b000000.
     • There is a control section of each DPM which is a "mailbox". It is in the region just above the base. Actual data begins at location 0xYY0401000 where YY is the base number (ie 08 etc).

Software

1. The FSCC is loaded from a boot node (the Sun) which loads VxWorks and runs a script file which loads the programs. The boot directory and IP address of each FSCC is set in non-volatile ram on the FSCC. It can be set after powerup from the console or after a boot from the network by typing bootChange at the command prompt.

2. Our FSCC's have accounts called goodman with password superker.

3. Executable routines are loaded from the VxWorks command prompt by typing ld<program_name.o
   • Note : sometimes this may not work right away from a telnet or rlogin network connection. If there are problems one possible cause is that the user name is not the same as the script name. Use the iam "username","password" command to reset the user name to an authorized name on the host sun.
   • Also when changing directories in VxWorks the directory name must be in "quotes". (cd "/superk")
   • Logging in via telnet or rlogin should be done through the local host sun or other machine in the same ip group.

4. FSCC routines for data taking and diagnostics:
   • Trigger interrupt routine for data taking: - sktrig_lcsr_fbc.o
     • This routine is called is with as sktrig_lcsr_fbc(nfbc) where nfbc is the hut number. It loads a specific configuration file for each hut which contains information of the location (slot number) of the TDC's and Latch Modules.
     • This routine is normally loaded at boot up and starts executing immediately.
     • This is the routine which initializes and reads the TDC's and latches after receiving a trigger from the backplane trigger card built by UW.
     • The trigger is held off until the BIP (buffering in progress) bit on all TDC's is off.
     • Once a trigger is received and the data is readout, it along with a header is passed through the aux card to the DC2.
     • Our experience shows that triggers should be inhibited during the initialization and boot process. Later versions of the code will make an output available to ensure this.

5. Fastbus Test routines
   • FB_1877test.o -
     • This is a simple to use routine that tests out an 1877 TDC. The user can select the TDC slot number, the test mode and the number of loops.
     • This routine displays the results of test to the terminal and can be used to test a variety of 1877 functions. I t initiates a test mode then reads the data out in block transfer mode. It checks headers and prints out channel 0.
     • Once loaded this routine can be run by typing FB_1877test Then follow instructions.

   • Dsource4b.o - FSCC->aux.card->DC2/DM115->DPMs
     • This routine is used to have the FSCC push test data downstream to the DC2. A block of data is sent with a user chosen pattern. The user chooses the number of events, the number of words per event, and the 32bit pattern.
     • Dsource is useful for checking out the DC2 or other downstrean components.

   • oport_test4b.o - TDC's->FSCC->aux.card->DC2/DM115->DPM's
     • User can choose a TDC as data source
     • TDC runs at test mode and provides data
     • The FSCC reads the TDC data and them to DM115/DC2. The detailed status in TDC, FSCC (especially the status of the output-port of FSCC) are reported.

6. The DC2 micro code is stored in the EPROMs. It will run automatically when the power is on or the DC2 is reset.
   • This code manages the data flow from DM115 through VSB bus into DPMs in a ping-pong mode.
   • Data are transferred into a DPM. When one DPM is nearly full, the data stream will be leads to another DPM automatically if it is available (i.e. not already full).
   • The BAF or buffer almost full mark is set in software and it is smaller than the size of the memory so that a block of data can be written even after the BAF is reached. In practice we have such a large memory that we set it well below the full mark. If we need to the current value can be raised.
   • Through an area in the DPM memory called a Mail-Box, the code provides information which is useful for monitoring and communication between DC2 and bit3 (VME/computer interface). When a DPM is ready for reading, it informs bit3 (through apage-flag) that it is ready to have data read out.
   • On the other hand, if the computer requires reading data, the DC2 code closes the current DPM after finishing the current event and set flag to inform bit3 interface to
   • read. The data stream then be switched into a another DPM if it is available.
   • In this way, the system can take data continuously.

7. DC-2 Software Diagram

   

8. The Mail Box
   • Communications between the DC2 and the Bit 3 (and therefore the Sun) take place by using a shared memory area in the DPM. This area is located at the beginning of the base address for the memo. An outline of this is show in a figure. Included in this shared memory are flags can be set by either the DC2 or the Sun (via Bit3). There is also other counters such as event numbers, page sizes etc. A full table is show at the end.
   • Important flags and counters
     • n_events (0x20) - shows the number of events in the current write buffer. This is reset when the pages is switched.
     • Page flag. (0x2c) - This 32bit word is the key element of communications between the DC-2 and Sun. This contains bits which tell which page is being written, which pages have data, and it contains control bits which cause certain actions to be taken. The details of this are shown in a table. A couple of examples of code that show the use of this page flag are listed here:

       void switch_page()
       {int   *pg_flag_addr;
               pg_flag_addr= (unsigned int *)0x0800002c ;
                *pg_flag_addr=*pg_flag_addr|0x8000;
               return;}
       void done_page(int pn)
       {int   *pg_flag_addr;
               pg_flag_addr= (unsigned int *)0x0800002c ;
               if(pn==1)
                *pg_flag_addr=*pg_flag_addr&0xffffff6f ;
               else   
                *pg_flag_addr=*pg_flag_addr&0xfffffff6 ;
       
       return;}
       void show_flag_stat()
       {int   *pg_flag_addr;
               pg_flag_addr= (unsigned int *)0x0800002c ;
               if(*pg_flag_addr & 0x1000)printf("Page 0 being written\n") ;
               if(*pg_flag_addr & 0x2000)printf("Page 1 being written\n") ;
               if(*pg_flag_addr & 0x0080)printf("Page 1 Buffer in use\n") ;
               if(*pg_flag_addr & 0x0008)printf("Page 0 Buffer in use\n") ;
               if(*pg_flag_addr & 0x0010)printf("Page 1 available for reading\n") ;
               if(*pg_flag_addr & 0x0001)printf("Page 0 available for reading\n") ;
               return;}
       

     • command (0x50) - This is the on/off switch for the DC-2 program. A 4 in this location starts data taking and a 5 stops it.
     • Heart-beat (0x40) this location shows the DC2 heartbeat. It "ticks" (inceases) to let you know the system is alive.(Note: this is a good location to check when there is trouble!)
     • Page-Size(0x30 /0x88) These locations tell the number of event in the page 1 (/2) buffers. Note: these are only filled when the page is finished being written. They are always zero while that page is being written.

Operations

FSCC

1
2
3
10
20
99
12

If there is no latch module. 
1
2
3
10
20
99
999       

B. DC2/DM115

The current version is UMD-v1.

Trouble shooting chart

DC-2

Action:

Send number 4 into "command" located at offset 0x50 from MBX base address (see next paragraph for the MBX base address defined) to active the data taking process.

The VME address of each DPM is marked on it's P.C. board. The lowest address of each odd number DPM is the MBX base address of this pair of DPM.

Checking:

If Page_flag (located at offset=0x2c) is read as 110f1008, DC-2 is running. Go to 2.

FSCC Part

a) A quick look :

Go to b.

b)Running FSCC diagnostics

Use oport_test4b.o to send a few events out.

Checking:

• "n_event" (located at offset=0x20) shows the number of events you have sent out.
• dump out the DPM memory from data area (starting at offset 0x00040100 from MBX base address).
• If the data are there, The whole system works. Finish up.

Other wise, go to C.

c) FSCC data transfer

Use oport_test4b.o to send out one event.

Checking:

The screen will show the following information :

------------------------------------------------
Before E-O-E :
OPORT_CSR0=[number 1] & OPORT_CSR1=[number 2]
After E-O-E:
OPORT_CSR0=[number 3] & OPORT_CSR1=[number 4]
------------------------------------------------

If [XX] & [07]

Type correct routine name and try c) again.

If [02] & [03]

If [02] & [03]

Go to e).

d) Aux-card of FSCC

Action :

Put switch number 2 at OPEN position.

Check :

Try c) again.

e) Cable

most likely is caused by the cables/connections.

Check the signals from input and out put of the cable.