Merged Upmu Reduction (umred) Technical Details

Table of Contents

Interpreting the Various Log Files

Interpreting upmu_run??????.log files

The upmu_run??????.log are intended to give an event-by-event summary, for every run that was analyzed, of what decisions the upmu reduction made for all events with total ID charge > 8000 p.e. These files have two types of lines:
  1. start subrun ??????

    This type of line indicates that all of the events listed in the lines which follow it were drawn from whatever subrun number took the place of ??????.

  2. A line with 12 columns which have the following meanings:

    event number
    Save or Reject
    total ID p.e.
    stopping/decay-e flag
    stopmu1st classification
    muboy classification
    muboy fit result
    stopmu2nd fit result
    thrumu1st fit result
    fstmu fit result
    thrumu2nd fit result
    nnfit fit result

    The vast majority of lines of this type do not have values listed for all 12 possible columns, but instead are blank after some point which corresponds to the point in the logic tree at which the reduction program decided what to do with the event. The meanings of the various different columns are summarized in the following tables.

    Stopping/Decay-e Flag
    code valuecode definition
    0the event is not part of a decay sequence
    1the event may be part of a decay sequence (either a stopping µ or a decay-e)

    Stopmu1st Event Classification Algorithm
    code valuecode definition
    0through-going
    1pedestal
    2decay-e candidate (within 30 microsec of parent stopping event)
    3total ID charge < 8000 p.e. (this value is only seen internally by the reduction program; such events are omitted from the log files in order to save space)
    4stop-mu/FC/PC
    5multiple muon
    6no result due to missing OD data

    Muboy Event Classification Algorithm
    code valuecode definition
    0no result
    1through-going
    2stopping
    3multiple
    4multiple
    5corner clipping

    Fit Results
    code valuecode definition
    0no result (failed fit or poor goodness-of-fit)
    1steeply downward-going
    2near-horizontal-downward-going (probably downward-going but possibly poor-resolution upward-going)
    3upward-going
    4steeply downward-going and entry point is in the upper lid of the tank
    5near-horizontal-downward-going but entry point is in the upper lid of the tank
    6inconsistent result
    9fitter was not run by the reduction

Interpreting subrun_summ.log files

These files are intended to give a one line overview of everything of importance that happened in processing each subrun. There are always 13 columns in every line, which have the following definitions:

run
subrun
day
month
date
hour:min:sec
year
seconds spent waiting for a new file to become available (gives a rough measurement of magnetic tape library backlog)
seconds spent processing the file (gives a rough measurement of CPU burden)
number of events processed
number of events with ID charge > 8000 p.e.
number of events saved because of ultra-high total light deposition (ID charge > 1750000 p.e.)
number of events saved as upward-going candidates or associated decay-e candidates (in the case of stopping muons)

Note that all times refer to when the subrun was processed by the upmu reduction, not when the data was acquired in the detector.

Interpreting umlive_run?????? files

This file provides much of the same live time information as the umlive.log files, but states most of it as raw counts of the 50 MHz clock that sukon9 uses to stamp the acquisition time of every event, rather than in a more human-friendly system of units, so that in case any errors were later discovered in the live time counting code, there would be a back-up resource that would be more likely to have correct, or at least more readily reconstructable, information. Because most of the integral numbers in this file are derived from a 50 MHz clock, it means that they can be converted to units of seconds by multiplying by 0.00000002. The exception to this rule are the numbers which appear in the first two rows, which have a format like:
(1st line) START: Run [RUN_NUMBER] Subrun [NUMBER_OF_FIRST_SUBRUN] Event [NUMBER_OF_FIRST_EVENT]
(2nd line) START: [YEAR]/[MONTH]/[DAY] [HOUR]:[MINUTE]:[SECOND].[FRACTION_OF_SECOND] [50_MHz_CLOCK_OF_FIRST_EVENT]
and the final two rows, which look like:
(1st line) END: Run [RUN_NUMBER] Subrun [NUMBER_OF_FINAL_SUBRUN] Event [NUMBER_OF_FINAL_EVENT]
(2nd line) END: [YEAR]/[MONTH]/[DAY] [HOUR]:[MINUTE]:[SECOND].[FRACTION_OF_SECOND] [50_MHz_CLOCK_OF_FINAL_EVENT]
with appropriate numbers substituted for capitalized quantities in square brackets.

The main body of the file is composed of two other types of lines, plus a third type which appears only once, at the end of the file, immediately preceeding the final two lines described above. For each of the three other types, there are always two nearly identical, but frequently slightly different, copies of the same line. The reason for this is that two different counting algorithms are used to calculate the live time in order to check the validity of the results, and both results are printed in the file. The difference between the two algorithms is explained below. The first of the other three types of lines appears typically about once (but as a double copy, of course) for approximately every minute of real data-taking time, and has the following format: [RAW_50_MHz_TIME_OF_FIRST_EVENT] [RAW_50_MHz_TIME_OF_FINAL_EVENT] [A] [B] [C] [D] [E]
The 50 MHz clock times refer to the time of the first and final events in a given one minute interval. The lettered quantities are the total elapsed counts of the 50 MHz clock for five different types of dead time, within the one minute interval. Their meanings are explained in a table below. The second type of line which appears in the main body of the file is printed once per subrun, and has a format like:
Subrun [SUBRUN_NUMBER]: [RAW_50_MHz_TIME_OF_FINAL_EVENT] [A] [B] [C] [D] [E]
The various times listed in this line all correspond in this case, of course, to the subrun as a whole, and not to the sub-intervals which are listed in the lines preceeding it. The last type of line, which is very similar to the other two, is:
Run [RUN_NUMBER]: [RAW_50_MHz_TIME_OF_FINAL_EVENT] [A] [B] [C] [D] [E]

Interpreting umlive.log files

The purpose of this file is to provide a subrun-by-subrun account of the total live time of the analysis in a somewhat human-readable format. Therefore, the output data is stored as one run per file, and all times are stated in hours. Each subrun and each run total has 8 columns and 2 rows of numbers associated with it. In most cases, for a given run or subrun, numbers in the same column but a different row will be identical to each other, because the only difference between the two rows was the counting method used to obtain the numbers (the numbers in column 7 are the one exception to this rule, for reasons explained below). The definition of the 8 columns is as follows:
Column NumberColumn Letter Live/Dead Time TypeConditions Under Which Live/Dead Time is Counted (for the definitions of the bitwise codes: onsite: see skhead.h offsite: see head.h)
1 data-taking timeall events
2A inner detector turned off or simply has no readable dataonsite: (iand(ifevsk,z'10000000') .ne. 0) .or. (nqisk .le. 0) offsite: ((event_flag & 0x10000000) != 0) || (cali->ngood <= 0)
3B pedestalonsite: iand(ifevsk,z'00000200') .ne. 0 offsite: (event_flag & 0x00000200)!= 0
4 ambiguous live/deadonsite: (iand(ifevsk,z'00100000') .ne. 0) .or. (iand(ifevsk,z'00200000') .ne. 0) .or. (iand(ifevsk,z'02000000') .ne. 0) offsite: ((event_flag & 0x00100000) != 0) || ((event_flag & 0x00200000) != 0) || ((event_flag & 0x02000000) != 0)
5C outer detector turned offonsite: iand(ifevsk,z'20000000') .ne. 0 offsite: (event_flag & 0x20000000) != 0
6D outer detector turned on, BIP not in effect, but nevertheless no available data onsite: ((iand(idtgsk, z'00000002') .ne. 0) .or. (iand(idtgsk, z'00000008') .ne. 0)) .and. (nhitaz .le. 0) .and. ((this_event_time .lt. bip_start_time) .or. (this_event_time .gt. bip_end_time)) offsite: (((trg_id & 0x00000002) !=0) || ((trg_id & 0x00000008) != 0)) && (calo->ngood <= 0) && ((this_event_time < bip_start_time) || (this_event_time > bip_end_time))
7E BIP onsite: (this_event_time .gt. bip_start_time) .and. (this_event_time .lt. bip_end_time) offsite: (this_event_time > bip_start_time) && (this_event_time < bip_end_time)
8 ambiguous live/deadonsite: (iand(ifevsk,z'00100000') .ne. 0) .or. (iand(ifevsk,z'00200000') .ne. 0) .or. (iand(ifevsk,z'02000000') .ne. 0) offsite: ((event_flag & 0x00100000) != 0) || ((event_flag & 0x00200000) != 0) || ((event_flag & 0x02000000) != 0)

The difference between the two counting methods is summarized in the following tables:

Data-taking Time (Column 1 of the Log File)
Upper Rowdata_taking_time = (time_event_n - time_event_1)
Lower Rowdata_taking_time = (time_event_2 - time_event_1) + (time_event_3 - time_event_2) + (...) + (time_event_n-1 - time_event_n-2) + (time_event_n - time_event_n-1)

Dead Time (Columns 2 through 8 of the Log File)
Event12 345 678 9
Live/Dead Status of EventL LDD DLL DD
Live/Dead Status of Interval Between Events: Upper Row LL DD DL LD
Live/Dead Status of Interval Between Events: Lower RowL DD DL LD D

The numbers in columns 6 and 7 require some special explanation. Here is a timing diagram which describes how dead time is typically attributed to these columns:

It turns out that the "OD missing" events tend to cluster, in time, a few microseconds after BIP signals; i.e., a few microseconds after the outer detector TDCs are read out by the data acquisition system, just as the above diagram suggests. As one can see from the diagram, this systematic time clustering will tend to cause algorithm 1 to have a systematically longer OD dead time than algorithm 2.

Interpreting ofl_run??????.summary files

These files are produced by tqreal, which was not written by anyone in the upmu group, and therefore the meanings of the numbers in these files are not well understood by any of the upmu personnel. The ofl_run??????.summary files are saved merely as a precaution; there is no plan ever to look at them again unless some unexpected reason manifests itself.

Flowcharts and Internal Details of the Source Code

The live time process

Flowcharts are available in three different formats:

Systematic Effects and Known Bugs

Missing Decay-Electron Events

There was a bug in the first version of the reduction program such that decay electron candidate events were rejected if they followed a stopping muon candidate which was
  1. fitted by muboy as 1 > cosz > 0 (upward-going).
A decay electron candidate is defined as any event following within 30 µsec of an event which has been flagged as a stopping muon candidate. Decay electron candidate events were saved if they followed a stopping muon candidate which was
    1. fitted by muboy as 0 > cosz > -0.1 (near-horizontal downward-going), and
    2. not fitted by stopmu2nd (smf) at a later stage as -0.12 > cosz > -1 (steeply downward-going), or
    1. not successfully fitted by muboy, and
    2. fitted by stopmu2nd (smf) at a later stage as 1 > cosz > -0.12 (upward-going or near-horizontal downward-going).
Muboy was always the first fitter to be run on all events. Because muboy had nearly 100% efficiency for identifying and saving upward-going stopping events, stopmu2nd was rarely needed for such events, and therefore very few upward-going muons were saved with their decay-electrons. Near-horizontal downward-going muons were always saved with their decay electrons, regardless of whether they were identified by muboy or by stopmu2nd.

The entire 386 day data sample (run 5510 to run 7199), plus runs 5492 and 5494, which were processed with the 386 day sample but have been counted for live time purposes as the final two runs of the 537 day data sample, were affected by this bug. But the bug has now been repaired, and all data sets processed after the 386 day sample will have all of their decay electrons.

In order to make up for the error in the 386 day sample, all of the potentially affected parent muon events were retrieved from tape and saved a 2nd time, with their decay electrons, if any existed, in special "make-up" files. At the time of this writing, the files are stored in suk*:/net/sukop3g/mnt1/upmu/386day/dke_makeup/decay00??.zbs.

The Cut on Events with a Fitted Entry Point in the Upper Lid of the Tank

A large number of the events are saved by the merged upward-going muon reduction due to being fitted into the near-horizontal downward-going region, which is 0 > cosz > -0.1 (with downward-going defined, therefore, as cosz = -1). In the azimuthal directions associated with the smallest overburden, the number of downward-going muons varies exponentially with zenith angle near cosz = -0.1, and therefore there are a substantial number of events which "spill over" from above into the near-horizontal region due to the limited resolution of the automatic fitters. (This fact should be quite obvious and unremarkable to any reader who has ever participated in hand-scanning data; simply recall that there were many many downward-going events in the data sample which were hand-fitted with cosz = -0.11, -0.12, -0.13, etc.) In order to cut the final hand-scan sample of data down to the smallest size possible, a cut was made on events with a fitted entry point in the upper lid of the tank (z = 1810 cm) and more than 300 cm from the 1690 cm edge of the inner detector (sqrt(x² + y²) < 1390). The cut was included in the reduction algorithm because a greater importance was placed, at the time the code was written, on eliminating non-upward-going events than on saving near-horizontal downward-going events for background and other types of studies. However, in the time since the reduction code was written, an upmu group member with an interest in such studies has asked about what kind of systematic effect this cut has on the events with 0 > cosz > -0.1. The answer to this question is summarized in the following, which is an extension of SuperK official plot um408 (note that we have redefined downward-going so that cosz = 1 in this plot, in order to better correspond to the original official plot):

It should be noted that the areas shown for various zenith angles in this figure were calculated with a grid method which used a 10 cm step size. Therefore, there is a small error associated with each point along the plotted curve of approximately sqrt((10/(2 X 1690))² + (10/(2 X 1810))²) = 0.4%. As can be seen from the plot, the effective area subtended by the inner 1390 cm of the upper lid is only about 3.5 X 10**5 cm, as opposed to 1.2 X 10**7 cm subtended by the detector as a whole, therefore the systematic depletion of near-horizontal downward-going muons due to this cut is always less than a 3% effect throughout the entire zenith angle parameter space of interest.

Conclusion: The cut on events entering through the upper lid of the tank has an almost negligible effect on the overall number of near-horizontal downward-going events, because the lid does not subtend very much effective area at such a steeply grazing angle and therefore the vast majority of such events must enter through the side wall.