Buffer Clear at start of run

r3dunlop
New Member

Posts: 30

Buffer Clear at start of run Feb 23, 2015 10:29:02 GMT -8

Quote

Post by r3dunlop on Feb 23, 2015 10:29:02 GMT -8

There is an issue where the first bunch of events coming from each GRIF-16 are left over events from the last run. I am working on ascript to decide which of these events do not belong to a run and reject them. It is not clear at this moment whether these events are from after a run is "stopped" or if they are the events that do not get cleared from a buffer before the run is "stopped". In the below example, it seems as though the first GRIF-16 cleared right away (it's first hit being at event 94). However, at around 2040 there is a flip over where everything seems to have reset, including the first GRIF-16. Following this, there are still a few events with a 15 time stamp that have everything reset. They seem to come at random, and they seem to be from a variety of channels. These only happen a few times and then go away completely. These events are easy to identify as they always come with a Channel Trigger Id of 0 for some reason.

************************************************************************
* Row * MidasId * Master Filter ID * Channel Trigger ID * Address * TimeStamp high *
************************************************************************
* 0 * 0 * 1 * 2894 * 4111 * 15 *
* 1 * 1 * 2 * 2721 * 4110 * 15 *
* 2 * 2 * 3 * 2723 * 267 * 14 *
* 3 * 3 * 4 * 2776 * 4356 * 15 *
* 4 * 4 * 5 * 2617 * 266 * 14 *
* 5 * 5 * 6 * 2725 * 267 * 14 *
* 6 * 6 * 7 * 2581 * 261 * 14 *
* 7 * 7 * 8 * 2606 * 4101 * 15 *
* 8 * 8 * 9 * 2805 * 4108 * 15 *
* 9 * 9 * 10 * 2607 * 4101 * 15 *
* 10 * 10 * 11 * 2778 * 4356 * 15 *
* 11 * 11 * 12 * 2686 * 260 * 14 *
* 12 * 12 * 13 * 2654 * 4109 * 15 *

* 92 * 92 * 93 * 2834 * 4359 * 15 *
* 93 * 93 * 94 * 2673 * 4364 * 15 *
* 94 * 94 * 95 * 2 * 1 * 0 *
* 95 * 95 * 96 * 2548 * 4099 * 15 *
* 96 * 96 * 97 * 2624 * 266 * 15 *
* 97 * 97 * 98 * 0 * 0 * 0 *
* 98 * 98 * 99 * 4 * 3 * 0 *
* 99 * 99 * 100 * 2836 * 4360 * 15 *
************************************************************************
* Row * MidasId * Master Filter ID * Channel Trigger ID * Address * TimeStamp high *
************************************************************************
* 2040 * 2040 * 8248 * 3315 * 4361 * 15 *
* 2041 * 2041 * 8249 * 0 * 3 * 0 *
* 2042 * 2042 * 8250 * 0 * 11 * 0 *
* 2043 * 2043 * 8251 * 0 * 13 * 0 *
* 2044 * 2044 * 8252 * 2 * 8 * 0 *
* 2045 * 2045 * 8253 * 5 * 3 * 0 *
* 2046 * 2046 * 8254 * 1 * 12 * 0 *
* 2047 * 2047 * 8255 * 3 * 13 * 0 *
* 2048 * 2048 * 8256 * 3 * 514 * 0 *
* 2049 * 2049 * 8257 * 2 * 1 * 0 *
* 2050 * 2050 * 8258 * 0 * 263 * 0 *
* 2051 * 2051 * 8259 * 1 * 514 * 0 *
* 2052 * 2052 * 8260 * 1 * 271 * 0 *
* 2053 * 2053 * 8261 * 0 * 269 * 0 *
* 2054 * 2054 * 8262 * 1 * 13 * 0 *
* 2055 * 2055 * 8263 * 3 * 7 * 0 *
* 2056 * 2056 * 8264 * 1 * 9 * 0 *
* 2057 * 2057 * 8265 * 1 * 263 * 0 *
* 2058 * 2058 * 8266 * 3 * 0 * 0 *
************************************************************************
* Row * MidasId * Master Filter ID * Channel Trigger ID * Address * TimeStamp high *
************************************************************************
* 2059 * 2059 * 8267 * 1 * 7 * 0 *
* 2060 * 2060 * 8268 * 1 * 15 * 0 *
* 2061 * 2061 * 8269 * 0 * 514 * 15 *
* 2062 * 2062 * 8270 * 0 * 14 * 0 *
* 2063 * 2063 * 8271 * 3 * 14 * 0 *
* 2064 * 2064 * 8272 * 2 * 263 * 0 *
* 2065 * 2065 * 8273 * 0 * 264 * 0 *
* 2066 * 2066 * 8274 * 5 * 2 * 0 *
* 2067 * 2067 * 8275 * 0 * 265 * 0 *
* 2068 * 2068 * 8276 * 4 * 8 * 0 *
* 2069 * 2069 * 8277 * 5 * 7 * 0 *

jsmith
New Member

Posts: 36

Buffer Clear at start of run Feb 23, 2015 13:31:45 GMT -8

Quote

Post by jsmith on Feb 23, 2015 13:31:45 GMT -8

So a very coarse way to fix this would be to eliminate all the high channel trigger IDs at the beginning of a run as well as all those with Channel Trigger ID's equal to zero? You'd lose a very small amount of good data, but I would expect it to be random.

r3dunlop
New Member

Posts: 30

Buffer Clear at start of run Feb 25, 2015 10:49:58 GMT -8

Quote

Post by r3dunlop on Feb 25, 2015 10:49:58 GMT -8

That is correct. I think the plan is to wait until I see a channel trigger ID < 10 and then start writing events for that channel (except for the ones that equal 0). I will also add some output to see if we for some reason never have a trigger Id < 10 for a channel, in which case we have other problems we need to figure out (like why we are missing > 90% of the data for a channel). There aren't many of these events, so typically leaving them in wouldn't be terrible, but we unfortunately need an algorithm for fixing time stamps, and these little buggers complicate that process!

r3dunlop
New Member

Posts: 30

Buffer Clear at start of run Mar 5, 2015 9:06:50 GMT -8

Quote

Post by r3dunlop on Mar 5, 2015 9:06:50 GMT -8

Here is the new table once I have removed the buffered events with my above algorithm
************************************************************************
* Row * MidasId * MasterFilter Id * Channel Trigger Id * Address * High Time Stamp *
************************************************************************
* 0 * 94 * 95 * 2 * 1 * 0 *
* 1 * 98 * 99 * 4 * 3 * 0 *
* 2 * 123 * 124 * 1 * 2 * 0 *
* 3 * 125 * 126 * 2 * 10 * 0 *
* 4 * 132 * 133 * 2 * 3 * 0 *
* 5 * 137 * 138 * 1 * 11 * 0 *
* 6 * 158 * 159 * 3 * 3 * 0 *
* 7 * 179 * 180 * 1 * 3 * 0 *
* 8 * 196 * 197 * 3 * 5 * 0 *
* 9 * 205 * 206 * 4 * 10 * 0 *
* 10 * 217 * 218 * 3 * 11 * 0 *
* 11 * 239 * 240 * 4 * 5 * 0 *
* 12 * 246 * 247 * 1 * 14 * 0 *
* 13 * 258 * 259 * 1 * 5 * 0 *
* 14 * 273 * 274 * 14 * 0 * 0 *
* 15 * 277 * 278 * 8 * 3 * 0 *
* 16 * 278 * 279 * 1 * 0 * 0 *

* 150 * 814 * 815 * 7 * 2 * 0 *
* 151 * 815 * 816 * 3 * 12 * 0 *
* 152 * 816 * 817 * 6 * 7 * 0 *
* 153 * 817 * 818 * 15 * 0 * 0 *
* 154 * 826 * 827 * 16 * 0 * 0 *
* 155 * 835 * 836 * 12 * 5 * 0 *
* 156 * 836 * 837 * 13 * 15 * 0 *
* 157 * 837 * 838 * 7 * 12 * 0 *
* 158 * 839 * 840 * 11 * 7 * 0 *
* 159 * 845 * 846 * 24 * 0 * 0 *
* 160 * 849 * 850 * 17 * 11 * 0 *
* 161 * 852 * 853 * 5 * 12 * 0 *
* 162 * 853 * 854 * 11 * 15 * 0 *
* 163 * 854 * 855 * 12 * 13 * 0 *
* 164 * 855 * 856 * 10 * 14 * 0 *
* 165 * 872 * 873 * 15 * 15 * 0 *
* 166 * 879 * 880 * 5 * 6 * 0 *
* 167 * 882 * 883 * 8 * 4 * 0 *
* 168 * 884 * 885 * 8 * 9 * 0 *
* 169 * 889 * 890 * 10 * 4 * 0 *
* 170 * 894 * 895 * 11 * 10 * 0 *
* 171 * 895 * 896 * 3 * 8 * 0 *
* 172 * 896 * 897 * 4 * 9 * 0 *
* 173 * 906 * 907 * 14 * 11 * 0 *
* 174 * 909 * 910 * 7 * 14 * 0 *

This had 1741 events removed from ~ 28000000 events. The code is called buffer clear.C but could use some multithreading. Success, and closed IMO.

jsmith New Member Posts: 36	Buffer Clear at start of run Mar 6, 2015 10:03:05 GMT -8 Quote Select Post Deselect Post Link to Post Member Give Gift Back to Top Post by jsmith on Mar 6, 2015 10:03:05 GMT -8 I agree that it is solved with regard to offline analysis. I'd like to leave this open until we confirm a DAQ fix for future experiments though.

r3dunlop
New Member

Posts: 30

Buffer Clear at start of run Apr 15, 2015 8:44:20 GMT -8

Quote

Post by r3dunlop on Apr 15, 2015 8:44:20 GMT -8

This problem is worse than originally thought. The events do not have a single "reset" point. There is no clear general way of removing these events as they do not come through Trigger Id ordered. The only way to accomplish this would be to set a run dependent threshold to remove them. I have just added something to offset fix, and I believe it will ignore these events. With the amount of events that make it through the original bufferclear script being small, I might just let this problem be. Unless anyone else has any other ideas.

jsmith New Member Posts: 36	Buffer Clear at start of run Apr 24, 2015 8:26:22 GMT -8 Quote Select Post Deselect Post Link to Post Member Give Gift Back to Top Post by jsmith on Apr 24, 2015 8:26:22 GMT -8 Status update from the DAQ side: We think we've eliminated this issue from the GRIF-16s, but it is still present in the GRIF-Cs and possibly the GRIF-4Gs.

Post by r3dunlop on Feb 23, 2015 10:29:02 GMT -8

Post by jsmith on Feb 23, 2015 13:31:45 GMT -8

Post by r3dunlop on Feb 25, 2015 10:49:58 GMT -8

Post by r3dunlop on Mar 5, 2015 9:06:50 GMT -8

Post by jsmith on Mar 6, 2015 10:03:05 GMT -8

Post by r3dunlop on Apr 15, 2015 8:44:20 GMT -8

Post by jsmith on Apr 24, 2015 8:26:22 GMT -8

Quick Reply