Skip to content

Veto definer padding not correctly applied

Created by: dethodav

When using DataQualityDict.from_veto_definer_file(), if a single flag in the veto definer file includes multiple definitions with different padding, the padding is applied differently depending on the time period queried.

If a time period covering multiple flag definitions is used when ingesting the veto definer, only the first padding definition is used for all segments when queried.

This behavior should be resolved or a warning should be presented to let users know of this unexpected behavior. Here is a minimal code and file example which shows padding being applied in only one of the two cases. (Only works using LSC resources)

from gwpy.segments import Segment,SegmentList,DataQualityFlag,DataQualityDict
veto_definer = 'vdf_ex.xml'
start = 1164585617
end = 1164672017
mid = 1164670017
flags_cut = DataQualityDict.from_veto_definer_file(veto_definer,start=mid,end=end)
flags_cut.populate(segments=SegmentList([Segment(mid,end)]))
flags = DataQualityDict.from_veto_definer_file(veto_definer,start=start,end=end)
flags.populate(segments=SegmentList([Segment(mid,end)]))
print(flags_cut['H1:DMT-ANY_PI_HIGH_RMS:1'].active - flags['H1:DMT-ANY_PI_HIGH_RMS:1'].active)
<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE LIGO_LW SYSTEM "http://ldas-sw.ligo.caltech.edu/doc/ligolwAPI/html/ligolw_dtd.txt">
<LIGO_LW>
	<Table Name="veto_definer:table">
		<Column Name="veto_definer:category" Type="int_4s"/>
		<Column Name="veto_definer:comment" Type="lstring"/>
		<Column Name="veto_definer:end_pad" Type="int_4s"/>
		<Column Name="veto_definer:end_time" Type="int_4s"/>
		<Column Name="veto_definer:ifo" Type="lstring"/>
		<Column Name="veto_definer:name" Type="lstring"/>
		<Column Name="veto_definer:process_id" Type="ilwd:char"/>
		<Column Name="veto_definer:start_pad" Type="int_4s"/>
		<Column Name="veto_definer:start_time" Type="int_4s"/>
		<Column Name="veto_definer:version" Type="int_4s"/>
		<Stream Delimiter="," Name="veto_definer:table" Type="Local">
			1,"High RMS in a PI mode",0,1164670017,"H1","DMT-ANY_PI_HIGH_RMS","process:process_id:1",0,1164585617,1,
			1,"High RMS in a PI mode",5,1164672017,"H1","DMT-ANY_PI_HIGH_RMS","process:process_id:2",-5,1164670017,1
		</Stream>
	</Table>
</LIGO_LW>