* Tags: 20121008B01 through 20121014C09
* Overall: End of 20121008 schedule aborted for thermal tests
Weather generally fairly good to good
Noisy blue pair in 20121008; not cut
as only symptom is high noise
Still seeing some nonlinear el nods: see analysis notes
More examples of pair-diff glitches and sinusoids after
round2 cuts; analysis forthcoming
* Odd scansets:
* Pair diff sinusoid after round2 cuts:
20121011C03_dk068 (pair diff sinusoid in some chans;
variable WX)
20121011C04, C05 (similar to C03)
20121011D01_dk068 (pair diff sinusoid)
20121011D02, D03 (similar to D01)
20121011F03_dk068 (pair diff sinusoid)
20121011F04, F05 (many chans)
* Pair diff glitches after round2 cuts:
20121008B03 (small, blue chan)
20121008B06 (small, re chan t ~ 0.24)
20121008I01_dk293 (complex, t ~ 0.2)
20121008I04_dk293 (complex, blue chan, t ~ 0.3)
20121011E01_dk068 (orange chan near t=0.27)
20121011F09_dk068 (complex, blue chan near t=0.6)
20121011I03_dk068 (blue chan, t~0.28)
* El nod nonlinearity in pair diff (see postings):
20121008C01_dk293 through C04
20121008D02, D05, D06
20121011F02_dk068
20121011F07, F08
20121011G03 through G07
* Cut by stationarity_ab:
20121008H05_dk293 (variable WX)
20121008I06_dk293 (variable WX)
20121011C06_dk068 (also cut by scanset_std)
20121011C07_dk068 (variable WX)
20121011C08_dk068 (WX doesn't look bad)
20121011E08_dk068 (WX variable; also cut by scanset_std)
20121011F05_dk068 (el nods look like bad WX)
20121011F09_dk068 (el nods look like bad WX)
20121011I07_dk068 (extreme burst of bad WX)
20121011I08_dk068 (bad WX)
20121011I09_dk068 (burst of bad WX)
20121011I10_dk068 (variable WX)
20121014B07, B08 (WX a bit noisy)
20121014C02, C03 (bad WX)
* Cut by scanset_std:
20121011E05_dk068 (trailing el nod looks bad)
20121011E06, E07 (looks OK)
20121011E09_dk068 (leading el nod looks bad)
20121011F09_dk068 (bad WX, also cut by el nod cuts)
* Cut by num_destep:
20121011B07_dk068 (no obvious symptoms in cuts=none)
Hi All,
Observing:
----------
121014 06:14:52 - still running
* 9_CMB_36b_dk113_016.sch
121014 06:09:07 - 121014 06:10:32
* mce_autotune.sch
* MASD restart, DSP driver reload & mce_reset_clean, GCP restart
after running schedule.
121014 00:29:47 - 121014 02:25:51
* starpoint_pole_bicep_summer.sch
* Tilts:
az 0° 180°
x -8.904 -9.333
y 9.723 10.349
121014 00:16:40 - 121014 00:22:19
* do_cycle.sch
* dk=0 mark OK
* cleaned GS, boot, UFB, window
* Membrane 0.14/0.44
121011 06:22:20 - 121014 00:16:40
* 9_CMB_36b_dk068_016.sch
* MCE error @ 20121013 03:53:24:
/home/bicep/gcp/control/../masd/specific/scripts/mce_fast_pipe:
line 77: 25252 / Reply does not match command. : syntax error: invalid
arithmetic operator (error token is ". ")
121011 06:15:57 - 121011 06:17:21
* mce_autotune.sch
* MASD restart, DSP driver reload & mce_reset_clean, GCP restart
after running schedule.
121011 00:53:49 - 121011 00:59:26
* do_cycle.sch
* Tilts:
az 0° 180°
x -8.904 -9.377
y 10.074 10.206
* Reset sync box serial number
* dk=0 mark OK
* cleaned GS, boot, UFB, window
* Membrane 0.16/0.48
121010 23:23:47 - 121011 00:53:49
* 8_PID_test_clean_001.sch
121010 23:07:30 - 121010 23:23:47
* 8_Filter_xfer_revisit_v4_001.sch
121010 22:51:12 - 121010 23:07:30
* 8_Filter_xfer_revisit_v3_001.sch
121008 06:41:43 - 121010 22:50:18
* 9_CMB_36a_dk293_016.sch
* Aborted before last two scan sets to run thermal tests
Telescope:
----------
- Ran Jon's thermal test schedules
- Reset sync box s/n. According to Walt, we should be good on this until
we shut down
- 1 pointing schedule
Weather
--------
- Good
Cryogenics:
-----------
* Tracking spreadsheet at
http://bicep.rc.fas.harvard.edu/bicep2/south_pole_helium_usage/Summer_2011_…
* Kathleen's Cryo Sitrep:
LHe:
• Approximately 3700 liters of useable LHe remain in the Wessington dewars.
• Approximately 250 liters of LHe were delivered to BICEP2.
• Based on a historical consumption rate of 35.7 liters/day, LHe
supplies are expected to be exhausted by January 2013
GHe:
• Out of 16 storage tanks for gaseous helium: 8 tanks are at 17.2 MPa
(2500 psi), one tank is at 13.8 MPa (2000 psi), one tank is at 11.4 MPa
(1650 psi), one tank is at 10.3 MPa (1500 psi), one tank is at 2.8 MPa
(400 psi), one tank is at 1.4 MPa (200 psi), one tanks is at 1.0 MPa
(150 psi), and two tanks are at 0 MPa (0 psi).
• Vent GHe from the LHe transfer was compressed into the storage tanks.
LN2:
• The liquid nitrogen plant off all week
• Approximately 450 liters of LN2 are currently stockpiled on station.
Analysis/software/computers:
----------------------------
- nothing to report
Cheers,
Steffen
The problem appears to be that the site latitude and longitude are zero in the
archive:
lat=median(double(d.antenna0.tracker.siteActual(:,2))/3.6e6);
lon=median(double(d.antenna0.tracker.siteActual(:,1))/3.6e6);
I do not know how the system can function without these variable being set -
they are used by the tracker thread to point the telescope.
I suspect this December data is worthless anyway. But it's still a puzzle.
Clem
--
**********************************************************************
Clem Pryke - Associate Professor - Physics
University of Minnesota,
Room 313 Tate, 116 Church Street S.E. Minneapolis, MN, 55455
Tel: 612-624-7578 Fax: 612-624-4578 email: pryke(a)physics.umn.edu
**********************************************************************
Hey,
healpix_interp complains when you try to interpolate to a coordinate that
does not exist in the healpix map. This is a nice feature. Before, though,
it was somewhat useless because we use cut-sky healpix maps, and we would
interpolate the entire timestream, not just the mapped indices provided by
make_mapind. Thus it would always complain. I've since changed the code to
interpolate only mapped indices, and this gets rid of the complaining.
However, that complain is still showing up, though in how many tags I don't
know. There is a lot of Dec 2011 where this is true. An example tag is
20111225H08_dk068
However, looking at the scanset browser, 20111225H08_dk068 looks fine. The
2nd round cut message is missing, but I'd be surprised if it's being cut.
Now, when I load in the TOD and plot the dec, it's very weird. Doing this...
load data/real/20111225H08_dk068_tod
mapind=make_mapind(d,fs);
subplot(1,2,1)
plot(d.pointing.cel.dec(mapind));title('dec');
subplot(1,2,2)
plot(d.pointing.hor.el(mapind));title('el');
...I get the attached plot. Something is very wrong with this. None of it
will end up in the map, of course. There are even tags where dec scans
between -15 and +15.
I've spot checked. Everything is fine in 20111213B01_dk113. But then
everything after 20111216B01_dk113 is deeply messed up. The starpointing
plots don't show any outliers big enough to account for the huge
discrepancy between el and dec, so I'm at a loss.
-Chris
--
**********************************************************************
Christopher Sheehy - Graduate Student - University of Chicago
University of Minnesota,
Room 220 Tate, 116 Church Street S.E. Minneapolis, MN, 55455
Tel: 612-625-1802 Fax: 612-624-4578 email: csheehy(a)uchicago.edu
**********************************************************************
Hi all,
The Harvard crew have recently run into a "bad sync" episode, which
gave me the chance to spend some time looking into the root of the
problem. I've found the root cause of this episode at Harvard and some
of the "bad sync" issues we've seen in both experiments at Pole. The
details are posted here:
http://bicep0.caltech.edu/~spuder/analysis_logbook/analysis/20121009_bad_sy…
The 2010 BICEP2 "bad sync" problems appear to be truly caused by sync
box / MCE miscommunication, but the ones in 2011 and 2012 are explained
by the GCP bug described in the note.
As for what to do about it, we should:
1) fix the underlying GCP bug for Keck during summer work;
2) add support for GCP to reset the sync number at schedule start;
3) we're not close to hitting this again in remainder of 2012, but we
should ask the operators to manually reset the sync number.
Instructions included in the note.
- Walt
On Mon, 8 Oct 2012 14:44:30 -0700
Walt Ogburn <ogburn(a)stanford.edu> wrote:
>
> * Tags: 20120929F09 through 20121005G02
>
> * Overall: Weather improves, then has another bad spell.
> Snow plots now uniformly very bright as sun comes up.
> 20121005G02 is last scanset before Jon's thermal tests
> started.
>
> * Odd scansets:
>
> 20120929F10, G01, G04, H02: leading el nod nonlinear in one channel
> (see pair diff)
> 20120929I08_dk068: glitch in pair diff around t=0.48
> 20121002D02, 03, 05: periodic signal in pair diff after round2 cuts
> 20121002E07_dk113: glitch in pair diff around t=0.43
> 20121005B04_dk248: weather changes abruptly in the middle, but
> stationarity cut doesn't kick in
> 20121005D03_dk248: weather pretty bad, but not cut; pair diff
> wobble on some chans
> 20121005E08_dk248: weather fairly bad, with glitches; cut by a
> combination of elnod_nancount, num_destep, stationarity_diff
> 20121005E10_dk248: very bright atmosphere w/ little noise;
> pair diff wobble on some chans
>
> * Cut by stationarity_ab:
> 20121002C08_dk113
> 20121002E08_dk113
> 20121002F01_dk113 - bad WX
> 20121002F02_dk113 - bad WX
> 20121002F02_dk113
> 20121002G04_dk113
> 20121002I01_dk113 - onset of bad WX
> 20121002I05_dk113
> 20120929H07_dk068 - variable WX
> 20120929I03_dk068 - variable WX toward end of scanset
> 20121005B09_dk248 - bad WX
> 20121005B10_dk248 - bad WX
> 20121005C09_dk248 - bad WX, also cut by scanset_std,
> stationarity_diff
> 20121005E05_dk248 - variable WX
> 20121005G01_dk248 - various oddities, also fails num_destep,
> stationarity_diff
>
> * Cut by scanset_std:
> 20121002E09_dk113 - bad WX, also elnod_nancount
> 20121002E10_dk113 - bad WX
> 20121005C01_dk248 - bad WX
> 20121005C02_dk248
> 20121005C10_dk248 - bad WX
>
> * Star-pointing bug:
> I saw a crash in reduc_starpoints that resulted from analyzing a
> star-pointing session that was either aborted, or present in the log
> files but not yet in the arc files. Besides the crash in
> reduc_starpoint, this also led to a line of NaNs in the pointing
> model file that crashed reduc_initial. Randol has seen something
> similar in the past. I traced this back to a bug in structcut.m and
> will put in a patch. I have also removed this line of NaNs, and a
> previous line of NaNs at 2012-april-2, from
> pointingmodel_complete_v1.csv.
Hi all,
BICEP2 team meeting today (Tues) will be 2-3pm Pacific, so that Walt can
join:
Phone: 1-866-890-3820 (toll: 1-334-323-7229) Passcode:59702175
group discussion terminates at 1 hour,
any wrap-up discussion < 90 min.
Here is today's agenda:
1) Weekly report from Pole [ Steff ]
- cryo
- bicep.rc / data transfer
- weather: cold and clear
2) Data quality / reduc czar report
This week's report [ Angiola's email ]
status of B2 2012 re-reduction [ Angiola/Randol ]
status of cut definitions / reduc to-do
Next week's czar?
3) General news and plans
- new mailing list working?
- SPIE papers posted...thanks.
4) Analysis pipeline news and plans
see notes below
- pipeline telecon Friday 10 Aug, 4 eastern, 3 central, 1 pacific
5) Other Postings
2012 Aug 1: Generating Synfast Maps with WMAP noise (JET) Updated Aug 3
2012 Aug 6: Sim 1205/1206 (CDS)
2012 Aug 6: Guide to running a simset (CDS)
end of today's agenda.
Notes from yesterday:
-----------------------------------------------------------------------
6 Aug 2012 B2 analysis/pipeline telecon
On the line: John, Randol, Colin, Abby, Chris, Angiola, Jon
B2 reduction (2:00-2:30 eastern) -----------------
status of Angiola's B2 2012 re-reduction
[ Angiola / Randol / Chris to summarize ]
disk space on bicepfs1 and panlfs
- John to ask P Edmon again for quotes [done, quotes for 56 TB more
are coming]
status of thermal stability from Jon:
- Jon is working on season-split (alternate days?) NTD jacks
- will post updated outline of thermal stability project/paper for
tomorrow
Pipeline telecon (2:30-3:30 eastern) ----------------------
WMAP-like noise added to synfast input maps
- Jamie Tolan not on, but briefly discussed his posting (further
discussion Tue/Fri)
- IDL-free, incl. matlab fits binary writer -- awesome.
- noise levels (e.g. 67uK) defined for what area?
- Should we consider more than W-band? Randol says yes, V-band is
what he used.
Abscal: Chris recently changed get_ukpervolt for B2 to 3071, from 3605.
- We need clear abscals for each different B2 epoch!
- Angiola to ask Justus for summary of work done on this, incl. vs.
ell consistency
Plan for closed loop sim/deproj tests:
1. input sim systematics @ best guess B2 level. What are these?
a. relgain--we need closure from per-det abscal or per-pair T/dif
leakage [ Justus / Walt?]
http://bmode.caltech.edu/~spuder/analysis_logbook/analysis/20120117_wmapxco…
b. diff pointing - beams_ab_cmb_bicep2_20100101.csv (per det fits
from CMB, Walt)
c. diff beam size - ?
d. diff ellipticity - ?
2. generate timestreams using 3 sim algorithms:
a1. linear systematic introduced using same template as deproj
a2. best realistic model
a3. best realistic model x2
a1 gives us a check to numerical accuracy (for deprojection on nopol
input) and a clean test of E-to-B only (for deprojection on EnoB input).
a2 or a3 is what we use by default; the ability to increase accuracy
by 2x (e.g. doubling nsides, or the number of beamwidths used for
interpolation) and verify negligible difference in coadded output maps
assures us that sim timestreams are “close enough to reality”.
3. Deproj using linear basis constructed from derivatives.
what is status of gen_templates? [Randol]
width/ellipticity? strategy for interpolation?
Randol's posting here serves as a model:
http://bicep.caltech.edu/~spuder/analysis_logbook/analysis/20120404_diffpoi…
We need to identify a limited set of tags to run this on, and run full
signal (nopol, EnoB, BnoE) and scaled-noise sims.
Each of the signal sims should be generated without systematics, and
with systematics using the 3 sim algorithms.
Each of these simsets should be mapped and processed to aps with and
without deprojection.
We do this first for relgain, diffpoint, and diffwidth separately, and
then in combinations.
Hi All,
Observing:
----------
121008 06:41:43 - still running
* 9_CMB_36a_dk293_016.sch
121008 06:28:40 - 121008 06:30:05
* mce_autotune.sch
* MASD restart, DSP driver reload & mce_reset_clean, GCP restart
after running schedule.
121008 01:42:55 - 121008 03:10:26
* starpoint_pole_bicep_summer.sch
* Tilts:
az 0° 180°
x -8.904 -9.377
y 10.217 10.250
121008 01:14:07 - 121008 01:19:42
* do_cycle.sch
* dk=0 mark OK
* cleaned GS, boot, UFB
* Membrane 0.13/0.44
121007 04:14:04 - 121008 01:14:07
* 8_Filter_xfer_revisit_v2_001.sch
121007 02:56:03 - 121007 04:14:04
* 8_PID_test_001.sch
121007 02:39:45 - 121007 02:56:03
* 8_Filter_xfer_revisit_v1_001.sch
121005 06:52:47 - 121007 02:37:58
* 9_CMB_36a_dk248_016.sch
* Aborted to run thermal tests
121005 06:44:45 - 121005 06:46:10
* mce_autotune.sch
* MASD restart, DSP driver reload & mce_reset_clean after running
schedule.
121005 01:17:01 - 121005 01:22:38
* do_cycle.sch
* Tilts:
az 0° 180°
x -9.849 -9.608
y 10.074 10.305
* bicep33 rebooted and GCP & masd restarted before starting schedule
* dk=0 mark OK
* cleaned GS, boot, window
* Membrane 0.12/0.40
121002 07:07:09 - 121005 00:52:04
* 9_CMB_36a_dk113_016.sch
121002 07:03:41 - 121002 07:05:05
* mce_autotune.sch
* MASD restart, DSP driver reload & mce_reset_clean, GCP restart
after running schedule.
Telescope:
----------
- Ran Jon's thermal test schedules
Weather
--------
- Good
Cryogenics:
-----------
* Tracking spreadsheet at
http://bicep.rc.fas.harvard.edu/bicep2/south_pole_helium_usage/Summer_2011_…
* Kathleen's Cryo Sitrep:
• Approximately 3900 liters of useable LHe remain in the Wessington dewars.
• Approximately 250 liters of LHe were delivered to BICEP2.
• Based on a historical consumption rate of 35.7 liters/day, LHe
supplies are expected to be exhausted by January 2013
GHe:
• Out of 16 storage tanks for gaseous helium: 7 tanks are at 17.2 MPa
(2500 psi), one tank is at 16.5 MPa (2400 psi), two tanks are at 13.8
MPa (2000 psi), one tank is at 10.3 MPa (1500 psi), one tank is at 2.8
MPa (400 psi), one tank is at 1.4 MPa (200 psi), one tanks is at 1.0 MPa
(150 psi), and two tanks are at 0 MPa (0 psi).
• Vent GHe from the LHe transfer was compressed into the storage tanks.
LN2:
• The liquid nitrogen plant tuned off 01 October 2012 and is still running
• Approximately 10L of LN2 were delivered to the machine shop
• Approximately 500 liters of LN2 are currently stockpiled on station.
Analysis/software/computers:
----------------------------
- Rebooted bicep33
Cheers,
Steffen
Hi All,
Just a heads up for those looking at the data. We changed the air
handler motor in the room below the mount. If anything, there should be
less vibrations, but one never knows. Please let me know if you see any
disturbances in the data, starting with the CMB schedule starting 121002.
Nasty, cold job. Didn't envy the guys. They did well and we didn't have
to send Cynthia down into the crawl space :-)
Cheers,
Steffen
Hi All,
Observing:
----------
121002 03:07:11 - 121002 04:15:29
* starpoint_pole_bicep_summer.sch
* Tilts after re-leveling mount:
az 0° 180°
x -9.168 -9.168
y 10.042 10.217
121002 01:44:12 - 121002 02:47:51
* starpoint_pole_bicep_summer.sch
* Tilts before re-leveling mount:
az 0° 180°
x -9.157 -9.168
y 9.393 10.536
121002 01:37:13 - 121002 01:42:51
* do_cycle.sch
* dk=0 mark OK
* cleaned GS, boot, UFB, LFB, window
* Membrane 0.10/0.40
120929 07:18:38 - 121002 01:03:51
* 9_CMB_36a_dk068_016.sch
* MCE Error @ 20120930 01:11:33:
/home/bicep/gcp/control/../masd/specific/scripts/mce_fast_pipe:
line 77: 25252 / Reply does not match command. : syntax error: invalid
arithmetic operator (error token is ". ")
* MCE Error @ 20120930 08:40:4:
/home/bicep/gcp/control/../masd/specific/scripts/mce_fast_pipe:
line 74: 50000000 / 33 / Reply does not match command. : syntax error:
invalid arithmetic operator (error token is ". ")
/home/bicep/gcp/control/../masd/specific/scripts/mce_fast_pipe:
line 77: / 140 : syntax error: operand expected (error token is "/ 140 ")
* MCE Error @ 20121001 02:11:03:
/usr/mce/mce_script//script/mce_run: line 133: 7495 Segmentation
fault mce_cmd -q -o / $lock_opts -X "acq_config${acq_config_suffix}
$filename rc$rc" -X "acq_go $numpts"
120929 07:10:32 - 120929 07:11:57
* mce_autotune.sch
* MASD restart, DSP driver reload & mce_reset_clean, GCP restart
after running schedule.
120929 01:37:14 - 120929 01:42:50
* do_cycle.sch
* Tilts:
az 0° 180°
x -8.904 -9.608
y 9.371 10.833
120926 07:15:48 - 120929 01:15:40
* 9_CMB_36d_dk293_015.sch
120926 07:11:50 - 120926 07:13:15
* mce_autotune.sch
* MASD restart, DSP driver reload & mce_reset_clean, GCP restart
after running schedule.
120926 02:02:29 - 120926 03:28:42
* starpoint_pole_bicep_summer.sch
* Tilts:
az 0° 180°
x -8.674 -9.641
y 9.723 10.437
120926 01:49:34 - 120926 01:55:11
* do_cycle.sch
* dk=0 mark OK
* cleaned GS, boot, UFB, LFB, window
* Membrane 0.12/0.40
120923 07:38:24 - 120926 01:27:28
* 9_CMB_36d_dk248_015.sch
* Station wide power outage 20120925 20:31:19 - 20120925 20:31:45
Telescope:
----------
- A few non-fatal MCE errors
- re-leveled mountl three pointing schedules
- Motor for load leveler room air handler being changed as I write this
- Power outage didn't affect the telescope. Just had to twiddle with the
cryo monitoring serial ports to get things working again.
Weather
--------
- Mostly windy and overcast. Cleared up today
Cryogenics:
-----------
* Tracking spreadsheet at
http://bicep.rc.fas.harvard.edu/bicep2/south_pole_helium_usage/Summer_2011_…
Boiling off remaining liquid in Simon and pumping it into the half rack.
* Kathleen's Cryo Sitrep:
LHe:
• Approximately 4100 liters of useable LHe remain in the Wessington dewars.
• Approximately 250 liters of LHe were delivered to BICEP2.
• Based on a historical consumption rate of 35.7 liters/day, LHe
supplies are expected to be exhausted by January 2013
GHe:
• Out of 16 storage tanks for gaseous helium: 8 tanks are at 17.2 MPa
(2500 psi), one tank is at 13.8 MPa (2000 psi), one tank is at 10.3 MPa
(1500 psi), one tanks is at 5.2 MPa (750 psi), one tank is at 3.2 MPa
(460 psi), one tank is at 1.4 MPa (200 psi), one tanks is at 1.0 MPa
(150 psi), and two tanks are at 0 MPa (0 psi).
• Vent GHe from the LHe transfer was compressed into the storage tanks.
LN2:
• The liquid nitrogen plant tuned on 28 September 2012 and is still running
• Approximately 35L of LN2 were delivered to A-119-S (Taylor)
• Approximately 300 liters of LN2 are currently stockpiled on station.
Analysis/software/computers:
----------------------------
- Nothing to report
Cheers,
Steffen
