Day by Day Schedule


daily plans/summaries
May 30th:
We had one overnight store after development with 1.1e9 each and transmissions of just above 90% in yellow and 95% in blue. A RHIC FEC kept us off until 2AM after development yesterday. An AtR cooling fault prevented us from refilling and this store was kept until 7 AM (instead of 5AM) when APEX began. APEX is given 1 hour more at the end to compensate (it ran over by 1 hour some other time).

May 29th:
During the development today we changed the dipole trim around transition (and actually some at injection) in yellow to try compensate for the horizontal mean orbit swing observed in yellow. In addition we had a successful yellow feedback ramp (another attempt and a blue one failed though) and did feedforward the coupling and tune corrections. Since the tunes used for the feedback ramp seemed too low (in vertical) we moved them up by 0.005 up to transition (gammatT4). The store stone was not changed (actually reverted to the last physics store). The last 6x6 ramp looked quite promising and we are in the process of ramping down right now to return to physics (our next fill will be 103x103). Despite of all the problems we still totalled at about 280 ub-1 last week and 80 hours at store!

May 28th:
We are not running so smoothly. We've had all kinds of problems with BtA vacuum,  QLIs, power supplies, fan faults and the stochastic cooling pick-up tank (not a complete list!). Since very early this morning the stochastic cooling is disabled awaiting repair of one of the limit switches and/or gear. Right now, May 28th,  we are off because of problems with the Y-arc cooling water (again) and a blue QPA problem (that ended our 30 minute store *very* prematurely). In about an hour we might get back to beam (if nothing else comes up). However, at the same time, we have a transmission efficiency problem since last Wednesday when the experimental magnet polarities were switched. Yellow dropped to an unacceptable low 80+%, blue to around 90+%. We have tried a few things (decouple at injection again, propagate into the ramp, increase yellow octupoles, do some tune changes on the ramp mostly before transition etc.) but we are pretty much stuck with 1.1e9 or even below to have transmissions of about 90%. Some of it is recovered by a little less transverse beam growth though. We'll have to consider some development time asap (i.e. tune and coupling feedback ramps in both rings).

May 25th:
There were only 3 stores since Wednesday afternoon, none on Thursday during the day, none on Friday during the day (so far) :(. The PHENIX vacuum recovered with the new polarity but STAR's got much worse and reached more than 2e-9 (hasn't done that before) causing very high background that cannot be collimated. During the days we have been fighting QLIs, PS failures and fan faults together with a reduced transmission efficiency causing aborted ramps. Intensities had to be reduced to ~ 1.1e9 max. to maintain stores. STAR vacuum seems to recover some as it didn't go quite to 1e-9 with the next 2 stores last night though the intensity was somewhat higher than with the first store. On Friday we also had STAR magnet and electrical problems and an AGS tune quad failure in addition to the "standard" set of QLIs and fan faults.

May 23rd:
STAR magnet polarity change and PHENIX magnet polarity change (+- until Sun, ++ Mon  & Tue, -+ after Wed). Should be the last magnet polarity change this run. Beam was back in RHIC by 5 PM and we had a first 6x6 ramp at 5:15 pm, the first store by 5:45 PM (103x103, ~ 1.1e9 each), resulting in 25 kHz initial rates.

May 22nd:
The weekend was pretty good even though we still had PS problems (sextupoles, bi12-qf9 ...), access control system problems and abort kicker problems. Even though we had less hours at store (~100 h!) we scored higher than during the week with 114 h and ended up with about 380 ub-1 integrated luminosity :).  We tried 111x111 again but abandoned it since it didn't give more luminosity. It took us a couple of hours to commission the ramp for PHENIX ++ magnet configuration. A vacuum problem in IR8 evolved with this configuration and backgrounds because of beam gas are so large (and non-collimatable) that we had to reduce intensity for this configuration. Monday night we were off because of booster access control and blue abort kicker problems. The first store is up around 11:30 am on Tue. We are in the process of trying to turn on stochastic cooling even earlier in the store. Yellow gap cleaning is no longer necessary with cooling ON.

May 18th:
Recovery from APEX was very straight forward (they were cut short because of another QLI). PHENIX had problems with their chiller system and had to turn off electronics because of over-heating.There were physics stores over night. The day was lost to the search of an assumed ground connection in the yellow main quad bus in the 1008 area and a STAR Magnet power fault. We came back out of this at around 22:00 and had physics stores over night. Friday morning we lost due to an access system fault. The next store came up around 2 PM. We are still running with 103x103. Blue was able to support bunch intensities of more than 1.25e9 while we had problems in yellow with just above 1.2e9. Octupoles were changed by another -0.5 but didn't eliminate the problem.

May 16th:
stochastic Cooling becomes part of the up sequence and should be on for every store now (yellow only of course).

May 15th:
beams are separated in IR2 and IR10, no collisions except for IR6&8.

May 9th - May 14th:
What a (horrible) week! Pretty much only 50% of what we achieved last week, approx. 50 hours at store and 190 ub-1 only :(. Ever since the maintenance day we had failure after failure, mainly PS related and with the injectors. However, the transmission efficiency is consistently high (92-96 %) and rates are quite reproducable and up to initial rates of about 30 kHz. We are still using 103 bunches, going back to 111 is still in the plans. Stochastic cooling was now on during several stores (keeps looking good!) and making it operational is in progress.

May 4th - May 9th:
There was beam and physics stores every day during this period resulting in a total of over 100 h at store last week and about 350 ub-1 delivered luminosity :). Our store length was kept at 5 hours. However, we reduced the number of bunches to 103 per ring as problems with the rebucketing procedure and subsequent debunching seemed to be correlated with 111 bunches (not fully understood). When trying to go back to 111 we  quenched the machine twice :(. We had a few power supply failures resulting in QLIs and a blue abort kicker misfire (first!). There was no development on Tuesday but a special optic measurement period of 2 h on Monday. This measurement confirmed a beta-beat of up to 30% in IR6 and IR8. We are still using the new gammaT configuration in blue but not in yellow and we are not ready yet to try yellow. Last Sunday yellow octupole strengths were increased resulting in considerably higher luminosities and more consistent transitions. Instantaneous (corrected) peak luminosities are now routinely above 28 10^26 and a store-average of about 8 10^26 over the period of several days could be achieved (a milestone for this run!).  We are still working on RF issues (rebucketing, debunching) and the beta* correction (not yet ready). Tue afternoon we had the first store with stochastic cooling for all bunches in yellow and the whole store! Luminosity lifetime could be increased visibly :).

Apr 27 - May 4th:
There was no beam on Friday and for most of Saturday. Sunday and Monday were going well. This resulted in an integrated luminosity of about 230 ub-1 for this week. Tuesday was spent on development, trying to implement a new gammaT scheme avoiding crossing 0 chromaticity three times around transition. This worked in blue but not in yellow. Wednesday was spent on APEX and we had the 1st store up after APEX at 6:30. Thursday we tried to make the gammaT scheme work in yellow but failed. The new ramp design turned out to be too sensitive to small deviations from the "perfect" model. During the week we had two more events where all of the beam debunched at rebucketing. RF is investigating but didn't find a smoking gun yet. Even though blue transmission efficiency is consistently high (above 95% even, with yellow around 90%) it seems that the upper limit in total intensity deteriorates. Typical rates at the beginning of the store are down to about 20 kHz. Preliminary measurements indicate a beta-beat of up to about 30%. We plan to confirm this measurement during a short development/measurement time on Tuesday.

Apr 24 - Apr 27:
We had one store in the evening before APEX and maitenance. Recovery from maintenance got delayed because of disconnected gammaT quads (a consequence from maintenance). A store got established later in the evening. During the night Wed-Thu ramp orbit corrections started failing and reverting to an earlier ramp got us back into business early morning. An RF operator failure ended that store in time to begin with development. During that time we successfully tested automatic orbit corrections, took out the IR2 orbit bump, tuned a local skew quad in IR8 and tested Landau cavities. They worked with 6x6 but still had debunching issues with 111. We will keep them off from now on until further notice. Due to the orbit changes on the ramp 111 bunch ramps later failed and we had to revert back to an older successful ramp to go back to physics before midnight. The night ended early due to a QPA fault in 2o'clock. By Friday noon we had a series of more failures (Booster, RTDL) and are still off.

Apr 20 - Apr 24:
We ran physics throughout the entire weekend with very few interruptions (some PS failures, blue main QLI, pass, AtR-PS etc.). The minimim turn-around time was just above 30 minutes, however, the average is still close to 2 hours. Stores were with 111x111 and between 1 and 1.1e9/bunch with 5 hours length. Debunching at rebucketing got more and more severe (up to 10e9 loss per ring). Tuesday was setup for RF development that didn't happend because of a lead flow fault and a QLI that kept us off until almost 2pm when we had agreed to return to physics. However, it turned out that turning of the landau cavities reduced the debunching considerably and we will run like this until Thursday morning (w/o Landau).

Apr 17 - Apr 20:
"Recovery" from APEX went quite well. We had physics the night after until Thu morning when we spent 4 hours on development. RF cavities got rephased and10 Hz oscillation feedback was made operational and is now running. A local coupling issue and insufficient bump closure was detected. A ramp with 1.3e9/bunch was tested and failed. However, the root cause is not clear at this point. Blue transmission started to deteriorate over the last few days and blue background issues got worse. Therefore we reverted on Friday morning to ramp 8527 from 10 days ago that achieved 94% transmission in blue. Because of the background issues also the store stone got copied. We plan to run physics over the weekend until Tuesday. Tuesday day we plan some more development.

Apr 13 - Apr 17:
Friday ended as it began. There was no beam before Saturday early afternoon. We had one more rebucketing-debunching event, problems with the Xarc power supply, and some other PS problems and power to the yellow RF. Yesterday, Mon Apr 16, the blue mains had another one of their spurious QL drops :(. The weekend yielded about 5 long stores and 111x111 bunches routinely. A power dip ended the weekend prematurely early Monday morning. We rocevered by about noon and had another 3 stores until Tuesday. We spent 3 hours on background investigations and went back to physics. Bunch intensity is typically between 1e9 and 1.1e9, transmission above 90%. There was one store on Sunday with 75% in yellow. Not clear why since there were other stores with higher intensity after that w/o that problem.

Apr 13:
(yes, it's Friday ;)). Recovery from maintenance wasn't too bad - unfortunately we had an X/Yarc PS failure that kept us off until 9:30 pm. We had an overnight store after midnight (with the reverted "old" ramp). Thursday during the day we used feeding forward and basically made all our changes from Monday "life". At the end of development we had a successful ramp with 111x111 and close to 1e9/bunch, transmission was above 94% in both rings :). Later that night we had a successful ramp with 111 and slightly above 1e9 but we lost it before rebucketing due to an RF failure. The loss of the totally debunched blue beam cause massive quenches. After midnight, until early afternoon on Friday we were off because of a cooling tower failure that feed AGS RF. The plan for the weekend, including Monday, is to run physics with 111 bunches while moderately increasing bunch intensity.

Apr 11:
During the development on Monday we had several successful TF ramps and made lots of chromaticity corrections and new rePlay ramps for both rings (at the expense of excursion attempts in the qTrims).  While 6x6 ramps succeeded, physics stores failed with the new rePlays. We reverted to the earlier ramp and provided two physics store after midnight until Tuesday around 2pm. We tried again Tuesday afternoon (after fixing some bugs) but failed again. We reverted to the earlier ramp in the afternoon but had no beam until about midnight because of the AGS and yellow RF cavity problems. APEX was shortened even more by an operator failure in the early morning after the injectors were locked up already. Maintenance began on schedule. Recovery in RHIC is planned for 6pm.


Apr 9:
There were physics stores Thursday, Friday and Saturday. Sunday we had limited success and were off for the most part of the day because of power issues of a yellow acceleration cavity. Stores were with 93x93 and up to 1e9 resulting in initical collision rates of up to 27 kHz. Monday was dedicated to development, cleaning up both ramps. By the end of Monday we had a few successful TF ramps in yellow and blue. Both will use replay from now on. Chromaticities were measured on a couple of ramps and adjusted. With transmissions in the order of 95% we should get ready to try above 1e9/bunch.

Apr 6:
The work on the blue mains ended late Wednesday night. APEX experiments didn't quite happen since we had more problems and QLIs with other PS, the RF problem was fixed and the route cause was found later during the day of Apr 4. By the end of Wed, Apr 4 we had undone all board swaps that were made to investigate the blue main problem. No smoking gun was found but the problem didn't reappear. We went into physics mode Wed evening and started to deliver collisions on owl  Apr 5. We stayed in physics mode all Thursday and as of this morning RHIC is still running w/o problems! :) However, we used some end of store time and we will do the same today. No major development planned for today but we will install the non linear chromaticity correction in blue. The plan is to do more development (high bunch intensity ramp etc.) beginning Monday during days. At this point we fill RHIC with 84x84 bunches and about 1e9/bunch. Peak collision rates are about 20 kHz.

Apr 3:
By Friday, Mar 30, night we finally got some beam up at store and started running physics collisions. Despite some failures (tq-PS, lead flows, blue main dipole QLIs etc.) we had a quite succesful weekend (including Monday) going up to 79x79 bunches and 1e9/bunch. The blue mains and yellow RF cavities began to fail during the night to Apr 3. As of this morning we are down w/o estimate trying to find the problem with the blue mains and the cavities.

Mar 30:
there were again no stores over night (but one of poor quality). There was no problem with the new ramp Au72 however. The q89 PS keeps getting too hot, two AC units are installed close to it now. Cavity problems caused beam debunching at injection as well as at store. These problem supposedly were fixed during the day.

Mar 29:
Recovery from maintenance day was typical, 1st beam in RHIC around 8pm. There were no stores over night because of continuous problems with the q89 PS. This morning we decided to change the ramp and squeeze to 5.7 beta* in IR12 rather than 7m. The development of this new ramp will go on today and possibly tomorrow.

Mar 28:
First maintenance day this run. The last two nights and evenings we had physics stores, the 1st physics store came up Monday, Mar 26, around 9pm with 51x51 bunches and a collision rate of about 14 kHz! Our record so far is 61x61 bunches (however, the bunch intensity was reduced some).

Mar 26:
Finally had two stores with 37 bunches over night. The q89 PS problem is fixed for now, some RF issues addressed. We injected 110 bunches per ring at injection for 1 1/2 h today for scrubbing. Vacuum got improved some. We decided to declare physics with the next decent store. That happened around 9 pm.


Mar 25:
While rebucketing continues to look OK now we were fighting PS failure all night. We still want to get ready for delivering stores. Blue transmission deteriorated some though.

Mar 24:
A series of problems beginning with blown up ramps due to BBQ kickers being on, a PS failure, fire alarms, exp. accesses and blown sweeps. This delayed rebuckting commissioning and store setup quite a bit. Therefore ramp/store development gets pushed into evening of Mar 24.

Mar 23:
We had more ramp development and intensity increases during the past 2 days. First store with 48x48 over night. We had some problems with timing shifts in blue. Rebucketing not yet operational, longitudinal emittance needs to be addressed. This work is going on today. RF will test rebucketing, collimators will be moved in by the sequencer, gap cleaning will start automtically too. Pressure rise at IR4 gets close to critical with 48 bunches.

Mar 20:
Lots of ramp and TF development over the past 3 days. Blue TF resulted in a reasonable rePlay ramp and good transmissions. Somehow yellow TF worked on a few ramps but didn't result in good transmissions. We are ramping yellow w/o rePlay. A lot of chromaticity changes on the ramp were made along with radius adjustments, orbit corrections and transition tuning. Intensity is up to a standard of 0.8 10^9 and today we had a ramp with 1.1 10^9/bunch an with 37x37 bunches. First collisions with  37x37 last night (Mar 19).

Mar 17:
First optics measurements at store look good. During the day both bea transmissions got up to above 90%. Yellow TF ramp was successful and both ramps are rePlayed. Transmission efficiency deteriorated and we are working on that before increasing intensity. Ready for RF work at store (rebucketing).

Mar 16:
BBQ work still going on. Blue transmission up to 95% and yellow around 50%. Since early morning we used rePlay to ramp blue while focussing on yellow. Exp. magnets were turned on today. Beam intensity rarely above .5 10^9.

Mar 14:
Instrumentation is all setup except BBQ where commissioning for the yellow ring continues. The radial loop was closed and ramp development in yellow began in parallel to blue radial shift work.

Mar 13:
We had beam back by midnight. We got circulating beam in yellow with decent lifetimes and did some blue ramps that failed at first. The last few stones were very different from yesterday morning. We are off for the most part of the day because of a yellow PS failure. Beam is back around 4pm when we begin to do yellow capture and instrumentation setup. We should be in 24/7 operation for the next 10 days.

Mar 12:
PS work continued yesterday (w/o need for access!). Vacuum went into IR2, there was (with one short exception) no need for experimental access: we had the machine back by 5 pm :). Beam for 15 hours over night, working on blue ramps. RHIC open on Monday, 9 am. Vacuum and experiments have access, should be done by 5 pm. PS works on yellow ramps. Begin injecting into yellow ring.

Mar 10:
Both rings are cold and alarms settled. Yellow PS at injection for the 1st time. Baking in IR2 still in progress. We began ramping this morning around 6 am. Some beam at flattop by early afternoon, good transmission before midnight. BBQ and feedback commissioning still in progress.

Mar 9:
IPMs are repaired and back in the yellow pipe. Beam activities started at 1:00 am until 8:00 am when vacuum started baking. Beam activities should resume after 6 pm, RHIC is in controlled/restricted access until then.

Mar 8:
blue is cold :). RHIC is swept in the afternoon and PS work is done. Yellow is getting cold faster and we anticipate yellow cold by Saturday night/Sunday morning. Yellow PS work should begin Sunday morning. Late afternoon we discover a short in the yellow IPMs. Repair begins immediately and baking will start tomorrow.

Mar 7:
Blue cool-down is progressing well. The 4K wave is in the last sector (4/5) and RHIC should be cold by tonight and will settle over night. PS work and check-out will begin tomorrow morning.

Mar 6:
Blue cool-down progressing nicely. Blue is expected to be cold by tomorrow night. Yellow should follow 5 days later (Tue, Mar 13). We expect one more He delivery (Fri). The RF section developed two vacuum leaks. Both are fixed and pumping down again is in progress. They should not prevent us from injecting beam. PS checkout is now planned for Thursday (day).

Mar 5:
The by-pass was installed and finished last week. Everything was cleaned, pumped and vacuum re-established. Warm turbines were started over the weekend and cooldown is on its way.

Mar 1:
Oil in the heat-exchanger is the probable cause for the problems. Since we can run without this newest exchanger we'll by-pass it (should be done by the end of today) and, after cleaning, scrubbing and re-establishing vacuum cool down the rings again. The rings are drifting right now. We could be done as early as Mar 9 (most optimistic) or up to Mar 12 (more conservative), depending on progress and actual magnet temperatures when we start recooling again.

Feb. 26:
The fix didn't work and the heat exchanger has to warm up completely. Both rings are drifting. Assuming that the contamination/problem can be cleared, we'll be back with blue cold in about 10-11 days.

Feb. 25
Since we had to pause yellow cooldown yellow isn't cold yet. No PS access needed and no STAR access needed. PHENIX needs one hour after 10 am. There was no blue beam available over night and we ask for beam time during the day:
keep ring open 'til 11:00 am for PHENIX
11 am - 8 am (next day) working with beam

Feb. 24
yellow ring shouldn't be completely cold yet. We'll need yellow PS work and magnet testing in STAR during the day:
8:00 am - 6 pm access
6 pm - 12 pm (next day) working with beam
 
Feb. 23
with the yellow ring getting cold (arc by arc) we'll need a full day of PS work in the ring:
~8:30 am - 8 pm access
~ 8 pm - ~ 8 am working with beam

Feb. 22
depending on progress over night we will repeat RF capture efforts between 7 am up to 11 am. Could slide some. Access until 6 pm after that (depending on start time and PS needs).

Feb. 21, 9 am - 1 pm+?
(needs some contingency)
RF works on capture with beam available in RHIC, no access. If capture is not achieved during the 4 h tomorrow, we'll add more hours until we get there to guarantee efficient use of beam time during the next days/nights.

Feb. 21,1 pm - 8 pm
access for experiments, AGS bunch merge work continues, blue PS work if needed

Feb. 21, 8 pm until Feb. 22, 8 am
work with captured beam at injection in RHIC, begin instrumentation setup etc.

Feb. 20, after 7 pm, until 9 am Feb. 21 morning:
- work with circulating uncaptured beam, fix orbits, work on mismatch, x-and y -arc etc. (prepare for capture)






History of the RHIC Au-Au Run 2007

 

Mar 26
 physics declared, 1st store 8403, 51x51 bunches, initial rate is 14 kHz
Mar 10
 both rings cold again
Feb. 26
 both rings have to warm up again
Feb. 20
 working with beam begins (blue only)
Feb. 17
 RHIC blue ring cold, getting ready for injecting into RHIC tomorrow
Feb. 12-13
 2nd RHIC Dry Run
02/01 2007
 Congress passes a modified 2007 budget. A RHIC run (shortened and Au only) is likely now!
Jan
 At this point we are still under continuing resolution and the word as to begin or not begin with cool-down hasn't come out (yet). The continuing resolution has been expended until Feb. 15 of this year and we do not expect to hear from DOE much earlier than that about a possible "GO" or "DON'T GO" order. In case we do get the GO-order we will need about 1 week of lead-time to begin cool-down, i.e. we'd need about 2 weeks before we would have one ring cold. This could result in a beginning of a setup around Mar 1st, allowing a total of 17 cryo-weeks and about 12 weeks of physics (including ramp-up weeks though).
Nov 15-17 RHIC Dry Run.
Nov. 2-3
 APEX workshop 2006
Oct 24 BBQ review (LARP)
Oct. 17 Assumption: The RHIC Run-7 will begin with an approximate 2 week set-up period on or about Dec. 8th 2006. At this point we will work with Blue at injection while Yellow is still cooling down. Power supply check-out will happen during day-shifts (8am-6pm), beam work will happen during the night-shift (7pm-8am). Once both rings are cold and all power supplies were checked out, we will resume 24/7 shifts with beam and begin ramping (hopefully around Dec. 18).
Sep 12-14
 PAC meeting
Jul 10-12
 RHIC Retreat 2006