Introducing: The Grid

Odds are, somewhere near you right now, computers are whirring all day and night frantically processing the latest ATLAS data. They’re part of a system called the Worldwide LHC Computing Grid, or just “the Grid” for short, and without them we’d drown in the data spat out by our detector. So what is the Grid, exactly, and how do we keep it so busy?

We always try to have a detailed simulation of the physics we’re trying to understand. This year, that means simulating about two billion proton collisions in ATLAS, which, if you were to start that on your laptop right now, would take somewhere around 15,000 years to finish. We don’t have that kind of time! Enter the Grid. The Grid ties together something like 100,000 laptop-equivalents sitting at universities and labs all around the world. We can ship chunks of data from our detector over to one of those computers, set it to work, and tell it to send us a message once it’s ready for the next chunk. It’s a kind of massive parallel-processing that allows us to churn through our data at a much higher rate than would ever be possible if we only used the computing at CERN.

We’ve come a long way in the last 30 years. For the discovery of the W boson at CERN in 1983, a picture of every event was sent to the “Megatek” facility in Switzerland, where one-by-one a person analyzed them by eye (really!). The Tevatron experiments need significant computing resources, but most of it can be hosted on-site at Fermilab near Chicago. Today, the LHC experiments are really the first physics experiments (that I know of!) that desperately need the Grid to stay operational. We could put the entire city of Geneva to work, and each person would have to process three collision events an hour, 24 hours a day, 365 days a year, to keep up with all our recorded and simulated events! The Grid makes that task much more manageable – but not easy, by any stretch of the imagination.

There are people around the world constantly working to take care of the Grid and keep it running smoothly for us. One year’s worth of operation costs around 15M Euros, give or take a bit, and we want to make sure we (and the taxpayers) get our money’s worth. About half of that goes straight to keeping the computers cool – without a fancy data center on the scale that Google and Apple build, it can be expensive keeping that many machines from overheating! No computer is retired if it can be upgraded or repaired cost-effectively. And we constantly work on improving our software, knowing that if we can make it 10 percent faster we can save over 1M euros. Well, that’s only half-true, of course; if our software runs faster, we’ll use the spare time to write even more papers!

Despite all this fancy computing, we aren’t anywhere close to developing SkyNet – don’t worry. Apple’s and Google’s new data centers, which reportedly run about $1B USD each, have computing resources that are 10-100 times as much as our entire beloved Grid. Of course, we don’t have that kind of money! But some times, late at night, waiting for a computer hundreds or even thousands of miles away to phone home and tell me it’s done with its little job for the day, I dream of all we could do with just one of those buildings…

Zach Marshall

Zach Marshall is a research fellow at CERN. Over the last five years he has alternated between developing software for ATLAS and abusing that software for the good of physics analyses.

Heavy ion time!

It’s heavy ion season at the LHC again: the final month before the Christmas shutdown, where the LHC experiments take a collective twist and switch from studying proton collisions to studying lead ion collisions instead. I wasn’t quite sure what to expect this year. In last year’s run, we were swept up in the fervor of publishing ATLAS’s dijet asymmetry result, where we first observed jet quenching. Getting a chance to study this phenomenon was the exact reason I wanted to do heavy ion physics at the LHC, and given that you rarely get exactly what you want, it isn’t an overstatement to say that those few weeks we’re the most exciting of my life. However, being none-too-gently tossed around by a swirling nexus of stress, pressure and exhilaration it is not a situation I am eager, or equipped to repeat on an annual basis.

Whether I was ready or not though, the perfect storm was brewing this fall, with the possibility of not only lead-lead collisions, but also proton-lead collisions during this year’s ion run. A year ago, even the most optimistic scenarios suggested that these collisions would not be a part of the LHC program until the 2012 run. However, in recent months the LHC team made rapid progress in preparing for these sorts of collisions. While nothing was guaranteed, we were told a slim chance existed that proton-lead collisions would occur for a short time during this year’s run. Like the situation with lead-lead collisions last year, many important measurements can be made even with a small amount of data. In ATLAS, our heavy ion group jumped on this opportunity and planned a comprehensive strategy for quickly analyzing this data, having learned from last year that if the physics was sound we would receive the full support of the collaboration. As the machine operation plans came into clearer focus, our expectations were raised from the potential of unstable, low-intensity beams (where we could measure simple properties of the collision system), to several hours of high-intensity running, which would enable us to do more advanced measurements.

In ion-ion collisions, the goal is to produce hot, dense nuclear matter called Quark Gluon Plasma. But studying this is complicated by the fact that we don’t fully understand some aspects of nuclear matter, even at much lower temperatures. Proton-ion collisions are of interest because Read more »

Joining forces in the search for the Higgs

Experimental Higgs boson exclusion limits for combined ATLAS and CMS data. *Full explanation of this plot underneath the blog!

Today we witnessed a landmark LHC first: At the HCP conference in Paris, friendly rivals, the ATLAS and CMS collaborations, came together to present a joint result! This ATLAS-CMS combined Higgs search was motivated by the fact that pooling the dataset increases our chances of excluding or finding the Higgs boson over those of a single experiment. This is the first example of this kind of scientific collaboration at the LHC, and the success of the whole endeavor hinged on a whole host of thorny issues being tackled…

Discussions about combining our Higgs search results with CMS’s first started over a year ago, but before we could proceed with any kind of combined analysis, we had first to jointly outline how on earth we were going to go about doing it. This was no small undertaking; although we’re looking for the same physics, the ATLAS and CMS detectors are very different beasts materially, and use completely independent software to define and identify particles. How can we be certain that what passes for an electron in ATLAS would also be picked out as such in CMS? Not only that – the Higgs working groups from ATLAS and CMS are made up of several hundred people apiece, making the challenge of combining results not only a technical one but also a sociological one.

From the start of the year, experts from both experiments started meeting regularly to try to converge on the combination procedure. First up, crucially, we had to ensure that we were both using consistent theoretical estimates of the rate we expected the Higgs to be produced (its ‘production cross section’) and of the relative probabilities of it decaying to each of the various signature collections of particles we use to spot it (so-called ‘branching ratios’). Read more »

Charming results that have got everybody thinking…

I’m writing from the annual Hadron Collider Physics Symposium, which began on Monday in Paris, France. It’s organised jointly by LPNHE and the University of Paris VI & VII, with an excellent location right in the heart of the Latin Quarter. HCP is a fun conference with only plenary talks, which means that I’ve had the chance to attend talks on a wide range of subjects including many quite remote from my usual areas of expertise.

The biggest surprise so far came not from ATLAS, but from LHCb. On Monday they showed a preliminary measurement of direct CP violation in the charm sector. That may sound rather impenetrable to the uninitiated, so let me elaborate: What they measured is the difference in the asymmetry between decays of D mesons (containing charm quarks) to kaons (containing strange quarks) to pions (containing only up/down quarks). By asymmetry here, we mean that the probability for a D0 to decay to kaons is not the same as the probability for a D0 to decay to pions. Then the final measurement is made by calculating the difference in each of these asymmetries between D0 mesons and anti-D0 mesons. They measure this compound quantity because a number of effects cancel to reduce the final uncertainty. The measured value of the CP violating asymmetry, A­_CP, is -0.82 percent, with a statistical uncertainty of +-0.21 percent and a systematic uncertainty of +-0.11 percent . Put simply, this means that the observed asymmetry is non-zero, with a significance of 3.5 sigma or a probability greater than 99.7 percent. This level of certainty is what particle physicists typically refer to as evidence.

Certainly measuring such CP violation in the first place is exciting, but what really matters is whether the value is consistent with what we expect from the Standard Model. Read more »

The Power of Perception

If you ask a child to draw a physicist, they’ll usually draw you a disheveled man in a lab coat. But looking around the hundreds of physicists eating lunch at CERN today, I saw many women, only one or two that could be classified as disheveled, and zero lab coats. Yet this image persists.

To bring a more modern and realistic image of physicists to school kids, myself and 12 other women from CERN participated in last weekend’s Expand Your Horizons conference. This all-day event is for girls between the ages of ten and fourteen to come and spend a day doing science experiments. From CERN we organized three different sessions: one involving experiments with liquid nitrogen; a second where the girls constructed their own cloud chamber; and a third where they did experiments which explained how the Higgs mechanism works, what is Dark Matter and other ‘Big’ questions. The message of this event is two-fold: science is fun and girls do science.

Manuela Cirilli helping girls build a cloud chamber at EHY Geneva 2011

This is my favorite of all outreach events. I love explaining the physics at CERN to this age group. One of my classic opening dialogs is the following…

Me: How old are you?
Girl: Ten
Me: How old is the universe?
Girl: Uh… 100?
Me: It is 13.7 billion years old!
Girl: Wow!!! That’s crazy! [A short pause] So prove it!

Ten to fourteen is a great age to be teaching science. It’s an age where kids are young enough to be wowed, old enough to ask tough questions, but not too old that they stop asking those questions. It’s an age where they’re not afraid to challenge the fundamentals that we adults often assume are proven truth.

As a woman in science, I also feel the importance in sharing my life and experiences with girls of this age. As I was sitting in the cafeteria with the 260 girls participating in this event, I asked myself: Who were my role models when I was this age? Read more »

ATLAS in Paris for a pop-up launch

It’s not every day you get to explain ATLAS to a group of journalists with just a pop-up book as a prop. But, as some readers might already know, this is no ordinary pop-up book. ATLAS and the LHC leap from the page in incredible detail thanks to paper engineer Anton Radevsky’s wonderful designs. A new edition of the book has just been released in French, so at the end of last month I found myself travelling in to the centre of Paris from Orsay for the press launch.

The press launch took place inside a tent in the gardens behind the Trocadéro Centre. Like the book, this was no ordinary tent – the area used for the launch had a full floor-to-ceiling view of the Eiffel tower! We were there as guests of the French research council, the CNRS, who run an exhibition of their research and a busy programme of activities during Science en Fête. The exhibitions were full of visitors, many of whom seemed very interested in what we were doing with the book…

The journalists attending had varied interests and included a correspondent for Les Echos, a writer for a music web-site, the editor of CNRS magazine and a freelancer for Le Figaro. Many others had asked for review copies off-line and the book will soon appear in magazines such as Science et Vie and Science et Avenir.

As the journalists gathered round tables over coffee and croissants, Emma Sanders, the book’s author, ran through the many pop-ups. She introduced the experiment, told some stories about the book’s design process and managed to manipulate the pages without dropping the calorimeters or creasing the muon big wheels in front of such an important audience. My role was to follow Emma and give an update of ATLAS results so far.

Just a few days before the press launch, ATLAS had exceeded 5 inverse femtobarns of recorded luminosity in 2011. I explained that it was an important milestone for ATLAS and demonstrates the outstanding performance of the LHC. The results now publicly available, Read more »

The Longest Shift

The clock just turned 2:00 a.m., again, on LHC Page One – the machine’s online status viewer –  and I’m pondering just how I ended up on the longest shift of the year. I normally love this evening, snuggling under a warm comforter for that extra hour of late-autumn sleep. But, this year, on the very hour we ‘fall back’, I am cuddling with the controls of ATLAS’s 46 meter long muon spectrometer, a bar of chocolate and an extra cup of coffee. So be it.

View from the Muon Spectrometer desk

To be perfectly honest, I don’t mind a bit. I’m playing my small role in the history of one of the greatest scientific accomplishments of mankind. If I do my job well and keep the muon subsystem running properly, it could very well measure the four muons coming from a ‘golden decay’ of the Higgs Boson. Or maybe it’ll measure the two muons coming from the decay of a new force carrier particle, or perhaps a whole plethora of muons coming from the decay of an extra-dimensional microscopic black hole, or maybe something I cannot even imagine. Yes, that would be the most fun, wouldn’t it?

Of course, the reality is that I’m never really going to see exactly which events were recorded during my shift, and neither would the existence of those events, on their own, be sufficient for such important discoveries. That requires the accumulation and analysis of the data, as well as its statistical combination with the trillions of other events taken while I was either home in bed or at work in my office across the street. But that can’t stop me from dreaming as I check the high voltage and the data quality plots.

No, reality might not disturb my dreams, but the sound of a toilet flushing nearly always does. And yes, I am still speaking about shift. That’s just one of the sounds our run coordination installed in the control room to keep us aware of the conditions of the LHC. Read more »

Spreading the idea of day-to-day life in extreme science

Recently I was invited to give a talk about ATLAS in a series of public talks called TEDx. It took me a long time to think of what to say because I wanted to say something about ATLAS that was interesting, but at the same time not give some Brian Cox-y sort of talk. Don’t get me wrong – I think what he’s doing for the field is great, but the fact of the matter is that I am not Brian Cox and I think even if I tried to give talks like he does, I wouldn’t get very far.

It’s a running joke at CERN that the BBC usually seems to get wind of results before we get the internal memo; I often end up reading about the latest results from the LHC in my lunch break on the Internet. This is the side of CERN that the public gets to see, that and those grand talks all about the detectors, the accelerator, the nature of the universe and why we’re here and all of these sorts of existentialist questions. But my experience isn’t sitting under a tree scribbling esoteric thoughts down on envelopes. Actually I’m sending emails like the rest of the world and spending a lot of time in meetings discussing which datasets should be replicated to which sites and fiddling with the video connection ’cause the folks in Japan can’t hear the audio. So I decided that I would talk about this: daily life in extreme science.

My first rehearsal didn’t go down very well, I told the committee all my detailed thoughts about ‘from the control room to the conference’, and then the first question someone asked me was: “What’s CERN?” Read more »

Dispatch from the dispatch: Musings from the Tevatron’s final run

The first time I drove to Fermilab as a grad student, I got kind of lost. However, once I remembered my adviser’s words of advice, it was suddenly easy to find the strangely shaped Wilson Hall, a.k.a. “the highrise sticking out of the Prairies”. During my PhD years with the CDF Collaboration, I went there many a time – to attend meetings, to take shifts. The Chicago area summers were harsh, and so were the winters. On early morning shifts over the Christmas week, I realized that all too well. CDF had over five hundred collaborators at that time and this was my first introduction to a big experiment. Despite its size, everyone still seemed to know each other and it was one big happy family.

And so it was that it almost seemed like a family reunion, not the funeral of the Tevatron, when I walked in to the CDF control room for one last time on September 30th. The proton/anti-proton beams kept on colliding and CDF continued to record data with its usual alertness, with ATLAS’s own Alison Lister (another ex-CDF-er) acting as shift leader (called sci-co in CDF parlance, and ‘psycho’ lovingly). As past and present CDF-ers gathered and talked, one could not help feeling how strong the connection between the Tevatron and the LHC is in general, and between CDF and ATLAS in particular.

Seeing so many ATLAS colleagues there when they hit the big red ‘stop’ button in the main control room, and at parties thereafter, I realized that even though the Tevatron is officially retired, ATLAS (and CMS) will carry its legacy forward. So many of us learned our basics there and got to know how it is to work in such big collaborations. Thankfully we did not adopt all the CDF terminology – we used to find ourselves getting congratulated for the wrong reason whenever we said we were ‘blessed’ (we use the word ‘approved’ for the same thing in ATLAS – our results getting the official seal of approval from the rest of the collaboration), and many were confused when we talked about ‘godparents’ (equivalent to ATLAS’s ‘Editorial Board members’).

The Tevatron’s final run was a sombre occasion for many who had worked in Fermilab all their lives and retired as the machine shut down. For others, analysis of the data will continue, and the results that come out of those analyses will certainly improve our understanding of nature. For those of us who moved across the ocean it was a homecoming, albeit sad, but also helped to put things into perspective. We are incredibly lucky to be part of a generation that has had the opportunity to work with two of the most unique and important experiments ever conceived.

Not everyone was so introspective though. One of our ATLAS colleagues casually remarked at the party that we should do this more often – anything for a free beer!

Deepak Kar

Deepak Kar is a postdoctoral research fellow with TU Dresden. His physics interest is soft-quantum chromodynamics, and he is currently involved in underlying event analysis activities in ATLAS.

The Tevatron: Goodnight but not goodbye

The Tevatron collider, the scientific predecessor of the LHC, was shut down last Friday after 26 years of operations. Situated at the Fermi National Accelerator Laboratory (Fermilab) outside of Chicago, Illinois, the Tevatron collided protons with antiprotons at a center-of-mass energy just shy of 2 TeV. While it still held the title of the world’s highest energy colliding beams, it was the intellectual home of hundreds of scientists and students working as part of the CDF and D0 collaborations.

I was one of those Tevatron students. I like to explain that I am the same age as the CDF collaboration – both of us came into being in 1979. Twenty-seven years later, I got my PhD for research done as part of that collaboration, which at the time included around 800 scientists. It’s a privilege to have been part of a scientific enterprise of such grand scope. After finishing a paper a couple of years ago though, I stopped being an active participant of CDF. Drawn by the prospect of higher energies and larger datasets at the LHC, I joined ATLAS at the beginning of my postdoc four years ago, but last week’s shutdown has me reflecting on the time I spent at Fermilab and the connections between the Tevatron and the LHC.

The shutdown was scheduled for around 9:30 in the evening Central European time, so on Friday night I got together with some friends to watch the webcast and toast to the final proton-antiproton collision events. Fermilab director Pier Oddone spoke about the past and future of the lab. Scientists unaccustomed to being in front of cameras gave short tours of the CDF, D0, and accelerator control rooms. Then Helen Edwards, who led the design and construction of the Tevatron, pushed a couple of buttons, one to dump the beam, and another to ramp down the superconducting magnets.

And that was it.

Many nice tributes to the Tevatron have been written, discussing its history and legacy. Of the ones I’ve read, two of my favorites are here and here. The legacy of the Tevatron is not just a striking list of scientific and technical achievements, some of which directly paved the way for the LHC, but also a whole generation of young particle physicists who learned how to be scientists at the Tevatron. The friends I spent Friday evening with were a mixture of people I knew from my time at Fermilab in graduate school and people I’ve met more recently, but all of us who are experimental physicists did our PhD work at D0 or CDF. And we’re not alone.

Over the last five years, there has been a mass migration of young scientists across the Atlantic, from the prairies of Illinois to the valley on the French-Swiss border between the Alps and the Jura mountains. Read more »