Game of Shadows, the new Sherlock Holmes movie starring Robert Downey Jr. and Jude Law, is doing well at box office. So congratulations to vfx firm Framestore which created 550 shots for the film, including a climactic encounter at the Reichenbach Falls, a battle on a speeding train and a scene taking place inside a gun.

Shooting took place from September 2010 to January 2011, with the majority of first unit material captured on location around the UK and at Elstree and Leavesden Studios. Supervising for Framestore were Visual Effects Supervisor Sirio Quintavalle, Compositing Supervisor Kyle McCulloch and CG Supervisor Ben White.

Spoiler Alert!

Perhaps the toughest sequence was the finale – which is outlined below. If you haven”t seen the film,beware! The section below contains movie spoilers.

As the Guy Ritchie-directed film reaches its conclusion, Holmes and Moriarty meet in a fortress high in the Swiss Alps, built above the source of the colossal Reichenbach Falls. They play chess then fight. Holmes realises that the only way to definitively best his nemesis is to seize and pull him over the balcony into the waterfall below, even at the price of near-certain death. So the camera is placed midway down the falls, descending and pointing upwards, as the two figures locked together plummet towards it. They are moving faster than the camera, though, and it pans as they pass it. The action switches to ultra-slow motion as the audience watches Holmes and Moriarty’;s heads in close-up, before the film speed picks up and they continue their fatal descent. It is, says Framestore, “29 seconds of virtuoso film-making, the sort that Guy Ritchie has made his own. But such visual audacity places the highest demands on a VFX team.”

Quest for the perfect falls

According to Framestore, VFX Supervisor Chas Jarrett’;s initial hope was that the sequence could be constructed using real waterfall elements alone. “Both Quintavalle and McCulloch travelled far and wide as part of the quest for the perfect falls. They both dangled perilously out of helicopters (Quintavalle in Switzerland, McCulloch in Norway), clutching their trusty Canon EOS 5D cameras as they captured plates and footage. In the end, no real waterfall proved up to the sequence’;s demands”.

To achieve the scene, the actors (Robert Downey Jr. and Jared Harris) were attached through harnesses to two gigantic KUKA robot arms, says Framestore. More commonly used for making cars, these powerful arms can be programmed to move the actors through space.

Cleaning up

But while offering great benefits in terms of precision positioning and performance capture from the actors, this technique does have its downside. “There was an enormous quantity of clean-up work to be done before the actors were ready to be placed in the shots,” says McCulloch, “We had an eight-strong paint and roto team putting in long hours for many weeks to remove all traces of the harnesses and the arms, as well as patching up bits of the outfits with CG elements.” This action was shot on a Phantom HD Gold camera.

The water was the biggest headache. Ben White was Framestore’;s CG Supervisor for the project. He says: “Waterfalls do not lend themselves brilliantly to fluid simulation software, which is at its best on rivers and seas – things with a surface. Also, the fact that it was such a long waterfall meant that we were dealing with colossal amounts of data, because we had to have pre-roll for an entire kilometre, and when you factor in the time-warp down to almost 500fps in the middle of the shot, and the fact that the camera is pointing straight up into the descending water, making it simply hard to see – all this meant we had our work cut out. In the end we broke up the water into several segments rather than trying to do it in one piece, with each segment being made of multiple layers to achieve the complexity we needed. We had a team working in Naiad, on the water coming down from above, and we had another team working in Houdini providing a different element for the lower part of the waterfall where it blends in to the live action elements. We also added CG and live action elements of water vapour and mist being pulled away from the surface which helped to give a special relationship to the falling actors in this huge space.”

Water droplets refraction

Another water element proved more susceptible to augmentation. As the protagonists pass the camera in close-up, Jarrett and Ritchie wanted water droplets in motion around their heads. The compositing team found that the latest version of Nuke gave them a set of tools that helped sidestep a 3D solution.

“We found photos of live action water droplets from reference libraries,” says Quintavalle, “And we put them onto cards – lots of droplets arranged in motion and distributed as we wanted. It was a temporary measure at first, while we waited for the ‘;proper’; particles to get rendered. Then as it went along we thought, ‘;Could we make them move a bit as well, add a bit of warping? Could we have a bit of refraction as you’;re looking through them? Can we get the time-warp working properly?’; and each time the answer was yes. The drops kept getting better within Nuke until we realised we had the shot we needed.”

The sequence was a major effort on the part of the Framestore team. Half of Framestore’;s 30-strong Shadows compositing team worked on it. It was one of the first to start in October 2010, and among the last to finish. “It was a labour of love, blood, sweat, tears – and 28 terabytes of data caches!” says Framestore.