Let's us look on how capturing reality will improve our own creative output.
In visual effects, animation, and games, capturing reality isn't just an add-on but a core part of what we do. For technical directors who bridge the creative and technical sides of a project, real-world data can simplify workflows, reduce guesswork, and help deliver visuals that truly "fit" the real or imagined worlds we're creating.
Jahirul Amin, founder of CAVE Academy, shared valuable insights on capturing reality and broke down the many ways it helps us create convincing CGI. Grounding our work in real-world references helps us stay consistent and realistic, whether we’re matching models, shaders, capturing textures, lighting, animation, fx or gathering on-set data.
On-Set Data Acquisition
A key in on-set data capture is to "keep it simple". When we collect lighting references, photogrammetry or HDRI, our goal is not to create "pretty" data - it's to get a clear, functional set of references that VFX artists can actually use down the line. For example, HDRI (high dynamic range imaging) can give us the "ground truth" of a scene's lighting. It's just the starting point with lighting maps, scene sketches, and even notes on lighting positions can help when re-creating the scene in CGI.
Building a Ground Truth
In VFX, we need a reliable "ground truth" - a baseline reference that anchors the digital work to physical reality. This ground truth gives TDs something to compare to, making it easier to verify that assets and the scene lighting are consistent and accurate. Imagine trying to recreate a car in a sunny environment: capturing the real lighting and reflections of that environment means we can accurately match them in CGI. It's like having a cheat sheet to keep us grounded.
Keep things simple - primarily because we can overcomplicate things.
Lighting Essentials and HDRI Limitations
HDRI is a great tool, but it's not a magic bullet. It's just a snapshot of the environment's lighting. HDRI is a first version to start but in most cases it needs work and additional light to make it work. For example, if an HDRI captures light coming through a window, the CG asset may look close to correct at first, but refining the placement and intensity of the light often gives it the true "natural" look. An HDRI alone may miss detailed shadows or exposure range, so additional measurements or notes on light sources (size, position, color) can make all the difference.
Be prepared to unwarp our HDRI
Environment and Material Reference
When working with a character, capturing both lighting and environmental reflections helps them to "sit" naturally in the scene. LIDAR scans, photogrammetry, and even quick environmental references help artists to simulate realistic shadows and light bounces to integrate CGI elements into the existing scene. Simple environmental captures - even with a phone - can go a long way in helping the lighting team create something that feels authentic (though phones are reserved as a last resort).
Notes, Documentation, and Communication
One of the simplest yet most powerful tools is the humble notebook. On set, things are constantly changing - lights move, gels are added, angles shift. A quick note of these details means lighting and lookdev artists aren't left guessing later. It's also helpful to note things like the type of light, its dimensions, or any special setup used, such as barn doors or diffusers. When it's time to match the shot in CGI, these little notes can save hours of trial and error.
Jahirul's meetup notes
Using Practical SFX as Reference
Sometimes practical effects (like fake blood or a fog) work best as real-world shots, providing invaluable reference footage for integration with CGI. Instead of spending hours trying to simulate something complicated, a practical shot can give us a reference that looks and moves naturally. For example "bloody footprints" can be achieved by simply photographing real footprints with red dye and then compositing them in.
Tools and Equipment: Grey Balls, Cameras, and Colour Charts
When we're capturing reality, we can't overlook the basics like gray balls, chrome balls, and color charts. These reference tools help us calibrate and match lighting and colors between real footage and CGI. However not all equipment is perfectly consistent, so calibration is important. A good practice is to set up a pipeline that processes all captured data consistently, so we can trust the accuracy of each department's work. Consistency means that the entire team can rely on the data without question.
Python: Your Secret Weapon in Data Capture
Technical directors who know Python have a huge advantage when dealing with data on set. Python allows us to automate many tedious tasks, from organizing HDRI brackets, calibrating color charts, streamline tasks like color calibration or automate parts of the photogrammetry process. This allows us to get to the creative work faster.
Resume
Capturing reality is a chance to combine technical precision with artistic vision. It's about creating a reference point that teams can rely on to ensure a seamless pipeline from on-set capture to final render. A solid technical foundation combined with Python automation, will allow us to create CGI that doesn't just look real - it is real!
Thank you for reading,
Vish Patel & Alex
Links
CAVE Academy - CAVE Academy offers a variety of training sessions and tutorials on practical data capture for VFX. Jahirul Amin emphasizes hands-on learning and simplicity, making these resources a great place for aspiring TDs to build foundational skills.
TD Newsletter - Perfectly tailored for you if you're a (future) artist or technician working in visual effects, animation, and games, and want to receive future TD Meetup notifications.