Fernando De la Torre’s 10-year-old son, Nico, wants to attend a FIFA World Cup match this summer in Atlanta, to cheer on Spain.

The problem: not only are the tickets themselves costly, they’re difficult to obtain through a high-demand lottery process.

“The question is, how can we offer a similar, immersive experience of sport without going to the field?” said De la Torre, a research professor at the Robotics Institute at Carnegie Mellon University.

De la Torre and a group of CMU graduate students think they’ve found the answer.

They’ve developed LiveSplats, a virtual reality technology and artificial intelligence-powered rendering that allows people to follow live scenes from any angle, regardless of where they’re located.

“You are not going to watch the game — you are going to navigate the game. This is the future of 3D sports,” De la Torre said. “This is how we are going to consume sport in five years. The technology is there, we just need to put it together.”

LiveSplats could also be used for concerts, graduations and other large-scale events.

The technology uses a 3D rendering technique called splatting. Also known as “Gaussian splatting” (after German mathematician Carl Friedrich Gauss), the process reconstructs scenes using millions of tiny, cloud-like elements rather than traditional geometric models.

Each element stores information about position, color and transparency, ultimately forming a complete representation of a scene. Splatting can render the same scene from many different viewpoints without being rebuilt each time.

So, in LiveSplats, a person would use a virtual reality headset and controllers to not only view a live, 3D replica of a stadium but also move around it — even as close as on the court.

“We believe it will be a new way to interact with the game, and navigate the game,” De la Torre said. “We have all the 3D information, we can capture the emotion of a player. We have the 3D representation of the game, so there are many things we can do.”

Researchers created a benchmark dataset that includes different camera views and information like player positions, depth and movements. Some scenes are captured from virtual cameras and others test challenges like lighting changes or rapid movements.

But, there’s much more work to be done before LiveSplats becomes mainstream. Hurdles include visual challenges like reflections, the motion of a ball or glare on a court surface.

“It’s very hard to get the experience you see on TV,” De la Torre said. “To provide the same fidelity but in 3D is not easy work.”

Members of De la Torre’s research team are CMU students Junkai Huang, Saswat Subhajyoti Mallick, Alejandro Amat, Marc Ruiz Olle, Albert Mosella-Montoro, Bernhard Kerbl and Francisco Vicente Carrasco.