Crazy ideas

What could technology do for us? Immersive experiences

Isambard Kingdom Brunel next to SS Great Eastern

Brunel standing next to the launching chains of the SS Great Eastern, a very big ship indeed!

Television history documentaries used to be simple. Take one respected historian, stand him or her on a rock with an ancient ruin dominating the background, then ask them to talk enthusiastically while gesturing selectively at the crumbling remains. The fatal flaw in this method was that viewers were expected to use their imagination to marry the faded decaying architecture in the distance with the florid dialogue of the foreground. If the ruins were erased by the sands of time, perhaps or the burning torches of an invading army, the viewer had to do even more mental work to turn the patchy green/brown bog behind the presenter into a thriving Viking or Saxon settlement.

When trying to convey concepts such as scale, tv historians resorted to standing ancient and modern side-by-side. How big was Brunel’s iron ship, the SS Great Eastern (aka The Leviathan)..? The famous photograph of Brunel standing next to the ship’s massive chains gives the audience a very crude sense of scale, but a direct comparison is better. The ship’s length was said to be 211 metres, and that can be compared to a famous landmark like the London Gherkin building (aka 30 St. Mary Axe) at 180 metres tall. This is much better than the Brunel photo, but still only gives an approximate impression to the audience of how large the ship actually was.

These days television history is a lot more high-tech. With the benefit of a few thousand pounds of hardware purchased from a local branch of PC World, the muddy field is transformed into a glorious 3D virtual representation of its former self, the dilapidated ruins get a lick of fresh paint, and Brunel’s ship can be seen moored right next to a famous landmark as if it had never been sold for scrap. The camera swoops majestically across the historic site, as dotted here-and-there (tv budget permitting) historic virtual people go about their historic virtual business.

Few people would deny that this kind of virtual reality time travel does a powerful job. When used effectively to underline and compliment engaging commentary, it can bring a historic scene alive in ways no static picture, diagram or crumbled ruin can. Of course, we still need the crumbled ruins and the photos; the virtual reality compliments the existing methods, it doesn’t replace them. But a problem remains: even with big tv budgets and state-of-the-art visual gimmickry, the nature of the experience is still fundamentally unchanged. Reconstructed history is certainly eye-catching, but ultimately the viewer is still left watching a tiny glass window showing a flat representation of a world, with someone else controlling the view.

The same issue is found throughout most forms of education. We devise ever more sophisticated means of broadcasting the message — print, photography, film (then talkies, then colour), television, online video — but ultimately it always amounts to more glossy forms of the same experience: a window on a flat representation of someone else’s view.

No matter how clever the technology gets, there’s always a fourth wall separating the learner from the content, and the learner is always on the wrong side of it. If only there was a technology that could take the whizzy 3D graphics we’ve become so used to on tv, and transport the learner inside!! Actually, at scale, next to virtual reconstructions of Victorian iron ships, atop a Mayan temple, or amidst a Viking settlement..!

Breaking the fourth wall (from the other side!)

All the trend-setters are wearing them: the Oculus Rift combines stereoscopic vision with head tracking.

All the trend-setters are wearing them: the Oculus Rift combines 3D graphics with head motion tracking.

The Oculus Rift is a prototype pieces of hardware targeted at the video games market. It combines two small video screens and motion tracking to create a headset that allows the users to step inside a 3D environment and look around by moving their head, just as they would in the real world. It is expected to be released to the public by 2015. Originally a Kickstarter project (crowd funded), Oculus Rift was subsequently purchased by Facebook, and at the time of writing is still under development, targeting various high-end games platforms. A similar product is under development at Sony, specifically for the Playstation, under the banner Project Morpheus.

We’ve had virtual reality before; most people were introduced to the term in the early 1990s thanks to ‘Virtuality‘, arcade games hardware created by UK tech start-up, W Industries. But that generation of hardware apparently cost roughly $70,000+ USD per unit. The interesting part of these new devices is that, while not cheap, both devices are expected to carry a price tag that is within the comfort zone of most dedicated video games players; initial models of the Oculus Rift might retail for less than $300 USD. If such accessories become standard issue for the next generation of gamers, competition and economies-of-scale should slowly drop the price down to an even more reasonable level. Already Google have, rather cheekily, come up with a cardboard alternative that makes use of a smartphone for the screen. The cardboard device is rather crude and disposable; its intent is to let people dabble with VR on the cheap in the hope that they will shell-out for a full headset, but Samsung is already rumoured to be creating its own cut price headset based around a smartphone.

Here’s the $64,000 question: what happens if ‘education’ was to sneak in and commandeers this hardware, and use it for its own benevolent purposes? How big was the SS Great Eastern? Just pop on the goggles and voilà, she’s right in front of you, to scale. What might the Colossus of Rhodes have looked like? Pop on the goggles and see for yourself! How does the placement of stones at Stonehenge interact with the position of the Sun in the sky? Pop on the goggles, stand in the centre of the circle, and fast-forward / rewind through the solar calendar using the game controller.

But moving around using a joystick ruins the experience, surely? When you’ve an Iron Age fort to explore, wouldn’t you rather be on foot?

The Virtuix Omni is another prototype piece of hardware that, like the headsets, is expected to be priced to cater for the gaming crowd. It is a low-friction circular platform that tracks direction and movement as users walk across it, while constrained by a frame and harness that holds them in place. The whole thing collapses down for easy storage, apparently. Already the Oculus Rift and Virtuix Omni have been combined, experimentally, to create game environments that allow players to run, walk, turn, and look all around them, immersed within the video game environment. The only thing these technologies don’t allow the player to do is touch anything (in the gaming world, hugging a zombie is probably not very high on the technical requirements list.)

Fun with cadavers (and other activities you’d never get away with in real life!)

Skeleton fight

Not every animated skeleton is dangerous. Some might be quite educational.

In this virtual environment the rules of health and safety don’t apply. In the real world if you wanted to fire a real trebuchet you’d be buried in mountains of paperwork and requirements to purchase all manner of protective head gear and flame-proof clothing. Thankfully virtual polygons don’t hurt when they hit you, so keen young history students can fire virtual boulders and flaming missiles at virtual medieval castle walls all day long, without fear of anyone losing an eye.

But health and safety isn’t the only way we might cheat the real world in a virtual reality. If we step away from the history based examples into the realm of medicine and science, we find there’s other gadgets that can be combined with virtual reality to create experiences that would be unparalleled in the real world. Already well established on the market is the Microsoft Kinect, another games-related add-on (this time for the X-Box console, although Microsoft have published software libraries to let the device also be used on a PC.) This gadget uses an infra-red scanner / camera to capture the environment immediately in-front of it. It analyses the data to locate shapes that look like humans, then attempts (with a high degree of success) to track arm, leg, hand and head movements. The more recent versions of the device also allow for voice input, just like electronic assistants such as Apple Siri.

Imagine yourself as a medical student at some undetermined point in the near future, putting on the VR headset and being confronted with a life-sized skeleton, complete with organs. The heart beats, the lungs expand and contract, and (with the aid of Kinect) its arms and legs mirror your own movements, exposing muscles tightening and relaxing. Give a verbal cue, and watch the 3D skeleton’s heart simulate a cardiac arrest right before your eyes. Rewind and replay; slow it down or speed it up; lean a bit closer if you want a better look. Load a different program and perhaps you find yourself shrunk down, inside the body, watching blood cells or viruses drift by and getting a sense of the relative scale of things at this microscopic level. Another program might place a trainee doctor or paramedic at the scene of a major disaster; and although they won’t be able to practice any hands-on treatment of the virtual casualties, they will be able to test their skills at assessing the complex scene and prioritising treatment for the wounded and injured they find.


Atomic weapons testing

Put on your headsets, boys and girls, today we’re visiting Mars

The most exciting thing about this hardware is that it isn’t millions of pounds worth of kit destined for some research lab or science museum; this technology is aimed at the regular gamer. This means that if VR hardware becomes popular in the gaming community, education will benefit from a ‘double whammy’: not only will the costs drop within the budget of most education establishments, but many students may also have access to the same technology at home. For once, the elephant in the room is not the cost of the hardware, but the cost of the content! The hardware is useless without good software to exploit it — the imaginary examples given above of giant ships and animating skeletons won’t just fall out of the sky (real or virtual!); they need to be designed and programmed by someone. The video games industry is now a multi-billion dollar proposition, and can afford to spend millions to produce its top-selling blockbuster titles. Education isn’t quite in that league.

But don’t despair, the situation isn’t entirely doom and gloom… The games industry has one more favour to pay education: by creating a demand for 3D graphics, the games industry has driven up the graphics power of the average computer, widened access to 3D modelling software and tools, and created a small army of artists and designers with 3D skills. Although television and movies have played their part, it is largely due to the games industry that 3D graphics have become accessible to everyday digital artists, rather than remaining a niche field inhabited by mathematicians and sci-fi movie directors.

The power of what could be done was demonstrated in 2013 when six De Montfort students re-imagined 17th Century London using the authentic street layout from the time of the great fire in 1666. The 3D environment itself is probably only detailed enough for a fly-through video; a closer inspection presumably exposes a lot of low-resolution models. But it demonstrates what can be done, right now, by just a few students. As the hardware continues to get more powerful, and the software becomes more flexible, we may reach a point were a reasonably high definition 3D world could be created by a relatively modest team. There is a possibility that in the not-too-distant future the production cost for detailed world models could drop to be comparable with producing an education textbook. In the same way one can purchase a textbook on Ancient Egypt, it might be possible to purchase a set of explorable 3D models of Egyptian settlements, tombs, homes, etc. Potentially publishers could create and sell these resources as they do education books and DVDs. A medically accurate model might take a bit more work, depending upon the sophistication required; this may best be done as part of a reasonably sized university project.

We’re at a time when various factors are converging outside of education to create a perfect storm: real-time computer graphics are getting more realistic, 3D worlds are getting easier and cheaper to create, and now we have a new generation of cheap VR headsets! The ability to place learners inside a virtual environment has almost boundless possibilities for education. These new VR headsets are certainly a technology worthy of keeping an eye on.



Comments are closed.

Previous Posts

%d bloggers like this: