Tactile Gaming Vest (TGV)

The TGV was demonstrated at HAPTICS SYMPOSIUM 2010 in Waltham, MA (Boston Area) and received some honorable mention in IEEE Spectrum and WIRED. The article in Penn Gazette gives a very good user response of the vest.

TGV is a haptic feedback device designed to increase the immersiveness of first- and third-person shooter games. The vest includes a multitude of solenoids, eccentric-mass motors, and Peltier elements controlled by custom electronics and a personal computer. When the player’s character experiences injury in the game, the vest simulates the appropriate sensation (bullet hit, vibration, and/or heat) in the correct location on the player’s body. Beyond shooter games, we believe similar systems could be used to enhance laser tag, military training, movie watching, and other games.

TGV begun as a class project for the ‘MEAM 625: Haptic Interfaces for Virtual Environments and Teleoperation’ in Spring 2009. Receiving positive response at the end of semester, we decided to continue this as our research. The project was undertaken by Me and Ruoyao Wang. Later Ruoyao (who worked on developing game) left for China after Summer 2009. I was joined my Ned Naukam in Fall 09 (Help in modelling the force’s of the solenoids and its effect on skin) and finally Edward Li in Spring 2010 (helping me to put the Vest together especially hardware placement and holding it together) to complete the research. This vest would not have been possible without constant motivation, belief and funding by Professor Katherine Kuchenbecker and Haptics Lab resources

Motivation

The motivation behind this project was the ongoing trend of making the gaming and cinema going experience as realistic as possible. There has been a huge growth in the 3-D movies and games, thus increasing the need for specialized gadgets and equipments to provide audio and visual effects. Also people’s likeness for realistic experiences of virtual world, inspired us to take a step further into 4-D (where the 4th dimension is Physical or Haptic feedback).

  • Movies: There has already been a small progress in 4-D films. But, since the physical effects are expensive to set up, 4-D films are currently presented only at special venues such as theme parks and amusement parks. The effects simulated in 4-D films include wind and vibration. The use of water sprays and air jets is also common. These effects are usually simulated on seats by causing them to vibrate or move a few inches. At the 2009 World Haptics Conference researchers from Philips Research Europe, presented a jacket which they have lined with vibration motors to study the effects of touch on a movie viewer’s emotional response to what the characters are experiencing.
  • Gaming: A company based in Redmond, WA, TN Games has developed “3rd Space Vest”, a wearable force-production system, which sends physical impacts directly from the virtual environment (a first person shooter game) to player’s bodies. They use the pneumatic system for the physical impact. The Jacket / vest consists of 8 pneumatic actuators (two on chest, two on stomach and four on the back) which pushes into the player’s body. The Vest is compatible with many first person shooter games. They are also in process of making a head gear for the FPS games. The main disadvantage of the vest we found while experimenting was that the pneumatic actuator is really slow, thus causing a poking effect or feeling rather than a gunshot. Also the air compressor used was bulky and noisy, making it difficult to play game without headphones.
  • Simulator/Training: Simunitions, short for “simulated munitions”, are used by police and military forces for realistic training. Simunitions are an expensive substitute for live ammunition — costs for simunitions cartridges are as much as three times the cost of live ammunition. Simunitions do, however, provide the most realistic training available. Other sports or activities like paint-ball and laser-tag are current substitute for masses to experience the war situation in real world, and serve more as entertainment activities.

Chosen Approach

When we started this project, the broad idea was to build a Vest for Gaming (as it was easier to specify the location of the physical impacts) and then further enhance it for movies and simulators. We decided to have some basic sensations like gunshot, punch or kick, body blows, surrounding environment (temperature, impacts due to artilleries and ammunitions). We also decided to include a sensation of blood flowing on the body from the location of the gun shot.

Unfortunately, due to lack of time, we were only able to simulate gunshots, slashing and blood flow sensation.

  • Gun Shot:We wanted to simulate the gun shot such that it feels almost real but without causing any harm or long term injury to the user. This part proved to me much more tedious then it seems. It was really difficult to imagine how gun shot really feels like. Searching for people who have been shot and are absolutely fine about describing it was proving to be next to impossible. Internet surfing only gave us information that the gunshot is actually felt a few moments after you have been hit and the area around the shot either has a burning sensation or becomes numb. Also the brain slowly becomes numb or dizzy due to loss of blood.
    We thus decided to follow our instincts and trust our judgment (and movies) as to how gun shot actually feels.
    Our Assumption: Gunshot feels like a sudden but momentary impact on the body lasting a few milliseconds. The gunshot is usually a point or a very small area contact on the body with a radius of a millimeter to about a centimeter, depending upon the bullet type.

  • Blood Flow:The basic idea was to enhance the experience of being shot by adding a sensation of blood flow. Again, we had no clue about how the blood flow feels. We experimented by testing flow of hot water on body, and also some moderately warm viscous liquids and wax. Our Assumption: Blood flow is similar to flow of heat on body. Thus we decided to use the sensation of thermal flow.
  • Slashing: The slash, as we assume, will be stimulated as a flow of vibration across the body.

Me testing the TN Game's 3rd Space gaming vest while also testing the effects of Peltier on my Arm

Me testing the TN Game's 3rd Space gaming vest while also testing the effects of Peltier on my Arm

Virtual World

FPS Game: HALF LIFE 2 – Gun Shot Moment

We felt that the vest cannot be just used to feel Haptic feedback unless the user is involved in a corresponding virtual world. For the Haptic Jacket to be really useful and more attractive, we utilized a First Person Shoot Game, HALF LIFE 2 – Gun Shot Moment, based on the Source Engine. Yeah, we are really serious about the gaming!!

Source Engine is one of the top 3 game engines around the world and famous for the popular games, such as Half Life and Counter Strike. We used the the Source Engine wiki to set up own Game Mode source code, complied it and ran it through the Vavle Steam platform. Next we designed our own map and game level and got our own 3D FPS Game!

When the player gets shot in the game, the program will turn on the specific solenoid based on the position of gun shot in the game and also trigger the blood flow simulation process. The gun shot can be felt almost instantly as it occurs in the game.

We initialized the gun shot program in the game code, and set it to send out a message whenever player got a shot in game. The source engine provides the information about the occurrence of shot and also its position and density. Thus, we could determine to turn on which solenoid and how much force should be output. However, the provided gun shot position was with reference to a global coordinate mapped on from the center of map. This information was insuffcient to determine if the shot was in front or back. So included a code to map this global damage position in to body fixed coordinate system, and send out different messages based on different position (no density temporally because of hardware limitation ).

The message from the game needs to be output to the controller board for Solenoid and Blood flow  actuators to be activated. To communicate this message from game to hardware we include a communication code module in our Mod Game, which transfer the message from computer to controller board through USB serial port.

We discovered that the serial port communication was making the game image stuck momentarily every time a gunshot occurs. To avoid this we used Multi-Thread Programming in Source Game Engine Style. Multi thread could separate the CPU using time for different thread properly, thus solving our problem.

Demonstration

The very first demonstration was held at the Penn Haptics Open House on April 30, 2009. This was prototype one consisting of only Blood flow and Gunshot sensation (3 location each).

We again held a demonstration at the  Haptics Symposium 2010 a Waltham, MA on March 25  & 26, 2010. Here we presented prototype 2 with Gunshots and Slash effect. (Blood flow was droped because of high current requirement).  Also we used a different game, not built by us but hacked into by us, at this demo, due to various problems with the custom game built by us.

Feedback from users

Demonstrations provided us an opportunity interact with people and show our project to them. About 250 people tried our vest each of them giving us some valuable feedback. The over all response was really positive, with many game addicts falling in love with the Vest. It was really difficult to ask them to leave so others can try on the Vest. Some of the important feedbacks are listed below:

  1. The Blood Flow cannot be felt unless notified by us. (Haptics Open House)
  2. The vest is bulky, warm and difficult to setup
  3. The solenoid on the stomach does not have enough impact. (Haptics Open House)
  4. Gunshot and Zombie Claw give same effects. (Haptics Open House)
  5. When and where can we buy this? :)
  6. “Ouch! That hurt!” [IEEE Spectrum]
  7. It is like a good body massage (Haptics Symposium).
  8. This is awesome stuff  (Gamer at Haptics Symposium).

Future Work

To scope of this project is really enormous. As mentioned earlier, we would like to make this vest as a multi purpose and portable one so it can be used in movie and simulators too.

In our original idea, we hope to simulate both bullet penetrate in and penetrate out effect, which will offer player much more realistic feeling and gaming fun. However, we have not discovered a effective method to simulate the penetrate out feeling. We tried several suction mechanisms like air pump or tape, but in vain.

The output force of solenoid is constant, but not depends on the damage density of gun shot in the game. This is mainly limited by the hardware for solenoid. We need to modify the hardwareso player could feel the different weapon types of gunshots and body blows in game.

We hope to add a new feature onto the Haptic Vest – sound. We will mount an array of speakers on to vest, and offer local gun shot sound. Local speaker array will create an interesting sound space around ears, which could be more realistic than the headphone. And such design is to prepare to modify the vest works in realism game like paint ball.

On the game side, there are lot possibilities there about improvement. In our old study of human gun shot feeling, many people mentioned that noise sound and slow motion scene. We could definite improve our game with such effects by changing the sound file and shader effects.

References

[1] W. Lindeman, R.W. and Cutler. Controller Design for a Wearable, Near-Field Haptic Display, Proc. of the 11th Symp. on Haptic Interfaces for Virtual Environment and Teleoperator Systems (2003), pp.397-403.

[2] W. Lindeman, Y. Yanagida, H. Noma and K. Hosaka, Wearable vibrotactile systems for virtual contact and information display, Virtual Reality (2006), Vol. 9 n.2, pp. 203-213.

[3] J. Teh, S. P. Lee and A. D. Cheok. Internet Pajama: A Mobile Hugging Communication System, ICAT (2005), Vol. 157, pp. 274-274.

[4] J. Teh, A. D. Cheok. Huggy Pajama: a mobile parent and child hugging communication system. Proc. IDC ’08. ACM Press (2008), pp. 250-257.

[5] T. O. Gilad, Y. Salzer, A. Ronen. Thermoelectric Tactile Display Based on the Thermal Grill Illusion. ACM (2007), Vol. 250, pp. 303-304.

[6] L. M. Brown , S. A. Brewster , H. C. Purchase, Multidimensional tactons for non-visual information presentation in mobile devices, ACM (2006), Vol. 159, Pages: 231–238.

[7] P. Lemmens, F. Crompvoets, D.Brokken. A body-conforming tactile jacket to enrich movie viewing. Third Joint Eurohaptics Conference and Symposium on Haptic Interfaces forVirtual Environment and Teleoperator Systems (2009). Salt Lake City, UT, USA

[8] S. Ooshima, Y. Hashimoto, H. Ando, J. Watanabe, H. Kajimoto. Simultaneous presentation of tactile and auditory motion to the abdomen to present the feeling of being slashed. SICE Annual Conference (2008). Pages: 467-471.

TGV in News/Media:

Approximately 30 more websites features “TGV” in upto 14 different countries and languages. It was a great exposure and a pleasure to read about the project in so many different languages. (thanks google for translating!!!)

SHORT TRAILER:

FULL VIDEO:

Tags: , , , , , , , , , , , , ,

8 responses to “Tactile Gaming Vest (TGV)”

  1. Manish Chauhan says :

    Cool stuff.
    Really nice work…
    I will look forward to buy it soon..
    Please launch it as a product asap…

    Congrats to the team…

  2. Chris says :

    would be interested to see how it compares to:

    http://www.engadget.com/2007/10/20/medical-force-feedback-vest-tweaked-for-gaming/

    and hopefully a step up above this:

    http://www.vrealities.com/intervest.html

    now all one needs to do is hook up via a remote iPhone app and really have virtual phone s3x!

  3. JD Mc Coy says :

    This is awesome stuff guys

  4. Odeon says :

    Strange, it reminds me A LOT about the TN Games vest…

    http://tngames.com/

    Coincidence?

  5. saurabhpalan says :

    TGV vs TN Games (3rd Space Gaming Vest) and Interactor Vest
    @Chris and @Odeon

    I have mentioned in the Background section about the TN Games vest. Infact we ordered a few of them and tested it before we started building ours.

    I was really inspired by their work and they being the first one’s to build something like this, have done a really good job. I especially liked the way they have got their jacket so rugged, light weighted and easy to setup. Also TN Games did a fair amount of job on the gaming side.

    TN Games vest is a pneumatic one requiring external air compressor (a rather noisy device). Also I was interested in doing some research about how we can use electronics for applications like this. We are not comparing our self to TN Games vest as they have made a product, and we are more on the research end yet trying to figure out how we can simulate various effects and studying human’s psychology when using it.

    Regarding Interactor vest: never got a chance to try this one so cannot say whats it like. But gathering the information from around web, I understand that it just vibrated and uses a sub-woofer on the back.

  6. Speed Dome says :

    Hey guys, this good wook!

  7. Elizbeth Taualii says :

    What a superb blog. I use up hours on the internet evaluation blogs, nearly tons of altered subjects. I like primarily award kudos to whoever created your website and secondly to you for writing what i know how to just express as an magnificentpost. I honestly think there is a skill to writing articles that just a hardly any posses and frankly you boast it. The blend of informative and characteristic content is beyond doubt exceptionally uncommon with the substantial sum of blogs on the internet.

  8. Tameika Tinson says :

    Hi there, please tell us when we will see a follow up!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,291 other followers

%d bloggers like this: