Unity 3D Integration with any application/code/tool [TUTORIAL]

So, have you got a very good algorithm for Real Time eye detection with webcam in MATLAB, and want to integrate it with Unity 3D like, the realtime coordinates of your eye alter a game object or are you having a cool hand detection algorithm in any language and want to integrate it with Unity 3D? You have come to the right place.


Any application/external code can be integrated with Unity 3D. The concept is very simple, we are going to make them (unity and the external app/code) speak the same language (i.e.) Socket Programming. If you are new to Socket Programming, I recommend you to read some tutorial. Anyways, here I have done a simple Unity 3D UDP Server/Client as example. This code can be used to integrate Unity 3D with other codes/applications/tools.

Tutorials::
Download the project files first, check the end of this post for download link

Unity 3D UDP Server
1. Open Unity 3D and create a new project
2. Right Click in "Project" pane and do: Create-->C Sharp Script
3. Now you get a new script called NewBehaviourScript in your project pane
4. Download the attachment with this post and copy the code of "server.cs" file from the downloaded file into the NewBehaviourScript
5. Make sure that you rename NewBehaviourScript to server
6. Attach this script(server) to any gameobject, for simplicity, Main Camera
7. Go to Edit-->Project Settings-->Player and check "Run In Background" (Unity 3D applications by default, freeze when the focus is removed from them, so to prevent it, we use this)
8. Build and Run this project

Unity 3D UDP Client

1. Open Unity 3D and create a new project
2. Right Click in "Project" pane and do: Create-->C Sharp Script
3. Now you get a new script called NewBehaviourScript in your project pane
4. Download the attachment with this post and copy the code of "client.cs" file from the downloaded file into the NewBehaviourScript
5. Make sure that you rename NewBehaviourScript to client
6. Attach this script(client) to any gameobject, for simplicity, Main Camera
7. Go to Edit-->Project Settings-->Player and check "Run In Background" (Unity 3D applications by default, freeze when the focus is removed from them, so to prevent it, we use this)


For testing, we will use both client and server from Unity 3D
Make sure that server application is running, Run the client application from within Unity 3D itself, because the code that I have given uses print() function and you can see it within Unity 3D only. Otherwise, you have to check the logs.

You can now observe that inside Unity 3D, client application prints integers, which is sent by none other than the independent and separate server application.

Now, use your imagination to build some cool Unity 3D applications like you can send your real time eye coordinates or say eye blink etc., from webcam through any application/language/tool and use UDP client of this project and use this data to control any gameobject inside Unity 3D
These are few example projects based on this idea:
http://www.youtube.com/watch?v=yN_YEnDs2uk
http://www.youtube.com/watch?v=-GxykrIB3yM

Project Files
Download Link 1:
http://www.megaupload.com/?d=IIV1H4WB

Unity 3D FaceAPI Integration Tutorials

First of all Im really sorry for taking a long time for releasing this awaited tutorials with all working codes.

Since I dont have web space of my own, I rely on free file hostings, I have uploaded the required files with the mirrors::
Demo:
http://www.youtube.com/watch?v=yN_YEnDs2uk


Tutorials PDF file: (Only the below two files/links are enough, other links are mirror links)
http://www.4shared.com/office/u4ZV8-ME/Unity_3D_FaceAPI_Integration.html
OR
http://www.megaupload.com/?d=G88L0PY1
OR
http://www.esnips.com/doc/2e10244f-ee05-473b-8c82-dbf1fc1aa244/Unity_3D_FaceAPI_Integration

Full Project (Unity Package File):
http://www.4shared.com/zip/f0-PHD1h/faceAPI.html
OR
http://www.megaupload.com/?d=YKX2B382
OR
http://www.esnips.com/doc/11acf06f-232a-4941-abe5-12a80aa713b0/faceAPI

Download the PDF file, that contains links of files to be downloaded...

IMPORTANT UPDATE:

In new Unity versions, there is an error "Null reference", this bug is fixed. Download this file
http://www.4shared.com/file/9E6oGLWg/script.html
OR
 http://www.mediafire.com/?ovz29bodd013gbk
OR

http://www.megaupload.com/?d=BOOK6WCD

and replace the original script.cs file from /scripts in the unity package...

Thank god, found this script from my video(10:37), dont forget to check the video. Check the link down.

Dont forget to view this post:
http://mypersonalsoft.blogspot.com/2011/05/unity3d-faceapi-integration-to-achieve.html

Enjoy....

Please don't forget to post your comments here. This is my first tutorial, and your comment will really help me in improving myself...


And Unity 3D handVU Integration Tutorials will take some more time...sorry again friends!!