Pages

getteammates.com

Thursday, 21 May 2015

Extracting video frames in unity 3d as textures


Video can be played using MovieTexture in Unity. There is no easy way of retrieving the video frame in unity though.

One of the hard way to retrieve the video frame is to render the video in to a RenderTexture and create video frame as texture by reading pixels from the render texture. Lets do it.

So let’s start and create a scene with a Camera, let’s call it MovieCamera, a cube and direction light.

Our scene will look like,




 
Let the camera be an orthographic one. We will use the cube to map the video on to it.


Playing the video



To play video you might need QuickTime Player, especially if you are not using ogg format. Keep the video in the Assets folder. Now drag and drop the video on to the cube object.

This will not start playing the video; for that you would need to write a script and call the Play method of MovieTexture.

script

Attach below given scripts to the cube game object.
using UnityEngine;
using System.Collections;

public class VideoPlayer : MonoBehaviour {

    // assign the video to this texture
    public MovieTexture movie;

    void Start () {
        GetComponent<renderer>().material.mainTexture = movie;

        // play the video
        movie.Play();
        movie.loop = true;

        // scale the cube so that it fills the screen
        float height = Camera.main.orthographicSize * 2.0f;
        float width = height * (float)Screen.width / (float)Screen.height;
        transform.localScale = new Vector3(width, height, 0.1f);
   }

   void Update () {
   }
}

The movie variable should be assigned with the video.
The inspector view of the Cube will be,

With this much of code we will be able to play the video. But our aim is to retrieve the video frames as textures.


To capture video frames we will render the video onto a RenderTexture first. For that instead of the camera rendering onto the screen we will make it render on to the RenderTexture.
 
Once video rendered on to the RenderTexture we can use ReadPixels method of Texture2D to retrieve the video frames.
Actually it is not so difficult to do, just attach the below shown script to the camera(MovieCamera) and we are done.

using UnityEngine;
using System.Collections;

public class MovieCamera : MonoBehaviour {
    private RenderTexture renderTexture = null;
    private Texture2D videoFrame;
    private Rect rect;   // Rect from where we need to read the pixels(entire  screen)
 
    void Start () {
    }

    void Update () {
    }

    void OnPreRender() {
        if (renderTexture == null) {
            Create();
        }

        // make camera’s targetTexture as current rendering target
        // and then read the pixels.
        RenderTexture orig = RenderTexture.active;
        RenderTexture.active = renderTexture;
        videoFrame.ReadPixels(rect, 0, 0, false);
        videoFrame.Apply();
        RenderTexture.active = orig;
    }

    void OnGUI () { 
        // draw the read texture on to the screen
        Graphics.Blit(videoFrame, (RenderTexture)null);
    }

    private void Create() {
        renderTexture = new RenderTexture (Screen.width, Screen.height, 8, RenderTextureFormat.ARGB32);
        videoFrame = new Texture2D(Screen.width, Screen.height, TextureFormat.ARGB32, false);
  
        // set targetTexture for the Camera
        GetComponent<camera>().targetTexture = renderTexture;
  
        // Rect to read the pixels; entire screen
        rect = new Rect(0, 0, Screen.width, Screen.height);
    }
}

Our Camera's(MovieCamera) inspector view,


In order to try this you would need a Unity pro version.
The video that I have used can be downloaded from https://peach.blender.org/download/

Thursday, 7 May 2015

Compiling C++11 and C++14 programs with GCC

Compiling C++11 programs

With gcc 4.7 and above we can use -std=c++11 or -std=c++0x or -std=gnu++0x or -std=gnu++11 option to compile the programs as shown below,
g++ -std=c++11 hellocpp11.cpp

For earlier version of GCC, try -std=c++0x or -std=gnu++0x
g++ -std=c++0x hellocpp11.cpp

Please see the link https://gcc.gnu.org/projects/cxx0x.html for more information on feature support.

Compiling C++14 programs

C++14 is supported from gcc 4.9 onwards, so please update your gcc if it is below 4.9.
g++ -std=c++14 hellocpp14.cpp

If the default g++ compiler is not of version 4.9 and you have updated it, please use the compiler with version suffix. For example,
g++-4.9 -std=c++14 hellocpp14.cpp

-std=c++1y or -std=gnu++1y or or -std=gnu++14 can also be used instead of -std=c++14

Please see the link https://gcc.gnu.org/projects/cxx1y.html for more information on feature support.

Updating GCC to version 4.9 in Ubuntu

We will use toolchain ppa test repository to update the GCC version. Use below given commands to complete the installation.
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install g++-4.9

After finishing the installation you can use g++-4.9 to compile programs written in c++14.

Proxy configuration

In case you are under proxy, try setting the following environment variables.
export http_proxy=http://<host>:<port>
export https_proxy=http://<host>:<port>

Also try add-apt-repository with sudo -E option if just sudo did not work for you.

In case your proxy server requires user name and password, try setting the proxy environment variables with user name and password
export https_proxy=<user_name>:<password>@<host>:<port>