Develop a working osgART application

What is a Scene Graph?

Tree-like structure for organising a virtual world, e.g. VRML
Hierarchy of nodes that define: Groups (and Switches, Sequences etc…) Transformations, Projections, Geometry …

And states and attributes that define:

•  Materials and textures
•  Lighting and blending

Benefits for performance

•   Structuring data facilitates 
• optimization:- Culling, state management, etc

What is a Open Scene Graph (OSG)?

Open-source scene graph implementation

Based on OpenGL

Object-oriented C++ following design pattern principles

Used for simulation, games, research, and industrial projects

Maintained by Robert Osfield | Documentation project: www.osgbooks.com

Uses the OSG Public License (similar to LGPL)

What is a osgART?

OSG + ARTookit = osgART

osgART adds AR to Open Scene Graph

Current version 2.0, Open Source

To add Video see-through AR:

•  Integrate live video
•  Apply correct projection matrix
•  Update tracked transformations in real-time

Develop a working osgART application from scratch?

Use ARToolKit 2.72 

     library for tracking and 

     video capture

Install OSG

Tutorial 1 : Basic OSG Viewer | OSG 

Install osg (.exe) to your computer

Run basic viewer - located in osg folder in C:\\Program Files (Windows)

Tutorial 2: Adding Video | OSG + ARToolkit

Add a video plugin

Load, configure, start video capture…

Add a video background

Create, link to video, add to scene-graph

Tutorial 3: Tracking | OSG + ARToolkit

Add a Tracker plugin

 Load, configure, link to video

 Add a Marker to track

 Load, activate

 Tracked node

 Create, link with marker via tracking callbacks

 Print out the tracking data

Tutorial 4: Adding Content | OSG + ARToolkit

Now put the tracking data to use!

Add content to the tracked transform 

Basic cube code

e.g. Wavefront (.obj), OpenFlight (.flt), 3D Studio (.3ds), COLLADA

Replace the simple cube with a 3D model

Models are loaded using the osgDB::readNodeFile() function

OSGART sample project by other researchers

Read More

Camera Resolution : Setting Resolution Problems

Camera Resolution: Bigger = Better Performance, Setting Resolution Problems

The larger the input camera image is, the smaller the markers can be and still be detected, and the less the JPEG artefacts will disturb the detection. Unfortunately the OpenCV cvcam interface that was used in these demos is not able to properly set the camera resolution, it usually defaults to a lower resolution such as 160x120 or 320x240.

The camera resolution should not be confused with the resolution of the graphics window, the demo program window can be 640x480 when the camera resolution is only 160x120. The picture of the background will just appear grainer and more pixelated. The system will work better if the camera can be set to a higher resolution.

Some users have found that if they run another program where you can set the camera resolution first, then it will run with a higher resolution when you run the ARTag demos. For some, the programs downloaded from http://www.shrinkwrapvb.com/ezvidcap.htm seem to do the trick on some computers.

Read More

Using ARToolkit to develop Augmented Reality

There are two parts to developing Augmented Reality applications that use ARToolKit; writing the application, and training image-processing routines on the real world markers that will be used in the application.

Developing the application

In writing an ARToolKit application the following steps must be taken:

1. Initialize the video path and read in the marker pattern files and camera parameters.
2. Grab a video input frame.
3. Detect the markers and recognized patterns in the video input frame.
4. Calculate the camera transformation relative to the detected patterns.
5. Draw the virtual objects on the detected patterns.
6. Close the video path down.

Steps 2 through 5 are repeated continuously until the application quits, while steps 1 and 6 are just performed on initialization and shutdown of the application respectively. In addition to these steps the application may need to respond to mouse, keyboard or other application specific events.

To show in detail how to develop an application we will step through the source code for the simpleTest program. This is found in the directory examples/simple/ The file we will be looking at is simpleTest.c. This program simply consists of a main routine and several graphics drawing routines. The main routine is shown below:


int main (int argc, char **argv)
{

glutInit (&argc, argv);
init();
arVideoCapStart();
argMainLoop (NULL, keyEvent, mainLoop);
return (0);

}

This routine calls an init initialization routine that contains code for initialization of the video path, reading in the marker and camera parameters and setup of the graphics window. This corresponds to step 1 above.

Next, the function arVideoCapStart() starts video image capture.
Finally, the argMainLoop function is called which starts the main program loop and associates the function keyEvent with any keyboard events and mainLoop with the main graphics rendering loop. The definition of argMainLoop is contained in the file gsub.c that can be found in the directory lib/Src/Gl/

In simpleTest.c the functions which correspond to the six application steps above. The functions corresponding to steps 2 through 5 are called within the mainLoop function.

Recognizing different patterns

The simpleTest program uses template matching to recognize the different patterns inside the marker squares. Squares in the video input stream are matched against pre-trained patterns.

These patterns are loaded at run time and are contained in the Data directory of the bin directory. In this directory, the text file object_data specifies which marker objects are to be recognized and the patterns associated with each object. The object_data file begins with the number of objects to be specified and then a text data structure for each object. Each of
the markers in the object_data file are specified by the following structure:

Name
Pattern Recognition File Name
Width of tracking marker

For example the structure corresponding to the marker with the virtual cube is:

#pattern 1
cone
Data/hiroPatt
80.0

Note that lines beginning with a # character are comment lines and are ignored by the file reader. In order to change the patterns that are recognized the sampPatt1 filename must be replaced with a different template file name.

These template files are simply a set of sample images of the desired pattern. The program to create these template files is called mk_patt and is contained in the bindirectory. The source code for mk_patt is in the mk_patt.c file in the util directory.


ARToolkit tutorial for beginersDownload latest version ARToolkit 2.7.2.1
Rujuk Documentation provided in HitLab website for more information/additional note.
Must know how to configure and setting up ARToolkit – set directories “Option properties” visual studio pada linker, include dll, path.

Download Tutorial 1 (.pdf)

Download Tutorial 2 (.pdf)

Read More

How to import VRML model in ARToolkit

Each pattern is associated with an image.
The mapping between pattern and image is found in
C:\ARToolkit\bin\data\vrml_data for simpleVRML.exe program
and C:\ARToolkit\bin\data\multi_vrml_data for the multiVRML.exe program.


For the simpleVRML.exe program, the patterns to be recognized and their associated models are specified in the Data/vrml_data file. In this file you will see a set of lines like this:

#pattern 1
VRML Wrl/bud_B.dat
Data/patt.hiro
80.0 0.0 0.0

In order to import your own model you will have to follow this pattern (Using your_file as example name):

1. Copy your_file.wrl into the Wrl directory

2. Make a your_file.dat file and associate it with your_file.wrl by typing Wrl/ your_file.wrl on the first line. (You may take a copy of one the existing .dat files and edit it to fit your wrl file). Place it in the Wrl directory.

3. Make a new marker (Se section 3.2) or use one that is not yet in use.

4. Edit the Data/vrml_data file by adding a new paragraph:
#pattern 3
VRML Wrl/ your_file.dat
Data/patt.sample1 -- Alternatively your new marker (See on How to make your own marker) 80.0
0.0 0.0

Start the simpleVRML.exe program. When the camera recognizes the pattern associated with your model, the model will be loaded and rendered.

Read More

How to generate marker

Marker





In order to create your own markers you have to start with a new blank rectangle. To make it simple you can use the blankPatt.gif (Picture 5) located in the patterns folder of the ARToolKit installation.

The next thing you have to do is start the mk_patt.exe file located in the bin folder. The file requires an input for the camera. You can use "data/camera_para.dat". This is the default setting for the camera.

Position the camera right over the marker. You should be able to see a red and green square around the pattern. This means that ARToolKit has found your marker. Rotate the camera until the red corner is in the upper left (Picture 7).

How to create Marker Using ARToolkit

In order to capture the marker, we must run an executable that is included in the software library. Make sure your camera is plugged in!

1. The file, mk_patt.exe, can be located in the ARToolKit\bin directory


2. Double click the mk_patt.exe, screen below will appear.


3. Attach the camera to detect your pattern of marker.


4. Enter your pattern name. For example patt.aku


5. Press Enter to saved the pattern.


6. Once you have created the pattern file using the mk_patt executable, copy the pattern file into the ARToolKit\bin\Data directory. This is important as the pattern files are accessed in this directory.



7. Once the pattern file has been copied, you will then need to edit the object_data_vrml file in the ARToolKit\bin\Data directory. Right click the object_data_vrml file and open it in WordPad.



8. The file required . dat file. Next you will learn how to create .dat file


How to create file .dat

Once you have the file in this directory, you will need to create a .dat file that is required from the toolkit. To create the .dat file, right click any empty space in the ARToolKit\bin\Wrl directory. Select New Text Document when the menu appears. Rename this document file to nameOfModel.dat. In our example, we have named it pasu.dat


In the .dat file you will specify the model file and the translation, rotation, and scale parameters.



Using your favourite 3D modelling program, export the model or animation into a .WRL file. If you are using any textures for your model, have the path of the texture set to “./textures/nameOfTexture.gif”. One thing to note is that ARToolKit only supports .GIF formats. Make sure your texture file is in gif format before you export the model!
Once you have the 3D model exported, copy it into the ARToolKit\bin\Wrl directory.

Model yang support
- Obj
- WRL // untuk VRML
Convert your model kepada WRL atau OBJ jika model itu format .3ds, .stl, .max, .dxf and so on

Save this file and run the SimpleVRML executable. Your model should now display on your
custom marker!

Read More

Augmented Reality Categories

Optical See-Through HMD

One of the devices used to merge real and virtual environment is an Optical See-Through HMD. It allows the user to interact with real world using optical technologies to superimpose virtual objects on the real world, as stated by Azuma (1997) and agreed by Ajune et al (2008).  The optical see-through is used transparent HMD to produce the virtual environment directly to the real world. Optical see-through HMDs’ function is placing optical combiners in front of the user's eyes. 

Video See Through

Again, as mentioned by Azuma (1997), Video see-through HMDs are able to give user a real world sight by combining a closed-view HMD with one or two head-mounted video cameras, due to this mixture give user a view of real world and virtual world in real-time through the monitors in front of the user's eyes in the closed-view HMD. Figure 2.4 shows a conceptual diagram of a video see-through HMD. Figure 2.5 shows a video see-through HMD. Video composition can be done using chroma-key or depth mapping (Silva, Oliveira & Giraldi 2003)

Virtual Retinal Systems

Virtual Retinal Systems aim to produce a full color, wide field-of-view, high resolution, high brightness and low cost virtual display (Ishii, 1994). This technology can be used in wide range of applications from head-mounted displays for military or aerospace applications to medical purposes. The Virtual Retinal Display (VRD) uses a modulated beam of light (from an electronic source) directly onto the retina to produce a rasterized image, as stated by Azuma (1997). 

Monitor Based AR

Monitor based AR uses one or two video cameras to view the environment where the cameras may be static or mobile. The video was produced by combining the video of the real world and graphic image generated by a scene generator and the product was shown to user by using monitor device. The display devices are not wearing by the user but when the images are presenting in the stereo on the monitor, it requires user to wear the display devices such as stereo glasses. 

Recently, AR is widely being used in many applications such as education, entertainment, simulation and games. In virtual heritage, AR is used to enhance the overall experience of the visitor of a cultural heritage site. Furthermore, with the interactive, realistic and complex AR system, it can enhance, motivate and stimulate students’ understanding of certain events

Read More