Unit-2 Dr.Vaishnavi Moorthy Asst Prof/Dept of NWC 18CSE304J-Building Applications using AR and VR opensource SDKs 1 Components of AR 18CSE304J-Building Applications using AR and VR opensource SDKs 2 Components of AR 18CSE304J-Building Applications using AR and VR opensource SDKs 3 Software Components of AR • High-level: Currently, several tools help in the development of applications, known as SDK (Software Development Kit), such as Vuforia, ARKit, ARCore, Wikitude, ARtoolKit, EasyAR, LayAR. • Low-level consists of different areas, which may vary according to the application or the developer’s needs, such as programming libraries, Computer vision, CG, Image processing, HCI. 18CSE304J-Building Applications using AR and VR opensource SDKs 4 Types of AR • Marker-based AR: It is tied to a specific physical image pattern marker in a real-world environment to superimpose the virtual 3D object, text, or animation on top of it. • The cameras continuously scan the input and put a mark for image pattern recognition to create its geometry. When the camera is not focused on a particular spot, the virtual 3D object is not shown properly. • Marker-based AR through Instagram, Facebook, and Snapchat filters and games like Pokemon Go. It is gradually entering into our daily lives through fun and social activities. • Marker-based AR image recognition system consists of several modules such as camera, image capturing, image processing, rendering as well as marker tracking. • It is easy and budget-friendly to implement on the filters through a custom app for recognizing specific patterns using its camera feed. 18CSE304J-Building Applications using AR and VR opensource SDKs 5 Types of AR • Markerless AR: It refers to the software application that does not require any camera to put a mark for image pattern recognition. • Markerless AR places virtual 3D objects in the real-life environment by examining the features present in the real-time data. It relies on the hardware of any smartphone including the camera, GPS, digital compass, and accelerometer for the AR software to complete the work efficiently. • There is no need for an object tracking system due to the advancement in cameras, sensors, and AI algorithms. It merges the digital data with input from real-time data that are registered to a physical space. • Markerless AR image analysis utilizes the Simultaneous Localisation And Mapping technology (SLAM) that scans the environment and creates appropriate maps to place the virtual 3D objects. • The user can increase the average range of motion while experiencing the Augmented Reality. There are four different categories of markerless AR such as location-based AR, Projection-based AR, Superimposition AR, and Outlining AR. 18CSE304J-Building Applications using AR and VR opensource SDKs 6 Types of AR • Location-based AR: It ties the augmentation to a specific place by reading the real-time data from a smartphone camera, GPS, digital compass as well as and accelerometer. • Location-based AR does not require any cue from an image or object to deploy as it can predict the user’s focus as a trigger to pair the real-time data with the present location. • It also allows developers to attach interactive and useful digital content to geographical points of interest. • It is beneficial to travelers to have a clear understanding of the whereabouts of a specific area through virtual 3D objects, videos, texts, links, and audio. 18CSE304J-Building Applications using AR and VR opensource SDKs 7 Types of AR • Projection-based AR: It is a method of delivering digital data within a stationary context as projectionbased AR focuses on rendering virtual 3D objects within or on a user’s physical space. • The user can freely move around the environment within a specified zone where the fixed projector and a camera for tracking are placed. It is used to create illusions on the depth, position, and orientation of an object by projecting artificial light onto the real flat surfaces. • Projection-based AR can be applied for proper guidance to simplify the complex manual tasks in companies or industries. It eliminates the need for computers and screens as the instructions can be placed on a particular task space. • Projection-based AR also provides feedback for process improvement and unique digital IDs for build cycles. 18CSE304J-Building Applications using AR and VR opensource SDKs 8 Types of AR • Superimposition AR: It is used either for partial or full replacement of the original view of an object with an updated augmented view of that object for the human eye. Superimposition AR provides multiple views of a target object with the option of showing extra relevant information on that object. • Outlining AR: Special cameras are built for human eyes to perform outlining of the specified objects like boundaries and lines to help in certain situations. Outlining AR utilizes object recognition for a better understanding of the current environment. It is specially used in-car navigation systems for safe driving after sunset. 18CSE304J-Building Applications using AR and VR opensource SDKs 9 Components of VR 18CSE304J-Building Applications using AR and VR opensource SDKs 10 VR Headsets A VR headset is a head-mounted device, such as goggles. A VR headset is a visual screen or display. Headsets often include state-of-the-art sound, eye or head motion-tracking sensors or cameras. There are three main types of headsets: • PC-Based VR Headsets: PC headsets tend to be the highest-priced devices because they offer the most immersive experiences. These headsets are usually cable-tethered from the headset and powered by external hardware. The dedicated display, built-in motion sensors and an external camera tracker offer high-quality sound and image and head tracking for greater realism. • Standalone VR Headsets: All-in-one or standalone VR headsets are wireless, integrated pieces of hardware, such as tablets or phones. Wireless VR headsets are not always standalone. Some systems transmit information wirelessly from consoles or PCs in proximity, and others use wired packs carried in a pocket or clipped to clothing. • Mobile Headsets: These shell devices use lenses that cover a smartphone. The lenses separate the screen to create a stereoscopic image that transforms a smartphone into a VR device. Mobile headsets are relatively inexpensive. Wires are not needed because the phone does the processing. Phones don’t offer the best visual experiences and are underpowered by game console- or PC-based VR. They provide no positional tracking. The generated environment displays from a single point, and it is not possible to look around objects in a scene. 18CSE304J-Building Applications using AR and VR opensource SDKs 11 Common VR devices VR Using Head Mounted Displays (HMDs) They create dynamic 3D images using built-in position sensors, stereo headphones and blackout blindfolds. It consist of two screens, each displaying varied stereoscopic images that create a realistic perspective of a 3D virtual world. However, HMDs are quite heavy and can become tiresome to wear for longer duration. VR Using Immersive Rooms As an alternate to HMDs, immersive rooms are used to project changing images on to the walls of a room. Flight simulations often use this technique where images of cities, landscapes and airports are projected on large screens that are positioned outside mock cockpits. VR Using Data Gloves The aim of data gloves is to allow user handling of virtual objects. These are nothing but ordinary gloves fitted with sensors that detect hand motions of the user. Fiber optics, piezoelectric sensors and strain gauges are some of the technicalities used to create data gloves. 18CSE304J-Building Applications using AR and VR opensource SDKs 12 VR Devices • 3D Mouse: A 3D mouse is a control and pointing device designed for movement in virtual 3D spaces. 3D mice employ several methods to control 3D movement and 2D pointing, including accelerometers, multi-axis sensors, IR sensors and lights. • Optical Trackers: Visual devices monitors the user’s position. The most common method for VR systems is to use one or multiple fixed video cameras to follow the tracked object or person. • Wired Gloves: This type of device, worn on the hands, is also known as cyber gloves or data gloves. Various sensor technologies capture physical movement data. Like an inertial or magnetic tracking device, a motion tracker attaches to capture the glove’s rotation and global position data. The glove software interprets movement. High-end versions provide haptic feedback or tactile stimulation, allowing a wired glove to be an output device. • Motion Controllers: These accessories allow users to act in mixed reality. Controllers allow for fine-grained interaction with digital objects because they have a precise position in space. • Omnidirectional Treadmills (ODTs): This accessory machine gives users the ability to move in any direction physically. ODTs allow users to move freely for a fully immersive experience in VR environments. • Smelling Devices: Smell devices are one of the newer accessories in the VR world. Vaqso, a Tokyo-based company, offers a headset attachment that emits odors to convey the size and shape of a candy bar. The fan-equipped device holds several different smells that can change intensity based on the screen action. 18CSE304J-Building Applications using AR and VR opensource SDKs 13 VR Software Components They include VR software development kits, visualization software, content management, game engines, social platforms, and training simulators. • VR Content Management Systems Software: Companies use this workplace tool to collect, store and analyze VR content in a centralized location. • VR Game Engine Software: Developers use the tools to create a VR video game experience. • VR Software Development Kit (SDK): SDKs offer a base to design, build and test VR experiences. • VR Social Platforms Software: Users collaborate from remote locations in VR with these tools. • VR Training Simulator Software: This software works for almost any industry for employee training in immersive environments. • VR Visualization Software: Users experience aggregated data in a virtual environment. to fully understand what data means. 18CSE304J-Building Applications using AR and VR opensource SDKs 14 Types of VR • Non-Immersive Virtual Reality: This category is often overlooked as VR simply because it’s so common. Nonimmersive VR technology features a computer-generated virtual environment where the user simultaneously remains aware and controlled by their physical environment. Video games are a prime example of nonimmersive VR. • Semi-Immersive Virtual Reality: This type of VR provides an experience partially based in a virtual environment. It will still give users the perception of being in a different reality when they focus on the digital image, but also allows users to remain connected to their physical surroundings. Semi-immersive technology provides realism through 3D graphics, a term known as vertical reality depth. This type of VR makes sense for educational and training purposes with graphical computing and large projector systems, such as flight simulators for pilot trainees. 18CSE304J-Building Applications using AR and VR opensource SDKs 15 Types of VR • Fully Immersive Virtual Reality: Fully-immersive simulations give users the most realistic simulation experience, complete with sight and sound. To experience and interact with fully-immersive virtual reality, the user needs the proper VR glasses or a head mount display (HMD). VR headsets provide high-resolution content with a wide field of view. The display typically splits between the user’s eyes, creating a stereoscopic 3D effect, and combines with input tracking to establish an immersive, believable experience. The possibilities for VR usage are endless. Car racing games are an example of immersive virtual reality that gives the user the sensation of speed and driving skills. Developed for gaming and other entertainment purposes, VR use in other sectors is increasing. 18CSE304J-Building Applications using AR and VR opensource SDKs 16
