Tag: Unity

Table-Top AR – Explore New York City in Your Living Room

By: Ben Simak

Geovisualization Class Project @RyersonGeo, SA8905, Fall 2018

Introduction:

For my geo-visualization project I decided to create a map that takes advantage of new interactive technologies. Augmented reality (AR) has been around since 1990’s and has been growing in use and capabilities. Augmented Reality was originally developed and implemented in Air Force Research and then heavily developed in the gaming industry to add a new way to interact with our surroundings to try and immerse the viewer on a new level. This same technology has migrated into different industry applications such as education and navigation.

The type of augmented reality that I decided to use is called “Table Top” augmented reality. It essentially allows the camera on the device you are using to find a flat surface and showcase a 2D or 3D model or rendering on that surface. You are then able to anchor it and walk around the model to see all the sides and get closer or further away as if it was actually there.

I utilized MapBox, Unity, ARKit, and Xcode to create an app that allows you to use an iPhones camera to render a 3-D scrollable model of  New York City (And anywhere in the world you want to scroll too). Mapbox provides the feed of building and elevation models for the building heights as well as terrain through its SDK. Unity is the platform that pulls together the MapBox SDK and generates the augmented world and allows for any physical altering of how the maps look and the starting point for the map. Unity is where the majority of the app components get bundled up before it is ready to be processed. The ARKit has coding that enables the app to be created on the iPhone and enables the use of the camera. Xcode is the final step that takes the bundled Unity file and generates an app that can be opened up on your personal iPhone for use.

How I Built the App:

*Requires Mac OSX and an iPhone (Android variant also available)

Step 1: Download MapBox SDK for Unity from https://www.mapbox.com/unity/.

Step 2: Download Unity (For Personal Use Free) https://store.unity.com/download?ref=personal

Step 3: Download Xcode on the Mac App Store https://itunes.apple.com/ca/app/xcode/id497799835?mt=12

Step 4: Open Unity and create a new 3-D project

Step 5: Go to Assets menu and go to import package and click custom package…

Navigate to you downloaded Mapbox SDK and click open. After it loads and opens click Import all and wait for it to load. A Mapbox setup window should open and looks like the below.

Step 6: Click the mapbox.com link highlighted in blue. It will take you to the Mapbox website to generate your access token. Click the copy button and then go back to your Unity Project and paste it in the field and click submit.

Step 7: Some layers that aren’t included by default in a Unity project and are needed to run this scene. Select ARTabletopKit in the Hierarchy view. Add the following layers by clicking Layer and selecting Add Layer

Specify the following layers:

  • ARGameObject
  • Map
  • Path
  • Both

Step 8: Click on MapRoot and find the Latitude Longitude settings in the GENERAL settings of the Abstract Map script. Click Search and enter the coordinates of anywhere in the world. For my example, I used New York City.

Step 9: Now when you press the Play button at the top center of the Unity window you should see a rendering appear on the Game tab

Step 10: go to edit, project settings, player. Make sure under the Camera Usage Description and Location Usage Description you put the following details.

 

Step 11: Go to File, Build Settings and open up the window seen below. Then click IOS (Or Android if you were making it on an Android device) and click Switch platform. Then make sure you click add open scene and select the Table Top AR with a check and uncheck Scene/main. Then click build. Save it to where ever you want.

Step 12: Download Xcode. Plug in Your Device. Before we can build to a device, we need to set up an Apple ID and add it to Xcode. Once you have obtained an Apple ID, you must add it to Xcode.

  • Open Xcode.
  • From the menu bar at the top of the screen choose Xcode > Preferences. This will open the Preferences window.
  • Choose Accountsat the top of the window to display information about the Apple IDs that have been added to Xcode.
  • To add your Apple ID, click the plus sign at the bottom left corner and choose Add Apple ID.

  • A popup will appear, requesting your Apple ID and password. Enter these.
  • Your Apple ID will then appear in the list. Select your Apple ID to see more information about it.
  • Under the heading Team, you will see a list of all Apple Developer Program teams that you are a part of. If you’re using a free Apple ID that isn’t enrolled in the Apple Developer Program, you will see your name followed by “(Personal Team)”.

Step 13: Go back to that original Unity file that you saved after pressing the build button. Double click the .xcodeproj file to open the project with Xcode.

  • In the top left, select Unity-iPhone to view the project settings. It will open with the General tab selected.
  • Under the topmost section called Identity, you may see a warning and a button that says Fix Issue. This warning doesn’t mean we’ve done anything wrong – it just means that Xcode needs to download or create some files for code signing.
  • Click the Fix Issue
  • Make sure that the correct team is shown in the dropdown – if you’re using a free Apple ID, it should be your name followed by “(Personal Team)”.

Step 14: Make sure the bundle identifier seen above is not a default name. if it is just change the default name to what ever you want.

Step 15: Click the play button and make sure your iPhone is still connected. The device must stay on and not lock during this process.

Step 16: Once completed you will see the app on your iPhone and you can open it and point at a flat surface and watch the magic happen!

Creating a Flight Simulator and Virtual Landscape Using LiDAR Data and Unity 3D

by Brian Mackay
Geovis Class Project @RyersonGeo, SA8905, Fall 2017

The concept for this project stems from the popularity of phone apps and computer gaming. It attempts to merge game creation software, graphic art and open source geographic data. A flight simulator was created using Unity 3D to virtually explore a natural landscape model created from Light Detection and Ranging (LiDAR) data collected from the US Geological Survey (USGS). 

The first step is to define the study area, which is a square mile section of Santa Cruz Island off the coast of California. This area was selected for the dramatic elevation changes, naturalness and rich cultural history. Once the study area is defined, the LiDAR data can be collected from the USGS Earth Explorer in an .LAS file format.

After data is collected, ArcMap is used to create a raster image before use in Unity 3D. The .LAS file was imported into ArcMap in order to create the elevation classes. This was done by using the statistics tab in the property manager of the .LAS file in ArcCatalog and clicking the calculate statistics button. Once generated an elevation map is displayed using several elevation classes. The next step is to convert the image to a raster using the .LAS dataset to raster conversion tool in ArcToolbox. This creates a raster image that must then be turned into a .RAW file format using Photoshop before it is compatible with Unity.

The data is now ready for use in Unity. Unity 3D Personal (free version) was used to create the remainder of the project. The first step is to import the .RAW file after opening Unity 3D. Click the GameObject tab → 3D Object → Terrain, then click on the settings button in the inspector window, scroll down, click import raw and select your file. 

Next, define the area and height parameters to scale the terrain. The USGS data was in imperial units so this had to be converted to meters, which is used by Unity. The Length and width after conversion were set as 1609m (1 sq mile) and the height was set as 276m (906ft), which was taken from ArcMap and the .LAS file elevation classes (seen right and below). Once these parameters are set you can use your graphic art skills to edit the terrain.

 

 

Editing the terrain starts with the inspector window (seen below). The terrain was first smoothed to give it a more natural appearance rather than harsh, raw edges. Different textures and brushes were used to edit the terrain to create the most accurate representation of the natural landscape. In order to replicate the natural landscape, the satellite map from Google was used as a reference for colours, textures and brush/tree cover.

This is a tedious process and you can spend hours editing the terrain using these tools. The landscape is almost finished, but it needs a sky. This is added by activating the Main Camera in the hierarchy window then going to the inspector window and clicking Add Component → Rendering → Skybox. The landscape is complete and it is time to build the flight simulator.

To build the plane you must create each of its parts individually by GameObject → 3D Object → Cube. Arrange and scale each of the planes parts using the inspector window. The final step is to drag and drop the individual airplane parts into a parent object by first clicking GameObject → Create Empty. This is necessary so the C# coding applies to the whole airplane and not just an individual part. Finally, a chase camera has to be attached to the plane in order for the movement to be followed. You can use the inspector window to set the coordinates of the camera identical to the plane and then offsetting the camera back and above the plane to a viewing angle you prefer.

C# coding was the final step of this project and it was used to code the airplane controls as well as the reset game button. To add a C# script to an object, click on the asset in the hierarchy you want to code and select Add Component → New Script. The C# script code below was used to control the airplane.

Parameters for speed and functionality of the airplane were set as well as the operations for the chase camera. Finally, a reset button was programmed using another C# script as seen below. 

The flight simulator prototype is almost complete. The last thing is inserting music and game testing. To insert a music file go to GameObject → Create Empty, then in the inspector window click Add Component → Audio → Audio Source and select the file. Now it’s time to test the flight simulator by clicking the game tab at the top of the viewer and pressing play.

Once testing is complete, it’s ready for export/publishing. Click File → Build Settings then select the type of game you want to create (in this case WebGL), then click Build and Run and upload the files to the suitable web host. The game is now complete.. Try it here!

There are a few limitations to using Unity 3D with geographic data. The first is the scaling of objects, such as the airplane and trees. This was problematic because the airplane was only about 5m long and 5m wide, which makes the scale of other objects appear overly large. The second limitation is the terrain editor and visual art component. Without previous experience using Unity 3D or having any graphic arts background it was extremely time consuming to replicate a natural landscape virtually. The selection of materials available for the terrain were limited to a few grasses, rocks, sand and trees in the Personal version. Other limitations include the user’s skills related to programming, particularly the unsuccessfully programmed colliders, which create invisible barriers to limit the airplane from flying off the landscape. Despite these limitations Unity 3D appears to provide the user with an endless creative canvas for game creation, landscape modeling, alteration and conceptual design, which are often very limited when using other GIS related software.