Month: January 2019

Using LiDAR to create a 3D Basemap

By: Jessie Smith
Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018

INTRO

My Geovisualization Project focused on the use of LiDAR to create a 3D Basemap. LiDAR, which stands for Light Detection and Ranging, is a form of active remote sensing. Pulses of light are sent from a laser towards the ground. The time it takes for the light pulse to be returned is measured, which determines the distance between where the light touched a surface and the laser in which it was sent from. By measuring all light returns, millions of x,y,z points are created and allow for 3D representation of the ground whether it be just surface topography or elements such as vegetation or buildings etc. The LiDAR points can then be used in a dataset to create DEMs or TINs, and imagery is draped over them to create a 3D representation. The DEMs could also be used in ArcPro to create 3D buildings and vegetation, as seen in this project.

ArcGIS SOLUTIONS

ArcGIS solutions are a series of resources made available by Esri. They are resources marketed for industry and government use. I used the Local Government Solutions which has a series of focused maps and applications to help local governments maximize their GIS efficiency to improve their workflows and enhance services to the public. I looked specifically at the Local Government 3D Basemaps solution. This solution included a ArcGIS Pro package with various files, and an add-in to deploy the solution. Once the add-in is deployed a series of tasks are made available that include built in tools and information on how to use them. There is also a sample data set included that can be used to run all tasks as a way to explore the process with appropriate working data.

IMPLEMENTATION

The tasks that are provided have three different levels: basic, schematic and realistic. Each task only requires 2 data sources, a las(LiDAR) dataset and building footprints. Based on the task chosen, a different degree of detail in the base map will be produced. For my project I used a mix of realistic and schematic tasks. Each task begins with the same steps: classifying the LiDAR by returns, creating a DTM and DSM, and assigning building heights and elevation to the building footprints attribute table. From there the tasks diverge. The schematic task then extracted roof forms to determine the shape of the roofs, such as a gabled type, where in the Basic task the roofs remain flat and uniform. Then the DEMS were used in conjunction with the building footprints and the rooftop types to 3D enable buildings. The realistic scheme created vegetation points data with z values using the DEMs. Next, a map preset was added to assign a 3D realistic tree shape that corresponds with the tree heights.

DEMs Created

DSM

DTM

Basic Scene Example

Realistic Scene

 

ArcGIS ONLINE

The newly created 3D basemap, which can be seen and used on ArcGIS Pro, can also be used on AGOL with the newly available Web Scene. The 3D data cannot be added to ArcGIS online directly like 2D data would be. Instead, a package for each scene was created, then was published directly to ArcGIS online. The next step is to open this package on AGOL and create a hosted layer. This was done for both the 3D trees and buildings, and then these hosted layers were added to a Web Scene. In the scene viewer, colours and basemaps can be edited, or additional contextual layers could be added. As an additional step, the scene was then used to create a web mapping application using Story Map template. The Story Map can then be viewed on ArcGIS Online and the data can be rotated and explored.

Scene Viewer

Story Map

You can find my story map here:
http://ryerson.maps.arcgis.com/apps/Styler/index.html?appid=a3bb0e27688b4769a6629644ea817d94

APPLICATIONS

This type of project would be very doable for many organizations, especially local government. All that is needed is LiDAR data and building footprints. This type of 3D map is often outsourced to planners or consulting companies when a 3D model is needed. Now government GIS employees could create a 3D model themselves. The tasks can either be followed exactly with your own data, or the general work flow could be recreated. The tasks are mostly clear as to the required steps and processes being followed, but there could be more reasoning provided when setting values or parameters specific to the data being used inside the tool. This will make it easier to create a better model with less trial and error.

 

 

 

Creating Effective Landsat Timelapses

When asked to describe a satellite time-lapse, it is extremely likely that one would extrapolate some sort of time-series creation that shows gradual change with the use of natural colour images. It is however in this method of displaying satellite imagery, that a great deal of information is lost. To elaborate, individuals viewing such time-series creations cannot interpret quantifiable descriptors that may aid in conveying a message. Take for example the time-lapse provided below.

For what it is, a simple visual attempt at identifying change, the time-lapse above does an effective job at conveying change. Having said this however it does not provide the viewer with quantifiable observations, like the area developed per year, or even the ratio of land to water, as an example. Such statistics would be extremely useful in not only further illustrating a point, but also further encapsulating a viewer in the creation made. With this being said there are methods in which information can be derived from satellite imagery to provide viewers with a greater level of detail.

Within the realm of remote sensing, or the field in which data are collected without having a physical presence, there are a number of methods that allow individuals to interpret quantifiable data from the images one may capture. The most significant method being, a change detection. As a brief summary of what a change detection is, it is a temporal analysis between two, or more, periods of time, where either multi-spectral, or hyper-spectral images, and subsequent band combinations, are used to identify quantifiable areas of change between temporal foci. A change detection is of significant importance as it imparts the viewer with a definitive measure of change, far from the interpreted observation of change found within a regular timelapse. Taking a step back, one could quite easily identify, and even argue that in order to create a more effective timelapse, one could combine aspects of a change detection, creating an enhanced, and ever effective geo-visualization. Such a creation would bridge the gap between the lay, and the academic.

But such a thought begs the question; how could this be accomplished? Better yet, how could a timelapse, comprised of hundreds of composite spectral images, that displays a quantifiable characteristic, be created in an timely, and effective manor?

Put simply, programming is the answer to such questions. 

By creating a number of programs, an effective timelapse can be created that can be an effective evolution, from what was previously used. In an attempt to elaborate such a plan, such an attempt follows three steps:

  1. Create a program, a web-driver, that downloads an entire dataset, in this case all Landsat 4 – 5, imagery for a specific row, and path, in the absence of an API, or database. For an effective geo-visualization to be created, a great deal of data are needed.
  2. Instantiate a program, that opens each file, in this case Landsat image, and pulls three bands to be used to create a composite image that will better show a specific type of geographic change..
  3. Develop a program, that classifies imagery based on a predefined signature class, a file containing information on predetermined training classifications, that will then find, and append, the total area of a classification to a CSV file.

Put simply, the series of programs that will be created will create the intended geo-visualization, as well as do so in an efficient manor.

Attached are two links to a Dropbox folder, both links are direct connections to both programs needed to, download, and compose the images in focus.

Image Downloader:

https://www.dropbox.com/s/63ig83vd30ydk0a/Composite_Composer.py?dl=0

Image Composer:

https://www.dropbox.com/s/63ig83vd30ydk0a/Composite_Composer.py?dl=0

*Note, that in-depth comments have been provided for each stage, within each program. A general idea, as well as flow of each program should be quite easy to interpret.

It should be noted however, in the process of creating such a geo-visualization, a significant error arose. To elaborate, in the process of creating the final program, the GIS software, in this case ArcGIS, through the manipulation of ArcPY, failed on multiple attempts to read a signature file, and create a classification for each image. After much research it was concluded that an internal, however still unresolved issue was the main cause. As such, the final, and most important program could not be created.

The images created, were then edited in Adobe Lightroom, a image manipulation software, to remove any significant errors, and stitched together using Adobe Premiere Pro. Additional peices of information, such as titles, summaries, as well as dates were added within Adobe After Effects. The final geo-visualization can be seen below.

In summary, the Landsat timelapse created is extremely unique, it can safely be argued that there are no other geo-visualization methods like it currently present. In regards to what was attempted to be solved, it however did not accomplish all the tasks set out. From the perspective of the viewer, it is an easier method of data interpretation, however, it still lacks the quantifiable data aspect.

With all of this being said however, it is hoped that in the future, when previously mentioned bugs in software are fixed, another, more successful attempt at this geo-visualization can be made.

Time-series Animation of Power Centre Growth in the Greater Toronto Area for the Last 25 Years

By: Jennifer Nhieu
Geovisualization Class Project @RyersonGeo, SA8905, Fall 2018

Introduction:

In 1996, there were 29 power centres with 239 retail tenants accounting for just under five million square feet of retail space (Webber and Hernandez, 2018). 22 years later, in 2018, there are 125 power centres with 2,847 retail tenants accounting for 30 million more square feet of retail space (Webber and Hernandez, 2018). In addition, power centres expand in an incremental manner, either through the purchase and integration of adjoining parcels or the conversion of existing parking space into new stores (Webber and Hernandez, 2018). This development process often leads to retail centres becoming “major clusters of commercial activity that significantly exceed the original approved square footage total” (Webber and Hernandez, 2018, pg. 3).

Data and Technology:

To visualize this widespread growth of power centres from 1996 to 2017, a time-series animation map was created on Kepler.gl. (beta version) using power centre growth data provided by the Centre for the Study of Commercial Activity (CSCA) at Ryerson University, who undertakes an annual field survey-based inventory of retail activity in the Greater Toronto Area. Kepler.gl was created by Uber’s visualization team and released to the public on the summer of 2018. It is an “open source, high-performance web-based application for visual exploration of large-scale geolocation data sets. Kepler.gl can render millions of points representing thousands of trips and perform spatial aggregations on the fly” and is partnered with Mapbox, a “location data platform for mobile and web applications that provides open-source location features”. (Uber Technologies Inc., 2018 and Mapbox, 2018).

Methodology:

The data provided by the CSCA includes information on the shopping centre’s name, a unique identification code for each power center, the longitude and latitude coordinates for each power centre, its square footage information from the year it was built to 2017 and etc. The data had to be restructured on Microsoft Excel to include a pseudo date and time column, which would include the years 1992 to 2017, as well as 1-hour time intervals to allow Kelper.gl to create an animation based on date and time. The table below is an example of the data structure required to create a time animation on Kepler.gl.

Table 1: Data structure example*

Table 1: Data structure example*
*The data in this table has been modified due to confidentiality reasons.

Below is the time series animation set up for visualizing power centre growth on Kepler.gl. This process can also be replicated to produce another time-series animations:

  1. Visit https://kepler.gl/#/
  2. Click Get Started.
  3. Drag and drop .csv file onto application.
  4. Hold and drag to navigate to the Toronto GTA.
  5. On the contents bar, click + Add Layer on the Layers
  6. Under Basic Layer Type, click Select A Type, then select Point.
  7. Under Columns, Lat* = COORDY and Lng* = COORDX.
  8. Under Color, click Color Based On, then select SQ FT.
  9. Under Color, click the color scheme bar, select a preferred light to dark colour scheme.
  10. Under Color, Color Scale, select quantize.
  11. Under Color, Opacity, set to 4.
  12. Under Radius, Radius Based on, Select a field, select
  13. Under Radius, Radius Range, set the range from 1 to 60.
  14. On the contents bar, click + Add Layer on the Filters
  15. Click Select a field, then select
  16. In the slider, drag the rightmost square notch to highlight only 2 bars with the left square notch.
  17. Press the play button the start the animation.

Notes:

  • The speed of the animation can be adjusted.
  • The legend can be shown by clicking the bottom circular button located to the top right corner of the screen.
  • Hover your mouse over a point to see the metadata of the selected power centre.
Figure 1: Power Centre Growth in the Toronto GTA (1992 – 2017)

Limitations:

During the implementation process, it became apparent that Kepler.gl is more focused on graphics and visuals than it does on cartographic standards. The program does not allow the user to manually adjust class ranges on the legend, nor does it accurately display continuous data. The proportional symbols used to represent power centre growth displays flashing or blinking symbols rather than a gradual growth in the symbols. There was an attempt to correct this problem by duplicating the values in the date and time column, and then adding additional pseudo date and time values between each year. However, when tested, the animation exhibited the same flashing and blinking behaviour, therefore it became apparent that is problem exists in the programming of Kepler.gl and not in the data itself. Furthermore, by duplicating these values, the file would exceed the maximum file size on chrome (250mb), and limit performance on Safari, the two web browsers it runs on.

Conclusion:

Regardless of the limitations, as the current Kepler.gl is still in its early beta version, it still has a lot of potential to incorporate user feedback from industry professionals and run additional testing before the final release.

References:

Webber, S. and Hernandez, T. (2018). Retail Development and Planning Policy.   Centre for the Study of Commercial Activity, Ryerson University. Toronto, CA.Uber Technologies Inc. (2018). Kepler.gl. Retrieved from http://kepler.gl/#/
Mapbox. (2018). About Mapbox. Retrieved from https://www.mapbox.com/about/

App Building for the Technically Challenged Cartographer

By Kevin Duffin

Geovis Project Assignment @RyersonGeo SA 8905 Fall 2018

Creating a custom web mapping application can seem like a daunting task for an individual without a whole lot of technical experience. These individuals, myself included, can feel as though they must first learn computer science before they are able to perform web mapping. However, there are a variety of web services which allow for the creation of powerful geovizualization application without the detailed knowledge of computer programming.

One such service is ESRI’s Web Appbuilder for ArcGIS.

Web Appbuilder is an application hosted on ArcGIS online which allows users to add functionality to custom web maps. An exciting feature of the Web Appbuilder is the ability to create maps not only in 2D, but also in 3D via the scene view in Web Appbuilder.

The scene view environment allows users to create a variety of 3D interactive mapping applications on a virtual globe. Navigation around the virtual world is incorporated, and various customization are possible via a variety of out of the box functionalities which can be easily incorporated to any application.

Pretty great right? Follow along and learn how to make a 3D web app!

Introduction to my data

The economies of many nations around the world, including Canada, rely very heavily on natural resource use. In recent times there has been a push by many countries to decouple their economies from natural resource use to both increase the sustainability of their economy, and to decrease their environmental impact.

Material footprint is a measure of domestic material use developed by Wiedmann, T. O., Schandl, H., Lenzen, M., Moran, D., Suh, S., West, J., and Kanemoto, K in 2015. The material footprint(MF) of a nation is the total amount global raw materials extraction that can directly attributed to the final demand of that nations economy.

When developing the metric, Weidmann et al determined the MF per capita of every nation in the world. The group also calculated the MF per capita for various material types, such as biomass materials, construction materials, fossil fuels, and metal ores. I joined Weidmann et al.’s MF data to a country shapefile using ArcMap and the resulting shapefile was exported and saved to my local computer. This is the data I used to create my geovisualization project.

The goal of my project was to create a web mapping application which allows the user to view the MF data on a virtual globe, and toggle between material types. By viewing which material types are important to various nations, the user can then make inferences about sustainability of those nations.

Getting Started: Publishing Layers to ArcGIS Online

In order to create the web application, I first needed to publish the spatial data to ArcGIS Online. To do this I first logged in to my ArcGIS online account. If you do not have an ArcGIS Online account, you can create a free personal account or sign up through your organization, such as university or business. Once signed in, navigate to the “My Content” tab and click the “Add Item” button. For my project I added the MF data from my computer. If your data is saved on your local computer, ensure that it is in ZIP format. I added the MF layer six times, as I had six layers I wanted to display on my application. I named one layer the Material Footprint, four layers using the material groups, and one representing the select few countries which did not have data.

Creating the Map

Once the layers were uploaded, I navigated to the “Create” button and hit “map”. This opened a map viewer tab where I was able to create the basemap for my application on a 2D surface. I added the six layers to the map viewer and began making the basemap. The “change style” tab was used to classify and select a colour scheme for each layer. I then configured pop- ups to display the MF value of a nation when a country is selected. Once I was happy with all the layers, I then saved the layers and the map, and navigated back to “My Content”.

Creating the Web Scene

I next needed to display the layers I just created in a 3D environment. From the “Create Tab”, I selected “Create Scene”. A scene viewer page opened and the virtual 3D globe which I used to display my layers was generated.  Using the “Modify Scene” tab, I added my six layers that I formatted in the map view from my content.  As this scene will become the base of the mapping application, it is important that I configured all the desired setting in the scene view, as these settings will not be able to be changed in the application itself. For example, I altered the order of the layers in my legend, chose a basemap, specified the suns position in the sky, and optimized the performance of the scene in scene settings by ensuring the 3D graphics slide bar was set to Performance rather than quality.  I then saved the scene and navigated back to “My Content”.

Creating the Web App

In the “Create” tab, I then created an App using the Web Appbuilder. In the following “Create a web app” pop up, I specified 3D and gave the app a title, a tag, and a brief summary.

I then needed to specify to Scene which I wanted to use as the baselayer for my app. I navigated to the Scene I just created through the “Choose web scene” button in the scene setting window. This then projected my 3D MF layers onto the virtual globe into the map. I then navigated to the theme window  and chose a graphic theme for my application.

Adding Functionality

Functionality is added to Web Appbuilder applications through widgets. Widgets are tools that can be added to the application. These tools perform a variety of functions. Also, the number of widgets you can incorporate into an app is based on the theme you have selected, so choose wisely. In my application I chose four main widgets, Legend, About, Layer List and 3D Extrusion. The legend widget simply adds a legend to the app which updates depending on the layer being displayed. I configured the About widget to display text introducing the application. The layer List is the widget which enables the toggling between MF layers. Fnally, the 3D widget allows for several different 3D functionalities. I selected the “Area Extrusion” visualization type to extrude the countries based on their Material Footprint per Capita Values. Along with the 3D extrusion, a display bar is added the app which displays the MF values of each nation. By clicking on a nation name in the display bar, the view automatically zooms to that country. Neat!

Finishing Touches

After all the functionality was added to the application, I added a few finishing touches. A proper summary and description were added to the Web App page, and a simple splash widget was added to introduce the application.

Try it out here!

Try the application out here, and thanks for following along!

https://ryerson.maps.arcgis.com/apps/webappviewer3d/index.html?id=b72c5f9cb9194a1abbff695a7b5b275f

 

New York City Automobile Collisions

Creating An Interactive Web Map

By: Joshua Ali

Geovis project Assignment @RyersonGeo, SA8905, Fall 2018

Data

The data used for this map was retrieved from New York City Open Data (https://opendata.cityofnewyork.us/) and automobile collisions data set, it has information on collision from 2011 to present.  This will have all information needed for the map.

Using Mapbox

The interactive map will be using map box GeoLibrary JavaScript, so an account must be created with map box.  This is a free sign up and a pay as you go account (pretty much if you use it a lot you have to pay) (https://www.mapbox.com/signup).

Creating the Base Map

The next step is to create the base map that was used to display my data.  To write the code I used a text writing software.  The two I switched between using is Sublime (https://www.sublimetext.com) and Codepen, (https://codepen.io) they are both free software’s that can be used.  Now you will need to write a html doc that will be used to display your map.  The doc was written below for optimal settings and will be built upon with more code later to customize the map.

Now that the setting style for the map functions have been in place, the map needs to be linked with a mapbox access token that is created from my account.  By doing this the html doc will be linked to my account.

A script was created using the var function to create a new map that will use as style option that is linked to your account.  In my case I decided to use the dark map background as my style for the map.  Also, in the script below the latitude and longitude was selected so when the map is opened it will be looking at New York City.

With all the current script within the text editor this document can be opened with chrome browser to show the base map.  The image below shows what would come up.

Customizing the Base Map

Now that the base map of the map is created, I can begin adding and customizing NYC automobile collisions to the map.  To connect the data downloaded to the map created they first need to be in the same project folder as the base map html doc made above.

To do this a local server has to be made on the computer so the base map made can draw information from the NYC database to be projected. The api also needs this to continuously update the projected data to the interactive tool that will be added later.  This was done by downloading python and running Python’s SimpleHTTPServer. Using the command control panel, the local server was run on my laptop.  This is useful because changes made in the coding on the text editor of the map can be seen immediately on the html map doc since the local host is constantly updating the files.

To connect the collision data to the base map, a map load function was used to link the id called collisions along with the data file url and the settings to display the collisions on the map as circles. Also the circle radius based on 0 to 5 was linked with their own selected colours and circle-opacity to 0.8 so depending on how many casualties occur they will have their own colour and partially transparent so they will not bock each other.

With the data now linked to the base map, a legend was created in the code by making a div section inside of the console.  Additionally, to this some CSS was added to style the colour gradient so it matches the colours of the circles.

This is what the map will look like with the data and legend.

Adding Time Slider and Interactivity

To add a time slider the slide bar function was added as a div to the body of the html document.  This will pull information of the time of accidents and display them on the map. To add the interactive a filter was added to obtain the time of collisions from the database.  The coding will be shown below along with a screen shot of the functional map.

Final Touches

The map is almost complete, the last function that was added to it was a filter that looks at the automobile collisions that occurred during the week compared to during the weekend.  To create this an if function was added to the text editor so that if a collision occurred on a Saturday it would be false, true.  This allows the data to show for week days compared to weekends. This is seen in the coding below. To add the function that allows you to decide on what part of the data to look at another div class session was added to filter the days.

The below script shows the div class session that created the slider bar and button selection for the final map legend.

Final Map

Below is screen shots different settings selected for the interactive map.

Mapping Toronto Green Space in Android

By Jacob Lovie | GeoVis Project Assignment                @RyersonGeo | SA8905 | Fall 2018

Introduction

With today’s technology becoming more and more mobile, and the ability to access everything you need on your mobile device, it is more important than even to ensure that GIS is evolving to meet these trends. My GeoVisualization project focused on designing an android application to allow users to explore Toronto’s green space and green initiatives, making layers such as parks and bike stations accessible in the palm of your hand. However, it is not just having access to this that is important. What’s important when working with these technologies is that a user can explore the map and retrieve the information seamlessly and efficiently.

Data and Hosting Feature Services

All the data for the project was retrieved from the city of Toronto’s open data portal. From there, all the data was uploaded to ArcGIS Online and set up as hosted feature services. A base map was also designed using ArcGIS for Developers and hosted. The application was able to target these hosted feature layer and use them in the map, making the size of the app small. The symbology and setup of the hosted feature layers was also done in ArcGIS online, so the app didn’t have to make any changes or set symbology when it wasn’t necessary.

Methods

The developer environment that I worked in to design my app was Android Studio, the baseline for designing any android apps. The programming language used in Android Studio is Java. Within Android Studio, the functionality of ArcGIS Runtime Software Developer Kit (SDK) for Android can be brought in, bringing in all the libraries and functions associated with ArcGIS Runtime SDK for Android. With this I was able to use ArcGIS functionality in android, designing maps, accessing hosted feature services, and perform geoprocessing.

Understanding how ArcGIS SDK for Android worked with in Android Studio was an important key in designing my app. When creating a map, I first had to create a Map object. An object is a variable that is of a certain datatype. If you were talking about having a text object as a variable that could be called, it would be of datatype string, and the word itself would be an object that is callable and referential. The Map object is what is displayed in an activity window (more on this later), which is what the user visualizes when using the app. The map can be set to view a certain area, which was Toronto in my app. A user can pan around the map like they would on any interactive map without having additional coding in Android Studio (it is natural to the Map datatype). The Map also has associated Layer objects that have their own set of parameters.

While designing my app, any time I would want something done in my design, such as creating a map object or adding a layer to a map object, I created a function that wold performs an action. This reduces repetition in the code when I attempted to do something complex multiple times. I designed 3 functions. The first was to create a Map, the second was to add a Layer that could be activated and deactivated in the Map through a switch that would be displayed in the main Activity Window. The final function added a layer that could be queried and would extract information from that layer.

When designing an android app, there are many fine details that are not necessarily considered when using an app on your phone. Simple things like having a window or text appear, opening a second window, or displaying information were things I very much appreciated after designing the app. Within my app, I wanted to display a second activity window to display information on neighbourhoods in Toronto when the user touched them. Within Android Studio this required creating a second activity window, and transferring the information obtained in the map to the second activity. This was done through my displayInformation function. I was then able to create a second activity and display this information using a custom list display to show the attribute data of a selected neighbourhood.

     >>>>>>>>>>>     

Setting up the display in Android Studio is relatively simple. There is an interface that allows you to anchor different objects to parts of the screen. This allows the app to run smoothly across all devices, not based on the size and ratio of the device. The switches in my Main Activity window were anchored to the top left, and to each other. My Map is in the background, but appears as white in this activity window.

The Application

Once all the coding and testing was completed, running the app was simple. I was able to bundle my code and send it to my personal phone, a Galaxy S9. The functions called the hosted service layers and displayed them in the map (Wifi or internet connection was required). I was also able to click on neighbourhoods and it would open my second activity that displayed the attribute information of that neighbourhood. If you want a more in-depth look at my code, it is available at https://github.com/jclovie/GeoVis-Ryerson/.

Urban Development of San Francisco

By Hannah Burdett

SA8905 Geovisualization Project, Ryerson University

The Development of San Francisco

San Francisco is located in the center of Northern California. It started as a base for the gold rush of 1849, the city quickly became one of the most populated cities in the United States. Shortly thereafter, San Francisco was devastated by the 1906 earthquake. Development peaked in the 1900’s as San Francisco rebuilt areas demolished by the earthquake and fires to compensate the growing population. During the 1930’s the San Francisco-Oakland Bay Bridge and the Golden Gate Bridge were opened. Additionally, during World War II, San Francisco was a major mainland supply point and port of embarkation for the war in the Pacific. Both factors led to another peak in construction. After World War II, many American military personnel who had fallen in love with the city while leaving for or returning from the Pacific settled in the city. This led to promoting the development of the Sunset District, Visitacion Valley, and the total build-out of San Francisco. Starting in the latter half of the 1960’s, San Francisco became most recognized for the hippie movement. Currently, San Francisco has become known for finance and technology industries. There is a high demand for housing, driven by its close proximity to Silicon Valley, and a low supply of available housing has led to the city being one of America’s most expensive places to live.

Data

The data used for the time series animation was imported from data.gov. Data.gov is a repository for the US Governments open source data. The imported data included a Land use Shapefile for San Francisco. The shapefile included information such as land use, shape area, street address, street number, etc. The land use shapefile also included the year the building was built. The building years range from 1848 to 2016 displaying 153 years of urbanization. The buildings were represented as polygons throughout San Francisco. Additionally, a grey scale base map from ArcGIS Pro was displayed to create a more cohesive map design.

 

 

Time Series Animation

To develop the reconstruction of San Francisco throughout the years, both QGIS and ArcGIS Pro were utilized. Both platforms were used so to provide a comparison between time series animation tools from an open source application and a non-open source application.

QGIS is an open source geographic information systems application that provides data visualization, editing, and analysis through functions and plugins. To create the time series animation the Time Manager plugin was utilized. The Time Manager plugin animates vector features based on a time attribute. For this study the time attribute was the years built.

ArcGIS Pro is the latest professional desktop GIS from Esri. ArcGIS Pro enables users to view, explore, analyze, edit and share maps and data. Unlike QGIS, no additional plugins are required to create the animated time series.

QGIS Methodology

To generate the time series in QGIS, the land use shapefile was downloaded and opened in QGIS. The attribute table from the land use shapefile was then exported and opened in Excel so that the yrbuilt column could be reformatted to meet QGIS Time Manager requirements. The yrbuilt column had the data presented as YYYY format for building dates. QGIS Time Manager requires timestamps to be in YYYY-MM-DD. To correct the format, -01-01 was added to the end of each building year. The modified values were then saved into a new column called yrbuilt1. The Excel sheet was then imported into QGIS and joined to the land use shapefile.

In QGIS, each of the buildings was presented as polygons. The shapefile symbology was changed from single symbology to quantified symbology. In other words, the symbology for each of the polygons was broken down to seven classes defined by years. Each class was then distinguished by color, so that one may differentiate the oldest building from the newest buildings. Furthermore, a grey scale basemap was added to create a more cohesive map.

Furthermore, in the Time Manager settings, “Add Layer” was selected. The land use shapefile was chosen as the Layer of interest. The start time was set to the yrbuilt1 attribute, whereas the end time was set to “No end time – accumulate features”. This allows newer buildings to be added without older buildings being removed from the map. For the animation, each time frame will be shown for 100 milliseconds. The Time Manager plugin was then turned on so that the time series may run.

 

In order to export the time series animation, Time Manager offers an “Export Video” option. However, this exports the animation as an image series, not as an actual video. To correct this, the image series was uploaded to Mapbox where additional Mapbox styles were used to render the map. It was then exported as a Gif from Mapbox.

ArcGIS Pro Methodology

In ArcGIS Pro, the land use shapefile was imported. The symbology for each of the polygons was then broken down to seven classes defined by years. The same colours utilized in QGIS were applied to the classes in ArcGIS Pro to differentiate between the building years. Within the layer’s properties, the Layers Time was selected as “each feature has a single time field”. Furthermore, the start and end times were set to the newest and oldest building years. The number of steps were assigned a value of sixteen. In View, the animation was added, and the Time Slider Steps were imported. The time frames were set to match the QGIS animation so that both time series animations would run at the same speed. The time series animation was then exported as a Gif.

Final Animated Map

Finally, to create a cohesive animated map the exported Gif’s were complied together in PowerPoint. Additional map features, such as a legend, were designed within PowerPoint. A bar graph was added along the bottom of the map to show years of peak building construction. The final time series map was then exported as a .mp4 and upload to YouTube.

Invasive Species in Ontario: An Animated-Interactive Map Using CARTO

By Samantha Perry
Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018

My goal was to create an animated time-series map using CARTO to visualize the spread of invasive species across Ontario. In Ontario there are dozens of invasive species posing a threat to the health of our lakes, rivers, and forests. These intruding species can spread quickly due to the absence of natural predators, often damaging native species and ecosystems, and resulting in negative effects on the economy and human health. Mapping the spread of these invasive species is beneficial for showing the extent of the affected areas which can potentially be used for research and remediation purposes, as well as awareness for the ongoing issue. For this project, five of the most problematic or wide-spread invasive species were included in an animated-interactive map to show their spatial and temporal distribution.

The final animated-interactive map can be found at: https://perrys14.carto.com/builder/7785166c-d0cf-41ac-8441-602f224b1ae8/embed

Data

  1. The first dataset used was collected from the Ontario Ministry of Natural Resources and Forestry and contained information on invasive species observed in the province from 1982 to 2012. The data was provided as a shapefile, with polygons representing the affected areas.
  2. The second dataset was downloaded from the Early Detection & Distribution Mapping System (EDDMapS) Ontario website. The dataset included information about invasive species identified between 2010 and 2018. I obtained this dataset to supplement the Ontario Ministry dataset in order to provide a more up-to-date distribution of the species.

Software
CARTO is a location-intelligence based website that offers easy to use mapping and analysis software, allowing you to create visually appealing maps and discover key insights from location data. Using CARTO, I was able to create an animated-interactive map displaying the invasive species data. CARTO’s Time-Series Widget can be used to display large numbers of points over time. This feature requires a map layer containing point geometries with a timestamp (date), which is included in the data collected for the invasive species.

CARTO also offers an interactive feature to their maps, allowing users control some aspects of how they want to view the data. The Time-Series Widget includes animation controls such as play, stop, and pause to view a selected range of time. In addition, a Layer Selector can be added to the map so the user is able to select which layer(s) they wish to view.

Limitations
In order to create the map, I created a free student account with CARTO. Limitations associated with a free student account include a limit on the amount of data that can be stored, as well as a maximum of 8 layers per map. This limits the amount of invasive species that can be mapped.

Additionally, only one Time-Series Widget can be included per map, meaning that I could not include a time-series animation for each species individually, as I originally intended to. Instead, I had to create one time-series animation layer that included all five of the species. Because this layer included thousands of points, the map looks dark and cluttered when zoomed out to the full extent of the province (Figure 1). However, when zoomed in to specific areas of the province, the points do not overlap as much and the overall animation looks cleaner.

Another limitation to consider is that not all the species’ ranges start at the same time. As can be seen in Figure 1 below, the time slider on the map shows that there is a large increase in species observations around 2004. While it is possible that this could simply be due to an increase in observations around that time, it is likely because some of the species’ ranges begin at that time.

Figure 1. Layer showing all five invasive species’ ranges.

Tutorial

Step 1: Downloading and reviewing the data
The Ontario Ministry of Natural Resources and Forestry data was downloaded as a polygon shapefile using Scholars GeoPortal, while the EDDMapS Ontario dataset was downloaded as a CSV file from their website.

Step 2: Selection of species to map
Since the datasets included dozens of different invasive species in the datasets, it was necessary to select a smaller number of species to map. Determining which species to include involved some brief research on the topic, identifying which species are most prevalent and problematic in the province. The five species selected were the Eurasian Water-Milfoil, Purple Loosestrife, Round Goby, Spiny Water Flea, and Zebra Mussel.

Step 3: Preparing the data for upload to CARTO
Since the time-series animation in CARTO is only available for point data, I had to convert the Ontario Ministry polygon data to points. To do this I used ArcMap’s “Feature to Point” tool which created a new point layer from the polygon centroids. I then used the “Add XY Coordinates” tool to get the latitude and longitude of each point. Finally, I used the “Table to Excel” conversion tool to export the layer’s attribute table as an excel file. This provided me with a table with all invasive species point data collected by the Ontario Ministry that could be uploaded to CARTO.

Next, I created a table that included the information for the five selected species from both sources. I selected only the necessary columns to include in the new table, including; Species Name, Observation Date, Year, Latitude, Longitude, and Observation Source. This combined table was then saved as an excel file to be uploaded to CARTO.

Finally, I created 5 additional tables for each of the species separately. These were later used to create map layers that show each species’ individual distribution.

Step 4: Uploading the datasets to CARTO
After creating a free student account with CARTO, I uploaded the six datasets as excel files. Once uploaded, I had to change the “Observation Date” column from a “string” to “date” data type for each dataset. A “date” data type is required for the time-series animation to run.

Step 5: Geocoding datasets
Each dataset added to the map as a layer had to be geocoded. Using the latitude and longitude columns previously added to the Excel file, I geocoded each of the five species’ layers.

Step 6: Create time-series widget to display temporal distribution of all species
After creating a blank map, I added the Excel file that included all the invasive species data as a layer. I then added a Time-Series Widget to allow for the temporal animation. I then selected Observation Date as the column to be displayed, meaning that the point data will be organized by observation date. I chose to organize the buckets, or groupings, for the corresponding time-slider by year.

Since “cumulative” was not an option for the Time-Series layer, I had to use CARTCSS to edit the code for the aggregation style. Changing the style from “linear” to “cumulative” allowed the points to remain on the screen for the duration of the animation, letting the user see the entire species’ range in the province. The updated CSS code can be seen in the screenshots below.

Step 7: Creating five additional layers for each species’ range
Since I could only add one Time-Series Widget per map, and the layer with the animation looks cluttered at some extents, I decided to create five additional layers that show each of the species’ individual observation data and range.

Step 8: Customizing layer styles
After adding all of the layers, a colour scheme was selected where each of the species’ was represented by a different colour to clearly differentiate between them. Colours that are generally associated with the species were selected. For example, the colour purple was selected to represent Purple Loosestrife, which is a purple flowering plant. The “multiply” style option was selected, meaning that areas with more or overlapping occurrences of invasive species are a darker shade of the selected colour.

A layer selector was included in the legend so that users can turn layers on or off. This allows them to clearly see one species’ distribution at a time.

Step 9: Publish map
Once all of the layers were configured correctly, the map was published so it could be seen by the public.

Traffic.me: Mapping live traffic with ArcGIS Runtime SDK and HERE Technologies using Android App Developer

by Nicholas Pulsone
Geovis Class Project @RyersonGeo, SA8905, Fall 2018

Topic & Background

Driving through congested parts of Toronto is a tedious and troubling problem that many people would like to avoid. The goal was to create a mobile application using android app developer that can use traffic data as a live input to map traffic patterns across North America. Many companies such as HERE Technologies record traffic information that updates regularly and can be used to map and observe traffic patterns across the entire world. Using android app developer, it is easy to add software developer packages such as ArcGIS Runtime SDK to develop new tools that can be used on a day-to-day basis.

Data

The first problem when creating an app for a purpose or goal is where to find the data. As previously mentioned, HERE technologies is a company owned by NOKIA and currently has its headquarters in Amsterdam. HERE technologies records live weather, routing and traffic information using a combination of both geolocation and intelligence algorithms. Geolocation services that HERE tracks include:

  • Devices with location or GPS tracking
  • Tables or other devices with WIFI and signal strength
  • Phones while measuring varying strength of reception via cell tower signals

HERE technologies contains a global database of over 93 million cellular towers and over 2.3 billion Wi-Fi hotspots which record and store data. The data needed to be able to map varying levels of traffic or traffic density as well as potential collisions or other disruptions affecting driving conditions. The data would need to be able to be displayed visually on a mapping platform and accessible by android app developer software.

Methods

There are multiple ways a live traffic application can be created using data from HERE technologies:

  1. Creating a live traffic app using HERE API and map creator
  2. Creating a live traffic app using HERE data in ArcGIS Runtime SDK (requires ArcGIS developer license)

The methods in this blog will be describing how to create an application using the data from HERE technologies with ArcGIS Runtime SDK.

The first step is to download the needed requirements. First, is to download the newest version of android app developer studio. Currently, the newest version of Android App developer studio is 3.2.1 and available online for Windows, Mac and Linux. Once android app developer is downloaded, the next step is download the second part of the software that will be used in this creation, which is the ArcGIS Runtime SDK for Android 4.0.

The second step is to set the back-end of the application. After specifying the operating system the application will work on, and inputting the name of the application, the first thing to set up is include the ESRI bintray for ArcGIS.

As ESRI’s repository is not open source, the url must be specified to manually add the url for the ESRI bintray. Then the app dependencies need to updated to include the ArcGIS Runtime SDK.

Once the Gradle scripts were synced, the next step was to add a map view for the app. By default, we can remove the text view element and manually create a map view with the following syntax:

After adding the map view for the data, the next step was to specify a basemap then access the data:

The above syntax is a sample of how a basemap and starting location can be specified upon opening. The final step was to be able to access the data. HERE technologies has collaborated with ESRI to develop a world traffic service that can be accessed from mobile and desktop services using the url:

http://traffic.arcgis.com/arcgis/rest/services/World/Traffic/MapServer

Additionally, ESRI and HERE technologies have also created a layer available on the ArcGIS developer portal to users with an ArcGIS developer license. Once the layer is accessible, it is important to open and save the layer in ArcGIS online and enable sharing and public access permissions. As layers used in ArcGIS require a login to be viewed, the next step is to setup a proxy to bypass the login error that would prevent the data from being used, even if permissions are set to public.

To setup a proxy using the ArcGIS developer server proxy, the application must be authenticated and registered in the ArcGIS developer platform. Once registered, the user has access to many services such as a proxy service which will be used along with the traffic layer.

To enable this proxy under services, we must specify what type of proxy service and request limit the proxy will use. Once the requirements are specified, the service outputs a URL which contains a proxy service from ArcGIS.

To use the proxy, simply add the link as a layer from web in ArcGIS online, and the proxy should be active.

Figure 1: Adding Services to ArcGIS Online Map

The final step was to add in the url for the ArcGIS online webmap which contains the traffic data, into android app developer.

Once the url was added into the Android App developer; just click Sync & Run and the app will appear on your device similar to the picture below!

Figure 2: Example of Traffic App

Limitations

A limitation that was experienced while coding the application was ease of use. Without using a legend or slider, it is very hard to distinguish which areas of Toronto are being affected by what kind of problem. The symbology can be changed, however integrating a legend as a button feature in android app developer was more useful and ultimately was included in the final iteration of the app shown in Figure 3.

Figure 3: Final Iteration of Traffic.me

 

Global Impacts of Earthquakes and Natural Disasters

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018

Author: Katrina Mavrou

Date: Tuesday November 13th 2018

Topic: Global Impacts of Earthquakes and Natural Disasters

Link: Natural Disasters: Earthquake

Purpose:

To educate an audience with a wide variety of educational backgrounds on natural disaster impacts and side effects through visualization of spatial data using Esri’s Story Map Applications. As well, to gain a personal understanding  and experience using ArcGIS online and available tools.

Data Source: USGS Earthquake Database, 2018; Global International Displacement Database, 2018.

Software: ArcGIS Pro, ArcGIS Online

Outline:

The topic of this project is Natural Disaster impacts, in specific Earthquakes, and analyzes the global displacement and infrastructure loss. The platform used is ArcGIS Online, using hosted feature layers, web maps and mapping applications, such as a variety of Esri’s story map applications. The data is provided from USGS Earthquake database and Global International Displacement Database. The purpose of this exercise is to utilize multiple Esri story map functions, and other online plat forms into one map.

The first step was to clean up the datasets. The CSV files were imported into Microsoft SQL Server Management Studio to preform spatial queries and extract data needed from each dataset for the displacement dataset data was extracted by year and imported into a new table. The subset data was imported into ArcGIS Pro, where is was joined to the global shapefile and exported as new layers. Each year created a new layer so that the animation tool could be used on the time enabled field to create a time lapse of displacement. Each new layer was exported as a shapefile zipped folder so that it could be imported into ArcGIS Online as a hosted feature layer.

The first section, monthly earthquake report, of the story map is created by using a live link to publish data. To do this a web map is created using the live updating CSV from USGS. The data is collected and published every 5 minutes, therefore each time the story map is opened or refreshed the map will look different from the previous time. The data is displayed using the magnitude and depth of the earthquake (Image 1). This process was repeated for the second section, the weekly earthquake report. Another feature that was used for this map is pop-ups. Each instance displayed on the map when clicked will introduce a pop-up window that gives the user more information on that specific earth quake.

Live data of Monthly Earthquakes

Image 1: Live data of Monthly Earthquakes

The next section introduces the user to fault lines, specifically San Andreas Fault located in California, USA (Image 2). Using KML and KMZ files LiDAR layers of the fault lines are displayed on a satellite image map. Data of the most tragic earthquakes are also displayed on the map. This is historic data of earthquakes with magnitude greater than 5. By clicking on each earthquake location a pop-up is enabled and gives historical facts about each instance.

San Andreas Fault and Earthquakes

Image 2: San Andreas Fault and Earthquakes

The following section of the story map regards displacement caused by earthquakes. The main screen includes a heat map of displaced people from 1980-2018. The side panel includes a story map slide show. This was created from a simple web map with time enabled layers. The presentation function was then used to create a time lapse of global displacement for each year from 2010 to 2017 (Image 3). The presentation was then embedded into the side panel of the displacement section of the story map.

Presentation of Timelapse

Image 3: Presentation timelapse

The slide displacement VS population includes a swipe and slide story map embedded into the main story map. This is a global map that swipes two layers global displacement due to natural disasters for 2017 and world population for 2017 (Image 4). The swipe and slide map allows the user to easily compare the two layers side-by-side.

Swipe and Slide Map

Image 4: Swipe and Slide Map

The last section uses a story map shortlist to display images of earthquake impacts. This uses geocoded images to place images on a map (Image 5). The main panel holds with map with pointers indicating the coordinates of the geocoded images. The side panel displays the images. When an image is clicked on it pops up with the map and information regarding the image and earthquake can be found.

Geocoded image and reference map

Image 5: Geocoded image and reference map

The purpose of this technology is to easily display spatial data for individuals who are unfamiliar with utilizing other Esri products. Story maps are an easy and interactive way for users will all backgrounds of knowledge to interact with spatial maps and learn new information on a topic.

Limitations and Summary:

The final project is a story map journal with easy navigation and educational purpose. In the future I would like to incorporate even more of Esri’s online functions and applications, to expand my understanding. As well, a limitation to the project is the amount of space allotted to each student at Ryerson with ArcGIS online. Some processing functions were unavailable or used too much space, therefore these procedures had to be processed using ArcPro and then import the final product into ArcGIS online. Overall, the project reached its purpose of providing contextual information on spatial impacts of earthquakes.