Time-Lapse of City of Toronto Precipitation and Beach E.Coli Count- July 2018

Kezia Weed- SA8905 Geovis Project, Fall 2021

Introduction

Toronto’s beaches are an incredible feature of the city, popular throughout warm months for water activities of, among other things, swimming, paddle boarding, and boating. While taking a plunge into Lake Ontario is a great way for residents to cool off, it is always important to be aware of the current water quality conditions. During the summer, the City of Toronto tests beaches for Escherichia coli (i.e., E.Coli) counts daily as a public health measure, posting results available on site and online. What many residents are unaware of, is that E.coli is a bacteria that lives naturally in the guts of warm blooded animals; only high concentrations of E.coli at beaches pose a danger of infections for swimmers. In Toronto, beaches are posted as unsafe to swim when E.Coli counts exceed 100 fecal coliforms per 100ml.

Water quality can be impacted by numerous factors including legacy contaminants (i.e., lead), industrial activity (i.e., direct effluent discharge), and by urban stormwater and sewage water. In this visualization, the purpose is to examine the impact of precipitation on fecal coliform counts. This visualization includes seven weather stations and 10 downstream beaches across the city, as to capture the effect of rainfall on water quality. As a general rule, cities believe residents should not swim within 24-48 hours of a rain storm as rain can mobilize urban contaminants into the surface flow, streams, and eventually beaches. This is especially important during extreme precipitation events in Toronto, when there is the risk of combined sewer overflows (CSOs); CSOs occur when the volume of water discharged exceeds wastewater treatment plant capacity, and must be directed into Lake Ontario untreated.

On this premise, this visualization displays both precipitation and beach water quality over the span of a month. This is to examine, if at all, there are clear impacts between precipitation and the observed fecal coliforms at beach sites.

Data

The base layers for this visualization are i) ‘Toronto Watercourses’ polyline shapefile and ‘Toronto Watershed’ polygon shapefile, downloaded from the TRCA; a ii) ‘Lake Ontario’ shapefile, downloaded from the United States Geological Survey (USGS) Open Data portal; and ii) a ‘City of Toronto boundary’ shapefile downloaded from City of Toronto Open Data.

Seven (7) weather stations with daily precipitation data were identified for the duration of July 2018. Five of the weather stations’ datasets were retrieved from Environment Canada’s (EC) ‘Historic Weather’ data catalogue, and two additional weather stations’ datasets were accessed from the City of Toronto Open Data portal, from the ‘2018 Precipitation Data’ file. The daily total fecal coliform tests (i.e., E.Coli concentration) were downloaded from the City of Toronto’s ‘Swimming Conditions History’ webpage.

All of the weather and total coliform datasets were placed into individual comma separated value (.csv) files, with a column for date, latitude, longitude and an entry for either precipitation or E.Coli concentrations.

Sample csv file in appropriate format

Technology

All spatial datasets were inputted and visualized within the open-source geographic information system (GIS), QGIS 3.10. To achieve a time-series visualization, this study used the QGIS plugin, ‘TimeManager’ (developed by Anita Graser). TimeManager allows for the creation of timelapse maps with temporally stamped data.

TimeManager Plug-in in QGIS

Process

To create the initial map, add the basemap shapefiles of i) ‘Toronto Watercourses’, ii) ‘Toronto Watersheds’, iii) ‘City of Toronto boundary’, and iv) ‘Lake Ontario’.

Import the .csv files for each individual Toronto weather station using the ‘Add Delimited Text Layer’ option in the ‘Add Layer’ menu in QGIS 3.10. Within the import manager, the ‘X’ and ‘Y’ geometry values have to be selected for defining point geometry; in the ‘Geometry Definition’ section of the import manager, assign the ‘X’ value to the weather stations’ longitude and the ‘Y’ value to latitude. As the latitude and longitude coordinates were in decimals, minutes second, the geometry type was specified as ‘DMS’ in the coordinates box. If the data file has coordinates in decimal format, leave the coordinates box unchecked. Finally, select ‘Add’ and a point for the weather station which will be placed on the map. Repeat this process for each of the Toronto weather stations and Toronto beaches .csv files.

The next step is to create a spatial buffer for the precipitation levels that surround each weather station. In the tools pane, select ‘Vector’, then the geoprocessing toolbox (i.e., ‘Geoprocessing’), then select the buffer operation (i.e., ‘Buffer’). Using the weather station point as the start for the buffer, then define the width of the buffer. For this map the value selected was 5km. To limit overlap of stations. Repeat this process for each of the Toronto weather station points.

Buffer options screen

Once Toronto weather stations are buffered, proceed to the ‘properties’ pane for the new point and select the variable ‘precipitation’, then specify layer symbology under ‘graduated symbols’. Within ‘graduated symbols’, set the symbol ranges and select the colour ramp. In this example, buffers were each made to be 50% opacity.

To create the beach points, proceed to the layer’s ‘properties’ pane, and select ‘symbology’. Selecting ‘E.Coli’ as the display variable, specify layer symbology using ‘graduated symbols’, and then select a corresponding symbol. For the purpose of this map, four categories of fecal coliform concentration were used: 0-50 (Green), 51-100 (Yellow), 101- 200 (Red), and 201-999 (Starburst). Graduating visualization breaks in four categories was chosen to be able to see the changes, they do not directly correspond to beach advisory levels.

Map with buffered weather station points and beach data points

Once weather station points and buffers, and beach points were set up, open up the ‘Time Manager’ plugin from the workbench in QGIS. Add in each vector layer, ensuring that the date format of the points and buffers is in ‘yyyy-mm-dd’, otherwise it will not work. For the purpose of this visualization, days were selected as the ‘time format’

TimeManager control panel

Finally, select export video. When exporting using TimeManager, it does not export as a compiled video; instead, TimeManager creates a different image for each day. While this isn’t ideal for a very large data set, it does automate map generation in a consistent layout form. Moreover, it is not possible to add map elements in the QGIS 3.10 ‘TimeManager’ plugin, therefore this must all be done as post-QGIS processing. In this case, Microsoft PowerPoint (v.2019) was selected, with the additional map elements (scale bar, legend, title, and north arrow) added at this point. A video was then compiled in Microsoft PowerPoint (v.2019) and uploaded to YouTube.

Results

Below is the final result, published as a video on YouTube.

Urban Development of San Francisco

By Hannah Burdett

SA8905 Geovisualization Project, Ryerson University

The Development of San Francisco

San Francisco is located in the center of Northern California. It started as a base for the gold rush of 1849, the city quickly became one of the most populated cities in the United States. Shortly thereafter, San Francisco was devastated by the 1906 earthquake. Development peaked in the 1900’s as San Francisco rebuilt areas demolished by the earthquake and fires to compensate the growing population. During the 1930’s the San Francisco-Oakland Bay Bridge and the Golden Gate Bridge were opened. Additionally, during World War II, San Francisco was a major mainland supply point and port of embarkation for the war in the Pacific. Both factors led to another peak in construction. After World War II, many American military personnel who had fallen in love with the city while leaving for or returning from the Pacific settled in the city. This led to promoting the development of the Sunset District, Visitacion Valley, and the total build-out of San Francisco. Starting in the latter half of the 1960’s, San Francisco became most recognized for the hippie movement. Currently, San Francisco has become known for finance and technology industries. There is a high demand for housing, driven by its close proximity to Silicon Valley, and a low supply of available housing has led to the city being one of America’s most expensive places to live.

Data

The data used for the time series animation was imported from data.gov. Data.gov is a repository for the US Governments open source data. The imported data included a Land use Shapefile for San Francisco. The shapefile included information such as land use, shape area, street address, street number, etc. The land use shapefile also included the year the building was built. The building years range from 1848 to 2016 displaying 153 years of urbanization. The buildings were represented as polygons throughout San Francisco. Additionally, a grey scale base map from ArcGIS Pro was displayed to create a more cohesive map design.

 

 

Time Series Animation

To develop the reconstruction of San Francisco throughout the years, both QGIS and ArcGIS Pro were utilized. Both platforms were used so to provide a comparison between time series animation tools from an open source application and a non-open source application.

QGIS is an open source geographic information systems application that provides data visualization, editing, and analysis through functions and plugins. To create the time series animation the Time Manager plugin was utilized. The Time Manager plugin animates vector features based on a time attribute. For this study the time attribute was the years built.

ArcGIS Pro is the latest professional desktop GIS from Esri. ArcGIS Pro enables users to view, explore, analyze, edit and share maps and data. Unlike QGIS, no additional plugins are required to create the animated time series.

QGIS Methodology

To generate the time series in QGIS, the land use shapefile was downloaded and opened in QGIS. The attribute table from the land use shapefile was then exported and opened in Excel so that the yrbuilt column could be reformatted to meet QGIS Time Manager requirements. The yrbuilt column had the data presented as YYYY format for building dates. QGIS Time Manager requires timestamps to be in YYYY-MM-DD. To correct the format, -01-01 was added to the end of each building year. The modified values were then saved into a new column called yrbuilt1. The Excel sheet was then imported into QGIS and joined to the land use shapefile.

In QGIS, each of the buildings was presented as polygons. The shapefile symbology was changed from single symbology to quantified symbology. In other words, the symbology for each of the polygons was broken down to seven classes defined by years. Each class was then distinguished by color, so that one may differentiate the oldest building from the newest buildings. Furthermore, a grey scale basemap was added to create a more cohesive map.

Furthermore, in the Time Manager settings, “Add Layer” was selected. The land use shapefile was chosen as the Layer of interest. The start time was set to the yrbuilt1 attribute, whereas the end time was set to “No end time – accumulate features”. This allows newer buildings to be added without older buildings being removed from the map. For the animation, each time frame will be shown for 100 milliseconds. The Time Manager plugin was then turned on so that the time series may run.

 

In order to export the time series animation, Time Manager offers an “Export Video” option. However, this exports the animation as an image series, not as an actual video. To correct this, the image series was uploaded to Mapbox where additional Mapbox styles were used to render the map. It was then exported as a Gif from Mapbox.

ArcGIS Pro Methodology

In ArcGIS Pro, the land use shapefile was imported. The symbology for each of the polygons was then broken down to seven classes defined by years. The same colours utilized in QGIS were applied to the classes in ArcGIS Pro to differentiate between the building years. Within the layer’s properties, the Layers Time was selected as “each feature has a single time field”. Furthermore, the start and end times were set to the newest and oldest building years. The number of steps were assigned a value of sixteen. In View, the animation was added, and the Time Slider Steps were imported. The time frames were set to match the QGIS animation so that both time series animations would run at the same speed. The time series animation was then exported as a Gif.

Final Animated Map

Finally, to create a cohesive animated map the exported Gif’s were complied together in PowerPoint. Additional map features, such as a legend, were designed within PowerPoint. A bar graph was added along the bottom of the map to show years of peak building construction. The final time series map was then exported as a .mp4 and upload to YouTube.

Displaying Brooklyn’s Urban Layers by Mapping Over 200 Years of Buildings

Renad Kerdasi
Geovis Course Assignment
SA8905, Fall 2015 (Rinner)

Growth in Brooklyn
Located at the far western end of Long Island, Brooklyn is the most populous of New York City’s five boroughs. The borough began to expand between the 1830s and 1860s in downtown Brooklyn. The borough continued to expand outwards as a result of a massive European immigration, the completion of the Brooklyn Bridge connecting to Manhattan, and the expansion of industry. By mid 1900s, most of Brooklyn was already built up as population increased rapidly.

Data
The data in the time series map are from PLUTO, which is a NYC open data site created by NYC Department of City Planning and released in 2015. The data contain information about each building located in the boroughs, including the year the construction of the building was completed (in numeric 4 digits format) and the building footprints. The building years range from 1800 to 2015, there are some missing dates in the dataset as well as some inaccuracy in the recorded dates. The data are available in Shapefile and Windows Comma Separated format, found on NYC Planning website: http://www.nyc.gov/html/dcp/html/bytes/dwn_pluto_mappluto.shtml

The Making of the Time Series
To present the structural episodes of Brooklyn’s built environment, QGIS 2.10 was utilized with the Time Manager plugin. QGIS is an open source GIS application that provides data visualization, editing, and analysis through functions and plugins (https://www.qgis.org/en/site/about/). The Time Manager plugin animates vector features based on a time attribute (https://plugins.qgis.org/plugins/timemanager/). This tool was effective in presenting a time series of Brooklyn’s building construction dates.

To create the time series, the PLUTO SHP was downloaded and prepared by removing any unnecessary fields. The columns of interest are: FID, Shape, and YearBuilt. Because we are interested in the time column, the formatting must fit with QGIS Time Manager. QGIS Time Manager requires timestamps to be in YYYY-MM-DD format whereas the building dates in the PLUTO SHP are in a four-digit format. Therefore, the date in the dataset must be modified to fit the Time Manager format before it can be brought into QGIS.

Table 1_BrooklynData

In QGIS, Time Manager plugin must be installed first. The SHP can then be added into QGIS as well as other Shapefiles needed: roads, highways, state boundaries, etc. Note: to use Time Manager, the data must be in SHP format.

Layer_BrooklynData

Once the data are added, the polygons (i.e. buildings) are styled based on age. This will be effective in distinguishing the oldest buildings from the newest. In QGIS, there are a large number of options available to apply different types of symbology to the data. The layer is styled based on the attribute Year Built, since the time series will show urban layers using building dates. Also, Graduated is chosen because features in the layer should not be styled the same way. The other data file, such as roads, highways, and state boundaries, are styled as well.

Once all the data are added and styled, it can be oriented and applied to the Time Manager plugin. To truly see the urban layers, the map is zoomed on the upper portion of Brooklyn. In Time Manager settings, the layer with building dates is added and the Start Time is the Year Built field, which includes the timestamp data. To get features to be configured permanently on the map, in the End Time option “No End Time” is selected. For animation options, each time frame will be shown for 100 milliseconds, and timestamp (i.e. built year) will be displayed on the map.

Layer_BrooklynData

In the Time Manager dock, the time frame is changed to years since the animation will be showing the year the construction of the building was completed. The size of the time frame will be 5 years. With these settings, each frame will display 5 years of data every 100 millisecond. Playing the video will display the animation inside QGIS, and one can see the time scrolling from 1800-2015 in the dock.

Dock_BrooklynData

Time Manager also enables you to export the animation to an image series using the “Export Video” button. Actual video export is not implemented in Time Manager. To play the animation outside of QGIS, various software applications can be used on the resulting image series to create a video file.

In addition, QGIS only allows users to insert a legend and title in the Composer Manager window. Currently, it is not possible to get the legend rendered in the main map window. One approach to generate a video with a legend is to create a dummy legend and add the image containing the legend into the PNGs that Time Manager produces. A dummy legend and a title for Brooklyn’s urban layers was created outside of QGIS, and added to each PNG.

Finally, to create a time-lapse and compile the images together, Microsoft Movie Maker was utilized. Other software applications can be used, including mancoder and avidemux.

Results

Link: https://youtu.be/52TnYAVxN3s

Working with WMS-T layers and Time Manager in QGIS

By: Lauren Blumberger
Geovis Course Assignment, SA8905, Fall 2015 (Dr. Rinner)

Are you ever troubled by the amount of large files that you need to download and store on your computer?  If so, a Web Map Service (WMS) may be useful as it delivers georeferenced map images over the internet that are generated by a map server. All you need is an internet connection to be able to access maps from a remote server and load them into your GIS. A request is made, and the response is one or more map images (returned as PNG, JPEG etc.). While WMS services are commonly used for retrieving base maps, they also provide a quick and easy way to work with data that has already been symbolized and styled by the author.  Many WMS servers also have time support, WMS-T, which enables users to define a temporal subset for the rendering of layers that are properly configured with a time dimension.

In this post, I explore how to work with WMS-T retrieved maps in QGIS and how to animate them using Anita Graser’s Time Manager plugin (http://anitagraser.com/projects/time-manager/ and https://plugins.qgis.org/plugins/timemanager/). Anita Graser’s Blog includes a guest post by Karolina Alexiou on using Time Manager for WMS-T layers. I will describe the process, as well as discuss the limitations.

The first step is to find the service layers that you want to work with. This is not always the easiest task as you must use the GetCapabilities URL to access the maps, which in some cases doesn’t reveal the image until the connection is made from QGIS to the server. Thus, there might be some trial and error in terms of retrieving the maps you are looking for.

The map I use for this demonstration is made available by the Fire Information Resource Management System (FIRMS), who offer fire-based map images. The WMS-T allows access to FIRMS Active Fire Hotspots for a specific day. The GetCapabilities URL is provided on the website:

https://firms.modaps.eosdis.nasa.gov/wms-t/?SERVICE=WMS&VERSION=1.1.1&REQUEST=GETCAPABILITIES

Navigate to Add WMS/WMTS Layer in QGIS and click on new. Add the above URL and name the connection FIRMS. Make sure to enable the Ignore GetMap/GetTile URI reported in capabilities, as this will redirect you to the WMS instead of the WMS-T. When you click connect, multiple layers will appear. Load the MODIS_Hotspots layer to the canvas.*2

Now that you have connected to the server you can overlay this image with other layers accessed through servers or available locally. Download any country boundary shapefile and load it to the canvas. Now you can change the style of this new layer and adjust the transparency to make sure that you can see the MODIS image underneath.

new

The next step involves the TimeManager plugin, which provides some support for stepping through this spatial temporal data.** Once you have installed the plugin, click on settings and add raster layer. The WMS-T standard uses many different time formats and the plugin requires you to know this format for inputting the start and end times. Therefore, you must examine the XML document with the web service description and find the section that defines the format of the time dimension. For this example, add the MODIS layer and input start time as 2005-11-16 and end time as 2015-11-16. Set the time step to months and press play. Now you will see that TimeManager renders each month interval by querying the FIRMS WMS-T.window

Since WMS-T is a live service that will automatically update on the map, there are significant delays when working with these layers. Querying the web service and waiting for response takes time, and thus, the interactive mode using TimeManager does not result in a visually pleasing animation. TimeManager includes an option to export all the frames, which you can then move into another software to create a smooth animation.

The resulting animation shows the MODIS Active Fire Hotspots globally for the past ten years. The visualization works best if you maximize the video to full screen view.

Overall, working with WMS-T servers can be tricky. As there are many different time formats that can be used, it is quite a complex task to support them all in a GIS. Also, while you want to be connected to the server to be able to access live, current data, this connection slows down the rendering process and creates choppy animations. This being said, exporting the frames to create an animation provides a nice result and a fast way to visualize current data. As FIRMS constantly provides access to active fire data with the MODIS satellite, through reconnecting to the WMS-T and adjusting the time I can update the animation to 3 hours ago.

*The MODIS satellite detects both flaming and smoldering fires ~1000m2

**Note that FIRMS WMS-T only supports one date per request and date ranges are not allowed. To make Time Manager send the correct request, the source code file wmstlayer.py must be adjusted. If Time Manager is installed, the file can be found in the user folder on your computer. Replace lines 59 to 63 with:

self.layer.dataProvider().setDataSourceUri(self.IGNORE_PREFIX + \
self.originalUri + self.addUrlMark() + “TIME={}” \
.format(
time_util.datetime_to_str(startTime, self.timeFormat)))

This ensures that Time Manager only sends a point in time. A time range will return a black image for this WMS-T.

Special thanks to Dr. Eric Vaz and Anita Graser for their help.