Fall2019 – Page 2 – Master of Spatial Analysis

Batman’s Trending: A Spatial Approach to Normalizing Google Trends and it’s Inferences on Geographic Data

With the growing fascination of map forums such as Map Porn on Reddit or Terrible Maps on Instagram, it’s evident that mapping has gone beyond the needs of the typical geographer. Open data is used every day and sometimes use to produce “cool” map that nevertheless, relays information. One of my obsessions is superheroes from the DC and Marvel universe. Comic books, movies and conventions have steadily increased and with Disney overseeing all Marvel movies this trend will not be dwindling anytime soon. Google Trends is one of the most useful, real time and accessible data resources that is freely available to anyone who has internet access. Google Trends is keyword-related data that uses the volume index as well as the amount of search engines within the geographical area. I wanted to explore this type of data to see its limitations, and its usefulness.

The limitation with Google Trends is the output data provided has already been normalized by Google themselves. Google’s normalization algorithm is not provided, instead, they state:

“While only a sample of Google searches are used in Google Trends, this is sufficient because we handle billions of searches per day. Providing access to the entire data set would be too large to process quickly.”

Instead, results are normalized through the following process:

Data points are divided by the total searches of the geographic location and time to compare relative popularity.* This eliminated locations with the most search volume being ranked highest.
Results are scaled from 0 to 100 range with consideration to the topic’s amount to all searches on every topic generated.
Regions that show the same search interest will not have the same total search volume.

*it should be noted that Google’s normalization method throughout time causes data to slightly changes depending on the day you gather the information despite the time frame you consistently use. To have a more accurate sets of data, it is best to gather all the data within the same week to avoid larger differences

The design of the Geovis project is that of a physical map and the aesthetic of an old fashion comic book pop art piece, that is simple, artistic and comprehensive of the data.

Laser Cut

After speaking with the laser cutting company Hot Pot, three laser cut pieces were produced, one of the United States, one with all the States, and one with Alaska and Hawaii. The maps are on a 1:20,000,000 scale which is then be blown up to 2.5x its size to fit the boards. Each state will be covered with its trending superhero or villain from within the past year.

In order to create the laser cut template, I imported a shapefile of the US states into ArcPro and used Albers Equal Area Conic projection. A dissolve tool was then used on the US shapefile to create an outline of the continental United States. The maps were then exported as an SVG in order to create a vector file the company could use. Since there are many intricate details of the US states there were some errors from the SVG file ArcPro produce. To correct this the US state file was imported into Adobe Illustrator and traced to create the vectors.

Next, the superhero/villain data needed to be gathered. Below is a step by step process on how I normalize Google Tends Data and create a visual physical map that any fan would love to hang in their living room.

Google Trends Superhero Data

17 characters were chosen from both universes to populate 50 states. 20 male characters and 14 female characters that have had a major motion picture or television appearance between 2017 and 2019, which exception to staple characters and upcoming movies in 2020.

DC Superheroes and antiheroes that have made an appearance in 2014 -2019	Marvel Superheroes and antiheroes that have made an appearance in 2014-2019
Batman (Movie 2017)	Iron Man (Movie 2019)
Superman (Movie 2017)	Captain America (Movie 2019)
Joker (Movie 2019)	Hulk (Movie 2019)
Aquaman (Movie 2018)	Spiderman (Movie 2019)
Flash Barry Allen (TV show 2019/Justice League 2018)	Black Panther (Movie 2019)
Shazam (Movie 2018)	Thor (Movie 2019)
Cyborg (Movie 2017)	Venom (Movie 2019)
Green Lantern *staple in the DC universe	Deadpool (Movie 2019)
Green Arrow (TV show 2019)	Groot (Movie 2019)
Dick Grayson (TV show 2019)	Hawkeye (Movie 2019)

WOMAN	WOMAN
Wonder woman (Justice League)	Black Widow (Movie 2019)
Harley Quin (upcoming movie 2020)	Captain Marvel (Movie 2019)
Catwoman *staple in the DC Universe	Scarlet Witch (Movie 2019)
Raven (TV show 2019)	Gamoura (Movie 2019)
Starfire (TV show 2019)	Storm (Movie 2019)
Supergirl (TV Show 2019)	Jean Grey (Movie 2019)
Bat Woman (TV show 2019)	Nebula (Movie 2019)

Once all characters were established each character was imputed into Google Trends to find there rating. A year from September 30, 2019, was used as the time frame and a score 75 and above was recorded on an excel sheet in order to keep track of all the rankings. The Related Topic and Related Queries tables were used to make sure the term was relating to the character and not something else. If the character showed unrelated topics and queries because of the ambiguity of their name, they were kicked out. To keep universes equal in numbers, if one character was kicked out of the list another from the opposite universe needed to be kicked out as well. Venom and Storm were kicked out of Marvel and Shazam and Raver were kicked out of DC.

Once all the characters were recorded the next was to determine how a superhero/villain won a state. Since I wanted all characters to be represented on the map, I first ranked the character scores and then the state’s score.

If a character scored 100% for a state and no other score the character instantly won that state. These included Aquaman, Hawkeye, Green Lanter and Green Arrow.

Utah could not be won by one superhero alone because of the large amount of 100% scores. Instead, Utah would be split into 4 characters who score the most 100% scores.

Next, I looked at superheroes with the lowest number of total scores. If a superhero had a low total number of scores but only one 100% ranking score the superhero would win the state. If there were multiple 100% scores for a state (Alaska) those states needed to be ranked later on.

Once all the characters won over a state the next step was to rank the state scores. Whichever character held the highest score within that state they won that state.

Once all the scores were established the next step was to make the physical map

Making the Map

First thing to do was gather all necessary comic books. After attaining over 30 I had all my characters.

After receiving the laser cut of the map each character was cut out to fit the shape of the state and glued and sealed onto the continental US map. Since Alaska was too big to fit on to the same map as the continental US, it was given its own separate board along with Hawaii.

After all characters were glue down and sealed, fabric reflecting the original pop art colours was glued down to foam board before mounting the maps. This step was done for both the continental map and Alaska/Hawaii map.

Since some east coast states were too small to have characters directly represented on the stated, yellow fabric was used as filler. The east coast states where then outline and used an array of coloured strings and pins to connect pictures of the state characters to the shape of the states then to the geographic location.

Once all the states were now represented by characters from both universes a scale bar was added to both maps as final touches.

The maps turned out amazing and something truly artistic. I could not make this map alone and would like to thank my friends in the program Jeremy, Miranda and Fana for all their help and support, my friend Kevin who helped with all Adobe Illustrator needs, and my cat for being himself.

Toronto Theft: A Neighbourhood Investigation

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2019

By: Julia DiMartella-Orsi

Introduction:

ESRI’s creation of the Story Map changed the way we could visualize data. Not only did it allow for a broader audience to interact and create their own maps due to its easy to use design, it also contained many new amazing functions, templates, and themes. Users can personalize their story by adding in their own images, text, videos, and map layers by creating their own free ArcGIS Online account. Popular templates include Map Series, Tour, Journal, and Cascade.

Get started making your own Story Map here: http://storymaps-classicqa.arcgis.com/en/app-list/

Creating Your Story Map:

Once you have selected the template you want to use the choice is up to you. By clicking the “+” symbol you can choose to include text, media sources such as a videos, a new title page, or immersive content such as a web map.

ESRI also designed Story Maps to link to outside content and various social media sites such as Flickr and Unsplash. ‘Link to Content’ is also extremely useful as it allows users to add photos and videos found on the internet directly to their story map by copying and pasting their link.

To add interactive web maps into your story map users can link map layers from their ArcGIS Online account. Layers can be created in ArcGIS Online, but also in ArcMap where layers are exported as a zip file and imported onto your ArcGIS Online base map. Map layers can also be found online using the ‘add layer from the web’ or ‘search for layers’ options. The layers that appear are based on the type of ArcGIS Online account you have created. Enterprise accounts contain additional layers provided by your organization, however ESRI also has free downloadable layers available for users without an organization.

Users also have the option to make their story maps public by clicking the globe icon, or private for their own personal use by clicking the lock icon. To save your story map select the floppy disk icon. Your saved map will appear under ‘My Content’ in your ArcGIS Online account.

My Story and Creating Web Maps:

Over the last few years, theft in Toronto has been increasing at a rapid rate. According to the Toronto Police Service, Toronto experienced a total of 5430 thefts between 2014-2018. However, these are only those that have been reported and documented by police. In order to analyze the distribution of theft across the city, the Toronto Police created a point dataset that summarized when and where each theft took place. Additional datasets were also created for prominent types of theft such as bicycle and auto theft.

To compare the number and types of theft in each Toronto neighbourhood I decided to create a story map using the Cascade template. This created a scrolling narrative that would allow viewers to observe the data in a clear, unique way. The reason why I chose to use a story map was due to the number of layers I wanted to compare, as well as use the ‘swipe tool’ to easily compare each neighbourhood. Therefore, I created a series of choropleth maps based on the 2014-2018 theft/crime data from the Toronto Police Open Data Portal.

The following steps were used to create each web map used in my Story Map:

Step 1: Download the point data and add the layer into ArcMap.

Step 2: Use the ‘spatial join’ analysis tool and select your neighbourhood boundary file as the target layer and the theft point data as the join feature. Make sure to select ‘join one to one’. This will produce a new layer with a ‘count’ field that counts the number of thefts in each neighbourhood – each neighbourhood is given a count.

Step 3: In order to produce accurate results, you must normalize your data. To do so add a new field into your attribute table (same layer with the count field) titled ‘Area’, and right click to select ‘calculate geometry’. Change the property to ‘area’ and choose the units you wish to use. Click ‘ok’ and the results will populate your new field.

Step 5: Export the layer and save it as a compressed zip folder. Import the data into ArcGIS Online by clicking the “Add” tab.

Step 6: Once you import your layer you are given a variety of styles to choose from. Select the one you like best (ex: choropleth) as well as the field you wish to map – in this case select ‘count’. To normalize ‘count’ select the ‘divided by’ dropdown and choose your ‘Area’ field. Change the colour of your map to your preference by clicking ‘symbols’.

Step 7: Save your layer to and select the tags that relate to your topic. The layer will now appear in ‘My Content’ where it can be added to your Story Map.

Step 8: To compare each layer add both layers you wish to compare to your story map by using the “+” symbol. Once you have done so, choose the transition type (ex: horizontal swipe) you want to use by clicking on the arrow below. The transition will take place as the user scrolls through your story map.

My Story Map titled “Toronto Theft: A Neighbourhood Investigation” can be viewed here:

https://arcg.is/uiemr

Putting BlogTO on the map (literally) – Tutorial

Kyle Larsen
SA8905 – Cartography and Geovisualization
Fall 2019

Instagram is a wealth of information, for better or worse, if you’ve posted to Instagram before and your profile is public, maybe even if it’s not, then your information is out there just waiting for someone, someone maybe like me, to scrape your information and put it onto a map. You have officially been warned.

But I’m not here to preach privacy or procure your preciously posted personal pics. I’m here to scrape pictures from Instagram, take their coordinates, and put them onto a map into a grid layout over Toronto. My target for this example is a quite public entity that thrives off exposure, the notorious BlogTo. Maybe only notorious if you live in Toronto, BlogTo is a Toronto-based blog about the goings on in the 6ix as well as Toronto life and culture, they also have an Instagram that is almost too perfect for this project – but more on that later. Before anything is underway a huge thank-you-very-much to John Naujoks and his Instagram scraping project that created some of the framework for this project (go read his project here, and you can find all of my code here)

When scraping social media sometimes you can use an API to directly access the back end of a website, Twitter has an easily accessible API that is easy to use. Instagram’s API sits securely behind the brick wall that is Facebook, aka it’s hard to get access to. While it would be easier to scrape Twitter, we aren’t here because this is easy, maybe it seems a little rebellious, but Instagram doesn’t want us scraping their data… so we’re going to scrape their data.

This will have to be done entirely through the front end, aka the same way that a normal person would access Instagram, but we’re going to do it with python and some fancy HTML stuff. To start you should have python downloaded (3.8 was used for this but any iteration of python 3 should give you access to the appropriate libraries) as well as some form of GIS software for some of the mapping and geo-processing. Alteryx would be a bonus but is not necessary.

We’re going to use a few python libraries for this:

urllib – for accessing and working with URLs and HTML
selenium – for scraping the web (make sure you have a browser driver installed, such as chromedriver)
pandas – for writing to some files

If you’ve never done scraping before, it is essentially writing code that opens a browser, does some stuff, takes some notes, and returns whatever notes you’ve asked it to take. But unlike a person, you can’t tell python to go recognize specific text or features, which is where the python libraries and HTML stuff comes in. The below code (thanks John) takes a specific Instagram user and return as many post URLs as you want and adds them to a list, for your scraping pleasure. If you enable the browser head you can actually watch as python scrolls through the Instagram page, silently kicking ass and taking URLs. It’s important to use the time.sleep(x) function because otherwise Instagram might know what’s up and they can block your IP.

But what do I do with a list of URLs? Well this is where you get into the scrappy parts of this project, the closest to criminal you can get without actually downloading a car. The essentials for this project are the image and the location, but this where we need to get really crafty. Instagram is actually trying to hide the location information from you, at least if you’re scraping it. Nowhere in a post are coordinates saved. Look at the below image, you may know where the Distillery District is, but python can’t just give you X and Y because it’s “south of Front and at that street where I once lost my wallet.”

If you click on the location name you might get a little more information but… alas, Instagram keeps the coordinates locked in as a .png, yielding us no information.

BUT! If you can scrape one website, why not another? If you can use Google Maps to get directions to “that sushi restaurant that isn’t the sketchy one near Bill’s place” then you might as well use it to get coordinates, and Google actually makes it pretty easy – those suckers.
(https://www.google.com/maps/place/Distillery+District,+Toronto,+ON/@43.6503055,-79.35958,16.75z/data=!4m5!3m4!1s0x89d4cb3dc701c609:0xc3e729dcdb566a16!8m2!3d43.6503055!4d-79.35958 )
I spy with my little eye, some X and Y coordinates, the first set after the ‘@’ would usually be the lat/long of your IP address, which I’ve obviously hidden because privacy is important, that’s the takeaway from this project right? The second lat/long that you can gleam at the end of the URL is the location of the place that you just googled. Now all that’s left is to put all of this information together and create the script below. Earlier I said that it’s difficult to tell python what to look for, and what you need is the xpath, which you can copy from the html (right-click an element and then right-click that html and then you can get the xpath for that specific element. For this project we’re going to need the xpath for both the image and the location. The steps are essentially as follows:

go to Instagram post
download the image
copy the location name
google the location
scrape the URL for the coordinates

There are some setbacks to this, not all posts are going to have a location, and not all pictures are pictures – some are videos. In order for a picture to qualify for full scraping it has to have a location and not be a video, and the bonus criteria – it must be in Toronto. Way back I said that BlogTO is great for this project, that’s because they love to geotag their posts (even if it is mostly “Toronto, Ontario”) and they love to post about Toronto, go figure. With these scripts you’ve built up a library of commands for scraping whatever Instagram account your heart desires (as long as it isn’t private – but if you want to write script to log in to your own account then I guess you could scrape a private account that has accepted your follow request, you monster, how dare you)

With the pics downloaded and the latitudes longed it is now time to construct the map. Unfortunately this is the most manual process, but there’s always the arcpy library if you want to try and automate this process. I’ll outline my steps for creating the map, but feel free to go about it your own way.

Create a grid of 2km squares over Toronto (I used the grid tool in Alteryx)
Intersect all your pic-points with the grid and take the most recently posted pic as the most dominant for that grid square
Mark each square with the image that is dominant in that square (I named my downloaded images as their URLs)
Clip all dominant images to 1×1 size (I used google photos)
Take a deep breath, maybe a sip of water
Manually drag each dominant image into its square and pray that your processor can handle it, save your work frequently.

This last part was definitely the most in need of a more automated process, but after your hard work you may end up with a result that looks like the map below, enjoy!

Greta Thunberg’s Journey, Mapped

Sahil Parikh
Geovis Project Assignment @RyersonGeo, SA8905, Fall 2019

Introduction

Climate change continues to grow in severity, and the younger generations are noticeably taking a stand. Among them is Greta Thunberg, a 16-year-old environmental activist from Sweden, who first made headlines in 2018 when she played a pivotal role in organizing the School Strike for Climate movement. Her decision to refrain from attending school and protest outside Swedish parliament for stronger climate change action inspired millions to follow suit around the world.

But Greta wasn’t done there. She received an invitation to speak at the 2019 UN Climate Action Summit in New York City, and decided to attend. Greta opted not to fly in order to promote her pro-environmental stance on reducing carbon emissions. The Malizia II sailing yacht crew offered to bring both her and her father across the Atlantic Ocean. Greta agreed, and the boat sailed from Plymouth, England, to New York City on a carbon-neutral trip that once again rocked headlines worldwide.

Once she arrived in New York City, Greta proceeded with her plan to speak at the Summit, but also made small trips all around North America before getting ready for the 2019 United Nations Climate Change Conference (COP25) in Santiago, Chile. Due to national protests, Chile backed out from hosting COP25, and the conference has now been moved to Madrid, Spain. Having completed her tour of North America, Greta set sail again, this time from Hampton, Virginia, to return to Europe and eventually attend COP25.

The focus of this geovisualization project is on Greta Thunberg’s voyage from Europe to North America, from August 2019 to the present. Her voyage is visually communicated through an interactive web map.

Technology

The interactive web map was built using two main technology platforms: Mapbox and OpenStreetMap. Mapbox is an open source mapping platform that allows the user to build custom web maps and applications. It has a very generous free-to-use pricing tier based on map loads that most users will be able to get by with. Mapbox also provides the GL JS library, which is a JavaScript library that can render interactive maps from Mapbox and vector basemap tiles. Its API can be accessed using a token and style URL. Two specific API method examples were used in this web map: displaying a pop-up on click and flying to a location.

Mapbox’s API documentation can be found here for more information.

OpenStreetMap (OSM) is an open source, collaborative global mapping project that provides vector basemap tiles. It typically has to be accessed using a style URL, but Mapbox automatically references an OSM vector tile, so no additional code was required.

The web map was launched with the aid of two additional platforms: Sublime Text and GitHub. Sublime Text is a text editor that allows the user to edit and save HTML, JavaScript, and CSS. It was used to write and save the web map as an HTML file. GitHub is a popular software development platform that allows for code storage using repositories, version control, and minimal web hosting. It was used to externally host the web map through the GitHub Pages feature.

Note: a general knowledge of HTML, JavaScript, and CSS is recommended if attempting to replicate this project.

Data

Greta Thunberg’s voyage was widely documented by the media, but unfortunately not in one single place since multiple publications were reporting on her activities. Therefore, the data for this map had to be manually obtained. A general scan of the major news publications was conducted to determine her location on a given date and what she was doing in that location. Then, that group of locations was inputted into LatLong.net to obtain their individual geographic coordinates in decimal degrees.

Additionally, brief summaries of Greta’s activities were written up with the intention of displaying them in a pop-up textbox at each location.

How to Build the Web Map

The first step in building this web map is to ensure that you have access to all required technologies. Create accounts for Mapbox and GitHub, and download Sublime Text or some other text editor that will allow you to edit and save HTML/JavaScript/CSS.

Next, you need to obtain a Mapbox access token. This token will allow you to use the GL JS methods. Create a new HTML file in your text editor (use w3schools’s HTML tutorial on how to do this if you haven’t already). and follow Mapbox’s instructions with regard to placing these code snippets into your file:

Next, copy the code from Mapbox’s pop-up example into your HTML file. You’ll see multiple pop-up entries under map.addLayer. Replace the coordinates within the [ ] brackets with the ones you’ve obtained from LatLong.net and the descriptions of Greta’s activities, ideally in chronological order as you move down the file. Here’s an example:

The pop-up code does have built-in map markers, but I recommend adding custom markers to the map since they’re more versatile. Use Mapbox’s Custom Markers tutorial to do this.

The next step is to enable the functionality of flying to a location with the click of a button. Mapbox does have an example, but I used a modified example that allows the user to loop through a series of coordinates. Once again, copy the code into your file. Once again, replace the coordinates with the ones you’ve obtained, this time within the arrayOfCoordinates:

Create a title textbox by creating a div element, adding text using HTML formatting, and then adding the two buttons that will allow the user to fly back and forth between locations:

The basic functionality is now built into your web map! What’s left is to add styling to the map, the title textbox, the location pop-up textboxes, and the location map markers. Styling can be changed using CSS, and w3schools’s tutorial is a good place to start. Here’s an example of my CSS for the fly buttons:

Play around with the styling until you’re satisfied with the map’s visual appeal.

Open the HTML file in a web browser to test its functionality. Click both buttons to fly back and forth between the locations that Greta visited. Click on the map markers to display pop-ups at each location with a description of when Greta was there and what she did.

The final step is to make the HTML file publicly accessible. Create a new Public repository in GitHub and upload your file to it by dragging it in. For example, I named my repository “geovis”. Make sure your file is called “index.html”:

Next, click on “Settings” at the top right, scroll down to the bottom section called “GitHub Pages”. This feature will allow you to host your file as a web map page from your repository.

Choose the “master branch” as the Source, and the page should refresh, now displaying a URL at the top where your page is now published:

You’re done! Click on the URL and it should open in your browser, displaying the web map you just created?

Hopefully this post was informative and helpful. If you have any questions about the map and the process I used, don’t hesitate to email me at s5parikh@ryerson.ca.

Greta Thunberg’s Voyage, Mapped: https://s5parikh.github.io/geovis/

Missing Migrants: The Mediterranean Sea

By: Austen Chiu

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2019

Background

The dangerous journey of migrants seeking a better life has existed as long as countries have experienced political unrest. Advancements in technology have brought greater visibility to migrant groups than ever before. However, those who failed to make the journey often go unseen. Due to the undocumented nature of migrant paths, accurate numbers of survivors and deaths is difficult to track.

The data used in this project were obtained from the Missing Migrants Project. A dashboard was created in Tableau desktop to visualize the locations of missing migrant reports across the Mediterranean Sea, and to improve awareness of the scale at which the migrant crisis is occurring.

Creating an Animated Time Series Map

Following the prompts in Tableau, import your data. The data imported from an excel file should appear like this.

Make sure the data contains a date column, and spatial coordinates. Tableau can read spatial coordinates such as latitude and longitude, or northing and easting, to create maps. You can designate a column to be read as a date, or assign its geographic role as a latitude or longitude, to draw a map.

The icon above the column reveals options for which you want to format the data.

Geographic roles can be assigned to your data, allowing Tableau to read them as a map.

Creating a new map can be done by clicking the new tab buttons at the bottom of your window.

This is a blank graph. You can create graphs by dragging data into the “columns” and “rows” fields.

Tableau will automatically generate a map if data assigned with geographic roles are used to populate the “Columns” and “Rows” fields. If the “Pages” field in the top right corner is populated with the date data, a time slider module will appear below the “Marks” module. The “Pages” field facilitates Tableau’s animation capabilities.

The “Filters” field has applied a filter to the data, so only cases that occur in the Mediterranean region are visualized in the map.

The “Pages” field in the top left has been populated by the date data and a time slider has appeared in the bottom left.

The time slider allows you to select a specific date to view. The right arrow below the slider starts the animation, and Tableau will run through each snapshot of time, much like a slideshow.

Tableau can produce many types of data visualizations to accompany the animated map. A histogram, live counter, and packed bubbles visuals accompany the map on my dashboard.

The final product of the dashboard I created has been shared to Tableau Online. However, Tableau online does not support the animation features. A gif of the animated dashboard in Tableau Desktop has been shared through google drive, and can be viewed here.

A Century of Airplane Crashes

Laine Gambeta
Geovisualization Project, @RyersonGeo, Fall 2019

Tableau is an exceptionally useful tool in visualizing data effectively. It allows many variations of charts in which the software suggests the best type based on data content. The following project uses a data-set obtained from the National Transportation and Safety Board identifying locations and details of plane crashes between 1908-2009. The following screenshot is a final product and a run through of how it was made.

Map Feature:

To create the map identifying accident location, a longitude and latitude is required. Once inputted into the Columns and Rows, Tableau automatically recognizes the location data and creates a map.

The Pages function is populated with the date of occurrence and filtered by month in order to create a time animation based on a monthly scale. When the Pages function is populated with a date the software automatically recognizes a time series animation and creates a time slide.

The size of the map icon indicates the total number of fatalities at a specific location and time. To create this effect, the fatalities measure is inputted into the Size function. This same measure is inserted into the label function to show the total number of occurrences with each icon appearance.

When you scroll over the icons on the map the details of each occurrence appear. To create this tool, the measures you want to appear are inserted into the Details function. In this function, Date, Sum Aboard, Sum Fatalities, Sum Survivors, and Summary of accident appears when you scroll over the icon on the map.

Vertical Bar Chart Feature:

To create the vertical bar chart you must insert the date on the Y axis (columns), and the X axis (rows) with people aboard and fatalities.

Next, we must create a calculation to pull the number of survivors by subtracting the two measures. To do so, right click on a column title cell and click create calculated field. Within this calculation you select the two columns

you want to subtract and it will populate the fields. We will use this to identify the number of survivors.

The next step is creating a dual- axis to show both values on the same chart. Right click one of the measures in the rows field and click dual-axis. This will combine the measures onto the same chart and overlap each other.

Following this we need to filter the data to move along the animation by month. It tallies the monthly numbers and adds it to the chart. In order to combine the monthly tallies to show on an annual bar chart, the following filters are used. First filter by year which tallies the monthly counts into a single column on the bar chart. The Page’s filter identifies the time period increments used in the time slider animation, this value must be consistent across all charts in order to sync. In this case, we are looking at statistics on a monthly basis.

To split the colours between green and red to identify survivors and fatalities, the Measure Names (which is created automatically by Tableau) is inserted into the colour function. This will identify each variable as a different colour.

When you bring your mouse over top the bar chart it selects and identifies the statistics related to the specific year. To create this feature, the measures must be added to the tooltip function and formatted as you please.

Horizontal Bar Chart Feature:

The second bar chart is similar to the previous one. The sum of fatalities is put in Columns and the Date is put in Rows to switch the axis to have the date on the Y axis. The Pages function uses the same time frame as other charts calculating monthly and adding the total to the bar chart as the time progresses.

Total Count Features:

To create the chart you must insert the date on the Y axis (columns), and the X axis (rows) with people aboard and fatalities.

Adding in running counts is a very simple calculation feature and is built into Tableau. You build the table by putting the measure into the text function, this enable’s the value to show as text and not a chart. You will notice below that the Pages function must be populated with a date measure on a monthly basis to be consistent with the other charts.

In order to create the running total values, a calculation must be added to the measure. Clicking the SUM measure opens the options and allows us to select Edit Table Calculation. This opens a menu where you can select Running Total, Sum on a monthly basis. We apply this to 3 separate counters to total occurrences, fatalities, and survivors.

Pie Chart Feature:

Creating a pie chart requires the following measures to be used. Under the marks drop down you must select pie chart. This automatically creates a function for angular measure values. The fatality and survivor measures are used and filtered monthly. The Measure Values which is automatically created by Tableau identifies the values of these measures and is inputted into the Angle function to calculate the pie chart. Again, the Measures Names are inputted into the colour function to separate the values by fatalities and survivors. The Pages function is populated with date of occurrence by month to sync with the other charts.

Lastly, a dashboard is created which allows the placement of the features across a single page. They can be arranged to be aesthetically pleasing and informative. Formatting can be done in the dashboard page to manipulate the colors and fonts.

Limitations:

Tableau does not allow you to select your map projection. Tableau Online has a public server to publish dashboards to, however it does not support timeline animation. Therefore, the following link to my project is limited to selecting the date manually to observe the statistics.

https://prod-useast-a.online.tableau.com/t/lainegambeta/views/ACenturyofAirplaneCrashes/Dashboard2?:origin=card_share_link&:embed=n

Ontario Demographics Data Visualization

Introduction

The purpose of this project is to visualize any kind of data on a webmap. Using open source software, such as QGIS, solves one aspect of this problem. The other part of this problem is to answer this question:

How and what data can be visualized? Data can be stored in a variety of formats, and organized differently. The most important aspect of spatial data is the spatial information itself and so we need to figure out a way to display the data using textual descriptions, symbols, colours, etc. at the right location.

Methodology

In this visualization, I am using the census subdivisions (downloaded from Statstics Canada website) as the basic geographical unit, plus the 2016 census profile for the census subdivisions (also downloaded from Statistics Canada website). Once these data were downloaded, the next steps were to inspect the data and organize them in a fashion so that they could be easily visualized by the shapefiles. In order to facilitate this task, we can use any relational database management system, however, my preference was to use SQL Server 2017 express edition. Once the 2016 census profile has been imported into SQL Server, the “SQL Queries” [1] file can be run to organize the data into a relational table that can be exported, or copied directly from the result-set on management studio and pasted, into excel/csv; the sheet/file can now be opened in QGIS and joined to the shapefile of Ontario Census Subdivisions [2] using CSDUID as the common field between the two files.

Using the qgis2web plugin, all data and instructions are manually chosen on a number of tabs. You can choose the layers and groups you want to upload, and then customize the appearance and interactivity of the webmap based on available options. There is the option to use either Leaflet, or OpenLayers styles on QGIS version 3.8. You can Update preview and see what the outcome will look like. You can then Export the map and the plugin will convert all the data and instructions into json format. The most important file – index.html – is created on the directory you have specified.

index.html [1] is the file that can be used to visualize the map on the web browser, however, you need to first download all the files and folders from the source page [1]. This will put all the files on your (client) machine which makes it possible to open index.html on localhost. If the map files are uploaded on a web server, then the map can be viewed by the world wide web.

Webmap

The data being visualized belongs to the population demographics (different age groups). The map of Ontario’s census subdivisions is visualized as a transparent choropleth map of 2016 population density. Other pieces of demographics information are embedded within the pop-up for each of the census subdivisions. If you hover your cursor on each of the census subsivisions, it will be highlighted with a transparent yellow colour so you can see the basemap information on the basemap clearer. If you click on them, the pop-up will appear on the screen, and you can scroll through it.

There are other interactive utilities on the map such as controllers for zooming in and out, a (ruler) widget to make measurements, a (magnifying glass) widget to search the entire globe, a (binocular) widget to search only the layers uploaded on the map, and a (layers) widget to turn layers and basemaps on and off.

Limitations

There are some limitations that I encountered after I created this webmap. The first, and most important limitation, is the projection of the data on the map. The original shapefile was using the EPSG code of 3347 which uses the Canada Lambert Conic projection with NAD 1983 datum. The plugin converted data into the most common web projection format, WGS 1984, which is defined globally by Longitude and Latitude. Although WGS 1984 prevents the hassle of using projected coordinate systems by using one unified geographic projection for the entire globe, nevertheless, it distorts the shapes as we move farther away from the equator.

The second limitation was the fact that my transparent colours were not coded into the index.html file. The opacities are defined as 1. In order to control the level of opacities, the index.html file must be opened in a text editor, the opacities changed to the proper levels, ranging between 0 and 1, and lastly save the edits on the same index.html file.

The next limitation is the size of files that can be uploaded on github [3]. There is a limit of 100 MB on the files that can be uploaded to github repositories, and because the size of the shapefile for entire Canadian census subdivisions is over 100 MB, when converted to json, it could not be uploaded to the repository [1] with all the other files. However, it is possible to add to geojson formatted file (of census subdivisions) to the data directory of the repository on the localhost machine, and manually add its location with a pair of opening and closing script tags on the index.html file on the body tag. In my case, the script was:

<script src=”data/CensusSubdivisions_4.js“></script>

The name of the file should be introduced as the very beginning line of the geojson file as a variable:

var json_CensusSubdivisions_4 = {

And don’t forget that the last line should be a closing curly braces:

}

Now index.html is aware where to find the data for all of the Canadian census subdivisions.

What’s Next?

To conclude with the main goal of this project, which was stated in the introduction, we now have a framework to visualize any data we want. Which data we want to visualize should change our methodology becasuase the scripts can be adapted accordingly. What is more important is the way we want the data to be visualized on the webmap. This tutorial presented the basics of qgis2web plugin. Once the index.html file is generated, other javascript libraries can be added to this file, and depending on your level of comfort with javascript you can expand and go beyond the simple widgets and utilities on this webmap.

  [1]  https://github.com/Mahdy1989/GeoVisualization-Leaflet-Webmap/tree/master  

 [2] There is a simple way to limit the extent of the census subdivisions for the entire Canada, to the Ontario subset only: filter the shapefile by PRUID = '35' which is the code for Ontario.

[3]  https://help.github.com/en/github/managing-large-files/what-is-my-disk-quota

GeoVis: Mapdeck Package in R

Gregory Huang
Geovisualization Project, @RyersonGeo, Fall 2019

Introduction

This project is a demonstration of the abilities of the mapdeck package in R, including its shiny interactive app compatibility.

Mapdeck is an R package created by David Cooley. Essentially, it integrates some of mapbox’s functionality into the R environment. Mapbox is a popular web-based mapping service that is community-driven and provides some great geovisualization functionalities. Strava’s global heat map is one example.

I am interested in looking at flight routes across global hubs and see if there are destination overlaps for these routes. Since the arc layer provided by mapdeck has impressive visualization capabilities of the flight routes, I’ve chosen to use mapdeck to visualize some flight route data around the world.

Example of a map generated by mapdeck: arcs, text, lines, and scatterplots are all available. Perspective changes can be done by pressing down Ctrl and clicking. The base maps are customizable with a massive selection of both mapbox and user-generated maps. This map is one of the results from longest_flights.R, which uses the “decimal” basemap.

The Map has some level of built-in interactivity: Here is an example of using a “tooltip” where if a user hovers over an arc, the arc highlights and shows information about that particular route. Note that mapdeck doesn’t want to draw flight routes across the Pacific – so if accuracy is key, do keep this in mind.

Software Requirements

To replicate this project, you’ll need your own mapbox access token. It is free as long as you have a valid email address. Since the code is written in R, you’ll also need R and R Studio downloaded on your machine to run the code.

Tl;dr…

Here’s the Shiny App

The code I created and the data I used can also be found on my GitHub repository, Geovis. To run them on your personal machine, simply download the folder and follow the instructions on the README document at the bottom of the repository page.

Screenshot of the shiny app: The slide bar will tell the map which flights to show, based on the longitudes of the destinations. All flights depart out of YYZ/KEF/AMS/FRA/DXB.

Details: Code to Generate a Map

The code I’ve written contained 2 major parts, both utilizing flight route data. The first part is done with longest_flights.R, demonstrating the capabilities of the mapdeck package using data I curated for the longest flights in the world. The second part is done with yyz_fra.R and shinyApp.R to demonstrate the shiny app compatibility and show how the package handles larger datasets (hint – very well). The shinyApp uses flight route data from 5 airports: Toronto, Iceland-Keflavik, Amsterdam, Frankfurt, and Dubai, pulled from openflights.org.

For the flight route data for the 5 airports, in particular, the data needed cleaning to make the data frame useable to mapdeck. This involved removing empty rows, selecting only the relevant data, and merging the tables.

Code snippet for cleaning the data. After the for loop completes, the flight route data downloaded from openflights.org becomes available to be used for mapdeck.

Once the data were cleaned, I began using the mapdeck functions to map out the routes. The basic parts of the mapdeck() function are to first declare your key, give it a style, and assign it a pitch if needed. There are many more parameters you can customize, but I just changed the style and pitch. Once the mapdeck map is created, use the “pipe” notion (%>%) to add any sort of layers to your map. For example, add_arc() to add the arcs seen in this post. Of course, there are many parameters that you can set, but the most important are the first three: Where your data come from, and where the origin/destination x-y coordinates are.

An example creating an arc on a map. In addition to the previously mentioned parameters, tooltip generates the little chat boxes when you hover over a layer entry, and layer_id is important when there are multiple layers on the same map.

Additional details on creating all different types of layers, including heatmaps, can be found on the documentation page HERE.

Details: Code to make a “Shiny” app

On top of the regular interactive functionalities of mapdeck, incorporating a mapdeck map into shiny can add more layers of interactivity to the map. In this particular instance, I added a slider bar in Shiny where the user can indicate the longitudes of the destinations they want to see. For example, I can filter to see just the flights going to East Asia by using that slider bar. Additional functions of shiny include using drop-menus to select specific map layers, and checkboxes as well.

The shiny code can roughly be broken down into three parts: ui, server, and shinyApp(ui, server). The ui handles the user interface and receives data from the server, while the server decides what map to produce by the input given by the user in ui. shinyApp(ui,server) combines the two to generate a shiny app.

Mapdeck integrates into the shiny app environment by mapdeckOutput() in ui to specify the map to display, and by renderMapdeck() and mapdeck_update() in server to generate the map (rendeerMapdeck) and appropriate layers to display (mapdeck_update).

Below is the code used to run the shiny app demonstrated in this blog post. Note the ui and server portions of the code bode. To run the shiny app after that, simply run shinyApp(ui,server) to generate the app.

Snippet of the Server creation section. Note that the code listens to what the UI says with reactive() and observeEvent().

This concludes my geovis blog post. If you have any questions, please feel free to email me at gregory.huang@ryerson.ca.

Here is the link to my GitHub repository again: https://github.com/greghuang8/Geovis

Visualizing Tornado Occurrences in the United States of America from 1950 to 2018.

Kellie Manifold

Geovis Project Assignment @RyersonGeo, SA8905, Fall 2019

With a large growing population in the United States of America more people are being affected by tornadoes each year. Tornadoes are a mobile, destructive vortex of violently rotating winds with a funnel-shaped cloud and immense wind speeds. Over the past several decades tracking tornadoes has become more common, and there has been an increase in the amount of tornadoes recorded each year. With this, tornado tracking has become more frequent, as more technology becomes available. This increase in tornado occurrences and the technology to track and record them has resulted in a large dataset of tornado occurrences throughout the United States of America. All recorded tornado occurrences from 1950 – 2018 are kept in a database on NOAA National Weather Service’s website. This data set is very large and difficult to visualize.

To help visualize the distribution of tornado occurrences in the USA, the dataset was used to make an ESRI Dashboard. ESRI Dashboard was chosen because it is a good data visualization tool. Dashboards are very user friendly and allows for the creation of many charts, tables, and indicators to display large amounts of data, on a single web page. The dashboard also allows for user interaction, so that they get a better understanding of the data at hand.

The following steps were used to create the ESRI Dashboard on tornadoes in the United States of America.

First it important to make sure that you have an ArcGIS online account.

The next step is to gather your data. The data for this dashboard was downloaded from the NOAA National Weather Service and the National Centers for Environmental Protection. The file that was downloaded contains the location of all recorded tornadoes in the USA from 1950 – 2018.

Next; Import this data into ArcGIS Pro. The points will be shown through their Latitude and Longitude location. For the purpose of this project only the contiguous USA was looked at, so Puerto Rico, Alaska and Hawaii were removed. This was done through a select by attribute query which can be seen below.

These states were then removed from the map, and attribute table.

The next step to creating the dashboard is to upload the layer to your ArcGIS online account. This is done through publishing your map as a web map. This will then add the layers used onto your ArcGIS online account.

The following steps are through ArcGIS online. Once you are signed into your account, and the published web layer is in your contents, you can now create the dashboard. First, it prompts you to choose a theme colour, light or dark. Dark was chosen for the tornadoes dashboard as it looks more visually appealing.

From here you can add the web layer you published, in this case the tornado layer, and add any other widgets you would like. Such as serial chart, pie chart, indicator, or a list.

Once you decide which widget you would like, it is important that in the Actions tab in the configuration window that all other widgets and filters are selected.

Having these selected will allow for all other map elements and widgets to correspond to changes selected by the user. Make sure this is done for all the widgets that are added. In the case of tornados the widgets used include category selectors for each state, year, or month, an indicator to show the amount of tornados given the selection criteria, a pie chart to show the months tornados occurred, and a serial chart to display the amount of tornados per year.

Once all the widgets are added you can drag them to rearrange the appearance on the dashboard.

Once you have configured all your widgets to get the layout you want, you have successfully created a dashboard.

Thank you for following along!

The final product for this process can be seen here: http://arcg.is/1bDib10

Please take a look and interact with the dashboard.

Create a Quick Web Map with Kepler.gl and Jupyter Notebook

Author: Jeremy Singh

SA8903

GeoVisualization Project Fall 2019

Background: This tutorial uses any csv file with latitude and longitude columns in order to plot points on the web map. Make sure your csv file is saved in the same folder this notebook is saved (makes things easier).

I recommend downloading the Anaconda Distribution which comes with jupyter notebook.

There are 3 main important python libraries that are used in this tutorial

Pandas: Pandas is a python library that is used for data analysis and manipulation.
kepler.gl: This a FREE open-source web-based application that is capable of handling large scale geospatial data to create beautiful visualizations.
GeoPandas: Essentially, geopandas is an extension of Pandas; fully capable of handling and processing of geospatial data.

The first step is to navigate to the folder where you want this notebook to be saved from the main directory when juypter notebook is launched. Then click ‘new’ -> Python 3, a tab will open up with your notebook (See image below).

Next, using the terminal it is important to have these libraries installed to ensure that this tutorial works and everything runs smoothly.

For more information on jupyter notebook see: https://jupyter.org/

Navigate back to the directory and open a terminal prompt via the ‘new’ Tab’.

A new tab will open up, this will function very similarly to the command prompt on windows. Next type “pip install pandas keplergl geopandas” (do not include quotes). This process will help install these libraries.