Create a Quick Web Map with Kepler.gl and Jupyter Notebook

Author: Jeremy Singh

SA8903

GeoVisualization Project Fall 2019

Background: This tutorial uses any csv file with latitude and longitude columns in order to plot points on the web map. Make sure your csv file is saved in the same folder this notebook is saved (makes things easier).

I recommend downloading the Anaconda Distribution which comes with jupyter notebook.

There are 3 main important python libraries that are used in this tutorial

  1. Pandas: Pandas is a python library that is used for data analysis and manipulation.
  2. kepler.gl: This a FREE open-source web-based application that is capable of handling large scale geospatial data to create beautiful visualizations.
  3. GeoPandas: Essentially, geopandas is an extension of Pandas; fully capable of handling and processing of geospatial data.

The first step is to navigate to the folder where you want this notebook to be saved from the main directory when juypter notebook is launched. Then click ‘new’ -> Python 3, a tab will open up with your notebook (See image below).

Next, using the terminal it is important to have these libraries installed to ensure that this tutorial works and everything runs smoothly.

For more information on jupyter notebook see: https://jupyter.org/

Navigate back to the directory and open a terminal prompt via the ‘new’ Tab’.

A new tab will open up, this will function very similarly to the command prompt on windows. Next type “pip install pandas keplergl geopandas” (do not include quotes). This process will help install these libraries.

Below you will find what my data looks like the map before styling

With some options

KeplerGL also allows for 3D visualizations. Here is my final map:

Lastly, if you wish to save off your web map as an HTML file to host somewhere like GitHub or AWS this command will do that for you:

Link to my live web map here:

https://jeremysingh21.github.io/

The code and data I used for this tutorial is located on my GitHub page located here:

https://github.com/jeremysingh21/GeoVizJeremySingh

Visual Story of GHG Emissions in Canada

By Sharon Seilman, Ryerson University
Geovis Project Assignment @RyersonGeo, SA8905, Fall 2018

Background

Topic: 

An evaluation of annual Greenhouse Gas (GHG) Emissions changes in Canada and an in-depth analysis of which provinces/ territories contribute to most of the GHG emissions within National and Regional geographies, as well as by economic sectors.

  • The timeline for this analysis was from 1990-2015
  • Main data sources: Government of Canada Greenhouse Gas Emissions Inventory and Statistics Canada
Why? 

Greenhouse gas emissions are compounds in the atmosphere that absorbs infrared radiation, thus trapping and holding heat in the atmosphere. By increasing the heat in the atmosphere, greenhouse gases are responsible for the greenhouse effect, which ultimately leads to global climate change. GHG emissions are monitored in three elements -its abundance in the atmosphere, how long it stays in the atmosphere and its global warming potential.

Audience: 

Government organizations, Environmental NGOs, Members of the public

Technology

An informative website with the use of Webflow was created, to visually show the story of the annual emissions changes in Canada, understand the spread of it and the expected trajectory. Webflow is a software as a service (SaaS) application that allows designers/users to build receptive websites without significant coding requirements. While the designer is creating the page in the front end, Webflow automatically generates HTML, CSS and JavaScript on the back end. Figure 1 below shows the user interaction interface of Webflow in the editing process. All of the content that is to be used in the website would be created externally, prior to integrating it into the website.

Figure 1: Webflow Editing Interface

The website: 

The website it self was designed in a user friendly manner that enables users to follow the story quite easily. As seen in figure 2, the information it self starts at a high level and gradually narrows down (national level, national trajectory, regional level and economic sector breakdown), thus guiding the audience towards the final findings and discussions. The maps and graphs used in the website were created from raw data with the use of various software that would be further elaborated in the next section.

Figure 2: Website created with the use of Webflow

Check out Canada’s GHG emissions story HERE!

Method

Below are the steps that were undertaken for the creation of this website. Figure 3 shows a break down of these steps, which is further elaborated below.

Figure 3:  Project Process

  1. Understanding the Topic:
    • Prior to beginning the process of creating a website, it is essential to evaluate and understand the topic overall to undertake the best approach to visualizing the data and content.
    • Evaluate the audience that the website would be geared towards and visualize the most suitable process to represent the chosen topic.
    • For this particular topic of understanding GHG emissions in Canada, Webflow was chosen because it allows the audience to interact with the website in a manner that is similar to a story; providing them with the content in a visually appealing and user friendly manner.
  2. Data Collection:
    • For the undertaking of this analysis, the main data source used was the Greenhouse Gas Inventory from the Government of Canada (Environment and Climate Change). The inventory provided raw values that could be mapped and analyzed in various geographies and sectors. Figure 4 shows an example of what the data looks like at a national scale, prior to being extracted. Similarly, data is also provided at a regional scale and by economic sector.

      Figure 4: Raw GHG Values Table from the Inventory
    • The second source for this visualization was the geographic boundaries. The geographic boundaries shapefiles for Canada at both a national scale and regional scale was obtained from Statistics Canada. Additionally, the rivers (lines) shapefile from Statistics Canada too was used to include water bodies in the maps that were created.
      • When downloading the files from Statistics Canada, the ArcGIS (.shp) format was chosen.
  3. Analysis:
    • Prior to undertaking any of the analysis, the data from the inventory report needed to be extracted to excel. For the purpose of this analysis, national, regional and economic sector data were extracted from the report to excel sheets
      • National -from 1990 to 2015, annually,
      • Regional -by province/territory from 1990 to 2015, annually
      • Economic Sector -by sector from 1990 to 2015, annually
    • Graphs:
      • Trend -after extracting the national level data from the inventory, a line graph was created in excel with an added trendline. This graph shows the total emissions in Canada from 1990 to 2015 and the expected trajectory of emissions for the upcoming five years. In this particular graph, it is evident that the emissions show an increasing trajectory. Check out the trend graph here!
      • Economic Sector -similar to the trend graph, the economic sector annual data was extracted from the inventory to excel. With the use of the available data, a stacked bar graph was created from 1990 to 2015. This graph shows the breakdown of emissions by sector in Canada as well as the variation/fluctuations of emissions in the sectors. It helps understand which sectors contribute the most and which years these sectors may have seen a significant increase or decrease. With the use of this graph, further analysis could be undertaken to understand what changes may have occurred in certain years to create such a variation. Check out the economic sector graph here!
    •  Maps:
      • National map -the national map animation was created with the use of ArcMap and an online GIF maker. After the data was extracted to excel, it was saved as a .csv files and uploaded to ArcMap. With the use of ArcMap, sixteen individual maps were made to visualize the varied emissions from 1990 to 2015. The provincial and territorial shapefile was dissolved using the ArcMap dissolve feature (from the Arc Tool box) to obtain a boundary file at a national scale (that was aligned with the regional boundary for the next map). Then, the uploaded table was joined to the boundary file (with the use of the Table join feature). Both the dissolved national boundary shapefile and the river shapefile were used for this process, with the data that was initially exported from the inventory for national emissions. Each map was then exported a .jpeg image and uploaded to the GIF maker, to create the animation that is shown in the website. With the use of this visualization, the viewer can see the variation of emissions throughout the years in Canada. Check out the national animation map here!
      •  Regional map -similar to the national one, the regional map animation was created in same process. However, for the regional emissions, data was only available for three years (1990, 2005 and 2015). The extracted data .csv file was uploaded and table joined to the provinces and territories shapefile (undissolved), to create three choropleth maps. The three maps were them exported as .jpeg images and uploaded to the GIF maker to create the regional animation. By understanding this animation, the viewer can distinctly see which regions in Canada have increase, decreased or remained the same with its emissions. Check out the regional animation map here!
  4. Final output/maps:
    • The graphs and maps that were discussed above were exported as images and GIFs to integrate in the website. By evaluating the varied visualizations, various conclusions and outputs were drawn in order to understand the current status of Canada as a nation, with regards to its GHG emissions. Additional research was done in order to assess the targets and policies that are currently in place about GHG emissions reductions.
  5. Design and Context:
    • Once the final output and maps were created, and the content was drafted, Webflow enables the user to easily upload external content via the upload media tool. The content was then organized with the graphs and maps that show a sequential evaluation of the content.
    • For the purpose of this website, an introductory statement introduces the content discussed and Canada’s place in the realm of Global emissions. Then the emissions are first evaluated at a national scale with the visual animation, then the national trend, regional animation and finally, the economic sector breakdown. Each of the sections have its associated content and description that provides an explanation of what is shown by the visual.
    • The Learn More and Data Source buttons in the website include direct links to Government of Canada website about Canada’s emissions and the GHG inventory itself.
    • The concluding statement provides the viewer with an overall understanding of Canada’s status in GHG emissions from 1990 to 2015.
    • All of the font formatting and organizing of the content was done within the Webflow interface with the end user in mind.
  6. Webflow:
    • The particular format that was chosen in for this website because of story telling element of it. Giving the viewer the option to scrolls through the page and read the contents of it, works similarly as story because this website was created for informative purposes.

Lessons Learned: 

  • While the this website provides informative information, it could be further advanced through the integration of an interactive map, with the use of additional coding. This however would require creating the website outside of the Webflow interface.
  • Also, the analysis could be further advanced with the additional of municipal emissions values and policies (which was not available in the inventory it self)

Overall, the use of Webflow for the creation of this website, provides users with the flexibility to integrate various components and visualizations. The user friendly interface enables uses with minimal coding knowledge to create a website that could be used for various purposes.

Thank you for reading. Hope you enjoyed this post!

Building Visualizations Using D3

By Jessica Whitehead, Geovis Course Assignment, SA8905, Fall 2015 (Rinner)


“D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG and CSS. D3’s emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.” 

d3js.org


OVERVIEW: D3 has a steep learning curve and for someone who has only dabbled in html and css, it did not come easy. Fortunately there is a growing community of D3 and javascript geniuses online, allowing relatively inexperienced programmers, like myself, to piece together chunks of code and create something (surprisingly) visually-pleasing and informative.


THE DATA

I decided to stick with a simple dataset. The World Food Programme (WFP) maintains an online database of international food aid deliveries that can generate reports and tables based on the variables chosen by the user. For this project I looked at food aid deliveries in the year 2012 and produced 3 separate tables that would translate well into data visualizations.

  1. The first table was an overview of international food aid deliveries. It provided the total amount delivered for the the year, as well as breaking it down by food aid types and delivery modes.
  2. The second table summarized all recipient countries, the metric tons received, and for which food aid type. I reduced the dataset by focusing on the top 15 recipient countries.
  3. The third table summarized all donor countries, the metric tons donated, and to which countries it was delivered. I reduced the dataset by focusing on the top 15 donor countries and amalgamating the recipient countries into regions using categories specified on the WFP website.*

*As this project had a geospatial focus, I did not include donors that were not countries, such as the UN or private organizations.


THE CREATIVE PROCESS

So, back to D3. Looking at my data tables, I started exploring the wide array of examples available on the D3.org and GitHub websites. It was important that I found codes that could incorporate my data, either within the code or through .csv file.

Dancing BubblesEventually I found the “dancing bubbles” example and thought this would be an interesting way to show the different types of food aid and delivery modes. I was able to download the code off GitHub, however, it included hundreds of additional files that were unnecessary. As a D3 novice, I did not realize most of the files were irrelevant for a long time.
Source code: http://vallandingham.me/bubble_charts_in_d3.html

To show dashboardthe top 15 recipients I liked the idea of an interactive dashboard. This would allow the user to hover over a country and see a breakdown by food category. For example, North Korea received 372,555 mt of food in 2012 and all deliveries were for emergency purposes.
Source code: http://bl.ocks.org/NPashaP/96447623ef4d342ee09b

mapitTo show the top 15 donors I wanted to create a choropleth map that would produce a donut graph of receiving regions when hovering over each donor country. This proved to be very difficult, so for this project I ended up creating  the map and donut infographic separately.
Source code: http://bl.ocks.org/mbostock/3888852


THE CODING

interactivedatacoverA very useful resource I found was a book called Interactive Data Visualization for the Web written by Scott Murray and is available online for free at Interactive Data Visualization Text

It provides you with all the information required to get started with D3, explaining everything from web fundamentals to D3 setup and creating a web server. Most browsers have issues when interpreting D3 on your local computer, so it is necessary to set up a local web server to view your output.

A very brief look at how D3 works

Essentially you have to download the D3 library and store in a “js” javascript folder in the home directory of your website. Screen Shot 2015-11-18 at 4.34.46 AM Here you will also have folders for css (the style code that makes your visualizations look cool), data (the csv and GeoJson files that give meaning to your visualizations and shape your maps), and index html file (home page of your website).Screen Shot 2015-11-18 at 4.43.24 AM

You start off with a simple index html file and reference the D3 library in the head of your html. Your actual script will go in the body so that it can append items into the Document Object Model (DOM). The DOM is essentially the structure of your html document. D3 uses javascript to then bind your csv and Json files to SVG elements in the DOM. These SVG objects can range from simple rectangles forming bar charts to extremely complicated interactive visualizations.


TOP 3 THINGS ABOUT D3

  1. Web-based – so people can easily access and view the visualizations you spend so long creating.
  2. You are not limited by pre-built charts – endless room for creativity.
  3. D3 programmers seem to be pretty generous when uploading their codes to the web. There is wide variety of open source codes available online to manipulate.

END PRODUCT

If you would like to see the final product please see below
http://www.jwdataviz.ca/sa8905-geovis/