13  Google Earth Engine

13.1 Learning Objectives

• Understand what Google Earth Engine provides and its applications
• Learn how to search for, import, manipulate, and visualize Google Earth Engine Data
• Learn about some real-world applications of Google Earth Engine in the geosciences

13.2 Introduction

Google Earth Engine (GEE) is a geospatial processing platform powered by Google Cloud Platform. It contains over 30 years (and multiple petabytes) of satellite imagery and geospatial datasets that are continually updated and available instantly. Users can process data using Google Cloud Platform and built-in algorithms or by using the Earth Engine API, which is available in Python (and JavaScript) for anyone with an account (Earth Engine is free to use for research, education, and nonprofit use).

GEE is just one of a number of cloud platform solutions developed for climate and geoscience research. Others include Microsoft Planetary Computer, Pangeo, & Amazon Sustainability Data Initiative (ADSI)

Image Source: Earth Engine Data Catalog

So what’s so exciting about platforms like GEE? Ryan Abernathey frames this nicely in his blogpost Closed Platform vs. Open Architectures for Cloud-Native Earth System Analytics…

• as Earth System data have gotten larger, the typical download-data-work-locally workflow is no longer always feasible
• those data are also produced and distributed by lots of different organizations (e.g. NASA, NOAA, Copernicus)
• researchers often need to apply a wide range of analytical methods to those data, ranging from simple stats to machine learning approaches

GEE offers web access (i.e. no need to download data to your computer) to an extensive catalog of analysis-ready geospatial data (from many different organizations) and scalable computing power via their cloud service, making global-scale analyses and visualizations possible for anyone with an account (sign up here!). Explore the public Earth Engine Data Catalog which includes a variety of standard Earth science raster datasets. Browse by dataset tags or by satellite (Landsat, MODIS, Sentinel).

In this lesson, we’ll first get some hands-on practice connecting to and using Google Earth Engine to visualize global precicpation data. We’ll then walk through a demonstration using GEE to visualize and analyze fire dynamics in the Arctic.

13.3 Exercise 1: An introductory lesson on using Google Earth Engine

13.3.1 Part i. Setup

1. Create a Google Earth Engine account (if you haven’t already done so)

• Please refer back to the Preface to find instructions on creating a GEE account.

2. Load libraries

import ee
import geemap
import pandas as pd

3. Authenticate your GEE account

• In order to begin using GEE, you’ll need to connect your environment (scomp) to the authentication credentials associated with your Google account. This will need to be done each time you connect to GEE, (but only be done once per session).

ee.Authenticate() # triggers the authentication process

• This should launch a browser window where you can login with your Google account to the Google Earth Engine Authenticator. Following the prompts will generate a code, which you’ll then need to copy and paste into the VS Code command palette (at the top of the IDE). This will be saved as an authentication token so you won’t need to go through this process again until the next time you start a new session. The browser-based authentication steps will look something like this:

1. Notebook Authenticator: choose an active Google account and Cloud Project (you may have to create one if this is your first time authenticating) and click “Generate Token”
   
2. Choose an account: if prompted, select the same Google account as above
3. Google hasn’t verified this app: You may be temped to click the blue “Back to safety” button, but don’t! Click “Continue”
   
4. Select what Earth Engine Notebook Client can access: click both check boxes, then “Continue”
   
5. Copy your authorization code to your clipboard to paste into the VS Code command palette

4. Lastly, intialize. This verifies that valid credentials have been created and populates the Python client library with methods that the backend server supports.

ee.Initialize() 

If successful, you’re now ready to begin working with Earth Engine data!

13.3.2 Part ii. Explore the ERA5 Daily Aggregates Data

We’ll be using the ERA5 daily aggregates reanalysis dataset, produced by the European Centre for Medium-Range Weather Forecasts (ECMWF), found here, which models atmospheric weather observations.

Reanalysis combines observation data with model data to provide the most complete picture of past weather and climate. To read more about reanalyses, check out the EWCMWF website.

ERA5 Daily Aggregates dataset, available via the Earth Engine Data Catelog

Take a few moments to explore the metadata record for this dataset. You’ll notice that it includes a bunch of important information, including:

• Dataset Availability: the date range
• Dataset Provider: where the data come from
• Earth Engine Snippet: a code snippet used for loading the dataset
• Description (tab): get to know a bit about the data
• Bands (tab): the variables present in the dataset; each band has its own name, data type, scale, mask and projection
• Image Properties: metadata available for each image band
• Example Code: a script to load and visualize ERA5 climate reanalysis parameters in Google Earth Engine (JavaScript)

13.3.3 Part iii. Visualize global precipitation using ERA5 Daily Aggregate data

Content for this section was adapted from Dr. Sam Stevenson’s Visualizing global precipitation using Google Earth Engine lesson, given in her EDS 220 course in Fall 2021.

1. Create an interactive basemap

• The default basemap is (you guessed it) Google Maps. The following code displays an empty Google Map that you can manipulate just like you would in the typical Google Maps interface. Do this using the Map method from the geemap library. We’ll also center the map at a specified latitude and longitude (here, 50N, 151E), set a zoom level, and save our map as an object called myMap.

myMap = geemap.Map(center = [60, -151], zoom = 4)
myMap

2. Load the ERA5 Image Collection from GEE

• Next, we need to tell GEE what data we want to layer on top of our basemap. The ImageCollection method extracts a set of individual images that satisfies some criterion that you pass to GEE through the ee package. This is stored as an ImageCollection object which can be filtered and processed in various ways. We can pass the ImageCollction method agruments to tell GEE which data we want to retrieve. Below, we retrieve all daily ERA5 data.

Earth Engine Snippets make importing ImageCollections easy!

To import an ImageCollection, copy and paste the Earth Engine Snippet for your dataset of interest. For example, the Earth Enginge Snippet to import the ERA5 daily aggregates data can be found on the dataset page.

weatherData = ee.ImageCollection('ECMWF/ERA5/DAILY')

3. Select an image to plot

• To plot a map over our Google Maps basemap, we need an Image rather than an ImageCollection. ERA5 contains many different climate variables – explore which variables the dataset contains under the Bands tab. We’ll use the select method to choose the parameter(s) we’re interested in from our weatherData object. Let’s select the total_precipitation band.

Plotting an Image will produce a static visualization (e.g. total precipitation at a particular point in time, or the average precipitation over a specified date range), while an ImageCollection can be visualized as either an animation or as a series of thumbnails (aka a “filmstrip”), such as this animation showing a three-day progression of Atlantic hurricanes in September, 2017 (source: Google Earth Engine).

# select desired bands (total_preciptation)
precip = weatherData.select("total_precipitation")

• We can look at our precip object metadata using the print method to see that we’ve isolated the total_precipitation band, but it’s still an ImageCollection.

print(precip)

Note

You may see a message in that says, “Output exceeds the size limit. Open the full output data in a text editor” when printing your image object metadata. Click here to see the entire output, which includes date range information.

• Let’s say that we want to look at data for a particular time of interest – e.g. January 1, 2019 - December 31, 2019. We can apply the filterDate method to our selected total_precipitation parameter to filter for data from our chosen date range. We can also apply the mean method, which takes whatever precedes it and calculates the average – this step reduces our ImageCollection to a single Image.

# initial date of interest (inclusive)
i_date = '2019-01-01'

# final data of interest (exclusive)
f_date = '2020-01-01'

# select desired bands (total_preciptation), dates, and calculate total precipitation across that date range
precip = weatherData.select("total_precipitation").filterDate(i_date, f_date).sum()

• Use the print method again to check out your new precip object – notice that it’s now an ee.Image (rather than ee.ImageCollection) and the start and end date values over which the average is taken are as we specified.

print(precip)

4. Add the precipitation Image to the basemap

• First, set a color palette to use when plotting the data layer. The following is a palette specified for ERA5 precipitation data (scroll down to the example code, available on the landing page for the ERA5 metadata in the Earth Engine Data Catelog). Here, we adjusted the max value to change the range of pixel values to which the palette should be applied – this will make our colors stand out a bit more when we layer our precipitation data on our basemap, below.

Learn more about GEE color palettes and Image visualization here.

precip_palette = {
    'min':0,
    'max':5,
    'palette': ['#FFFFFF', '#00FFFF', '#0080FF', '#DA00FF', '#FFA400', '#FF0000']
}

Note

GEE has lots of pre-defined color palettes to choose from based on the type of data you want to visualize. Also check out Crameri et al. 2020 for recommended best practices when choosing color gradients that accurately represent data and are readable by those with color vision deficiencies.

• Finally, plot our filtered data, precip, on top of our basemap using the addLayer method. We’ll also pass it our visualization parameters (colors and ranges stored in precip_palette, the name of the data field, total precipitation, and opacity (so that we can see the basemap underneath).

myMap.addLayer(precip, precip_palette, 'total precipitation', opacity = 0.7)
myMap

13.3.4 Part iv. Extract annual sum of precipitation data from points

We probably don’t want to just plot the data, extracting it at points of interest to use in analysis would be helpful too. I’ll show a simple example here of how to extract raster values from a set of point geometries.

First, we’ll create a pandas data frame with some locations in Alaska.

# initialize data frame of points
data = [['anc', 61.2, -150.1 ], ['fai', 64.8, -147.6], ['jun', 58.3, -134.4]]
  
# Create the pandas DataFrame
df = pd.DataFrame(data, columns=['loc', 'lat', 'lon'])

Next we need to convert the rows in the data frame to a GEE FeatureCollection. To do this, we will run a loop over every row in the data frame which:

1. creates a point geometry from the lat/lon coordinates
2. creates an attribute dictionary from the row
3. uses the attribute dictionary and geom in the Feature function

# convert data frame to list of GEE features
features=[]

for index, row in df.iterrows():
    p_geom = ee.Geometry.Point([row['lon'], row['lat']])
    # construct attributes for each row
    p_props = dict(row)
    p_feature = ee.Feature(p_geom, p_props)
    features.append(p_feature)

We then pass that features list into the FeatureCollection function

ee_fc = ee.FeatureCollection(features)

Now, we can use the sampleRegions method on our precip image, passing the feature collection we just created to the collection argument.

res = precip.sampleRegions(collection = ee_fc, scale = 30)

This returns another feature collection, which we can convert back to a pandas data frame with just one function call.

res_df = geemap.ee_to_pandas(res)

Of note, by default GEE data is in maps mercator (EPSG:3857). The documentation has many more details on how to handle projections, but in this case it is okay for us to use the default value since our input data were also in EPSG:3857.

This was a simple example but you can do much more with these methods, including extracing timeseries data, extracting data from regions with polygon geometries (as opposed to point geometries), buffering features, and more. The GEE documentation is excellent, and examples abound.

13.3.5 Part v. Takeaways

In just about five lines of code (and mere seconds of execution time), we’ve applied and visualized a global precipitation model (that’s pretty cool, right??). We can zoom in/out across our interactive map while Google Earth Engine recalculates and revisualizes our model in near real-time.

GEE does have some limitations. To name a few:

• it’s a closed platform service, so GEE software can only be run on Google’s infrastructure, as opposed to other open source options, like Pangeo
• it’s only free for non-commercial use
• there are memory and storage limits

Despite these limitations, GEE’s data catelog and cloud computing resources can vastly streamline and expedite analyzing and visualizing large geospatial datasets.

13.4 Exercise 2: Visualize fire dynamics in the Arctic using GEE

In this exercise, use the Google Earth Engine datasets listing to find a dataset showing burn areas, that includes a day burned layer.

Create a map of your dataset using methods similar to above. Look to the dataset landing page on GEE for palette help, or create your own!

Below is one solution. Don’t peek unless you are stuck!

Show the code

burn = ee.ImageCollection("ESA/CCI/FireCCI/5_1")
burn_2020 = burn.select('BurnDate').filterDate('2019-01-01', '2019-12-31').max()
fire_pal = {
  'min': 1,
  'max': 366,
  'palette': [
    'ff0000', 'fd4100', 'fb8200', 'f9c400', 'f2ff00', 'b6ff05',
    '7aff0a', '3eff0f', '02ff15', '00ff55', '00ff99', '00ffdd',
    '00ddff', '0098ff', '0052ff', '0210ff', '3a0dfb', '7209f6',
    'a905f1', 'e102ed', 'ff00cc', 'ff0089', 'ff0047', 'ff0004'
  ]
}


myMap2 = geemap.Map(center = [66,-145], zoom = 7)

myMap2.addLayer(burn_2020, fire_pal, "Burned area with date burned", opacity = 1)
legend_dict = {
    "January 1" : '#ff0000',
    "March 1" : '#7aff0a',
    "June 1" : '#00ddff',
    "Sept 1" : '#a905f1'
}

myMap2.add_legend(legend_title="Burn Day", legend_dict=legend_dict)

13.5 Additional Resources

Textbook:

• Cloud-Based Remote Sensing with Google Earth Engine Book

Tools:

• GEE Code Editor, a web-based IDE for using GEE (JavaScript)

Data:

• Earth Engine Data Catalog, the main resource for “official” GEE Datasets
  
• awesome-gee-community-datasets, the main resource for “community” GEE Datasets

Documentation, Tutorials, & Help:

• earthengine-api installation instructions
  
• Creating and managing Google Cloud projects
  
• Troubleshooting authentication issues
  
• An Intro to the Earth Engine Python API
  
• geemap documentation
  
• Qiusheng Wu’s YouTube channel for GEE & geemap Python tutorials
• GEE on StackExhange

Other:

• Climate Engine App, a no-code user interface to Google Earth Engine for quickly and easily visualizing various Earth observation processes and variables
• Qiusheng Wu