mirror of
https://github.com/giswqs/earthengine-py-notebooks.git
synced 2022-05-07 18:24:49 +03:00
Updated template.py
This commit is contained in:
Qiusheng Wu
382
AssetManagement/export_ee_data.ipynb
Normal file
382
AssetManagement/export_ee_data.ipynb
Normal file
@@ -0,0 +1,382 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<table class=\"ee-notebook-buttons\" align=\"left\"><td>\n",
|
||||
"<a target=\"_blank\" href=\"https://colab.research.google.com/github/giswqs/qgis-earthengine-examples/blob/master/Folium/export-ee-data.ipynb\">\n",
|
||||
" <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /> Run in Google Colab</a>\n",
|
||||
"</td><td>\n",
|
||||
"<a target=\"_blank\" href=\"https://github.com/giswqs/qgis-earthengine-examples/tree/master/Folium/export-ee-data.ipynb\"><img width=32px src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /> View source on GitHub</a></td><td>\n",
|
||||
"<a target=\"_blank\" href=\"https://nbviewer.jupyter.org/github/giswqs/qgis-earthengine-examples/blob/master/Folium/export-ee-data.ipynb\"><img width=26px src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/3/38/Jupyter_logo.svg/883px-Jupyter_logo.svg.png\" />Notebook Viewer</a></td><td>\n",
|
||||
"<a target=\"_blank\" href=\"https://mybinder.org/v2/gh/giswqs/qgis-earthengine-examples/master?filepath=Folium%2Fexport-ee-data.ipynb\"><img width=58px src=\"https://mybinder.org/static/images/logo_social.png\" />Run in binder</a></td></table>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Exporting Data from Google Earth Engine\n",
|
||||
"\n",
|
||||
"This notebook demonstrates how to export data from Google Earth Engine using the [Earth Engine Python API](https://developers.google.com/earth-engine/python_install).\n",
|
||||
"\n",
|
||||
"**Contact:** Qiusheng Wu (<https://wetlands.io>)\n",
|
||||
"\n",
|
||||
"**Table of Contents**\n",
|
||||
"* [Install Earth Engine API](#install-ee-api)\n",
|
||||
"* [Import Earth Engine API and authenticate](#import-api)\n",
|
||||
"* [Export ee.Image](#export-image)\n",
|
||||
"* [Export ee.ImageCollection](#export-image-collection)\n",
|
||||
"* [Export vector data](#export-vector)\n",
|
||||
"* [Export tabular data](#export-table)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Install Earth Engine API <a class=\"anchor\" id=\"install-ee-api\"></a>\n",
|
||||
"\n",
|
||||
"The Earth Engine API is installed by default in [Google Colaboratory](https://colab.sandbox.google.com/notebooks/welcome.ipynb). You only need to install it if you are running this notebook using your local Python interpreter or other cloud computing platforms. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"!pip install earthengine-api"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Import Earth Engine API and authenticate<a class=\"anchor\" id=\"import-api\"></a>\n",
|
||||
"\n",
|
||||
"The Earth Engine API is installed by default in Google Colaboratory so requires only importing and authenticating. These steps must be completed for each new Colab session, if you restart your Colab kernel, or if your Colab virtual machine is recycled due to inactivity.\n",
|
||||
"\n",
|
||||
"### Import the API\n",
|
||||
"\n",
|
||||
"Run the following cell to import the API into your session."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import ee"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Authenticate and initialize\n",
|
||||
"\n",
|
||||
"Run the `ee.Authenticate` function to authenticate your access to Earth Engine servers and `ee.Initialize` to initialize it. Upon running the following cell you'll be asked to grant Earth Engine access to your Google account. Follow the instructions printed to the cell."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"## Trigger the authentication flow. You only need to do this once\n",
|
||||
"ee.Authenticate()\n",
|
||||
"\n",
|
||||
"# Initialize the library.\n",
|
||||
"ee.Initialize()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Test the API\n",
|
||||
"\n",
|
||||
"Test the API by printing the elevation of Mount Everest."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Mount Everest elevation (m): 8729\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Print the elevation of Mount Everest.\n",
|
||||
"dem = ee.Image('USGS/SRTMGL1_003')\n",
|
||||
"xy = ee.Geometry.Point([86.9250, 27.9881])\n",
|
||||
"elev = dem.sample(xy, 30).first().get('elevation').getInfo()\n",
|
||||
"print('Mount Everest elevation (m):', elev)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"`ee.Image` objects can be displayed to notebook output cells. The `IPython.display` module contains the `Image` function, which can display the results of a URL representing an image generated from a call to the Earth\n",
|
||||
"Engine `getThumbUrl` function. The following cell will display a thumbnail of the global elevation model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/html": [
|
||||
"<img src=\"https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/thumbnails/a83efcc1cdfd60835e6ed06b44440762-38048e532765d7d100a05b05d1a82d6c:getPixels\"/>"
|
||||
],
|
||||
"text/plain": [
|
||||
"<IPython.core.display.Image object>"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Import the Image function from the IPython.display module. \n",
|
||||
"from IPython.display import Image\n",
|
||||
"\n",
|
||||
"# Display a thumbnail of global elevation.\n",
|
||||
"Image(url = dem.updateMask(dem.gt(0))\n",
|
||||
" .getThumbUrl({'min': 0, 'max': 4000, 'dimensions': 512,\n",
|
||||
" 'palette': ['006633', 'E5FFCC', '662A00', 'D8D8D8', 'F5F5F5']}))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Exporting ee.Image <a class=\"anchor\" id=\"export-image\"></a>\n",
|
||||
"\n",
|
||||
"You can export images from Earth Engine in [GeoTIFF](https://trac.osgeo.org/geotiff/) or [TFRecord format](https://www.tensorflow.org/api_guides/python/python_io#TFRecords_Format_Details). See [Configuration Parameters](https://developers.google.com/earth-engine/exporting#configuration-parameters) for more output options.\n",
|
||||
"\n",
|
||||
"To export an image to your Drive account, use `Export.image.toDrive()`. For example, to export portions of a Landsat image, define a region to export, then call `Export.image.toDrive()`:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Exporting m_4609915_sw_14_1_20100629\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import ee\n",
|
||||
"\n",
|
||||
"image = ee.Image('USDA/NAIP/DOQQ/m_4609915_sw_14_1_20100629')\n",
|
||||
"region = image.geometry() # specify the roi \n",
|
||||
"scale = image.projection().nominalScale().multiply(10) # specify the image resolution\n",
|
||||
"description = image.get('system:index').getInfo() # set the output image filename\n",
|
||||
"\n",
|
||||
"# Set configration parameters for output image\n",
|
||||
"task_config = {\n",
|
||||
" 'folder': 'gee-data', # output Google Drive folder\n",
|
||||
" 'region': region, # roi \n",
|
||||
" 'scale': scale, # image resolution\n",
|
||||
" 'crs': 'EPSG:4326',\n",
|
||||
" 'maxPixels': 1.0E13,\n",
|
||||
" 'fileFormat': 'GeoTIFF'\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
"# Export image to Google Drive\n",
|
||||
"task = ee.batch.Export.image.toDrive(image, description, **task_config)\n",
|
||||
"task.start()\n",
|
||||
"print(\"Exporting {}\".format(description))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Exporting ee.ImageCollection <a class=\"anchor\" id=\"export-image-collection\"></a>\n",
|
||||
"\n",
|
||||
"To export images from an ee.ImageCollection, use a for loop. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Total number of images: 6\n",
|
||||
"Exporting 1/6: m_4609915_sw_14_1_20090818\n",
|
||||
"Exporting 2/6: m_4609915_sw_14_1_20100629\n",
|
||||
"Exporting 3/6: m_4609915_sw_14_1_20120714\n",
|
||||
"Exporting 4/6: m_4609915_sw_14_1_20140901\n",
|
||||
"Exporting 5/6: m_4609915_sw_14_1_20150926\n",
|
||||
"Exporting 6/6: m_4609915_sw_14_h_20170703\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import ee\n",
|
||||
"\n",
|
||||
"# Create an ImageCollection\n",
|
||||
"loc = ee.Geometry.Point(-99.2222, 46.7816)\n",
|
||||
"collection = ee.ImageCollection('USDA/NAIP/DOQQ') \\\n",
|
||||
" .filterBounds(loc) \\\n",
|
||||
" .filterDate('2008-01-01', '2020-01-01') \\\n",
|
||||
" .filter(ee.Filter.listContains(\"system:band_names\", \"N\"))\n",
|
||||
"\n",
|
||||
"count = int(collection.size().getInfo())\n",
|
||||
"print(\"Total number of images: {}\".format(count))\n",
|
||||
"\n",
|
||||
"for i in range(0, count):\n",
|
||||
" image = ee.Image(collection.toList(count).get(i))\n",
|
||||
" region = image.geometry() # roi\n",
|
||||
" scale = image.projection().nominalScale().multiply(10) # specify the image resolution\n",
|
||||
"\n",
|
||||
" task_config = {\n",
|
||||
" 'folder': 'gee-data', # output Google Drive folder\n",
|
||||
" 'region': region, # specify roi\n",
|
||||
" 'scale': scale, # specify image resolution\n",
|
||||
" 'crs': 'EPSG:4326',\n",
|
||||
" 'maxPixels': 1.0E13,\n",
|
||||
" 'fileFormat': 'GeoTIFF'\n",
|
||||
" }\n",
|
||||
" \n",
|
||||
" description = image.get('system:index').getInfo()\n",
|
||||
" print('Exporting {}/{}: {}'.format(i+1, count, description))\n",
|
||||
" task = ee.batch.Export.image.toDrive(image, description, **task_config)\n",
|
||||
" task.start()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Exporting vector data <a class=\"anchor\" id=\"export-vector\"></a>\n",
|
||||
"\n",
|
||||
"You can export a `FeatureCollection` as CSV, SHP (shapefile), GeoJSON, KML, KMZ or TFRecord using `Export.table`. The `FeatureCollection` may represent vectors or simply a table of data. In the latter case, the features in the collection will have null geometry."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"scrolled": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Exporting vectorsToDriveExample\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import ee\n",
|
||||
"\n",
|
||||
"fc = ee.FeatureCollection('TIGER/2018/States') \\\n",
|
||||
" .filter(ee.Filter.eq('STUSPS', 'TN'))\n",
|
||||
"\n",
|
||||
"desc = 'vectorsToDriveExample'\n",
|
||||
"\n",
|
||||
"# Set configration parameters for output vector\n",
|
||||
"task_config = {\n",
|
||||
" 'folder': 'gee-data', # output Google Drive folder\n",
|
||||
" 'fileFormat': 'SHP', # output formats: SHP, GeoJSON, KML, KMZ, TFRecord\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
"print('Exporting {}'.format(desc))\n",
|
||||
"task = ee.batch.Export.table.toDrive(fc, desc, **task_config)\n",
|
||||
"task.start()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Exporting tabular data <a class=\"anchor\" id=\"export-table\"></a>\n",
|
||||
"\n",
|
||||
"To export tabular data, you can set the `selectors` parameter to specify a list of properties/attributes to export. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Exporting tableToDriveExample\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import ee\n",
|
||||
"\n",
|
||||
"fc = ee.FeatureCollection('TIGER/2018/States') \\\n",
|
||||
" .filter(ee.Filter.eq('STUSPS', 'TN'))\n",
|
||||
"\n",
|
||||
"desc = 'tableToDriveExample'\n",
|
||||
"\n",
|
||||
"# Set configration parameters for output vector\n",
|
||||
"task_config = {\n",
|
||||
" 'folder': 'gee-data', # output Google Drive folder\n",
|
||||
" 'fileFormat': 'CSV', \n",
|
||||
" 'selectors': ['ALAND', 'NAME', 'STUSPS'] # a list of properties/attributes to be exported\n",
|
||||
" }\n",
|
||||
"\n",
|
||||
"print('Exporting {}'.format(desc))\n",
|
||||
"task = ee.batch.Export.table.toDrive(fc, desc, **task_config)\n",
|
||||
"task.start()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.7.3"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
1
GetStarted/10_get_started_with_EE.ipynb
Normal file
1
GetStarted/10_get_started_with_EE.ipynb
Normal file
File diff suppressed because one or more lines are too long
@@ -115,28 +115,28 @@ for index, filename in enumerate(files):
|
||||
with open(out_py_script_path, 'w') as f:
|
||||
f.writelines(out_text)
|
||||
|
||||
basename = os.path.basename(out_py_script_path)
|
||||
if basename in changed_files:
|
||||
print(basename)
|
||||
out_nb_path = out_py_script_path.replace('.py', '.ipynb')
|
||||
cmd = 'ipynb-py-convert ' + out_py_script_path + ' ' + out_nb_path
|
||||
print(os.popen(cmd).read().rstrip())
|
||||
cmd2 = 'jupyter nbconvert --to notebook --execute ' + out_nb_path + ' --inplace'
|
||||
print(os.popen(cmd2).read().rstrip())
|
||||
# basename = os.path.basename(out_py_script_path)
|
||||
# if basename in changed_files:
|
||||
# print(basename)
|
||||
# out_nb_path = out_py_script_path.replace('.py', '.ipynb')
|
||||
# cmd = 'ipynb-py-convert ' + out_py_script_path + ' ' + out_nb_path
|
||||
# print(os.popen(cmd).read().rstrip())
|
||||
# cmd2 = 'jupyter nbconvert --to notebook --execute ' + out_nb_path + ' --inplace'
|
||||
# print(os.popen(cmd2).read().rstrip())
|
||||
|
||||
# modTimesinceEpoc = os.path.getmtime(out_py_script_path)
|
||||
# modificationTime = datetime.datetime.utcfromtimestamp(modTimesinceEpoc).strftime('%Y-%m-%d %H:%M:%S')
|
||||
# print("Last Modified Time : ", modificationTime , ' UTC')
|
||||
|
||||
# out_nb_path = out_py_script_path.replace('.py', '.ipynb')
|
||||
# print('{}/{}: {}'.format(i, len(files), out_nb_path))
|
||||
# i = i + 1
|
||||
out_nb_path = out_py_script_path.replace('.py', '.ipynb')
|
||||
print('{}/{}: {}'.format(i, len(files), out_nb_path))
|
||||
i = i + 1
|
||||
|
||||
# cmd = 'ipynb-py-convert ' + out_py_script_path + ' ' + out_nb_path
|
||||
# # print(cmd)
|
||||
# print(os.popen(cmd).read().rstrip())
|
||||
cmd = 'ipynb-py-convert ' + out_py_script_path + ' ' + out_nb_path
|
||||
# print(cmd)
|
||||
print(os.popen(cmd).read().rstrip())
|
||||
|
||||
# cmd2 = 'jupyter nbconvert --to notebook --execute ' + out_nb_path + ' --inplace'
|
||||
# # print(os.popen(cmd2).read().rstrip())
|
||||
cmd2 = 'jupyter nbconvert --to notebook --execute ' + out_nb_path + ' --inplace'
|
||||
print(os.popen(cmd2).read().rstrip())
|
||||
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -11,7 +11,7 @@
|
||||
'''
|
||||
## Install Earth Engine API
|
||||
Install the [Earth Engine Python API](https://developers.google.com/earth-engine/python_install) and [geehydro](https://github.com/giswqs/geehydro). The **geehydro** Python package builds on the [folium](https://github.com/python-visualization/folium) package and implements several methods for displaying Earth Engine data layers, such as `Map.addLayer()`, `Map.setCenter()`, `Map.centerObject()`, and `Map.setOptions()`.
|
||||
The magic command `%%capture` can be used to hide output from a specific cell.
|
||||
The magic command `%%capture` can be used to hide output from a specific cell. Uncomment these lines if you are running this notebook for the first time.
|
||||
'''
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ import geehydro
|
||||
# %%
|
||||
'''
|
||||
Authenticate and initialize Earth Engine API. You only need to authenticate the Earth Engine API once. Uncomment the line `ee.Authenticate()`
|
||||
if you are running this notebook for this first time or if you are getting an authentication error.
|
||||
if you are running this notebook for the first time or if you are getting an authentication error.
|
||||
'''
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user