Python for ArcGIS Pro

Copyright © 2021 Packt Publishing

This is an Early Access product. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the content and extracts of this book may evolve as it is being developed to ensure it is up-to-date.

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

The information contained in this book is sold without warranty, either express or implied. Neither the author nor Packt Publishing or its dealers and distributors will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Early Access Publication: Python for ArcGIS Pro

Early Access Production Reference: B17951

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK

ISBN: 978-1-80324-166-1

www.packt.com

Table of Contents

1. Python for ArcGIS Pro: Automate cartography and data analysis using ArcGIS Python modules, ArcGIS Pro Notebooks, Jupyter Notebooks, and pandas

2. Python: The Beginning

1. Python: Built Different

2. Why Python is different

1. Python Versions

2. What is Python?

3. Python Interpreter

4. Python 3 is very similar:

5. Versions included

6. IDLE development environment

7. Locate and double-click on the Path variable (or press edit when selected):

3. Basics of programming

1. Variables

2. Iteration

3. Counter and enumerators

4. Conditionals

5. If vs Else

6. List Position (or why programmers count from 0)

7. Data Types

8. Checking the data type

9. To check the data type of a Python variable, use the type() function:

10. Conversion between data types

11. In these examples, a character is converted from String to Integer to Float to String using the int and str and float functions:

12. Data Structures or Containers

13. Functions

14. Classes

4. Installing and importing modules

1. Using pip

2. Installing modules

3. The setup.py file

4. Installing in virtual environments

5. Importing custom code

6. The __init__.py file

7. Example custom module

8. Import your module

9. The site-packages folder

5. Basic style tips for writing scripts

1. Indentation

2. Define global variables

3. Define functions

4. Include print statements

5. Write the executable parts of the script

6. If __name__ == ‘__main__’

6. Summary

3. Basics of ArcPy

1. Checking your ArcPy Install

1. Python IDLE Shell for ArcGIS Pro Install

2. Python Window in ArcGIS Pro

3. ArcPy Environment Settings

4. ArcPy Tools - Geoprocessing using ArcPy

5. Built in ArcPy Functions

6. Introduction to ArcPy Modules

4. ArcGIS API for Python

1. What is the ArcGIS API for Python?

1. What does it do? And why use it?

2. How to use it?

2. Using the gis module to Manage your GIS

1. Search for data

2. Publish Data

3. Organize data, manage groups, and users

5. Feedback

1. Cover

2. Table of contents

Python for ArcGIS Pro: Automate cartography and data analysis using ArcGIS Python modules, ArcGIS Pro Notebooks, Jupyter Notebooks, and pandas

Welcome to Packt Early Access. We’re giving you an exclusive preview of this book before it goes on sale. It can take many months to write a book, but our authors have cutting-edge information to share with you today. Early Access gives you an insight into the latest developments by making chapter drafts available. The chapters may be a little rough around the edges right now, but our authors will update them over time. You’ll be notified when a new version is ready.

This title is in development, with more chapters still to be written, which means you have the opportunity to have your say about the content. We want to publish books that provide useful information to you and other customers, so we’ll send questionnaires out to you regularly. All feedback is helpful, so please be open about your thoughts and opinions. Our editors will work their magic on the text of the book, so we’d like your input on the technical elements and your experience as a reader. We’ll also provide frequent updates on how our authors have changed their chapters based on your feedback.

You can dip in and out of this book or follow along from start to finish; Early Access is designed to be flexible. We hope you enjoy getting to know more about the process of writing a Packt book. Join the exploration of new topics by contributing your ideas and see them come to life in print.

1. Chapter 1: Introduction to Python for GIS

2. Chapter 2: Basics of ArcPy

3. Chapter 4: Data Access Using Cursors

4. Chapter 3: Introduction to the ArcGIS API for Python

5. Chapter 5: Publishing to ArcGIS Online

6. Chapter 8: Vector Analysis with Pandas

7. Chapter 6: ArcToolbox Script Tools

8. Chapter 9: Raster Analysis with pandas

9. Chapter 7: Automated Map Production

10. Chapter 10: Geospatial Data Science with SciPy

11. Chapter 11: Case Study: ArcGIS Pro Data Management

12. Chapter 12: Case Study: Advanced Map Production

13. Chapter 13: Case Study: Interactive Data Science Web Map

Python: The Beginning

Programming with computers is one of the most rewarding and frustrating of human endeavors.

Those rewards can be in the form of money, as we can see with today’s high-tech salaries. I would argue, however, that the most rewarding part of mastering programming is to make yourself into a computer power user who can execute both simple and complex applications and analyses, written in reusable code, with ease.

The frustrations will come and go, and it is a good thing: you, like me and millions before you, will learn from each mistake (it helps to be a pedant, perhaps, but not being one myself I can’t be sure). You will grow and learn with each exercise in this book, and by asking the right questions and paying close attention you can avoid some of these issues.

If you are an ArcGIS expert or novice, seeking to expand on your skillsets: congratulations, you are in the right place. In this book you will learn how to take your existing GIS expertise (or interest) and multiply its potential using a deceptively simple programming language called Python.

Computer programming is its own vast field that cannot be captured in one chapter, of course. In this chapter I will explain the basic knowledge necessary to read, write and run Python scripts. We’ll leave the ArcGIS tools for later chapters and focus on Python: its beginnings, its current state, how to use it, and importantly, what Python is and what it is not.

We will cover the following topics:

- Basics of Python

- Basics of computer programming

- Installing and importing modules

- Writing and executing scripts

Python: Built Different

Guido Van Rossum, the creator of the Python programming language, was frustrated with the state of computer programming in the late 1980s. Programming languages were too complex, and at the same time, too loose with their formatting requirements. This led to large codebases with complex scripts poorly written and rarely documented.

Merely running a simple program could take a long time, as the code would need to be type-checked (variables declared correctly and assigned to the correct data type) and compiled (converted from high-level code written in text files into the assembly language or machine code understood by the CPU).

As the Dutch programmer completed professional work on the ABC programming language, where he had learned much about language design, he decided he wanted to turn his grips about the limits of ABC and other languages into a hobby.

With a master’s degree in mathematics and computer science from the University of Amsterdam, his hobbies tended towards the computer, but he did have a love for Monty Python, the British comedy series. So, he combined his passions and created Python, which is now used for all kinds of programmatic solutions. Today Python is everywhere, in the internet and appliances and cars and so much more. Because of its ubiquity and its simplicity, it has been adopted by the GIS software ecosystem as a standard programming tool.

Why Python is different

Because of Van Rossum’s extensive experience with the state of computer languages in the 1980s, he was well positioned to create a language that solved many of their deficiencies. He added features that he admired from many other languages and added a few of his own. Here is an incomplete list of Python features built to improve on other languages:

Issue

Improvement

Python Feature

Memory overrun

Built-in memory management

Garbage collection and memory management

Slow compiler times

One line testing, dynamic typing

Python Interpreter

Unclear error messages

Messages indicating the offending line and affected code

Error Traceback

Spaghetti code

Clean importation and modularization

Importation

Unclear code formatting and spacing making code unreadable

Indentation rules and reduced brackets

Forced whitespace

Too many ways to do something

There should be only one way: the Pythonic way

The Zen of Python

Python Versions

The original Python version release in 1991 by Van Rossum, Python 1.0 and its successors, was eventually superseded by the widely popular Python 2.x. Care was taken to ensure that version 2.0 and beyond were backwards-compatible with Python 1.x. However, for the new Python 3.0 and beyond, backwards compatibility with Python 1 and Python 2 was broken.

This break has caused a divergence in the Python ecosystem. Some companies chose to stick with Python 2.x, which has meant that the “sunset” date or retirement date for the older version was extended from 2015 until April 2020. Now that the sunset date has been passed, there is no active work by the Python Software Foundation (PSF) on Python 2.x. Python 3.x development continues and will continue into the future, overseen by the PSF.

Van Rossum served as the Benevolent Dictator for Life of the PSF until he resigned the position in 2018.

Check out more about the history of Python: https://docs.python.org/3/faq/general.html

Figure 1:Divergence of Python 3 from Python 2

ArcGIS Python Versions

Since ArcMap version 9.x, Python has been integrated into the ArcGIS software suite. However, ArcGIS Desktop and ArcGIS Pro now both depend on different versions of Python.

ArcGIS Desktop: Python 2.x

ArcGIS Desktop (or ArcMap) version 9.0 and above ships with Python 2.x included. The installer for ArcGIS would automatically install Python 2.x and would add the  arcpy  module (originally  arcgisscripting ) to the Python path variable, making it available for scripting.

ArcMap, ArcCatalog, ArcGIS Engine, and ArcGIS Server all depend on  arcpy  and the Python 2.x version included when the ArcGIS Desktop or Enterprise software is installed.

ArcGIS Pro: Python 3.x

ArcGIS Pro, which was designed after the decision to sunset Python 2.0 was announced, was divorced from the Python 2.x ecosystem and instead shipped with Python 3.x.

Instead of  arcpy , ArcGIS Pro uses the ArcGIS API for Python.

Managing both versions

The sunsetting of ArcGIS Desktop has been extended to March 2025, meaning that Python 2.7 will be included by Esri until that time despite it being officially retired by the Python Software Foundation.

Because of this, we will learn use virtual environments to manage the versions, and you will learn about the PATH and PYTHONPATH environmental variables, which control which version of Python is used to execute a script.

IMAGE CREDIT: https://media.geeksforgeeks.org/wp-content/uploads/20190502023317/TIMELINE.jpg

What is Python?

In short, Python is an application:  python.exe . This application is also an executable file, meaning it can be run by itself to interpret code, or it can be called from other applications to run custom scripts. This standard interoperability is part of why it is included in applications such as ArcGIS Pro. When ArcGIS is installed, Python is also installed on your computer, along with a series of supporting files and folders.

Python includes a large standard library of tools or “modules”. These include support for internet requests, advanced math, CSV reading and writing, JSON serialization, and many more modules included in the Python core. While these tools are powerful, Python was also built to be extensible, meaning that third-party modules can be easily added to a Python installation. The ArcGIS Python modules are both good examples of extending the capabilities of Python. There are hundreds of thousands of others, covering almost any type of programming need, of varying quality.

Python is written in the programming language C. There are variants of Python written in other languages for a variety of technical reasons, but most implementations of Python are built on top of C. This means that Python is often expanded through modules built on top of C code, usually for speed improvement reasons. A Python code “layer” or “wrapper” is put on top of C code to make it work with normal Python packages, gaining the simplicity of Python and the processing speed boosts of precompiled C code. NumPy and SciPy are examples of this type of module, and are included with the ArcGIS installation of Python.

Python is free and open software, which is another reason it is packaged with so many other software applications for automation purposes. Python can also be installed separately, using a free installer from the Python Software Foundation.

Check out the Python Software Foundation on the internet: https://www.python.org/psf

Download Python versions directly from the PSF: https://www.python.org/downloads/

Where is it installed

On Windows machines, Python is not included by default – it must be installed along with ArcGIS or separately using an installer from the Python Software Foundation. Once the ArcGIS Installer is run, you will see a folder inside the  C:\ drive . You can set a custom location or use the default.

Python Interpreter

When you start python.exe by double-clicking on it (see below for multiple other ways to run the executable), it starts what is known as the Python Interpreter.

This is a useful interface, allowing you to enter, one line at a time, bits of code for testing and confirmation. Once the line is entered, push Enter/Return and the code will be executed. This tool helps you both learn coding and test code in the same environment.

Starting the Interpreter

Double-clicking on  python.exe  from the folder or starting Python (command line) from the Start Menu, will start the interpreter, which allows for one-line commands to be executed.:

Python 3 is very similar:

What is a Python script?

The  python.exe  executable file, along with being a program where code can be run, will also execute Python scripts. These scripts are simple text files that can be edited by any text editing software. Python scripts are saved with the  .py  extension.

When a Python script is “run”, it is passed as the first command line argument to the Python executable ( python.exe ). This program will read and then execute the code from the top to the bottom as long as it is valid Python and it contains no errors. If there is an error encountered, the script will stop and return an error message. If there is no error, nothing will be returned unless you have added “ print ” statements to return messages from the main loop to the Python window as the script is running.

In this example the script is executed by “passing” the script as an argument to the executable (python.exe), which is explicitly called with the full folder path to the python.exe file to avoid path issues:

C:\Projects>C:\PythonArcGIS\ArcGIS10.5\python.exe chapter1.py

In this example the script is executed by “passing” the script as an argument to the executable, along with optional parameters that are accepted by the script itself before being run:

C:\Projects>C:\PythonArcGIS\ArcGIS10.5\python.exe chapter1.py arg1 arg2

Versions included

Python comes with two versions of the  python.exe  file. These are the same version of Python, to be clear, but each file has a different role. Python.exe is the main file, and the other version is pythonw.exe. This file will not open an interpreter if double-clicked, as the normal python.exe will. No interpreter is available from pythonw.exe, which is the point: it is used to execute scripts more “silently” than  python.exe . Use python.exe for to start the interpreter.

How to call the executable

The Python “executable” ( python.exe ) is accessed to run the Python Interpreter or to run a custom Python script. There are many different ways to “call” or start the Python executable:

- Double-click on python.exe

- Starts the Python Interpreter

- Open IDLE, the included integrated development environment (IDE)

- Should be accessible in your Start menu on Windows in the ArcGIS folder

- Open a CMD terminal and type “python”

- Only works if the Python executable is in the PATH environment variable

- Using a third-party IDE such as PyCharm

- Each PyCharm project can have its own virtual environment, and therefore its own executable, or it can use the one installed by Esri when ArcGIS is installed.

- There are a lot of IDEs but PyCharm is the one I recommend for a variety of reasons.

- Using a Jupyter Notebook, which we will discuss extensively in this book

- This requires the installation of Jupyter, which is not included in the standard Python installation.

- Inside ArcGIS Desktop or ArcGIS Pro

- There are menu buttons that allow you to start a Python interpreter window inside ArcMap or ArcCatalog or ArcGIS Pro.

- Run code one line at a time or by using the load script command in the right-click menu.

IDLE development environment

The included IDE called IDLE is a useful environment that comes standard with every Python instance. IDLE is useful for the Python Interpreter, but also because you can create and execute scripts in this environment easily by opening a new script from the File menu, and then using the script’s Run menu to execute the script.

The Path environment variable

On Windows there is a system environment variable known as the Windows Path environment variable. This variable is available to all applications installed on the machine. Other programs use it for different purposes, but for Python it is used to find all available Python executables and modules.

This is important to understand because you may end up with multiple versions of Python on your computer one day, or after just one install of ArcGIS Desktop or Pro. When

If a script is run in a CMD window using the “ python script.py ” (passing the script to Python as an argument), and it contains import statements, then there are three things that have to happen.

First, Windows will look for an executable called python.exe in the Path. If it is there, it will then confirm that the script is valid. If it is, then Python will run the script and the Path environment variable will be checked to look for allowed locations for all modules you are trying to import.

So the Python executable cannot be run by name (instead of file location) until the  python.exe  is in the Path. Here is how you edit the Path variable:

Open up the Advanced System Settings in the Control Panel:

Locate and double-click on the Path variable (or press edit when selected):

Add a new line to the Path environment variable in the interface. If you have multiple version of Python and you are not using virtual environments, be sure to order the folders in the Path so that the correct version of Python is called when you type “ python ” into a CMD line window:

If you are not allowed to edit the Path variable, you can still run Python in the command line by referring to it using the whole path to the executable:  C:\ArcGIS10.8\Python\python.exe script.py

The operating system and Python system modules

Two modules (code libraries) built into Python need to be mentioned first. The  os  and  sys  modules, also called the operating system module ( os ) and the Python system module ( sys ) are used to control Windows system operations and Python system operations respectively.

The OS module

The  os  module is used for many things, including folder path operations such as creating folders, removing folders, checking if a folder or file exists, or executing a file using the operating system-associated application used to run that file extension. Getting the current directory, copying files, and more is possible with this module.

In this example, a string is passed to the  os.path.exists  method, which is Boolean. If it returns False, the folder does not exist, and is then created using the  os.mkdir  method:

import os

folderpath = "C:\Test_folder"

if not os.path.exists(folderpath):

os.mkdir(folderpath)

Read about the os module here: https://www.geeksforgeeks.org/os-module-python-examples/

The sys module accepts arguments

The sys module allows you to accept arguments to a script at runtime, meaning when it is executed. This is done by using the  sys.argv  method, which is a list containing all arguments made to Python during the executing of the script.

If a name variable is using the sys module to accept parameters, here is what the script looks like:

import sys

name = sys.argv[1]

print(name)

The System path

The sys module contains the Python path or system path (system in this case means Python). This is a list that Python uses to search for importable modules, after accessing the Windows path. If you can’t edit the Windows path as explained above (due to permissions usually), you can alter the Python path at runtime using the system path.

The  sys.path  list is a part of the  sys  module built into Python:

Read more about the sys module here: https://www.geeksforgeeks.org/python-sys-module/

Basics of programming

Computer programming varies from language to language in terms of implementation, but there are remarkable similarities among these languages in how their internal logic works. These programming basics are applicable for all programming languages with specific code implementations shown in Python.

Key Concepts

Variables

Names assigned to Python objects of any data type. Variables must start with a letter. Underscores are encouraged.

x=0

y=1

xy = x+y

xy_str = str(xy)

Iteration

For loops are used to iterate through an iterable data object (e.g. a list). While loops are used to loop until a condition has been met

for item in datalist:

print(item)

x=0

while x < 1:

x+=1

Conditionals

If/Elif/Else statements that interpret if an object meets a condition.

list_var = [1,’1’,1.0]

for item in list_var:

if type(item) == type(0):

print(‘Integer’)

elif type(item) == type(‘a’):

print(‘String’)

else:

print(‘Float’)

Zero-based indexing

Data containers are accessed using indexes that start with 0. The indexes are passed to the list or tuple using square brackets []. String characters can be access using the same pattern.

list_var = [‘s’,’m’,’t’]

m_var = list_var[0]

name_var = “logan”

g_var = name_var[0]

Data Types

Strings are for text. Integers are for whole numbers. Floats are for floating point numbers. Data containers such as lists and tuples and dictionaries are used extensively to organized data.

Str_var = “string”

int_var = 4

float_var = 5.7

list_var = [45,43,24]

tuple_var = (87.’a’,34)

dict_var = {‘key’:’value’}

Code Comments

Comments in code are encouraged. They help explain your thinking to both other readers and yourself. Comments are created by using the “#” symbol. Comments can be on a line by themselves or can be added to the end of a statement as anything after the # symbol will be ignored.

# This is a comment

x = 0 #also a comment

Errors

Error messages of many types are built into Python. The error traceback show the affected lines of code and the type of error. It’s not perfect.

>>> str_var = 'red"

File "<stdin>", line 1

str_var = 'red"

^

SyntaxError: EOL while scanning string literal

Counters/Enumerators

Using a variable to keep track of the number of loops performed by a for loop or while loop is a good idea. Some languages (including Python) have some built-in enumeration functionality. Counters are reassigned to themselves after being increased.

In Python the shortcut “x += y” is the same as “x = x +y”

counter = 0

list_var = [34,54,23,54]

for item in list_var:

print(item, counter)

counter += 1

Variables

Variables are used to assign objects to labels or identifiers. They are used to keep track of pieces of data, to organize the flow of the data through the script, and to help programmers read the script.

variable = 1 # a variable assignment

It is recommended (by me) to use descriptive variables that are neither too long nor too short. When variables are too short, they can become confusing to read. When they are too long, they can be confusing to write. Using underscores to separate words in variables is a common practice.

Assigned to vs is equal to (value comparison)

In Python, variables are assigned to an object using the equals sign “=”. This means that there is another way to check if a value is equal to another value: using a double equals sign “==”.

variable = 1 # a variable assignment

variable == 1 # a comparison (that is True)

Variable formatting rules

Variables must start with a letter. They cannot start with a number or other symbol, otherwise a Syntax Error will occur. However, numbers and underscores can be used in the

>>> 2var = 34

File "<stdin>", line 1

2var = 34

^

SyntaxError: invalid syntax

>>> two_var = 34

>>> two_var

34

Read more about variables here: https://realpython.com/python-variables/

Iteration

The core of computer programming is  iteration : recursively performing the same action or analysis or function call or whatever your script is built to process. Computers excel at this type of task: they can quickly iterate through a dataset to perform whatever action you deem necessary, on each data item in the set.

For loops

A “for loop” is an iteration implementation that, when presented with a data list, will perform an operation on each member of the list.

In this example, a list of integers are assigned to the variable name  data_list . The list is then used to construct a for loop using the format “ for {var} in {iterable} ” where {var} is a variable name that is assigned to each object in the list, one at a time as the loop progresses. One convention is to use “ item ” but it can be any valid variable:

data_list = [45,56,34,12,2]

for item in data_list:

print (item * 2)

90

112

68

24

4

While loops

A “while loop” is an iteration implementation that will loop until a specific threshold is met. While loops can be dangerous as they can cause an infinite loop in a script if the threshold is never met.

In this example, the while loop will run (doing nothing but adding 1 to x until it reaches 100, upon which the threshold is met and the while loop will end

x = 0

while x < 100:

x = x + 1   #same as x += 1

Read more about loops here: https://www.geeksforgeeks.org/loops-in-python/

Counter and enumerators

Iteration in for loops or while loops often requires the use of counters (also called enumerators) to track loops in an iteration.

For loops have the option to use the enumerate function by passing the iterator to the function and using a count variable (can be any valid variable name but count is logical) in front of the item variable. The count variable will keep track of the loops, starting at index zero:

>>> data_list = ['a','b','c','d','e']

>>> for count,item in enumerate(data_list):

...     print(count, item)

...

0 a

1 b

2 c

3 d

4 e

In Python the shortcut “x += y” is used to increase the value of x while keeping the same variable name, which is the same as “x = x +y”:

>>> x = 0

>>> while x <100:

...    x = x + 1

>>> x

100

>>> x = 0

>>> while x <100:

...    x += 1

>>> x

100

Conditionals

If statements and Elif statements (short for else if) and Else statements are used to create conditions that will be used to evaluate data objects. If statements can be by themselves ( elif  and  else  are optional) and is used by declaring the keyword if and then the condition the data must meet:

list_var = [1,’1’,1.0]

for item in list_var:

if type(item) == type(0):

print(‘Integer’)

elif type(item) == type(‘a’):

print(‘String’)

else:

print(‘Float’)

Read more about conditionals here: https://realpython.com/python-conditional-statements/

If vs Else

If statements are usually specific to one condition, while else statements are used as catch-alls to ensure that any data that goes through the if statement will have some way of being dealt with, even if it doesn’t meet the condition of the if statement. Elif statements, which are dependent on the if statement existing and are also condition specific, are not catch-all statements.

List Position (or why programmers count from 0)

Iteration occurs over lists that contain data. Within the list, these data are differentiated by list order or position. Items in a list are retrieved by item index, the (current) position of the data in the list.

Zero-based indexing

In Python, like most computer programming languages, the first item in a list is at index 0, not index 1.

This is a bit confusing to beginners but is a programming standard. It is slightly more computationally efficient to retrieve an item in a list that starts with 0 than a list that starts with 1, and this became the standard in C and its precursors, which meant that Python (written in C) uses zero-based indexing.

Data extraction using index position

This is the basic format of data retrieval from a list. This list of strings has an order, and the string “Bill” is the second item, meaning it is at index 1. To assign this string to variable, we pass the index into square brackets:

names = [“Silas”, “Bill”, ”Dara”]

name_bill = names[1]

Data extraction using reverse index position

This is the second format of data retrieval from a list. List order can be used in reverse, meaning that the indexing starts from the last member of the list and counts backwards. Negative numbers are used, starting at -1, which is the index of the last member of the list, and -2 is the second-to-last member of the list and so on. This means that the “Bill” string is at index -2 when using reverse index position, and so -2 must be passed to the list in square brackets:

names = [“Silas”, “Bill”, ”Dara”]

name_bill = names[-2]

Read more about indexing here: https://realpython.com/lessons/indexing-and-slicing/

Data Types

The data type of a variable determines its behavior. For instance, the character 5 could be an integer type (5) or a float (5.0) or a string (“5”). Each version of 5 will have different available tools, such as the  replace  method for strings which can replace characters in the string with other characters.

Key Data Types

Data Type

Python Data Type Object

Text data is stored as a String data type

str

Numeric data is stored as an Integer or Float or Complex type

int  ,   float  ,   complex

Sequence data (lists or arrays) can be stored as a list or tuple. Range is a special generator

list  ,   tuple  ,   range

Mapping or key/value pair data types are also known as dictionaries in Python

dict

A Set is a data type that contains distinct, immutable objects

set  ,   frozenset

Boolean is either True or False, 1 or 0

bool

Binary data types are used to access data files in binary mode.

bytes  ,   bytearray  ,   memoryview

Checking the data type

To check the data type of a Python variable, use the  type()  function:

>>> x = 0

>>> type(x)

<class ‘int’>

Strings

All text data is represented as the String data type in Python. These are known as strings. Common data stored as strings includes names, addresses, or even whole blog posts.

Strings can be also templated in code to allow for “fill-in-the-blank” strings that are not set until the script is run. Strings are technically immutable but can be manipulated using built-in Python string tools and the separate String module.

Key Concepts

Quotation Marks

Single or double quotation marks can be used to designate a string, as long as it is the same at the beginning and end. Triple quotation marks are used for strings with multiple lines. Quotes within a string can be indicated using the opposite mark as the one opening and closing the string.

String addition

Strings can be “added” together to form a larger string. Strings can also be “multiplied” by an integer to repeat the string X times.

String formatting

String templates or placeholders can be used in code and filled in at run-time with the data required.

String manipulation

Strings can be manipulated using built-in functionality. Characters can be replaced or located. Strings can be split or joined.

Quotation marks

Strings must be surrounded by quotation marks. In Python, these can be either single or double quotes, but they must be consistent. If a single quote is used to start the string, a single quote must be used to stop it:

>>> string_var = 'the red fox"

File "<stdin>", line 1

string_var = 'the red fox"

^

SyntaxError: EOL while scanning string literal

>>> string_var = 'the red fox'

>>> string_var

'the red fox'

Multiple line strings

Multiple line strings are created by pair three single quotes or double quotes at the beginning of the string, and three at the end.

In this example the variable  string_var  is a multiple line string (“ \n ” is a Python character representing a new line):

>>> string_var = """the red fox chased the

... dog across the yard"""

>>> string_var

'the red fox chased the\ndog across the yard'

String addition (and more)

Strings can be “added” together to create a new string. This process allows you to build strings from smaller strings, which can be useful for populating new fields composed of other fields in a data file and other tasks.

In this example the string “forest” is assigned to  string_var . Another string is then added to string_var to create a longer string.

>>> string_var = "forest"

>>> string_var += " path" #same as string_var = string_var+ “ path”

>>> string_var

'forest path'

String formatting

Strings in code often make use of “placeholders” for data that will be filled in later. This is known as string formatting, and there are multiple ways to perform string formatting using Python.

Key Concepts

Format function

All strings have a built-in function called format that allows the string to have arguments passed. It will accept all data types and format the string from a template.

String literals

For Python 3.6+, there is a new tool called string literals, which allow you to insert variables into strings directly. An “f” is placed in front of the string.

Data type string operators

An older but still useful tool are the string operators, which are used in strings as placeholders for specific data types (either strings or floats or integers).

String format function

This method of formatting is the preferred form for Python 3 (it is also available in Python 2.7). It allows you to pass the variables to the  format  function (which is built into all strings) and to have them fill up placeholders within the string. Any data type can be passed to the format function.

In this example, the string template is filled with details contained in other variables using the  format  string function. The placeholders are filled in the order that the variables are listed, so they must be in correct order. The curly brackets are the placeholders, and the format function will accept arguments and fill in the string:

>>> year = 1980

>>> day = "Monday"

>>> month = "Feb"

>>> template = "It was a cold {} in {} {}"

>>> template.format(day, month, year)

'It was a cold Monday in Feb 1980'

In this example, the placeholders are named, and are passed to keyword arguments in the format function. The arguments are named and do not need to be in order in the format function:

>>> template = 'It was a cold {day} in {month} {year}'

>>> template.format(month=month,year=year,day=day)

'It was a cold Monday in Feb 1980'

In this example, the placeholders are numbered, which makes it much easier to repeat a string:

>>> template = "{0},{0} oh no,{1} gotta go"

>>> template.format("Louie", "Me")

'Louie,Louie oh no,Me gotta go'

String literals

There is a new (as of Python 3.6) method of formatting strings known as formatted string literals. By adding an “ f ” before strings, placeholder variables can become populated by variables without using the format function.

In this example, the variables are formatted directly into the string literal, which has an “ f ” before the string to indicate that it is a string literal:

>>> year = 1980

>>> day = "Monday"

>>> month = "Feb"

>>> str_lit = f"It was a cold {day} in {month} {year}"

>>> str_lit

'It was a cold Monday in Feb 1980'

Read more about string formatting here: https://realpython.com/python-string-formatting/

String manipulation

String manipulation is common and lots of tools are built into the String data type. These allow you to replace characters in a string or find their index location in the string.

Find and index are similar methods but find is able to be used in conditional statements. If the character is not found in the string, find will return -1, while index will return an error.

The join method is used to join together a list of string data. The split method is the opposite: it splits a string into a list based on a supplied character or the default empty space.

Method

Example

join

string_list = [‘101 N Main St’,’Eureka’,’Illinois 60133’]

address = ‘, ’.join(string_list)

replace

address = ‘101 N Main St’.replace(“St”,”Street”)

find, rfind

str_var = ‘rare’

str_index = str_var.find(‘a’) #index 1

str_index = str_var.find(‘r’) #index 0

str_index = str_var.rfind(‘r’) #index 2

upper, lower, title

name = “Laura”

name_upper = name.upper()

name_lower = name.lower()

name_title = name_lower.title()

index, rindex

str_var = ‘rare’

str_index = str_var.index(‘a’) #index 1

str_index = str_var.index(‘r’) #index 0

str_index = str_var.rindex(‘r’) #index 2

str_var.index(‘t’) #this will cause an error

split

latitude,longitude = “45.123,-95.321”.split(“,”)

address_split = ‘101 N Main St’.split()

String indexing

String indexing is similar to list indexing, as explained above. Individual characters, or groups of characters, can be selected from a string by passing the index of the character needed to the string in square brackets.

In this example, the “ d ” from “ readiness ” is accessed by passing the index [3] to square brackets next to the string:

>>> str_var = "readiness"

>>> d_var = str_var[3]

>>> d_var

'd'

Groups of characters are selected by passing a start and end index, where the end index is the index of the first character you do not want to include:

>>> str_var = "readiness"

>>> din_var = str_var[3:6]. #index 6 is e

>>> din_var

'din'

>>> dine_var = str_var[3:7]. #index 7 is s

>>> dine_var

'dine'

Integers

The Integer data type represents whole numbers. It can be used to perform addition, subtraction, multiplication, and division (with one caveat as noted below).

>>> int_var = 50

>>> int_var * 5

250

>>> int_var / 5

10.0

>>> int_var ** 2

2500

Convert a string to an integer

To convert a string (or a float) to an integer, use the  int  function:

>>> x = '0'

>>> y = int(x)

>>> y

0

>>> type(y)

<type 'int'>

>>> type(x)

<type 'str'>

Integer math issue in Python 2

A well-known and well-intentioned design issue in Python 2 is the integer division issue. It means that performing division math with integers will result in a (usually) unwanted result where no remainder is returned. It is encouraged to convert integers into floats before dividing.

Here is an example of the issue:

Python 2.7.16 (default, Dec 21 2020, 23:00:36)

>>> 5/3

1

This issue has been fixed in Python 3:

Python 3.8.2 (default, Apr  8 2021, 23:19:18)

>>> 5/3

1.6666666666666667

Read more about integers in Python here: https://realpython.com/python-numbers/

Floating Numbers

Floating point numbers in Python are used to represent real numbers as 64-bit double-precision values. Sometimes using binary systems to represent decimal based numbers can be a bit odd, so keep an eye out, but in general these will work as expected.

>>> x = 5.0

>>> x * 5

25.0

>>> x ** 5

3125.0

>>> x/2.3

2.173913043478261

Convert a string to a float

To convert a string (or an integer) to a float, use the  float  function:

>>> x = '5'

>>> y = float(x)

>>> type(y)

<type 'float'>

Read more about floating point numbers in Python here: https://www.geeksforgeeks.org/python-float-type-and-its-methods

Conversion between data types

Conversion between data types is possible in Python using built-in functions that are part of the standard library. To start, the type function is useful to find the data type of an object. Once identified, the data object can be converted from Integer ( int  function) to String ( str  function) to Float ( float  function), as long as the character would be valid in that data type.

In these examples, a character is converted from String to Integer to Float to String using the  int  and  str  and  float  functions:

>>> str_var = "5"

>>> int_var = int(str_var)

>>> int_var

5

>>> float_var = float(int_var)

>>> float_var

5.0

>>> str_var = str(float_var)

>>> type(str_var)

'<class 'str'>'

Data Structures or Containers

Data structures, also called data containers and data collections, are special data types that can hold, in a retrievable order, any data item of any data types (including other data containers). Data containers are used to organized data items by index in tuples or lists, or by key:value pair in dictonaries.

lData retrieval from data containers

To get data out of data containers, square brackets are used to pass either indexes (lists and tuples) or keys (dictionaries). If there is more than one level of data container (i.e. one container contains another), first the data container inside is referenced using an index or key inside a first square bracket, and then the data inside the container is accessed using a second.

Data Container

Example

Tuple

tuple_var = (“blue”, 32,[5,7,2],’plod’,{‘name’:’magnus’})

plod_var = tuple_var[-2]

magnus_var = tuple_var[-1][‘name’]

List

list_var = [‘fast’,’times’,89,4.5,(3,8),{‘we’:’believe’}]

times_var = list_var[1]

Dictionary

dict_var = list_var[-1]

believe_var = list_var[-1][‘we’]

Tuples

Tuples are ordered lists that can hold any data type, even in the same tuple. They are immutable, meaning they cannot be altered, and data cannot be added to or removed from the tuple once it has been created. They have length and the built-in  len  function can be used to get the length of the tuple.

In Python they are declared by using round brackets () or the  tuple  function. Data is accessed using zero-based indexing by passing the index to square brackets next to the tuple.

In this example, a tuple is assigned to the variable name  tuple_var , and data is accessed using indexing:

>>> tuple_var = ("red",45,"left")

>>> type(tuple_var)

<class 'tuple'>

>>> ("red",45,"left")[0]

'red'

>>> tuple_var[0]

'red'

Read more about tuples in Python here: https://www.geeksforgeeks.org/python-tuples/

Lists

Lists (often called Arrays in other programming languages) are data containers that can hold any other type of data type, even in the same list, meaning they do not have to be only one data type. Lists can be altered after they are created. In Python they are declared by using square brackets [] or the  list  function. Data is accessed using zero-based indexing by passing the index to square brackets next to the list.

In this example, a list is assigned to the variable name  list_var , and data is accessed using indexing:

>>> list_var = ["blue",42,"right"]

>>> type(list_var)

<class 'list'>

>>> ["blue",42,"right"][0]

'blue'

>>> list_var[0]

'blue'

Read more about lists in Python here: https://www.geeksforgeeks.org/python-list/

Convert between lists and tuples

Lists can be copied into a new tuple object using the  tuple  function. Conversely, Tuples can be copied into a list data type using the  list  function. Technically this does not convert the original data item, but instead creates a copy of the data item in the new data type.

In this example, the list is copied into a tuple data type, and then the tuple is copied into a list data type. Note that the brackets change with each new data type created:

>>> tuple_copy = tuple(list_var)

>>> tuple_copy

('blue', 42, 'right', 'ankle')

>>> list_copy = list(tuple_copy)

>>> list_copy

['blue', 42, 'right', 'ankle']

List operations for both tuples and lists

Lists and tuples can be iterated using for loops. They can both be “sliced” as well, creating a subset of the list or tuple that will be operated on for the for loop or other operation.

Slicing

Slicing a list or tuple will create a new list or tuple. The slice is created by passing indexes to the list or tuple in square brackets, separated by a colon. The first index is the start index, and it can be ignored if it is index 0 (i.e. the beginning of the original list). The second index is the index of the first value that you do NOT want to include (it can be blank if it the rest of the original list).

In this example we see a tuple with three data items sliced to only include the first two items. The string “left” is at index 2 in the tuple, meaning that the last index in the slice will be 2. The slice is assigned to variable name tuple_slice:

>>> tuple_var = ("red",45,"left")

>>> tuple_slice = tuple_var[:2]

>>> tuple_slice

('red', 45)

In this example we see a list with four data items sliced to only include the last two items. The first index is the index of the first data item we want (the string “ right” ). The last index is blank:

>>> list_var = ["blue",42,"right","ankle"]

>>> list_slice = list_var[2:]

>>> list_slice

['right', 'ankle']

List operations for only lists

A list can be appended (one data item added) or extended (a list or tuple of data items are all added to the main list). The list order can be reversed or sorted. Built-in functions allow for the calculation of the maximum or minimum value of a list or even the sum of a list (given the data type of the items in the list is correct).

Sets

Sets represent a collection of distinct objects. In Python, sets are unordered, no duplicates are allowed, and all data items inside a set must be immutable.

Set operations

Sets are especially useful for getting all distinct members of a list. They cannot be accessed using indexing (they are unordered) but they can be iterated:

>>> orig_list = ["blue","pink","yellow","red","blue","yellow" ]

>>> set_var = set(orig_list)

>>> set_var

{'pink', 'yellow', 'blue', 'red'}

>>> set_var[0]

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

TypeError: 'set' object is not subscriptable

>>> for item in set_var:

...     print(item)

...

pink

yellow

blue

red

Dictionaries

Dictionaries are key:value stores, mean they are data containers that use unordered key and value pairs to organize data. Keys are used as reference points for organization and retrieval. When a key is supplied to a dictionary in square brackets, the value is returned.

>>> dict_var = {"key":"value"}

>>> dict_var['key']

'value'

>>> dict_var = {"address":"123 Main St", "color":"blue"}

>>> dict_var["address"]

'123 Main St'

>>> dict_var["color"]

'blue'

Read more about dictionaries in Python here: https://www.geeksforgeeks.org/python-dictionary/

Keys and values

Keys can be any immutable data type (meaning lists cannot be used as keys, but strings and integers and floats and tuples can be used as keys. Values can be any type of data, including other dictionaries.

All keys in a dictionary can be accessed as a list using the dictionary  keys  function. In Python 2.x this is a list. In Python 3.x it is a generator.

All values in a dictionary can be accessed as a list using the dictionary values function. In Python 2.x this is a list. In Python 3.x it is a generator.

Functions

Functions are sub routines defined by code. When “called” or run, functions will do something (or nothing if written that way). Functions often accept parameters, and these can be required or optional.

Functions make it easy to perform the same action over and over without writing the same code over and over. This makes code cleaner, shorter and smarter. They are a good idea and should be used often.

Read more about functions here: https://realpython.com/defining-your-own-python-function/

Def keyword

Functions are defined using the “ def ” keyword, which is short for “define function”. The keyword is written, and then the name of the function and round brackets (), into which expected parameters can be defined.

Return statement

Functions allow for data to be returned from the subroutine to the main loop using  return  statements. These allow the user to calculate a value or perform some action in the function, and then return back a value to the main loop.

Parameters

Parameters or arguments are values expected by functions and supplied by the code at runtime.

Namespaces

In Python, there is a concept called namespaces. These are refined into two types of namespaces: global and local.

All variables defined in the main part of a script (outside of any functions) are considered to be in the global namespace. Within the function, variables have a different “namespace”, meaning that variables inside a function are in a  local   namespace  and are not the same as variables in the main script, which are in the  global   namespace . If a variable name inside a function is the same as one outside of the function, changing values inside the function (in the local namespace) will not affect the variable outside the function (in the global namespace)

Function Examples

In this example, a function is defined and written to return “hello world” every time it is called. There are no parameters, but the  return  keyword is used:

def new_function():

return "hello world"

In this example, an expected parameter is defined in the brackets. When called, this value is supplied and the function then returns the value from the local namespace back to the global namespace in the main loop:

def accept_param(value):

return value

In this example an expected parameter has a default value assigned, meaning it only has to be supplied if the function uses a non-default parameter:

def accept_param(value=12):

return value

Doc strings

Functions allow for a string after the definition line that is used to declare the purpose of the function for documentation purposes.

def accept_param(value=12):

'this function accepts a parameter if different from default'

return value

Classes

Classes are special blocks of code that organize multiple variables and functions into an object with its own methods and functions. Classes make it easy to create code tools that can reference the same internal data lists and functions. The internal functions and variables are able to communicate across the class, so that variables defined in one part of the class are available in another.

Classes use the idea of “self” to allow for the different parts of the class to communicate. By introducing self as a parameter into each function inside a class, the data can be called.

Classes are called or “instantiated” to create a class object. This means the class definition is kind of like a factory for that class, and when you want one of those class objects, you call the class type and pass the correct parameters if required.

class Object():

def __init__(self, name):

'accepts a string'

self.name = name

def get_name(self):

'return the name'

return self.get_name

Read more about classes here: https://www.geeksforgeeks.org/python-classes-and-objects/

Installing and importing modules

To extend the capabilities of the included standard Python library of modules, Python was built to be extensible. Third-party modules are downloaded in some format from a provider (often PyPI, the Python Package Index, where most are held) using either the built-in  pip  program or another method. For us modules such as  arcpy  and the ArcGIS API for Python are perfect examples: they extend the capabilities of Python to be able to control the tools that are available within ArcGIS Desktop or Pro respectively.

Using pip

To make Python module installation easier, Python is now installed with a program called pip. This name is an recursive acronym which stands for Pip Installs Programs. It simplifies installation by allowing for one line command line calls both locates the requested module on an online repository and runs the installation commands.

Pip connects to the Python Package Index (or PyPI). Stored on this repository are hundreds of thousands of free modules written by other developers. It is worth checking the license of the module to confirm that it will allow for your use of its code.

Pip lives in the Scripts folder, where lots of executable files are stored:

Installing modules

We will cover the

The setup.py file

Often Python 2.x and sometimes in Python 3.x a module is includes a “ setup.py ” file. This file is not run by  pip ; instead, it is run by Python itself.

Usually, a module will have a downloadable  zip  file that should be copied to the  /sites/packages  folder. This should be unzipped, and then the Python executable should be used to run the  setup.py  file using the  install  command:  python setup.py install

Installing in virtual environments

Virtual environments are a bit of an odd concept at first, but they are extremely useful when programming in Python. Because you will probably have two different Python versions installed on your computer if you have ArcGIS Desktop and ArcGIS Pro, it is convenient to have these versions located in a virtual environment.

The core idea is to use one of the Python virtual environment modules to create a copy of your preferred Python version, which is then isolated from the rest of the Python versions on your machine. This avoids path issues when calling modules, allowing you to have more than one version of these important modules on the same computer.

Here are a few of the Python virtual environment modules:

Name

Description

Example virtual environment creation

venv

Built into Python 3.3+

python3 -m venv

virtualenv

Most be installed separately. It is very useful and my personal favorite.

virtualenv namenv --python=python3.6

pyenv

Used to isolate Python versions for testing purposes. Must be installed separately.

pyenv install 3.7.7

Conda  /Anaconda

Used often in academic and scientific environments. Must be installed separately.

conda create --name snakes python=3.9

Read more about virtual environments here: https://towardsdatascience.com/python-environment-101-1d68bda3094d

Importing modules

To access the wide number of modules in the Python standard library, as well as third-party modules such as  arcpy , we need to be able to import these modules in our script (or in the interpreter).

To do this you will use import statements. These declare the module or submodules (smaller components of the module) that you will use in the script, and as long as the modules are in the  /sites/packages  folder in your Python installation, or in the PATH (as  arcpy  is after its been installed).

import csv

from datetime import timedelta

from arcpy import da.SearchCursor

Three ways to import

There are three different and related ways to import modules. These modules, from either the standard library or from third-parties, are all imported the same in a script.

Method 1: import the whole module

This is the simplest way to import a module, by importing its top-level object. Its sub-methods are accessed using dot notation (e.g.  csv.Reader , a method used to read CSV files):

import csv

reader = csv.Reader

Method 2: import a sub module

Instead of importing a top-level object, you can import only the module or method you need, using the “ from X import Y ” format:

from datetime import timedelta

from arcpy import da.SearchCursor

Method 3: import all sub modules

Instead of importing one sub-object, you can import all the modules or methods, using the “ from X import * ” format:

from datetime import *

from arcpy import *

Read more about importing modules here: https://realpython.com/python-import/

Importing custom code

Modules don’t have to just come from “third-parties”: they can come from you as well. With the use of the special  __init__.py  file, you can convert a normal folder into an importable module

The __init__.py file

This special file, which can contain code but mostly is just an empty file, indicates to Python that a folder is a module that can be imported into a script. The file itself is just a text file with a  .py  extension and the name  __init__.py  (that’s two underscores on each side), which is placed inside a folder. As long as the folder with the  __init__.py  is either next to the script or in the Python Path (e.g. in the site-packages folder), the code inside the folder can be imported.

Example custom module

In this example, we see some code in a script called example_module.py:

import csv

from datetime import timedelta

print('script imported')

Create a folder called  mod_test . Copy this script into the folder. Then, create an empty text file called  __init__.py :

Import your module

Create a new script next to the  mod_test  folder. Call it “ module_import.py ”:

Inside the script you will import the function “ test_function ” from the  example_module  script in the  mod_test  folder using the format below:

Scripts inside the module are accessed using dot notation (e.g.  mod_test.example_module ). The functions and classes inside the script called  example_module.py  are able to be imported by name.

Because the module is sitting next to the script that is importing the function, this import statement will work. But if you move your script and don’t copy the module to somewhere that is on the Python Path, it won’t be a successful import

That is because the way import statements work is based on the Python Path. This is a list of folder locations that Python will look for the module that you are requesting. By default, the first location is the local folder, meaning the folder containing your script. The next location is the site-packages folder.

The site-packages folder

Most modules are installed in a folder inside the Python folder. This is called the site-packages folder and it sits at  */Lib/sites-packages .

To make your module available to for import without needing it to be next to your script, put your module folder in the site-packages folder. When you run “ from mod_test.example_module import test_function ” it will locate the module called mod_test in the sites packages folder.

Basic style tips for writing scripts

To make clean, readable code, it is encouraged to follow these basic tips about how the code should be written and organized. The major rule enforced by Python is the required indentation, which is intended to make the code easier to read and write.

Read more about Python code style here: https://realpython.com/python-pep8/

Indentation

Python code has strict indentation rules that are enforced by all IDEs. These rules relate to functions and loops especially.

As a standard, 4 spaces are used after a function is declared or a loop is created. This is just a standard, as it could be only one space, but that indentation level becomes important when scripts get big and it helps to have 4 spaces for all indented lines so that they can be more easily read.

It is encouraged to not mix tabs and spaces when indenting.

Read more about indentation here: https://www.python.org/dev/peps/pep-0008/ - indentation

Add a comment at the top with script details

This is an optional but recommended way to start your scripts: write a comment at the top with your name, the date, and some quick explanation about what the script is supposed to do. This is especially nice when other people have to read your code.

Add lots of other comments throughout the script as well, to make sure you know what is happening throughout the script.

Follow with Import statements

It is encouraged but not required to put the import statements at or near the top of the script. Imports must happen before the module objects are called in the script, but the import statements can be placed anywhere. It is best to put them at the top so that people reading the script can understand what is being imported.

Define global variables

After the import statements, define the necessary variables that will be used in this script. Sometimes it is necessary to define variables later in the script but it is best to put major variables near the top.

Define functions

By placing function definitions below the global variables, it is easy to read and understand what the functions do when reading them. It is sometimes hard to find a function that is called in another part of the script if the function is not in a known location in the script.

Include print statements

The built-in function called print is used to send messages from the script to the command window while the script is running. Pass any valid data to the print statement and use it to track progress or to debug if there are issues.

>>> print("blueberry")

blueberry

>>> x = 0

>>> print(x)

0

Read more about print statements here: https://realpython.com/python-print/

Write the executable parts of the script

After importing modules and defining functions, the next part of the script is where the action takes place. The for loops are run, the functions are called, and the script is then done.

Make sure to add lots of comments to help yourself understand what is happening throughout the script, and print statements as well to help while the script is running.

If __name__ == ‘__main__’

Often at the end of scripts you will see this line,  if __name__ == “__main__” . What it means is that the indented code below this line will be run if the script is executed directly, but if the code in the script is imported by another script, the code will not execute until called in the second script.

Read more about this here: https://www.geeksforgeeks.org/what-does-the-if-__name__-__main__-do/

Summary

In this chapter, we did a fast but comprehensive overview of computer programming and the Python programming language. We reviewed the basics of computer programming, including variables and iteration and conditionals. We reviewed the Windows Path environment variable and the Python system path. We explored the data types of Python, including Integers and Strings and Float, and the data containers of Python such as lists and tuples and dictionaries. We learned some basic code structure for scripts, and how to execute those scripts.

In the next chapter we will discuss the basics of  arcpy  and the ArcGIS API for Python. We will learn how to import these modules and access their methods and submodules. We will begin to execute Python code to automate ArcGIS Desktop and ArcGIS Pro.

Basics of ArcPy

Now that you have an understanding of python syntax you can start working with the ArcPy package. ArcPy is the Python package provided by ArcGIS to perform and automate geoprocessing and map production tools tasks. In addition to the geoprocessing tools available in ArcGIS you have access to additional modules, functions, and classes. When combined you can create workflows and standalone tools that simplify and automate complex analysis and map production.

This chapter will cover:

- Ensuring your Python Environment is set up for ArcPy

- ArcPy Tools and how to use them in ArcGIS Pro

- Accessing environment settings in ArcPy

- Functions in ArcPy

- ArcPy Modules

Checking your ArcPy Install

The ArcPy package allows you access to the geoprocessing function of ArcGIS Pro. Python packages contain multiple modules, functions, and classes. A package is set up with a hierarchical structure and uses dot notation for the modules and functions.

ArcPy is installed with ArcGIS for Pro and ArcGIS Desktop. ArcPy has been used to write Python scripts in ArcGIS since ArcGIS 10.0. It is the official ArcGIS scripting language and can be used to automate analysis and map production workflows. ArcGIS desktop uses Python 2.7, which is currently up to release 2.7.18. ArcGIS Pro uses a new version of Python, Python 3. The most recent release of Python 3 is Python 3.9.5. You can check what version of Python you have installed by going to the Project>Python>Installed Packages and finding Python.

To use the ArcPy package it must be imported. Most scripts start with import statements to allow access to all the modules that are part of the package. To import ArcPy use the following line of code:

import arcpy

Importing ArcPy gives access to all of the geoprocessing tools and the modules included.

Some of the modules are:

- Arcpy.sa (Spatial Anlayst)

- Arcpy.geocoding (Geocoding)

- Arcpy.na (Network Analyst)

- Arcpy.da (Data Access)

- Arcoy.mp (Mapping)

Later chapters will take a closer at the data access and mapping modules. Both of those are very powerful and can automate work.

Python IDLE Shell for ArcGIS Pro Install

Since Python and ArcPy are installed with ArcGIS Pro, if you are using the IDLE Shell for the version of Python installed with ArcGIS Pro you will not get an error when importing ArcPy. If you are using a different Python interpreter you will have to set it up to find the ArcPy module. Most of the time you will be writing scripts either directly into ArcGIS Pro’s Python window, an ArcGIS Notebook in ArcGIS Pro, or to turn into script tools, so it is easiest to use the Python IDLE that comes with the ArcGIS Pro install.

The easiest way to ensure accessing the Python IDLE that comes with the install of ArcGIS Pro is to create a shortcut, as one is not created on install.

- Find the path to run IDLE, for a typical ArcGIS Pro installation it is here:  C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\idlelib\idle.bat  Double clicking on that will open IDLE:

- To create a shortcut, right click on your desktop and click New>Shortcut and paste the full path of the idle.bat file:

- Click Next, and then give your shortcut name:

I suggest using a name that will allow you to remember this is the Python environment that was installed with ArcGIS Pro.

- The icon will be the default shortcut icon

.

- To change the icon to the standard Python IDLE Icon:

- Right click on it

- Click Properties.

- On the Shortcut tab click on Change Icon.

You may get an alert that there are no icons and that you need to choose an icon from a different file. If so click okay so you can navigate to the icon location.

- Navigate to the Python IDLE location, which should be located here:  C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\idlelib\Icons

- A shortcut to access the IDLE install for ArcGIS Pro is now installed on the desktop. This install is a place to test snippets of code when working on complex scripts tools. Something that will be explored in later chapters.

When you want to use an IDLE to work with ArcGIS Pro you need to use this one as it is associated with your install of ArcGIS Pro.

A good way to ensure the correct IDLE is being used is to import arcpy. If three carets >>> will be displayed after the install was successful

If not an error will be displayed:

The Python IDLE Shell

The Python IDLE Shell is a good place to try out code as it is interactive and will display immediate results to code:

The IDLE Shell also shows how elements of the code will be interpreted using different colors. Strings are shown in green, functions are in purple, loops and condition statements are in orange, and results are in blue.

While it is useful to get immediate results from IDLE Shell it is not meant to be used for saving code. It is possible to copy it out if needed, but it is better practice to write it into a script file for saving.

To start a script file you click in the menu bar of the IDLE Shell on File>New File. This opens a new window that is an empty Python script file called Untitled. Unlike the IDLE Shell it doesn’t have a command prompt and the menu bar is different.

You can create a script file

- Copy the previous code you wrote in IDLE into the script file

- Remove the carets

- Fix the indentation to match below.

- Save the file.

This new file  HelloLoop.py  has  .py  extension signifying that it is a Python file. It can be run by clicking Run > Run Module. Sending the results to the Python IDLE Shell.

Now that you have an understanding of

- How to ensure your Python environment is set up to work with ArcPy

- How to use the IDLE Shell

- How to start a new script file

You are now ready to look at the Python window in ArcGIS Pro and what you can do there.

Python Window in ArcGIS Pro

ArcPy can be accessed not just through IDLE but by using the Python window in ArcGIS Pro.

The Python window allows you to write and run code directly in ArcGIS Pro and see the results of any geoprocessing tool as you run them. This can be an advantage when testing out new code to see what it is doing and how it is working. Code written in the Python window can then be copied or saved into large script tools. You will learn more about script tools in later chapters. For now let’s look at the Python window in ArcGIS Pro and see how powerful it can be.

Once you click on the icon the Python window will pop up, usually the first time it will be docked at the bottom of your screen.

Like all windows in ArcGIS Pro the Python window can be docked anywhere or can be floating. You can move it to where it is best for you to work using the same process of dragging and hiding as you would any ArcGIS Pro window.

The top part of the window is called the transcript and is where code you have previously written is. The bottom where it says Enter Python code here is called the prompt and is where you would type code into. When you first open the Python window the transcript is blank as you haven’t written any code yet.

Try some of the code you wrote in IDLE to see how it works in the Python window. Just like in IDLE when you enter a line of code you need to press Enter.

- Type in  x = 10 a nd Press Enter

- Type in  y = 3  and Press Enter

- Type  x + y a nd press Enter

You can see that this is working just like IDLE. This means that all standard Python functions and tools will work in the Python window the same as they would in the IDLE Shell.

The transcript can be cleared at any time by right clicking in the transcript box and selecting Clear Transcript. This does not remove your code or your variables from memory.

- Click in the transcript and select Clear Transcript.

- Type  x + y  And press Enter

As you can see the variables for x and y were saved in memory and still usable even after clearing the transcript. These variables are even available if you save and close the project and open it again. They are saved into the memory of the project so they can be used again later on the same project. This can be useful but you will look at better ways to save code for reuse in the same and other projects in a later chapter.

Just like in the IDLE Shell the Python Window understands when you are writing multiple lines of connected code. You can see this by writing in the for the HelloLoop.py script you wrote earlier in this chapter.

- Type in  String = “Hello”  and press Enter

- Type in  i = 1  and press Enter

- Type in  while i < 5:  and press Enter

Notice that the prompt window gets bigger and your cursor is indented. The Python Window understands that while statement is starting a block of code and is part of a multiline construct. As you continue to type in your code the prompt window will get bigger as additional lines are needed. The if statement is part of a multiline construct so it too will get the additional lines and indent as the while statement does

- Type in  if i == 1:  and press Enter Enter

- Type in  print(string)  and press Enter

- Type in  else:  and press Enter

- Type in  print(i)  and press Enter

- Type in  i+=1  and press Enter

- Press Enter

When you are doing writing your multiple lines of code and hit Enter you will just move down another line

What happens if you forget the i += 1 and get an endless loop? How do you stop the loop?

You can either click the X in the bottom of the Python Window to stop the execution.

Or type Ctrl+C in the prompt window to stop the execution.

ArcPy Environment Settings

- ArcPy environment settings allow you access to general geoprocessing settings as well as the geoprocessing settings of a specific tool.

- For tools they act as parameters you can set to change the results of a tool.

- There are many that can be used but you will find yourself using a few of them commonly.

You are going to look at the most common one and see how to set it:  arcpy.env.workspace .

Using the workspace property of the environment class you can check and change your workspace or scratch workspace

arcpy.env.workspace

arcpy.env.scratchWorkspace

You can check and set your workspace

- Type in  arcpy.env.workspace  and press Enter

The return you see is your current workspace.

Now you can set your workspace

- Type in  arcpy.env.workspace = r"C:\PythonBook\Project_2\Project_2.gdb"  and press Enter

You can check your scratch workspace in a similar way

- Type in arcpy.env.workspace  and press Enter

Now you can set your scratch workspace

- Type in arcpy.env.scratchWorkspace = r”C:\PythonBook\Project_2\Project_2.gdb”  and press Enter

Setting your scratch workspace and workspace is a good idea as it allows you to have a default location to send data you are creating. It is also the workspace that will be used when using the List functions you will explore below. In the examples you have set your workspace and scratch workspace to geodatabases. You could set them to folder or a feature dataset or any workspace you wanted.

What is the r in front of the path for the geodatabase?

Note the way you type in the locations. It has an r followed by the location enclosed in a double quote. The r stands for “raw string” and means that Python will read everything within the quotes exactly as it is written. This is important because the \ character in Python is an escape character and can be used to insert characters otherwise not allowed in a string. You have already seen how to use the escape character in a query to get quotes within the quotes of a string. Here though you don’t want the escape character so there are 3 options.

Use an r in front of the quotes to create a raw string

Change all the single \ to double \

Change the single to \

There are many other environment settings that could be useful to you depending on the process you are running. Most of the settings that you find for a tool in the properties of a tool can be set in the environment settings. Things like analysis extent can be set with  arcpy.env.extent , or a snap raster when doing raster analysis with  arcpy.env.snapRaster .

It is important to remember that once you set an environment setting it stays set until you change it. You can change it and in more advanced script tools you will change it or have it set and reset throughout the code.

ArcPy Tools - Geoprocessing using ArcPy

Now that you have some of the basics of how to use the Python Window it is time to look at how to use geoprocessing tools.

- You will learn how to use select, buffer, make feature layer, and select by feature layer tool, add field, and calculate field tools in the Python window.

- The end result will be a feature class of bus stops within 1,000 feet of parks.

To do some geoprocessing you will need some data. Make sure you have a map open in ArcGIS Pro and add in the CPAD_2020b_Units.shp data from where you downloaded it to. This data is California Protected Areas Database data that shows parks and other protected areas throughout the state of California. For more information about the dataset please go to https://www.calands.org/.

You are going to use the Python window to select the a parks in Oakland, buffer them by 1,000 feet and select the bus stops within that 1,000 foot buffer, and write them to out to a new feature class.

Right click in the table of contents and select attribute table and see what the data shows.

1. You are going to use the AGENCY_NAME field to run a Select tool to create a new feature class of just the protected areas in the City of Oakland. You are going to do this in the Python Window by typing in the following:  arcpy.Se  The Python window shows you some autocomplete options to help you find the tool you want. You are using the Select tool from the Analysis Toolset so you want Select() analysis.

2. After selecting the tool you can see what parameters the tool expects. Hover on the tool to get a help window popping up showing the tool parameters. The select tool takes the following mandatory parameters

1. in_features

2. out_features,

- And the following optional parameter

3. where_clause . The where clause is in curly braces {} because it is optional.

- The  in_features  is bold because it is the parameter the tool is currently looking to be entered. Type in the following in between the parenthesis to create the selection query  “CPAD_2020b_Units”,”CPAD_2020b_units_Oakland”,’”AGNCY_NAME = \’Oakland, City of\’  And press Enter

After running it will look like this and you should have a new feature class that is just the protected areas in Oakland.

How to write the query in the where clause so it works? Using the escape character “\” properly

The \ marks are escape characters that are necessary when you need to use multiple single or double quotes. In this instance since you are running a selection query on a shapefile the attribute field being selected on needs double quotes and the string value needs single quotes. The entire where clause needs to be inside a single or double quote. So the easiest option is wrap the entire query in a single quote and use the escape clauses around the string being selected. If you wanted to wrap the entire where clause in double quotes it would look like this  “/”AGNCY_NAME” = ‘Oakland, City of’”  Both will work the same.

If you are working in an ArcGIS Pro project then a new file will be created in that project’s geodatabase as that is the default workspace. If you have set a workspace through the environment settings it will write to that workspace. If you are not then it will be stored in a temp space and not written to disk.

How do you specify a different workspace if you don’t want to use the default workspace?

To specify a different location you need to write the full path when saving. To write out a shapefile to the  MyProject  folder you would write the following  arcpy.analysis.Select(“CPAD_2020b_Units”,r”C:\PythonBook\MyProject\CPAD_2020b_Units_Oakland.shp”,’”AGNCY_Name” = \’Oakland, City of\’’)

3. Now you can take the selected parks and buffer them by 1,000 feet. The buffer tool is in the analysis toolbox so the call it you type  arcpy.analysis.Buffer()  You can see the buffer tool parameters the buffer tool takes by hovering in the parenthesis. It takes the following mandatory parameters in this order

1. in_features

2. out_features

3. buffer_distance_or_field

And the following optional parameters in this order

1. line_side

2. line_end_type

3. dissolve_option

4. dissolve_field

5. method

You want to have a buffer of 1,000 feet for the parks, dissolved to just the  UNIT_NAME ,  AGENCY_NAME , and  LABEL_NAME . To get this you will be typing in the  in_features ,  out_featuers ,  buffer_distance ,  dissolve_option , and  dissolve_field paramaters . The  in_features ,  out_features , and  buffer_distance  are the first 3 parameters, but the  dissolve_option  and  dissolve_field  are the 6th and 7th parameters. To make sure they are in those positions you will type a pair of single or double quotes in the 4th and 5th parameters. This signifies to the function that those optional parameters are blank, just as if they weren’t entered; and allows you to enter parameters after them. Type in  “CPAD_2020b_Units_Oakland”,”CPAD_2020b_Units_Oakland_1000ft”,”1000 FEET”, “”,””,”LIST”,[“UNIT_NAME”,”AGNCY_NAME”,”LABEL_NAME”]  And press Enter

4. The buffers should have been added to your map. You can explore them and see what they look like. When you are ready you are going to now make a feature layer of the bus stops feature class with the Make Feature Layer Tool type in  arcpy.management.MakeFeatureLayer().  You can see that the Make Feature Layer tool takes two mandatory parameters

1. in_features

2. out_layer

And the following optional parameters

1. where_clause

2. workspace

3. field_info

5. You are going to enter the following parameters to make a feature layer of all the bus stops by typing in the following inside the parenthesis.  “UniqueStops_Summer21”,”AC_TransitStops_Summer21”  And press Enter

6. The feature layer should have been added to your map. You can explore it and see that it is just like the UniqueStops_Summer21 feature class. But because it is a feature layer you can use the Select Layer By Location tool to select all of the bus stops within the buffer. Type in  arcpy.management.SelectLayerByLocation() . You can see that the Select Layer By Location tool takes one mandatory parameter

1. in_layer

And the following optional parameters

1. overlap_type

2. select_features

3. search_distance

4. selection_type

5. invert_spatial_relationship

7. You are going to enter the following parameters to select the bus stops within the 1,000 foot buffer by typing in the following inside the parenthesis.  “AC_TransitStops_Summer21”,”INTERSECT”,”CPAD_2020b_Units_Oakland_1000ft”  And press Enter

8. You should see the bus stops get selected. You can explore the data and see if this is what you were looking for. From here you can export your data to table, csv, or feature class. Or just use it for a map display. Right now you are going to export to a feature class for later use when you explore the data access module and using cursors. To do this you will use the Copy Features tool. Type in  arcpy.management.CopyFeatures() . The Copy Features tool has two mandatory parameters

1. in_features

2. out_features

And one optional parameter

1. config_keyword

9. You are going to enter the following parameters in the parenthesis to copy the selected bus stops out to a new feature class  “AC_TransitStops_Summer21”, “AC_TransitStops_Within1000ft_OaklandPark”

Your resulting feature class is displayed in your map and written to your current workspace. The next steps with this data will be explored in Chapter 4: Data Access and Rows. In there you will learn how to do this process all in memory and add the park names to the bus stops.

Built in ArcPy Functions

ArcPy has many built in functions to help with geoprocessing. ArcPy functions look like geoprocessing tools in the way they are written. When you wrote the code to create a selection feature class you wrote  arcpy.analysis.select(in_features,out_features,{where_clause} ). By encasing the in features, out features, and where clause in parenthesis you were calling the function by passing to it those parameters. That is all a function is, code that you can call later by passing different parameters to it.

ArcPy has functions to assist with things such as the environment settings, describing data, licensing, ArcGIS Online, raster, listing data, along with functions for specific modules like the spatial analyst of mapping modules. In this section you will explore two of the more commonly used built in functions

- Describe function

- List functions

These are common because they help you in setting up and completing iterative processes like doing the same analysis on different feature classes in one location.

Describe Function

The describe function will return different properties depending on what type of element it is. The describe function can be called on a variety of elements, including but not limited to: shapefiles, geodatabases, feature classes, feature datasets, tables, LAS files, rasters and map documents.

The describe function returns an object with all of the properties so you need to create a variable to hold those properties and then call them later. You can try this on CPAD data.

- Type  desc = arcpy.Describe(r”C:\PythonBook\cpad_2020b\CPAD_2020b\CPAD_2020b_Units.shp  And press Enter

- It looks like nothing has happened but now you can use that desc variable to get information about the shapefile. You can see is what data type this is by typing  desc.dataType  and press Enter

- You can also see what type of geometry a feature class has by typing in  desc.shapeType  and press Enter

You can see that if you don’t know anything about a file you can call a describe function on it and use the properties to find out information about the file.

In the sample above you know the data is a shapefile you can see how this information could be useful if you were searching through a folder and only wanted to run an analysis on the shapefiles.

List Functions

List functions are very powerful as they allow you to create a list of data in a workspace, which you can then iterate over. For these examples you are going to use the data in the  MyProject.gdb  geodatabase.

List functions take the current workspace you are in and will create a list of all the datasets for that type of list function. There are the following list functions

- ListDatasets

- ListFeatureClasses

- ListFields

- ListFiles

- ListIndexes

- ListRasters

- ListTables

- ListVersions

- ListWorkspace

ListDatasets ,  ListFeatureClasses ,  ListFiles ,  ListRasters ,  ListTables , and  ListWorkspaces  need to have the workspace set before being run as they will only run on the current workspace.

- Start by listing the workspaces that your  MyProject.gdb  is in. You need to first set your workspace to the location of the geodatabase.  arcpy.env.workspace = r”C:\\PythonBook\MyProject”  and press Enter

- Next assign the ListWorkspaces function to a variable called wksp,  wksp = arcpy.ListWorkspaces()  and press Enter

The  ListWorkspaces  function has two optional parameters,

Wildcard - used to limit the values returned to the list to those that match what you put in.

Workspace Type - used to limit the types of workspace.

- You can see what that looks like by just typing in  wksp  and press Enter

- You can see it is all of the workspaces that are standard when creating a new project in ArcGIS Pro. They are a bit hard to read in that list. So to make them easier to read iterate through the list printing out each one. Type in  for w in wksp:  and press Enter  print(w)  and press Enter and press Enter

Now you can really read what you have

This is great as you can see all the workspace in the folder. But you just want the geodatabases in the folder. This is where you can use the parameters. For this you can use the workspace type parameter. The workspace parameter takes the following values as strings

- Access - personal geodatabases will be selected

- Coverage - coverage workspaces will be selected

- FileGDB - file geodatabases will be selected

- Folder - shapefile workspaces will be selected

- SDE - enterprise databases will be selected

- All - all workspaces - this is the default

To select just the file geodatabase you need to write this.

wksp = arcpy.ListWorkspaces("","FileGDB")

Why is there a “” followed by a ,?

The first parameter is for the wildcard and writing “” will leave it blank. That needs to be there though as functions take the parameter in the order they are written in. If you write it like this  wksp = arpcy.ListWorkspaces(“FileGDB”)  the function will still run. But when you call it you won’t have any data in the list because there is no workspace called “FileGDB”

You can see when calling the  wksp  variable you now have a list of just one value the  MyProject.gdb

While there is just the one value in the list it is still a list and acts as such in Python. That means that functions in ArcPy that expect a string will fail if given a list. For example you cannot update the workspace to this geodatabase location by using the  wksp  variable

The elements of the list can be set as the workspace through by using different methods on the list. If you wanted to set each element of the list as a workspace you could do the following

Type  for w in wksp:  press Enter

Type  arcpy.env.workspace = w  press Enter

Type  print(arcpy.env.workspace)  press Enter press Enter

What if you know you only have a single workspace that you are targeting?

In this example you only have one item in the list so it only runs once. In these cases where you do only have one item you could just write this  w = wksp[0]

In fact when you know you only have one item in your list you can just write the following to set you workspace  arcpy.env.workspace = ListWorkspaces(“”,”FileGDB”)[0] Be careful with using this notation as if you have more than workspace you will only be setting the workspace to the first one on the list

Now that the workspace is set to your geodatabase you can use the ListFeatureClasses function to get a list of all the feature classes in the geodatabase and assign it to a variable.

You are going to write the code to get a list of feature classes and then write a for loop to iterate through the list so you can easily read what feature classes are in your list

- fcs = arcpy.ListFeatureClasses() press Enter

- for fc in fcs:  press Enter

- print(fc)  press Enter. press Enter

You now have a list that contains all the feature classes in your geodatabase. This list can be iterated through to give you a single feature class which you can run through other ArcPy functions or geoprocessing tools. You could use the Describe function from above to find only the feature classes of a certain geometry to make sure you only run your analysis on that.

You are starting with your list of feature classes stored in the variable fcs and will iterate through it like above when you just printed out the name. You will then determine use the ShapeType Property of feature classes to determine what the shape is of each feature class and print out a statement saying that.

The following code is what you will write into the Python window

for fc in fcs:

desc = arcpy.Describe(fc)

fcName = desc.name

if desc.shapeType == "Polygon":

print("Shape Type for " + fcName + " is " + desc.ShapeType)

elif desc.shapeType == "Polyline":

print("Shape Type for " + fcName + " is " + desc.ShapeType)

elif desc.ShapeType == "Point":

print("Shape Type for " + fcName + " is " + desc.ShapeType)

else:

print(fcName + " is not a Point, Line, or Polygon")

These are the steps to write it in

- for fc in fcs:  and press Enter

- desc = arcpy.Describe(fc)  and press Enter

- fcName = desc.name  and press Enter

- if desc.shapeType == “Polygon”:  and press Enter

- print(“Shape Type for “ + fcName + “ is “ + desc.ShapeType)  and press Enter

- elif desc.ShapeType == “Polyline”:  and press Enter r

- print(“Shape Type for “ + fcName + “ is “ + desc.ShapeType)  and press Enter

- elif desc.ShapeType == “Point”:  and press Enter

- print(“Shape Type for “ + fcName + “ is “ + desc.ShapeType)  and press Enter

- else:  and press Enter

- print(fcName + “ is not a Point, Line, or Polygon”)  and press Enter press Enter

The for loop will iterate through each feature class. For that feature class you are creating a variable to hold the Describe properties of that feature class. You then create a variable to hold the name of that feature class. Then you write if/elif/else statements to test the ShapeType property of the Describe object. The output statement will look like this.

Could the if/elif statement be condensed?

Yes it could be, depending on what you wanted. But if you don’t know what you are looking for sometimes it can be better to be specific. And now that you have this code written out you can save it and copy and paste certain sections. For example if you wanted to run separate geoprocessing tools on the point, lines, and polygons in your geodatabase you now have the sample if/elif statement to use.

Another way to select elements in the List functions before putting them into the list is to use the wildcard parameter. The wildcard limits what the function returns. It is not case sensitive and uses the * to include any number of characters before or after the *. Let’s look at some examples of this works using our current geodatabase workspace.

You can select all of the CPAD data

- type in  cpad_fcs = arcpy.ListFeatureClasses(“CPAD*”)  and press Enter

- View what is the list using a for loop. Type in  for fc in cpad_fcs:  and press Enter

- Type in  print(fc)  and press Enter and press Enter

The wildcard is finding any feature class that starts with CPAD the * then tells it to match anything after CPAD. You can see in the results that you now have a list that contains all the CPAD feature classes.

The * can be used after, before, or before and after any characters.

You could select just the CPAD units and super units by writing this

cpad_fcs_units = arcpy.ListFeatureClasses(“*Units”)

And verify using the for loop to print out

for fc in cpad_fcs_units:

print(fc)

And the wildcard could be used to select just the census feature classes with 2019 in them by writing this

census_fcs = arcpy.ListFeatureClasses(“*2019*”)

And verify using the for loop to print out

for fc in census_fcs:

print(fc)

The wildcard is one of the optional parameters in many of the list functions and can be used with the other parameters. The  ListFeatureClasses  function has a parameter for feature type, and feature dataset. The feature type parameter can be used to select specific feature types, including but not limited to: point, polygon, polyline. The feature dataset can be used to specify the feature dataset in a geodatabase that is to be searched for feature classes. If no feature dataset is specified the  ListFeatureClasses  function will only look for stand alone feature classes in the geodatabase.

Using your  MyProjects.gdb  you can see how to combine both the wildcard and feature type parameters to get a specific feature class.

You can select just the census polygon feature class by writing the following

census_fc_poly = arcpy.ListFeatureClasses(“*2019*”, “Polygon”)

And verify by entering the variable

census_fc_poly

Note that the feature class is stored within the [] as it is in a list. To do any geoprocessing tasks on this you either need to iterate through the list and do the tasks in the for loop or extract the feature class using the list index to grab whichever list index you need.

census_county = census_fc_poly[0]

Note that what the  census_county  variable returns is the name of the feature class. As long as your workspace is still the geodatabase you can use just that name to do geoprocessing tasks. But if you reset your workspace ArcPy won’t know where to find the feature class with that name. So it is good practice to use the  os  library to create a variable that contains the full path for your feature class. To use the  os  library it needs to be imported like ArcPy when working in IDLE.

- Type in  import os  and press Enter

- Now you can create a variable with the census feature class full path. Type in  gdb = wksp[0]  and press Enter

- Type in  census_county_full = os.path.join(gdb,census_county)  and press Enter

Now you have the full path for the census county shapefile in a variable that you can use throughout any further code you may write.

Introduction to ArcPy Modules

ArcPy comes with a set of modules in addition to the geoprocessing tools and functions. Modules are just files that contain additional Python definitions and statements, including things like functions and variables. They are used to help organize code more logically.

ArcPy comes with the following modules

- Charts module ( arcpy.charts )

- Data Access module ( arcpy.da )

- Geocoding module ( arcpy.geocoding )

- Image Analysis module ( arcpy.ia )

- Mapping module ( arcpy.mp )

- Metadata module ( arcpy.metadata )

- Network Analyst module ( arcpy.na  or  arcpy.nax )

- Sharing module ( arcpy.sharing )

- Spatial Analyst module ( arcpy.sa )

- Workflow Manager module ( arpcy.wmx )

Some of the above modules do require specific licenses to use the functions and tools within them. For example the network analyst module and spatial analyst module would require you to have network analyst and spatial analyst extensions available. The two you will look at in depth in later chapters, the data access module and the mapping module do not. The data access module can help you to streamline your data cleaning and analyses processes. The mapping module can streamline mass map production and make creating 100s of maps a simple process.

Spatial Analyst Module

The spatial analyst module contains all of the geoprocessing tools associated with the spatial analyst extension. Because it is uses the spatial analyst extension you need to import the extension.

from arcpy.sa import *

Just like with wild cards the star means everything. You could import just specific modules, classes, or functions from the spatial analyst module but most of the time you will be importing the entire module.

You will learn how to write the code to run spatial analyst tools in the Python window using the FVEG data from CalFire. The data is available here: https://frap.fire.ca.gov/mapping/gis-data/

- Load the  fveg15_1  data into the map

- Right click on it and select Symbology

- Click the drop down where it says Stretch and select Unique Values

- Click on the drop down for Field 1 and select WHR10Name. You can play with the colors and color scheme if you want. But also explore the data a little to see what is showing. It is a land cover dataset for the entire state of California. You are going to extract the data in the protected areas in the City of Oakland

- Type in  from arcpy.sa import *  and press Enter

- If you created the feature class for the parks in the City of Oakland you can skip the next step

- Type in  arcpy.analysis.Select(“CPAD_2020b_Units”,r”C:\PythonBook\MyProject\MyProject.gdb\CPAD_2020b_Units_Oakland”,’”AGNCY_NAME” = \’Oakland, City of\’’)  and press Enter

- Check if a spatial analyst extension is available by typing in  arcpy.CheckExtension(“Spatial”)  and press Enter. It should return  ‘Available’  if not you need to either enable your spatial analyst license or if on a shared license network have someone release theirs.

- Once you have confirmed a license is available type in  arcpy.CheckOutExtension(“Spatial”)  and press Enter. It should return  ‘Checked Out’

- Type in  oaklandParksLandCover = ExtractByMask(“fveg15_1”,“CPAD_2020b_Units_Oakland”)  and press Enter

- Type in  oaklandParksLandCover.save(r”C:\MyProject\MyProject.gdb\OaklandParksLandCover”)  and press Enter

Why do you have to save the raster? Could it be done in a single line of code?

When working with the spatial analyst extension in ArcPy and you run the geoprocessing tool it is actually a function. The function returns the extraction raster. That raster is created in memory, if you want to save you have to use the save property of that returned raster.

It could be done in one line of code. That would look like this  oaklandParksLandCover_2 = ExtractByMask("fveg15_1","CPAD_2020b_Units_Oakland").save(r"C:\PythonBook\MyProject\MyProject.gdb\OakalndParksLandCover_2")

Now that you have this data you can find just the areas of the parks that are not urban by using the Con tool.

- Start by giving your raster a name, type in  oaklandParksNonUrban =

- Call the Con tool by typing  con(

- The first parameter is the input raster, it is our newly created parks land cover, type in  oaklandParksLandCover,

- The next parameter is the true raster or constant, this is the value that the new raster will get. This is going to be the same raster, as you want to keep the different land covers that are not urban in the new raster, type in  oaklandParksLandCover,

- The next parameter is an optional parameter and it is the false raster or constant. This is what the new raster will get in areas that are urban. You want it to be No Data so that all the areas of parks that are urban will be blank in the new raster, type in  “”,

- The final parameter is an optional parameter and it is a where clause. This is written just like a query for running a select tool, type in  “WHR10NAME <> ‘Urban’”)

- The full code should look like this  oaklandParksNonUrban = Con(oaklandParksLandCover,oaklandParksLandCover,””,”WHRNAME <> ‘Urban’”)

- Open up the attribute table on the new raster in your table of contents. It only has a Value field and not the rest of the attributes. That is because the con tool just takes the Value attribute from the true raster. This can be fixed by doing a join.

- The Join Field tool is in the management tool box and works on rasters as well as feature classes and creates a permanent join. Type in  arcpy.management.JoinField(

- The first parameter is the input layer, type in  oaklandParksNonUrban,

- The second parameter is the field to join on, type in  VALUE,

- The third parameter is the table to be joined, type in  oaklandParksLandCover,

- The fourth parameter is the field to join on from the join table, type in  VALUE,

- The last parameter is optional and is the list of fields you want to join from the join table to the input layer. You can select the fields to join by using a list. If left blank it will join all of the fields. You are going to join the WHR fields, type in  [“WHRNUM”,”WHRNAME”,”WHRTYPE”,”WHR10NUM”,”WHR10NAME”,”WHR13NUM”,”WHR13NAME”]

- The full code should look like this  arcpy.management.JoinField(oaklandParksNonUrban,”VALUE”,oaklandParksLandCover,”VALUE”,[ [“WHRNUM”,”WHRNAME”,”WHRTYPE”,”WHR10NUM”,”WHR10NAME”,”WHR13NUM”,”WHR13NAME”])

- Again this needs to be saved to store it. Type in  (oaklandParksNonUrban.save(r”C:\MyProject\MyProject.gdb\OaklandParksLandCover”)  and press Enter

In the following chapters you will learn how to use search cursors to create lists of the different attributes and iterate through them to create multiple rasters and do further analysis on them.

ArcGIS API for Python

What is the ArcGIS API for Python?

The ArcGIS API for Python is a Python package designed to work with web GIS. It allows you to work directly with data hosted on ArcGIS Online or ArcGIS Enterprise. Previously you have been using ArcPy, which is excellent for desktop work, but has limited capabilities when working with hosted data. The ArcGIS API for Python provides tool to do many of the same functions that ArcPy does such as create maps, geocode, manage data, and geoprocessing but within your ArcGIS Online or ArcGIS Enterprise account. In addition to this you can manage your organizations web GIS data by managing users, groups, and items. It is important to note that while all of the examples you will work through are within ArcGIS Pro Notebooks, you don’t have to work through ArcGIS Pro. You could install a stand-alone environment with conda and access everything through a Jupyter Notebook environment. This book will not cover that as it is focused on working with Python within ArcGIS Pro.

The ArcGIS API for Python is like ArcPy in that it is a Python package. It contains classes, modules, and functions. But it is not just a Python package: it is also an application programming interface (API). An API is code that allows different applications and software to talk to each other. It interacts primarily with the ArcGIS REST API. This means that you can use the module to make requests of data hosted on ArcGIS Online or ArcGIS Enterprise. This data is either in your own organization or is publicly available. It is a pythonic API in that it is designed to Python standards and best practices. As a pythonic API it allows for python programmers to easily use ArcGIS, and ArcGIS users familiar with Python to automate web GIS tasks.

The API is organized into different modules for your use. Each module has different functions and types to assist in your GIS.

Modules:

- arcgis.gis  – this module is the one you will use the most. It allows for entry to the GIS and provides the functionality to create, read, update and delete GIS users, groups or content.

- arcgis.features  – this module works with feature data, feature layers, collections of feature layers, and feature sets. It contains spatial analysis functions for working with feature data. All feature data is data with a set of geometric representations.

- arcgis.raster  – this module works with raster data. It is contains classes and functions for working with raster and imagery data

- arcgis.realtime  – this module works with real-time data feeds. It is for use with streaming data to perform continuous analysis. It allows for Python scripts that can subscribe to data that is streamed, broadcast updates, or alerts.

- arcgis.network  – this module is for completing network analysis. It is for use on network layers and can be used to find best routes, closest facility, cost matrix, and service areas.

- arcgis.schematics  – this module is for working with schematics. Schematics are simplified networks. It is for explaining the structure and way schematics work.

- arcgis.geoanalytics  – this module is for creating distributed analysis of large datasets, feature and tabular. The tools are designed to work with big data as well with feature layers.

- arcgis.geocoding  – this module is for geocoding and reverse geocoding. It creates points of addresses with the output visualized on a map, or used as input data for spatial analysis.

- arcgis.geometry  – this module is for working with geometry types. It has functions that use geometry types as input and output, and for converting geometries to different representations.

- arcgis.geoprocessing  – this module is for creating methods to create and share geoprocessing tools. Users can create their own geoprocessing tools to share and this module provides tools that can assist with that.

- arcgis.geoenrichment  – this module is for providing data about an area or location. Users can get information about people and places in an area or within a distance. It can assist by easily providing demographic data for models.

- arcpy.env  – this module is for having a shared environment to be used by the different modules. It stores the currently active GIS and the environment settings.

- arcgis.mapping  – this module is for providing visualization capabilities for GIS data. It includes WebMap and WebScene to enable 2D and 3D visualization.

- arcgis.widgets  – this module is for providing visualization of GIS data. It includes the Jupyter Notebook MapView widget to assist with display of maps and layers

- arcgis.apps  – this module is for providing the ability to manage the web based applications available in ArcGIS.

In this chapter the focus will be mostly on the  arcgis.gis  module to manage the data in your organization, the  arcgis.features  module to work with that data, and the  arcgis.mapping  module to visualize the data.

What does it do? And why use it?

The ArcGIS API for Python allows you to access your data in ArcGIS Online without having to be in ArcGIS Online. You can manage your ArcGIS Online or ArcGIS Enterprise organization, its users, and its data from either a Jupyter Notebook or an ArcGIS Pro Notebook. By having the ability to do this in a Notebook and not through ArcGIS Online web interface you can use the full functionality of Python to iterate over data to run the same process multiple times, and schedule tasks to be run. The ArcGIS API for Python complements ArcPy as it allows you to automate your organization’s web GIS processes. Just like using ArcPy to automate a process in ArcGIS Pro you would use the ArcGIS API for Python when you need to automate a process on your ArcGIS Online or ArcGIS Enterprise organization

How to use it?

The ArcGIS API for Python comes with the install of ArcGIS Pro. If you are using ArcGIS Pro 2.5.x or later the conda package should have the arcgis package pre-installed. The default conda environment arcgispro-py3, which you have seen in the ArcPy chapter, includes the ArcGIS for Python API. If you need to upgrade your package you can do it through either the Python Package Manager or the Python Command Prompt. If you only have the default Python environment installed and have not created a new one or cloned it you cannot update it.

Default Python Package Install

- Python Package Manager upgrade process

- Open ArcGIS Pro with a blank Project

- Select Project Tab

- Select Python menu

- Click the Manage Environment

- Select the package you want to update (You cannot update the default install this way)

- Select Update Packages

- Choose the  arcgis  release from the list of packages with updates

- Python Command Prompt upgrade process

- Under the ArcGIS folder in the start menu click on Python Command Prompt

- Activate the environment to be upgraded

- Type in  conda upgrade -c esri arcgis

Python Package Manager with Cloned Environment to Upgrade

Once you have checked that you have an updated version of the  arcgis  package installed you can test to see if it is installed properly.

1. Go to Project>Open>Computer>Open another Project and navigate to where you downloaded Chapter5.aprx and open it.

2. Right click on Chapter5 folder and select New>Notebook. Rename the Notebook  TestArcGISAPI

3. In cell 1 type in the following

from arcgis.gis import GIS

gis = GIS()

map1 = gis.map("Oakland, California")

map1

4. Click Run to run the cell. You should see the following result.

Map created in ArcGIS Pro Notebook

Let’s take a look at each line of code to understand what it did:

1. from arcgis.gis import GIS  this imports GIS from the  arcgis.gis  module

2. gis = GIS()  this connects you to ArcGIS Online as an anonymous user.

3. map1 = gis.map(“Oakland, California”)  this is creating a map centered on Oakland, California

4. map1  this is printing the map to the output so you can see the map you created.

Connecting to the ArcGIS Online or ArcGIS Enterprise

There are different ways to connect to ArcGIS Online or ArcGIS Enterprise through constructing a GIS object. The GIS object takes three parameters, all of them optional.

1. url – this is the url for your organizations ArcGIS Online or ArcGIS Enterprise. If none is given it is ArcGIS Online (www.arcgis.com)

2. username – this is your username, if left blank you will be logged in as anonymous

3. password – this is your password, if left blank and a username is specified you will be prompted to input your password.

Logging into ArcGIS Online with a username but no password

Using this set up you can share your Notebook but not your login credentials.

Anonymous Users

In the previous example to test your ArcGIS API for Python install you connected to ArcGIS Online as an anonymous user. That gives you the ability to query and view publically available data. But it does not allow you to create or modify content or perform analysis.

Built-In Users

ArcGIS Online and ArcGIS Enterprise come with built-in identity store. This is done by writing the following code in the cell

gis = GIS(‘home’)

That will connect your Notebook into the ArcGIS Online account with credentials you are currently using to log into ArcGIS.

ArcGIS Pro Connection

Connecting through ArcGIS Pro using the pro authentication scheme. This is done by writing the following code in the cell:

gis = GIS(‘pro)

This will connect your Notebook to your ArcGIS Online portal using the credentials used to sign into ArcGIS Pro. This is similar to a built-in user. The difference is that the pro authentication scheme only works when ArcGIS Pro is installed locally and is running concurrently.

Summary of Connection Options

Anonymous User

gis = GIS()

Allows only access to query and view publically available data

Built-In User

gis = GIS(‘home’)

Connects to ArcGIS Online with account you are currently logged in with, and allows full access to that data based on your user assignment.

ArcGIS Pro

gis = GIS(‘pro’)

Connects to active portal when logged into the ArcGIS Pro app. This only works within ArcGIS Pro with a local install of ArcGIS Pro

Credentials

gis = GIS({url}, username = ‘example’)

Connects to a url, or  arcgis.com  if left blank, using a specific username. Leaving the password parameter blank will protect your credentials when sharing as you are prompted to enter it when the cell is run

Most of the time you will be connecting either using the  gis = GIS(‘home’)  or  gis = GIS(‘pro’) . Through this book you will be using connecting to either anonymously or to your organization by using  gis = GIS(‘home’) .

Using the gis module to Manage your GIS

Using the  gis  module you can access and manage your folders, your content, your group, and your users. If you have any repetitive tasks and workflows you can automate them as scripts. In this section you are going to see how search for data, publish items, organize data into folders, access and manage groups, access and manage users, download data, delete data, and download attachments.

Search for data

You can use the content property of your GIS to use the  search()  method. This allows you to search through your GIS and find data. The  search()  method will return a list of items based on the arguments given. The  search()  method has several arguments that it can take. Most of the time you will be using the query, and type of item arguments. The query has multiple types of queries it can take. In this section you will look at how to query based off an items title or owner.

In previous samples you were connecting to your GIS in your ArcGIS Online account. For the search example you will be connecting anonymously to understand how to search for data. You are going to search for publically available feature layers of Oakland.

Search for Public Data as an Anonymous User

1. Go to Project>Open>Computer>Open another Project and navigate to where you downloaded Chapter5.aprx and open it.

2. Right click on Chapter5 folder and select New>Notebook. Rename the Notebook to  SearchForDataSample

3. You are going to create your GIS using an anonymous log in, and you are going to import a display module that will make viewing the data returned easier. Type in the following:

from arcgis.gis import GIS

from IPython.display import display

gis = GIS()

Click the Run button to run.

4. In the next cell you will search for feature layers associated with Oakland limited to just 5 items, and display the results.

oaklandResults = gis.content.search(query="Oakland",item_type="Feature Layer",max_items=5)

for item in oaklandResults:

print(item)

for result in oaklandResults:

display(result)

The two for loops will return the data details to you in two different ways. The first is just printing out the results and the second is using the display module to show more details. Click the Run button to run

Results of Search for Oakland data

The results are just the first five feature layers returned associated with Oakland. There are many more arguments that could be used to find different data. You can apply any or all of the following arguments

- query  – this can be used to query for title or owner and can use wildcards.

- item_type  – this can be used to query any type of item that can be on an ArcGIS Online portal. It can find shapefiles, feature layers, feature collections, csvs, tables, maps, web scenes and more. It also can take wildcards.

- sort_field  – this can be used to sort the data on a field such as the title, owner or number of views

- sort_order  – this can be used with the sort field to sort ascending or descending.

- outside_org  – this can be used when logged into your organization to search for data outside of it.

You are going to test some of the arguments to see how you can get different results from the  search()  method.

First you will take the last search and modify to find feature layers or collections that have Oakland in the title, and sort them by the number of views.

1. You are now searching for data with Oakland in the title. You do not have to use the wild card in the title as this will search for the word Oakland. It is searching for just the word Oakland though so anything like OaklandCounty will not be returned. You are using a wildcard for the  item_type  to return all types that begin with Feature. This will return feature layers and feature collections. You are also sorting this by the number of views descending to get the most viewed items, and retuning only 5 items. In the same Notebook type in the following:

oaklandResults2 = gis.content.search(query="title:Oakland",item_type="Feature *",

sort_field="numViews",sort_order="desc",max_items=5)

for item in oaklandResults2:

print(item)

for result in oaklandResults2:

display(result)

Click the Run button to run. The output should look something like this.

Results of search query for Oakland with different search arguments

You can also search for data by the owner of the data. Your query argument is structured like this  query=”owner:username”  This will only return data that the owner has made publically available.

2. You are taking the owner of two of the datasets from the previous cell and searching for all the feature data types they own. Since you don’t know how many there are and don’t want to print all that data you just print out the length of the search results list returned. In the same Notebook in the next cell type the following

oaklandResults3 = gis.content.search(query="owner:antievictionmapdev",item_type="Feature *")

print(len(oaklandResults3))

Click the Run button to run and see the results

Results of search for owner query

3. Now that you know there are 10 feature layers or feature collections in the next cell you can type the following.

for result in oaklandResults3:

display(result)

Click Run to see the layers display.

Display of layers from the owner query

You can click on the layer name in the Output cell and a browser will open showing you the overview page for the item you clicked on.

Searching for data when connected to your organization

In this section you have seen how to use the  search()  operation to search for public data as an anonymous user and how to search for data within your organization. As you have seen there are different ways to connect to your organization, depending on how you are using the ArcGIS API for Python, and if you are using ArcGIS Online or ArcGIS Enterprise. Because you will be connected to your organization in these samples there will be limited figures showing the out cells as those will depend on what data you have in your organization.

To search for data in your organization you are going to continue to use the  SearchForDataSample  Notebook from the above samples.

1. In the next cell you are going to create another connection ArcGIS Online using the account you are logged into in ArcGIS Pro by typing in the following

gis2 = GIS(‘home’)

This will create a GIS under gis2 that you can use to access and manage content and users in your ArcGIS Online. If you are using an ArcGIS Enterprise you need to type the following

gis2 = GIS(“https://portal/domain.com/webadapter”,”username”,”password”)

Click the Run button to run the cell.

2. You can see properties of the user you are signed in under by typing the following

gis2.properties.user

Click the Run button to run the cell. The results will be a data dictionary containing all the information about the user.

User details

3. All of which can be further accessed and assigned to variables if needed. To access the first name you just need to type in

firstName = gis2.properties.user.firstName

firstName

Click the Run button.

Extract a user’s name

4. Searching through your content is the same as when logged in anonymously. The only difference is you are searching through data within your organization. In the next cell type in the following

searchResults = gis2.content.search(query="*",item_type="Feature Layer")

for result in searchResults:

display(result)

Click the Run button to run the cell. It will display all of the feature layers in your organization.

5. To search for just the items owned by you write the following

searchResults = gis2.content.search(query="owner:"+gis2.users.me.username,item_type="Feature Layer")

for result in searchResults:

display(result)

Click the Run button to run the cell.

The only argument in the  search()  that is required is the query. Because wildcards can be used you can search for everything by just writing query=”*”. But be careful, if you have a lot of layers the search could be slow.

6. When connected to your organization you can still search for publically available data by setting the  outside_org  argument to  True  You can find the same Oakland datasets in  gis2  by writing the following code.

oaklandResultsHome = gis2.content.search(query="title:Oakland",item_type="Feature *",

sort_field="numViews",sort_order="desc",max_items=5,outside_org=True)

for result in oaklandResultsHome:

display(result)

Click the Run button to run the cell. The results should be the same as when connected anonymously.

Results from searching outside your organization

In this section you have seen how to search for data both anonymously and when connected to your organization. Now that you can find data you will see how to add data to your organization and how to organize that data.

Publish Data

Much of the data you publish to ArcGIS Online or ArcGIS Enterprise is done within ArcGIS Pro. This is useful and convenient when you are publishing single maps or feature layers. It is less convenient if you are going to publish a csv with latitude and longitude fields as you would need to display the coordinates in ArcGIS Pro before publishing it. Using the ArcGIS API for Python you can take a csv and add it to your organization and publish it with a few lines of code.

Adding data from a csv.

To add data to your GIS you are going to use the  add()  method. Like the  search()  method the  add()  method is part of the  content()  property of the GIS object. To use the add method you will need to create a data dictionary of the properties of the item to be created. The properties are the same ones that you would need to fill in to add items from ArcGIS Pro: title, description, tags. You can add an optional thumbnail as well using the  add()  method.

First open up the csv AlamedaCountyFarmersMarket.csv to see what you will be adding. It is a basic csv with a market name, days, hours, location, city, latitude, and longitude. Right now it only has the famers markets for Oakland and Berkeley and not all of Alameda County. Later you will append the rest of the data to the feature layer you are creating from this csv data.

Farmers Market CSV

1. Right click on  Chapter3  folder and select New>Notebook. Rename the Notebook to  AddPublishData

2. In the first cell type in your import statements and create your GIS. You are going to create the GIS logged in to your ArcGIS Online account that you are logged into ArcGIS Pro with. Type the following

from arcgis.gis import GIS

from IPython.display import display

gis = GIS(‘home’)

3. Click the + button to add a cell. In this cell you are going to create a variable for the  csv .

csvFM = r"C:\PythonBook\Chapter3\AlamedaCountyFarmersMarket.csv"

If your csv is saved to a different location make sure you are using its location.

4. Click the + button to add a cell. In this cell you are going to create the data dictionary of the  csv  properties. You will fill in the properties for the title, description, and tags as the keys with their properties as the values. Type the following

csvProperties = {

"title":"Farmers Markets in Alameda County",

"descrption":"Location, days, and hours of Farmers Markets in Alameda County",

"tags":"Farmers Market, Alameda County, ArcGIS API for Python"

}

If you were going to add a thumbnail you would do it in this cell by defining a variable with the location of the thumbnail.

thumbnail = r”PATH\TO\THUMBNAIL.png”

5. Click the + button to add a cell. In this cell you are going create a variable to hold the  csv  item being added. You use the add function from the content module. The arguments passed are the properties dictionary and the path for the  csv . Type the following

addCsvFM = gis.content.add(item_properties=csvProperties,data=csvFM)

6. Click the + button to add a cell. In this cell you will publish the csv item you just added by calling the publish method. Type the following

farmersMarketFL = addCsvFM.publish()

farmersMarketFL

By setting the publish method to a variable that variable contains the feature layer. You can call that variable to display the feature layers properties.

7. Click the + button to add a cell. In this cell you are going to create a quick map to visualize your data to verify the feature layer was created. Type the following

map1 = gis.map("Oakland, California")

map1.add_layer(farmersMarketFL)

map1

8. Click Cell>Run All to run all the cells. The output map should look like below

Map Widget showing Farmers Market Feature Layer.

Adding and Publishing Summary and Tips

You have seen how to publish a csv that has latitude and longitude columns for point data. This process could be turned into an iterative process using a loop to publish multiple csvs. All you would need to write is a properties data dictionary for each csv. But your data is not always going to be a csv of point locations with latitude and longitude. Below are some tips for publishing other types of data.

- When publishing a csv with latitude and longitude fields make sure they are named latitude and longitude. The module is looking for those field names. If they are not found it will not locate the points correctly. You can specify the field names to use with the optional  publish_paramaters  dictionary in the  publish()  method. To do that you would need to create the following dictionary

publishParam = {

"locationType":"coordinates",

"latitudeFieldName":"LatX",

"longitudeFieldName":"LongY"

}

- The  locationType  value to use is coordinates, and then the  latituedFieldName  and  longitudeFieldName  are the field names for each in your csv.

- Csvs without latitude and longitude but with address can be geocoded. To geocode data from a csv you would again use the  publish_paramaters  dictionary.

publishParam = {

"locationType":"address",

“addressTemplate”:”{address},{city},{state},{zip}”

}

- The  locationType  field is set to address. The  addressTemplate  field is then set to the fields that have the different address components. In this example there is a field with the street address, a field with the city, a field with the state, and a field with the zip. The set-up of this will depend on how you have your data in your csv.

- Shapefile and file geodatabses can be added and published using this same method but they must be zipped. If you have a large amount of shapefiles or file geodatabases that are unzipped you could automate the process of zipping them up.

Adding and publishing data using the ArcGIS API for Python can be very useful for quickly adding data to your organizations ArcGIS Online or ArcGIS Enterprise account. You have seen how to add and publish a csv. You have seen that the csvs can be geocoded on publishing using the ArcGIS Online geocoder. In the next section you will see how to organize the data into folders, create groups, and manage access to groups.

Organize data, manage groups, and users

Organizing your data within your ArcGIS Online or ArcGIS Enterprise is important. You want to be able to find your data. In addition to folders to hold your data in you also can create groups to share specific data with. In large organizations this is important as not everyone needs to have access to the same data. In this section you will see how to create folders and move data into them, create groups and manage access to them, and create and manage users.

Organize data into folder

One of the first things to do after adding data or publishing it should be finding a folder to place it in. It is good practice to use folders to organize your data. It not only helps you find data but also other members of your organization. You can add folders and move data over using the ArcGIS API for Python. In the sample below you will create a new folder and move the farmer’s market data from above.

1. Right click on  Chapter3  folder and select New>Notebook. Rename the Notebook to  CreateFolderMoveData

2. In the first cell type in your import statements and create your GIS. You are going to create the GIS logged in to your ArcGIS Online account that you are logged into ArcGIS Pro with

from arcgis.gis import GIS

from IPython.display import display

gis = GIS(‘home’)

3. Click the + button to add a cell. In this cell you will create a new folder

gis.content.create_folder(folder="AlamedaFarmersMarkets")

4. Click the + button to add a cell. In this cell you search for that you need to move over

alamedaFM = gis.content.search(query="title:Farmers Markets in Alameda County”)

5. Remember that the  search()  method returns a list of items. To confirm what you have in your list you will run a for loop to iterate through the list and display the data. Type into the same cell as above

alamedaFM = gis.content.search(query="title:Farmers Markets in Alameda County")

for item in alamedaFM:

display(item)

6. Click Cell>Run All to run all your cells to this point. Your Notebook should similar to this now.

Output of creating folder and finding data to move.

7. Click the + button to add a cell. In this cell you will move the feature layer and csv into the new folder. You will do that by looping though the search results and using  move()  method.

for item in alamedaFM:

item.move(folder="AlamedaFarmersMarkets")

print(item)

Click the Run button to run this cell. You should see an output like below confirming that your data has been moved. If you go to your ArcGIS Online you will see that you now have a new folder and both datasets are there.

Output of moving data to a new folder

Creating a folder and moving data into that folder is a process that ArcGIS API for Python can be used to help you organize data. In the sample you were able to find all the data sets by their name and move them to a newly created folder. Being able to search your GIS for data and move it to folders using the ArcGIS API for Python is a valuable tool that will save you time.

If you need to move data back to the root directory it is just the following code  item.move(“\”)

Access and Manage Groups

Groups are spaces where you share data with other users. It is how you can create a collaborative GIS by allowing other users access to your data and maps. Using the ArcGIS API for Python you can create and manage groups in a programmatic way to save you time while fostering better collaboration within and outside of your team. In this section you will see how to search for groups, find the properties of a groups, create new groups, and manage the group by sharing data to it, and adding and removing users from the group by using the ArcGIS API for Python.

Search for Groups

Searching for groups is very similar to searching for data. You can search for groups that are open to all when logged in anonymously or search for groups within your organization when logged into your organization. You are going to first search for groups anonymously and then access the properties of that group you created. Then you will search for groups within your organization.

1. Right click on  Chapter3  folder and select New>Notebook. Rename the Notebook to  SearchForGroups

2. In the first cell type in your import statements and create your GIS. You are going to create the GIS anonymously.

from arcgis.gis import GIS

from IPython.display import display

gis = GIS()

Click the Run button to run.

3. In the next cell you will create your search and then display the results. Just like with the feature layers you are going to limit your data search to the first 5 records. You will also be using the display module to better show the group information. Type in the following

oaklandGroups = gis.groups.search('title:Oakland',max_groups=5)

for group in oaklandGroups:

display(group)

Click the Run button to run. You should have results that look like the figure below

Results of group search for Oakland groups.

4. Just like with items you can search for groups by owner instead of title. You will use one of the group owners from the search results. Type in the following code

oaklandGroups2 = gis.groups.search('owner:DebusB@oakgov.com_oakgov',max_groups=5)

for group in oaklandGroups2:

display(group)

Click the Run button to run. You should have results that look like the figure below

Results of search for group search by owner.

5. Just like with searching for items the group search returns a list. To look further at a group’s properties you need to select it using the list index. You are going to select the first group from the first search to look at its properties. Type in the following code

oaklandGroup1 = oaklandGroups[0]

oaklandGroup1

Click the Run button to run. You should have results that look like the figure below

Results of selecting a group from the group list

6. Now you can see some the properties of the group. You are going to print out the values using  .format()  to add some context to the values being printed out

print("Group Access is : {}".format(oaklandGroup1.access))

print("Group id is: {}".format(oaklandGroup1.id))

print("Group Tags are : {}".format(", ".join(oaklandGroup1.tags)))

print("Group is Invitation only: {}".format(oaklandGroup1.isInvitationOnly))

Click the Run button to run. You should have results that look like the figure below

Results from Group Properties

There are more properties that you can access for a group. A full list of them is here, https://developers.arcgis.com/rest/users-groups-and-items/group-search.htm

7. To search for groups within your organization you need to be logged into your GIS. Create a new GIS in this workbook by typing the following in the next cell

gis2 = GIS('home')

Click the Run button to run

8. In the next cell you will run a search for all groups in your organization that you have access to.

myGroups = gis2.groups.search(query="*",max_groups=5)

for group in myGroups:

display(group)

Click the Run button to run. You should see at most 5 groups. If you are not a member of 5 groups you will only see those that you are a member of. If you want to see all the groups you are a member of you would remove the  max_groups=5

Creating a Group

Now that you have seen how to search for groups in your organization you need to create one. You can create groups to share data publically or with just members of the group. You are going to walk through how to create a group for sharing data publically. You will also see the arguments needed to create private groups and how to change the setting.

1. Right click on  Chapter3  folder and select New>Notebook. Rename the Notebook to  CreateGroupMoveData

2. You are going to log into your organizations GIS through the user you are currently logged into ArcGIS Pro with by typing in the following

from arcgis.gis import GIS

from IPython.display import display

gis = GIS("home")

Click the Run button to run

3. In the next cell you will create a group by using the create method of the groups module. The create method takes 5 arguments explained in the table below. Type in the following

farmerMarketGroup = gis.groups.create(title="Alameda County Farmers Markets",

tags="Alameda County, Farmers Market",

description = "Group with data for Alameda County Farmers Markets.",

access = "public",

is_invitation_only = "False"

)

Click the Run button to run

You should get no out message, but you have created a new public group. To check this you can go to your ArcGIS Online account and look in Groups. You can also type in  farmerMarketGroup  and click the Run button to see the group. It should look something like below.

Newly created group.

To create the group you used 5 arguments: title, tags, description, access,  is_invitation_only . Those are the minimum arguments you should use when setting up a new group as they give the group a title, tags, description and set basic access. The table below has those arguments along with the values they can be.

title

String between single or double quotes that will be the title of your group

tags

String between single or double quotes with a comma separating all the tags. When returned it is a list

description

String between single or double quotes that will be the description of your group

access

String between single or double quotes that sets the access. The access values can be: org, private, public. Org is a group that everyone in your organization can see. Private is a private group that only invited users can see. Public is a public group available to everyone.

is_invitation_only

String between single or double quotes that is a boolean value. When set to True users can only gain access if invited. When set to False users can request access or be invited.

You can verify any of the settings by typing the variable for the group a dot and the value. To verify the access of the group you just created you would type the following in the next cell

farmerMarketGroup.access

And click Run to run.

You should see the following output.

Verifying a groups access.

To change any of the values of a group you can do that by using the update function. The update function takes all the same arguments used to create the group. To update the access type in the following

farmerMarketGroup.update(access = "private")

And click Run to run.

You have now created a new group and seen how to change the values of that group. The next step is sharing data with a group.

Sharing content to a Group

An empty group is not very useful. The point of creating a group is to share data either publically or with other users. In this section you will see how to share data with a group.

1. You are going to need to access the feature layer containing the Alameda County Farmers Markets. In the next cell of the same Notebook from above you will use the  search()  method to get the Farmers Markets in Alameda County feature layer. The search method returns a list but you know you only have one item with that title so you can add a [0] at the end of the search method to return just the first value from the list to your variable. Type the following

alamedaFM = gis.content.search(query="title:Farmers Markets in Alameda County")[0]

alamedaFM

Click Run to run. You should see the display for the Alameda County Farmers Market Feature Layer

2. Now you can check the access of the feature layer by typing in the following

alamedaFM.access

Click Run to run. You should see it return  ‘private’

3. Now that you have a feature layer you can share it with your group. To share with the group you are going to use the  share()  method, and you are going to use two arguments to set the organization sharing level when sharing it with the group. The org argument can be set to True or False, when set to True it shares the item with your entire organization, when false it share it just with the group. The group argument takes the id from the group. Since you have a variable holding the group you just have to access its id by using the id method. Type in the following

alamedaFM.share(org=False,groups=farmerMarketGroup.id)

Click Run to run. You should see the result below

Results of sharing an item to a group

4. You can check the sharing level of any of your items by calling the  shared_with  method on it. To see that type in the following

alamedaFM.shared_with

Click Run to run. You should see the result below

Results of shared_with

The result is a dictionary with values for everyone, org, and groups that the item is now shared with.

Now that you have shared some data with the group you need to add or invite users to join your group.

Adding, Inviting, and Removing Users from a Group

Adding, inviting and removing users from a group all use similar code. They all apply a method to the group. The method takes a list of strings of usernames as its argument. The table below displays the syntax using our farmers market group from the example.

add_users

farmerMarketGroup.add_users([“user1”,”user2”, . . .])

invite_users

farmerMarketGroup.invite_users([“user1”,”user2”, . . .])

remove_users

farmerMarketGroup.remove_users([“user1”,”user2”, . . .])

When any of the above code is run the output is a dictionary with a list of users that were added, invited, or removed along with the details. The group owner cannot be removed from the group.

Which user can belong to group is dependent on your organization type. In some cases organizations do not allow users from outside your organization or public users to be part of a group.

Now you have created a group, shared data with it and added or invited new users to that group. Next you will see how to manage users

Manage Users

Managing users in your organization through the ArcGIS API for Python can be a time saver as you can have Notebooks created that can be run to quickly create new users, access user data, reassign user content and delete users. The first step is understanding the User class to see what information you can see about users.

User Properties

In order to learn more about the properties of users you are going to look at yourself and explore the different user properties.

1. Right click on  Chapter3  folder and select New>Notebook. Rename the Notebook to  UserProperties

2. You are going to log into your organizations GIS through the user you are currently logged into ArcGIS Pro with by typing in the following

from arcgis.gis import GIS

from IPython.display import display

gis = GIS('home')

Click Run to run

3. In the next cell you are going to view your own account by using the me property. Type in the following

me = gis.users.me

me

Click Run to run. You should see the following returned with your user information

User information

4. You can identify many different aspects of a user’s profile. Things like a user’s first and last name, email address, when they last accessed their account, what groups they are a member of, and how much storage they are using, You are going to extract and write out all of this information. You will need to import the time module to convert the time returned to month/date/year. The groups property returns a data dictionary. In order to access the name you create an empty list, and iterate through the data dictionary accessing the title of each, and append it to the group, then use the join function to write the contents of the group to string. In the next cell type the following

import time

firstName = me.firstName

lastName = me.lastName

email = me.email

accessedLast = time.localtime(me.lastLogin/1000)

groups = me.groups

myGroupsList =[]

for group in groups:

groupName = group["title"]

myGroupsList.append(groupName)

groupsName = ", ".join(myGroupsList)

storageAssigned = me.storageQuota

storageUsed = me.storageUsage

prctStorage = round((storageUsed/storageAssigned)*100,4)

print("First Name:           {0}".format(firstName))

print("Last Name:            {0}".format(lastName))

print("email:                {0}".format(email))

print("Last Accessed:        {0}/{1}/{2}".format(accessedLast[1],accessedLast[2],accessedLast[0]))

print("Groups:               {0}".format(groupsName))

print("Storage Assigned:     {0}.".format(storageAssigned))

print("Storage Used:         {0}.".format(storageUsed))

print("Percent Storage Used: {0}".format(prctStorage))

Click Run to run. You should see the following returned with your user information

User Information returned

Another useful item to be returned would be credits available. For this you need to have credit budgeting turned on. In a future section you will look more at managing credits as an administrator and you can add that code to your above code to find the credits available.

Search For Users

You can search for users just like you would with items or groups. You can set a query to look for a user by username or find users by an email address. You will set up a Notebook with samples of both that you can use to further search for users within your organization.

1. Right click on  Chapter5  folder and select New>Notebook. Rename the Notebook to  SearchForUsers

2. You are going to log into your organizations GIS through the user you are currently logged into ArcGIS Pro with by typing in the following

from arcgis.gis import GIS

from IPython.display import display

gis = GIS('home')

Click Run to run

3. First you will search for users by username. You will be writing in your username below in the code where it says { userNameSearch }. The search for users works just like all the previous searches in that it returns a list of values. Since you are searching for a specific username you should have a list of one. To be sure you will run a test to print out the length of the list. Type in the following

userNameSearch = gis.users.search(query="username:{userNameSearch}")

len(userNameSearch)

Click Run to run

Result of username search

4. To access the user returned you need to use the list index to extract the first user. Then display those results. Type in the following

userNameSelect = userNameSearch[0]

userNameSelect

Click Run to run

Results of selecting a single user

5. You can also search by email using the wildcard *. This allows you to search for all email addresses from the same email provider. The code is the same as for searching by username except for the query. In this sample you will put in your own email provider in the  {@email.com} . You are again going to find the length of the list returned to you before extracting a user. Type in the following in the next cell.

emailSearch = gis.users.search(query="email: *{@email.com}")

len(emailSearch)

Click Run to run

6. Depending on how many people in your organization have that email host you may have a large number. You could iterate through them using a for loop and print out the user information from the last Notebook. Type in the following

import time

for user in emailSearch:

firstName = user.firstName

lastName = user.lastName

email = user.email

accessedLast = time.localtime(user.lastLogin/1000)

groups = user.groups

myGroupsList =[]

for group in groups:

groupName = group["title"]

myGroupsList.append(groupName)

groupsName = ", ".join(myGroupsList)

storageAssigned = user.storageQuota

storageUsed = user.storageUsage

prctStorage = round((storageUsed/storageAssigned)*100,4)

print("--------------------------------------------")

print("First Name:           {0}".format(firstName))

print("Last Name:            {0}".format(lastName))

print("email:                {0}".format(email))

print("Last Accessed:        {0}/{1}/{2}".format(accessedLast[1],accessedLast[2],accessedLast[0]))

print("Groups:               {0}".format(groupsName))

print("Storage Assigned:     {0}.".format(storageAssigned))

print("Storage Used:         {0}.".format(storageUsed))

print("Percent Storage Used: {0}".format(prctStorage))

Click Run to run. It will print out the user information for each user in that search, separated by a  ----------- .

User search information.

You have seen how to search for users within your organization, and print out user information. In the next section you will see some of the admin privileges that can be applied to users.

Administering Your GIS

In this section you will explore what you can do if you are the administrator and have admin privileges. With admin privileges you can add and remove users, manage user licenses, and manage ArcGIS Online credits. In this section you will see how to do each of those.

You must have admin privileges to do any of the following. To add users you must have credentials available for the new user

Creating and Deleting users

To create new users in AcGIS Online you will be using the  create()  method. The  create()  method takes the following arguments:  username ,  password ,  firstname ,  lastname ,  email ,  description ,  role ,  user_type , and  provider .

username

String value of the username

password

String value of the password, this can be changed by user at any time when they log in

firstname

String value of the user’s first name

lastname

String value of the user’s last name

email

String value of the user’s email address

description

String value of the description of the user

role

String value of the role for the user in the organization.

ArcGIS Online has the three options

org_admin  is for admins with the organization that have admin privleges

org_publisher  is for publishers in the organization that can publish web layers and service layers

org_user  is for users in the organization that can create groups and content and use the organizations data

user_type

String value for the user type in the organization

viewer_UT  is for users that need to view data shared with them. They can’t create or edit data

fieldworkerUT  is for users that need to view and edit data and is for users that interact mostly through field apps.

editor_UT  is for users that need to view and edit data shared with them. They can’t create or share data

creator_UT  is for users that need to create and share content, such as web maps and apps.

GISProfessionalBasicUT  is for users with all the capabilities of a creator and access to ArcGIS Pro Basic

GISProfessionalStdUT  is for users with all the capabilities of a creator and access to ArcGIS Pro Standard

GISProfessionalAdvUT  is for users with all the capabilities of a creator and access to ArcGIS Pro Advanced

provider

String value for the provider

arcgis  for ArcGIS Online

enterprise for ArcGIS Enterprise

With this table you can create your new user.

1. Right click on  Chapter5  folder and select New>Notebook. Rename the Notebook to  AdminsteringYourOrg

2. You are going to log into your organizations GIS through the user you are currently logged into ArcGIS Pro with by typing in the following

from arcgis.gis import GIS

from IPython.display import display

gis = GIS('home')

Click Run to run

3. You are going to create a sample new user. You will assign them a username, password, first name, last name, email, description, role, user type, and provider. Type in the following

newUser = gis.users.create(username = "newUser1",

password = "1234PleaseChange",

firstname = "New",

lastname = "User",

email = "newUser@company.com",

description  = "Creating a new user",

role = "org_publisher",

user_type = "GISProfessionalAdvUT",

provider = "arcgis"

)

Click Run to run. If it runs you should get no result.

4. To view the user type in  newUser  in the next cell and click Run to run

5. When it comes time to delete a user you first want to make sure you don’t lose any data. As an admin you can reassign a user’s data to a different user. The first step is to find all of that user’s data. Recall from earlier you can search for content based on the owner’s username. In the next cell type the following

items = gis.content.search(query="owner: newUser1")

Click Run to run.

6. To reassign all the items to a new user you loop through the list of items calling the  reassign_to  method. The reassign_ to  method takes a  target_owner  and  target_folder  argument. If the name of the  target_folder  is not assigned it will put the items in the root folder. You can now reassign all the data from newUser to yourself by typing in the following

for item in items:

item.reassign_to(target_owner = "{yourUserName}")

Click Run to run. You should get no output

7. Now you can delete the user as all of their data has been reassigned. In the next cell type in

newUser1.delete()

Click Run to run. You should get no output

You can reassign all of a user’s items when you delete the user. But you are only able to reassign it to the root folder. To do that you would write  newUser.delete(reassign_to ={userNameToAssignTo}”

You have seen how to create a user and the different attributes you can assign a user when you create them. You have also seen how to reassign a user’s data and delete that user. Next you will see how to assign licenses and monitor credits for your users

Assign Licenses and Credits

You can manage both user licenses and credits through Notebooks. Users can be assigned a license and have that license revoked. You can view the available credits in your organization, and you can manage those credits through credit budgeting. You are first going to see how to check the apps in your organization and then view the licenses in your organization and see how to assign them.

1. Continue working in the  AdminsteringYourOrg  Notebook by finding all the apps licensed in your organization. You will create variable to hold the list of apps and then a loop to print out the apps each on a single line. Type in the following

license = gis.admin.license.all()

for l in license:

print(l)

Click Run to run. You will see the apps your organization has licensed.

2. In the next cell you will use the get method to get the license to your ArcGIS Pro app so you can see what individual license you have. Type in the following

proLic = gis.admin.license.get("ArcGIS Pro")

Click Run to run. You should get no output

3. Now that you have the Pro app license you can see what extensions you have licensed and if any are assigned. You are using the report object to get a table showing you what licenses you have, how many are assigned and how many are remaining. Type in the following

proLic.report

Click Run to run. You should see a table similar to below, just with the licenses in your organization

ArcGIS Pro Licenses in your organization

4. Now that you know what licenses you have available you can assign one. You use the assign method with the username and the extension as the entitlements argument. Assign yourself a spatial analyst license.

proLic.assign(username = me.username,entitlements="spatialAnalystN")

Click Run to run. If assigned your result should be True

5. You can check on what extension a user has assigned. You use the  user_entitlement  method passing in the user name as the argument. Check on what extensions you have assigned to you by typing in the following, putting in your user name between the quotes.

proLic.user_entitlement("{yourUserName}")

Click Run to run. The results are a dictionary with the username, last login, connected, and list of entitlements as the key/value pairs.

Extensions assigned to a user

6. To revoke a license you use the revoke method on the license object. The arguments are the same as assign, a username and the extensions to be revoked. You can use the * wildcard to revoke all licenses. Revoke all the licenses assigned to you.

proLic.revoke(username = me.username,entitlements="*")

Click Run to run. If assigned your result should be True

7. To verify you have no license use the  user_entitlement  method again

proLic.user_entitlement("{yourUserName}")

Click Run to run. The results are an empty dictionary.

8. You can use the credits property to view the amount of credits available in your organization. Type in the following

gis.admin.credits.credits

Click Run to run. The results will be the number of credits in your organization.

9. You can manage the credits available to each user by using credit budgeting. Turn on credit budgeting by using the enable method on the credits property.

gis.admin.credits.enable()

Click Run to run. Your result should be True

10. Now you can allocate credits to a user by using the allocate method. The allocate method takes the username and number of credits as its arguments. Allocate 10 credits to yourself

gis.admin.credits.allocate(username=me.username, credits=api_acc_credits)

Click Run to run. Your result should be True

You can also remove credits from a user by using the deallocate method. The deallocate method removes all the credits from a user. If you want to remove all the credits you would type in the following  gis.admin.credits.deallocate(username='{username}')

11. When credit budgeting is enabled you can check the available credits for each user as they get an assigned credits and available credits property. Check the credits you have available

me.availableCredits

Click Run to run. The results will be the number of credits available to you. If you are using single user account you cannot assign credits to yourself as you have access to all the credits

Summary

In this section you have seen how to search for content inside and outside of your organization. You can now create new folders for organizing your content. You can create groups and set different sharing levels of your groups for sharing content by inviting users to your group. Using the admin settings you can create new users with different roles and types. You can search for existing users and pull up information about their roles and data owned, groups they are members of, and storage used. You can manage licenses for users and budget credits. In all you have the tools and Notebooks to manage your content and users of your organization.

Feedback

Have your say! Help our authors to provide useful information to customers just like you. Fill out our quick surveys to give your feedback. We’d love to know more about your experience as a reader.

- Chapter 1: https://forms.office.com/r/f9Tq95wCLn

- Chapter 2: https://forms.office.com/r/bbXf2Qi9JP

- Chapter 3: https://forms.office.com/r/QNrPzUY164

Remember that Early Access chapters are first drafts, so they haven’t received a final polish from our editors yet.