QGIS – Creating new column from existing using Python

Yesterday, I was working on the ward level parks map of Chennai I had to join a CSV data layer with the boundary polygon layer, but there was one issue while my CSV file has the ward numbers as integers (1,2,3..etc), the polygon layer had them as strings (Ward 1, Ward 2, Ward 3 …etc.,) So I was thinking, wouldn’t it be nice just to strip the word Ward and put it in a new column, so that I can make a join by matching the ward numbers. Turns out Python integration in QGIS is so good that, I did it without even searching the internet. Here is how.

  1. Open the Attribute table
  2. Open Field Calculator.
  3. Enter the “Output field name”
  4. Switch to “Function Editor”
  5. Click the [+] button to create a new function file.
  6. Changed the function name, parameter and return the value after stripping “Ward ” from the string. Read the docs given below the function editor to understand what’s going on the file.
QGIS Field Calculator
QGIS Field Calculator
from qgis.core import *
from qgis.gui import *

@qgsfunction(args='auto', group='Custom')
def strip_ward(name, feature, parent ):
    return name.split(" ")[-1]

Now switch back to the Expression tab and call the function to calculate the new field

strip_ward.png

Click OK. Now the new field with the computed value would be created.

I had a simple use case, by one can use the power of Python to calculate anything from existing data and generate a new field based on it. I was really blown away by the level of Python integration in QGIS.

Python Tip #9 – sorting

Sorting is simplified in Python with sorted(). You can even sort with complex rules.

>>> strings = ['alice', 'bob', 'donald', 'cathy']
>>> sorted(strings)
['alice', 'bob', 'cathy', 'donald']

>>> sorted(strings, key=len)
['bob', 'alice', 'cathy', 'donald']

>>> def secondchar(word):
...    return word[1]

>>> sorted(strings, key=secondchar)
['cathy', 'alice', 'bob', 'donald']

Python Tip #8 – reducing looping by using dicts

In situations where you have a list of objects and have to retrieve then in random order, dictionaries can act as lookup tables.

users = list of class User 

# Get users one by one by looking up ids
user_1 = next((u for u in users if u.id == user_1_id), None)
user_2 = next((u for u in users if u.id == user_2_id), None)
...

# Simpler solution using lookup table
lookup = dict((u.id, user) for u in users)
user_1 = lookup[user_1_id]
user_2 = lookup[user_2_id]
...

This tip is not very obvious, hence this explanation:

user_1 = next((u for u in users if u.id == user_1_id), None)

This method employs a iterator looping through the list of users every time we have to find a user, which means we have to run this loop a hundred times. This poses a complexity of O(N2).

lookup = dict((u.id, user) for u in users)
user_1 = lookup[user_1_id]

This method on the other hand iterates through the users list one time and create a lookup table that we can again and again without having to iterate through the list every time. This reduces the complexity to O(N) which could theoretically lead up to 10 times faster execution of the program.

Python Tip #7 – getattr()

Sometimes we have to deal with external objects and their attributes. getattr() can save you at those times.

# Get the attribute name
name = obj.name  # AttributeError if name is not present

# Check if the attribute is present before fetching
try:
    name = obj.name
except AttributeError:
    name = "Guest"

# Simpler solution
name = obj.name if hasattr(obj, "name") else "Guest"

# Simplest Solution
name = getattr(obj, "name", "Guest")

Python Tip #6 – Merging Dictionaries

Merge or combine dictionaries

d1 = { "a": 1 }
d2 = { "b": 2 }

# Adding elements of one dictionary to another
d1.update(d2)  # d1 => { "a": 1, "b": 2 }

# Create a new dict with values from other dictionaries
d3 = { **d1, **d2 }  # d3 => { "a": 1, "b": 2 }
d4 = { **d3, "c": 3 }  # d4 => { "a": 1, "b": 2, "c": 3 }

** is the unpacking operator

Python Tip #5 – Get value from dict if key is present

Check for existence of a key in dictionary and retrieve its value if present.

dictionary = { "key": "value" }

# checking for the presence and key and getting the value
wanted = None
if "key" in dictionary:
    wanted = dictionary["key"]

# Simpler version
wanted = dictionary.get("key", None)

Python Tip #4 – Find an element in a list satisfying condition

What if you want to find the first item that matches the condition instead of getting a list of items?

selected = None
for i in items:
if condition:
selected = i
break

# Simpler version using next()
selected = next((i for i in items if condition), None)

next() is a built in function which is not that well known.