Python list comprehensions offer a concise method of interacting with each element of a list. Even though they’ve been available since Python 2.0, their syntax often demotivates people from using them. This article aims to introduce List Comprehensions in a friendly way and offer you one more Python feature to add to your scripting toolbox.
What is a List Comprehension
According to Python 3.7’s documentation, list comprehensions: […] provides a concise way to create lists. Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable or to create a subsequence of those elements that satisfy a certain condition.
Simple List Comprehension Example
The official list comprehension definition and its example may not be immediately clear. Let’s use an alternative example. Let’s say we had the numbers 0 through 5 and we wanted to multiply each number by 2. This means the numbers 0, 1, 2, 3, 4, and 5 are our multiplicands and 2 is our multiplier. We know the multiplication operations should be 0*2, 1*2, 2*2, 3*2, 4*2, and 5*2. The products of the multiplication operations should be 0, 2, 4, 6, 8, and 10.
In Python, we can use a list comprehension to perform the same calculation. Let’s describe our objective again but in a way that lends itself to scripting: for each, individual multiplicand (or number) in our multiplicands 0-5, we want to multiply by 2. Here’s the Python code and result:
>>> multiplicands = [0, 1, 2, 3, 4, 5] >>> multiplier = 2 >>> products = [number*multiplier for number in multiplicands] >>> products [0, 2, 4, 6, 8, 10]
Our first line of code defined the list of multiplicands, 0-5. Our second line of code defined our multiplier, 2. Our third line of code defined the Python list comprehension, which performed our multiplication. The final line of code simply returned our product values.
More Complex List Comprehension Example
It’s possible I became consumed with excitement and copied the entire list of dog breeds from a Wikipedia page into a spreadsheet and then used cell formulas to generate the syntax for an equivalent Python list. Furthermore, a few important notes about the data set:
- There are special characters like an apostrophe; therefore, do not change the double quotes to single quotes/apostrophes unless you know what you’re doing.
- There are international characters that cannot be represented in ASCII. We’ll revisit this soon in our example.
Source Code:
# Author: Jeoffri Davis
# Purpose:
# - Example of using Python List Comprehensions
# Tested With:
# - Python 3.7
# - MacOS Mojave
# Inputs:
# - All inputs are self-contained
# Outputs:
# - Two different lists for Heelers
# - One list for breed names containing international characters
# Define a massive list of dog breeds. Not attempting to contain
# doggo love in PEP 8 coding style.
dogs = ["Affenpinscher","Afghan Hound","Afghan Shepherd","Aidi","Airedale Terrier","Akbash","Akita","Alano Español","Alaskan husky","Alaskan Klee Kai","Alaskan Malamute","Alaunt","Alopekis","Alpine Dachsbracke","Alpine Mastiff","Alpine Spaniel","American Akita","American Bully","American Bulldog","American Cocker Spaniel","American English Coonhound","American Eskimo Dog","American Foxhound","American Hairless Terrier","American Pit Bull Terrier","American Staffordshire Terrier","American Water Spaniel","Anatolian Shepherd Dog","Andalusian Hound","Anglo-Français de Petite Vénerie","Appenzeller Sennenhund","Argentine Polar Dog","Ariegeois","Armant","Armenian Gampr dog","Artois Hound","Australian Cattle Dog","Australian Kelpie","Australian Shepherd","Australian Stumpy Tail Cattle Dog[10]","Australian Terrier","Austrian Black and Tan Hound","Austrian Pinscher","Azawakh","Bakharwal dog","Barbado da Terceira","Barbet","Basenji","Basque Shepherd Dog","Basset Artésien Normand","Basset Bleu de Gascogne","Basset Fauve de Bretagne","Basset Hound","Bavarian Mountain Hound","Beagle","Beagle-Harrier","Bearded Collie","Beauceron","Bedlington Terrier","Belgian Shepherd Dog (Groenendael)","Belgian Shepherd Dog (Laekenois)","Belgian Shepherd Dog (Malinois)","Belgian Shepherd Dog (Tervuren)","Bergamasco Shepherd","Berger Blanc Suisse","Berger Picard","Bernese Mountain Dog","Bichon Frisé","Billy","Black and Tan Coonhound","Black and Tan Virginia Foxhound","Black Norwegian Elkhound","Black Russian Terrier","Black Mouth Cur","Bloodhound","Blue Heeler","Blue Lacy","Blue Paul Terrier","Blue Picardy Spaniel","Bluetick Coonhound","Boerboel","Bohemian Shepherd","Bolognese","Border Collie","Border Terrier","Borzoi","Bosnian Coarse-haired Hound","Boston Terrier","Bouvier des Ardennes","Bouvier des Flandres","Boxer","Boykin Spaniel","Bracco Italiano","Braque d'Auvergne","Braque de l'Ariege","Braque du Bourbonnais","Braque du Puy","Braque Francais","Braque Saint-Germain","Brazilian Dogo","Brazilian Terrier","Briard","Briquet Griffon Vendéen","Brittany","Broholmer","Bruno Jura Hound","Brussels Griffon","Bucovina Shepherd Dog","Bull and Terrier","Bull Terrier","Bulldog","Bullenbeisser","Bullmastiff","Bully Kutta","Burgos Pointer","Cairn Terrier","Canaan Dog","Canadian Eskimo Dog","Cane Corso","Cantabrian Water Dog","Cão da Serra de Aires","Cão de Castro Laboreiro","Cão de Gado Transmontano","Cão Fila de São Miguel","Carolina Dog","Carpathian Shepherd Dog","Catalan Sheepdog","Caucasian Shepherd Dog","Cavalier King Charles Spaniel","Central Asian Shepherd Dog","Cesky Fousek","Cesky Terrier","Chesapeake Bay Retriever","Chien Français Blanc et Noir","Chien Français Blanc et Orange","Chien Français Tricolore","Chien-gris","Chihuahua","Chilean Terrier","Chinese Chongqing Dog","Chinese Crested Dog","Chinese Imperial Dog","Chinook","Chippiparai","Chiribaya Dog","Chow Chow","Cierny Sery","Cirneco dell'Etna","Clumber Spaniel","Collie, Rough","Collie, Smooth","Combai","Cordoba Fighting Dog","Coton de Tulear","Cretan Hound","Croatian Sheepdog","Cumberland Sheepdog","Curly-Coated Retriever","Cursinu","Czechoslovakian Wolfdog","Dachshund","Dalbo dog","Dalmatian","Dandie Dinmont Terrier","Danish-Swedish Farmdog","Deutsche Bracke","Doberman Pinscher","Dogo Argentino","Dogo Cubano","Dogue de Bordeaux","Drentse Patrijshond","Drever","Dunker","Dutch Shepherd","Dutch Smoushond","East Siberian Laika","East European Shepherd","Elo","English Cocker Spaniel","English Foxhound","English Mastiff","English Pointer","English Setter","English Shepherd","English Springer Spaniel","English Toy Terrier (Black & Tan)","English Water Spaniel","English White Terrier","Entlebucher Mountain Dog","Estonian Hound","Estrela Mountain Dog","Eurasier","Eurohound","Field Spaniel","Fila Brasileiro","Finnish Hound","Finnish Lapphund","Finnish Spitz","Flat-Coated Retriever","Fox Terrier, Smooth","Fox Terrier, Wire","French Brittany","French Bulldog","French Spaniel","Gaddi Kutta","Galgo Español","Galician Shepherd Dog","Garafian Shepherd","Gascon Saintongeois","Georgian Shepherd","German Longhaired Pointer","German Pinscher","German Roughhaired Pointer","German Shepherd Dog","German Shorthaired Pointer","German Spaniel","German Spitz","German Wirehaired Pointer","Giant Schnauzer","Glen of Imaal Terrier","Golden Retriever","Gordon Setter","Gran Mastín de Borínquen","Grand Anglo-Français Blanc et Noir","Grand Anglo-Français Blanc et Orange","Grand Anglo-Français Tricolore","Grand Basset Griffon Vendéen","Grand Bleu de Gascogne","Grand Griffon Vendéen","Great Dane","Great Pyrenees","Greater Swiss Mountain Dog","Greek Harehound","Greek Shepherd","Greenland Dog","Greyhound","Griffon Bleu de Gascogne","Griffon Fauve de Bretagne","Griffon Nivernais","Guatemalan Dogo","Gull Terrier","Hamiltonstövare","Hanover Hound","Hare Indian Dog","Harrier","Havanese","Hawaiian Poi Dog","Himalayan Sheepdog","Hokkaido","Hortaya borzaya","Hovawart","Huntaway","Hygen Hound","Ibizan Hound","Icelandic Sheepdog","Indian pariah dog","Indian Spitz","Irish Red and White Setter","Irish Setter","Irish Terrier","Irish Water Spaniel","Irish Wolfhound","Istrian Coarse-haired Hound","Istrian Short-haired Hound","Italian Greyhound","Jack Russell Terrier","Jagdterrier","Swedish Elkhound","Japanese Chin","Japanese Spitz","Japanese Terrier","Jindo","Jonangi","Kaikadi dog","Kai Ken","Kangal Shepherd Dog","Kanni","Karakachan dog","Karelian Bear Dog","Karst Shepherd","Keeshond","Kerry Beagle","Kerry Blue Terrier","King Charles Spaniel","King Shepherd","Kintamani","Kishu Ken","Komondor","Koolie","Koyun dog","Kromfohrländer","Kumaon Mastiff","Kunming Wolfdog","Kurī","Kuvasz","Kyi-Leo","Labrador Husky","Labrador Retriever","Lagotto Romagnolo","Lakeland Terrier","Lancashire Heeler","Landseer","Lapponian Herder","Lapponian Shepherd","Large Münsterländer","Leonberger","Lhasa Apso","Lithuanian Hound","Louisiana Catahoula Leopard Dog","Löwchen","Mackenzie River husky","Magyar agár","Mahratta Greyhound","Ratonero Mallorquin","Maltese","Manchester Terrier","Maremma Sheepdog","Marquesan Dog","McNab dog","Miniature American Shepherd","Miniature Bull Terrier","Miniature Fox Terrier","Miniature Pinscher","Miniature Schnauzer","Miniature Shar Pei","Molossus","Molossus of Epirus","Montenegrin Mountain Hound","Moscow Watchdog","Moscow Water Dog","Mountain Cur","Mucuchies","Mudhol Hound","Mudi","Neapolitan Mastiff","Nederlandse Kooikerhondje","Nenets Herding Laika","Newfoundland","New Guinea singing dog","New Zealand Heading Dog","Norfolk Spaniel","Norfolk Terrier","Norrbottenspets","North Country Beagle","Northern Inuit Dog","Norwegian Buhund","Norwegian Elkhound","Norwegian Lundehund","Norwich Terrier","Nova Scotia Duck Tolling Retriever","Old Croatian Sighthound","Old Danish Pointer","Old English Bulldog","Old English Sheepdog","Old English Terrier","Old German Shepherd Dog","Old Spanish Pointer","Old Time Farm Shepherd","Olde English Bulldogge","Otterhound","Pachon Navarro","Pandikona","Paisley Terrier","Papillon","Parson Russell Terrier","Pastore della Lessinia e del Lagorai","Patterdale Terrier","Pekingese","Perro de Pastor Mallorquin","Perro de Presa Canario","Perro de Presa Mallorquin","Peruvian Inca Orchid","Petit Basset Griffon Vendéen","Petit Bleu de Gascogne","Phalène","Pharaoh Hound","Phu Quoc Ridgeback","Picardy Spaniel","Plummer Terrier","Plott Hound","Podenco Canario","Poitevin","Polish Greyhound","Polish Hound","Polish Hunting Dog","Polish Lowland Sheepdog","Polish Tatra Sheepdog","Pomeranian","Pont-Audemer Spaniel","Poodle","Porcelaine","Portuguese Podengo","Portuguese Pointer","Portuguese Water Dog","Posavac Hound","Potsdam Greyhound","Pražský Krysařík","Pudelpointer","Pug","Puli","Pumi","Pungsan dog","Pyrenean Mastiff","Pyrenean Shepherd","Rafeiro do Alentejo","Rajapalayam","Rampur Greyhound","Rastreador Brasileiro","Rat Terrier","Ratonero Bodeguero Andaluz","Ratonero Murciano de Huerta","Ratonero Valenciano","Redbone Coonhound","Rhodesian Ridgeback","Romanian Mioritic Shepherd Dog","Romanian Raven Shepherd Dog","Rottweiler","Russian Salon Dog","Russian Spaniel","Russian Toy","Russian Tracker","Russo-European Laika"
,"Russell Terrier","Saarloos Wolfdog","Sabueso Español","Sabueso fino Colombiano","Saint Bernard","Saint John's water dog","Saint-Usuge Spaniel","Sakhalin Husky","Salish Wool Dog","Saluki","Samoyed","Sapsali","Šarplaninac","Schapendoes","Schillerstövare","Schipperke","Schweizer Laufhund","Schweizerischer Niederlaufhund","Scotch Collie","Scottish Deerhound","Scottish Terrier","Sealyham Terrier","Segugio Italiano","Segugio Maremmano","Seppala Siberian Sleddog","Serbian Hound","Serbian Tricolour Hound","Seskar Seal Dog","Shar Pei","Shetland Sheepdog","Shiba Inu","Shih Tzu","Shikoku","Shiloh Shepherd","Siberian Husky","Silken Windhound","Silky Terrier","Sinhala Hound","Skye Terrier","Sloughi","Slovakian Wirehaired Pointer","Slovensky Cuvac","Slovensky Kopov","Smålandsstövare","Small Münsterländer","Small Greek Domestic Dog","Soft-Coated Wheaten Terrier","South Russian Ovcharka","Southern Hound","Spanish Mastiff","Spanish Water Dog","Spinone Italiano","Sporting Lucas Terrier","Stabyhoun","Staffordshire Bull Terrier","Standard Schnauzer","Stephens Cur","Styrian Coarse-haired Hound","Sussex Spaniel","Swedish Lapphund","Swedish Vallhund","Tahitian Dog","Tahltan Bear Dog","Taigan","Taiwan Dog","Talbot Hound","Tamaskan Dog","Teddy Roosevelt Terrier","Telomian","Tenterfield Terrier","Terceira Mastiff","Thai Bangkaew Dog","Thai Ridgeback","Tibetan Mastiff","Tibetan Spaniel","Tibetan Terrier","Tornjak","Tosa","Toy Bulldog","Toy Fox Terrier","Toy Manchester Terrier","Toy Trawler Spaniel","Transylvanian Hound","Treeing Cur","Treeing Tennessee Brindle","Treeing Walker Coonhound","Trigg Hound","Tweed Water Spaniel","Tyrolean Hound","Cimarrón Uruguayo","Vanjari Hound","Villano de Las Encartaciones","Villanuco de Las Encartaciones","Vizsla","Volpino Italiano","Weimaraner","Welsh Corgi, Cardigan","Welsh Corgi, Pembroke","Welsh Sheepdog","Welsh Springer Spaniel","Welsh Terrier","West Highland White Terrier","West Siberian Laika","Westphalian Dachsbracke","Wetterhoun","Whippet","White Shepherd","Wirehaired Pointing Griffon","Wirehaired Vizsla","Xiasi Dog","Xoloitzcuintli","Yakutian Laika","Yorkshire Terrier"]
def isHeeler(breed: str):
"""This function lets us define more robust logic for
determining whether a dog is a Heeler breed. This comes
in handy when dog breeds have multiple nicknames.
"""
if 'Heeler' in breed or breed == 'Australian Cattle Dog':
return True
else:
return False
def isAscii(breed: str):
"""This function checks if a breed name can be encoded
with ASCII, which is a character set that supports all
standard alphanumeric characters found in American English.
If the breed name cannot be encoded with ASCII, it is a
good indicator that the breed name contains international
characters. While we like our dogs well traveled, we
would run into encoding issues if we expand our script to
integrate with other systems or solutions that expect
ASCII encoding.
"""
result = True
try:
breed.encode('ascii')
except UnicodeEncodeError as caught_exception:
result = False
return result
# List comprehension that searches the list contained
# in variable "dogs", finds all breeds with the word
# 'Heeler' in it and saves the matching breeds to a
# a new list called "heelers1"
heelers1 = [dog for dog in dogs if 'Heeler' in dog]
# List comprehension that uses the isHeeler function
# for its conditional logic. Using a function keeps
# the list comprehension concise.
heelers2 = [dog for dog in dogs if isHeeler(dog)]
# List comprehension that finds all dog breeds with
# international characters in its name. Sometimes
# in scripting it's good to watch for international
# characters because they can lead to encoding issues.
# You normally wouldn't force ASCII encoding but its
# character set is familiar, which helps understand
# the concept of encoding.
international_doggos = [dog for dog in dogs if not isAscii(dog)]
international_doggos = [dog for dog in dogs if not isAscii(dog)]
print("\nBreeds names containing the word 'Heeler':" + str(heelers1))
print("\nMore robust list of Heeler breeds:" + str(heelers2))
print("\nBreed names containing international characters:" + str(international_doggos))
Output
Breeds names containing the word 'Heeler':['Blue Heeler', 'Lancashire Heeler'] More robust list of Heeler breeds:['Australian Cattle Dog', 'Blue Heeler', 'Lancashire Heeler'] Breed names containing international characters:['Alano Español', 'Anglo-Français de Petite Vénerie', 'Basset Artésien Normand', 'Bichon Frisé', 'Briquet Griffon Vendéen', 'Cão da Serra de Aires', 'Cão de Castro Laboreiro', 'Cão de Gado Transmontano', 'Cão Fila de São Miguel', 'Chien Français Blanc et Noir', 'Chien Français Blanc et Orange', 'Chien Français Tricolore', 'Galgo Español', 'Gran Mastín de Borínquen', 'Grand Anglo-Français Blanc et Noir', 'Grand Anglo-Français Blanc et Orange', 'Grand Anglo-Français Tricolore', 'Grand Basset Griffon Vendéen', 'Grand Griffon Vendéen', 'Hamiltonstövare', 'Kromfohrländer', 'Kurī', 'Large Münsterländer', 'Löwchen', 'Magyar agár', 'Petit Basset Griffon Vendéen', 'Phalène', 'Pražský Krysařík', 'Sabueso Español', 'Šarplaninac', 'Schillerstövare', 'Smålandsstövare', 'Small Münsterländer', 'Cimarrón Uruguayo']
How List Comprehensions Can Apply to LogicMonitor
I often find myself using List Comprehensions when programmatically managing LM portals through the LM REST API. Two examples are:
- Maximizing REST API and data processing efficiency. Rather than submitting multiple API calls with filter query parameters, I can use one REST API call to pull all device groups and then filter the results into subsets of data using Python list comprehensions. Getting equivalent subsets directly from the REST API would require multiple API calls, which would be slower than accessing local memory, and, risk API usage throttling.
- Sanity checking CSV files used to build device groups or dashboards in a LogicMonitor portal. During Professional Services engagements where I need to help a customer, especially MSPs, onboard, I will often have them define their desired device and dashboard group structure in a CSV. Then I can use the LM REST API to build out their portal. There are some restricted special characters as well as character count limitations. These can both be checked early in the Python script with a list comprehension before making REST API calls that would ultimately fail.
Conclusion
Python List Comprehensions can offer concise, yet powerful, list manipulation. You can perform operations on each element in a list and even include conditions as to when the operations should take place. I commonly use list comprehensions to cache data returned from the LogicMonitor REST API and split it into subsets. I also reach for list comprehensions when I need to sanity check CSV files before using them for other operations.
LogicMonitor powers monitoring of global business services you probably use every day. Operating at that level takes flexibility. That’s why LogicMonitor DataSources support Python, Groovy, PowerShell and any other language as long as you configure it in your collector host environment.
