Python · Syntax · Intermediate

Regular expressions

10 tasks

Pattern matching with the re module. Covers search, match, findall, groups, and common patterns.

The re module: search, match, fullmatch, findall, sub

#
Regular expressions are a mini-language for describing text patterns. Python's `re` module compiles these patterns and applies them to strings. The four most-used functions differ in *where* they look: ```python import re text = "User: alice, age 30" re.search(r"\d+", text) # finds first match ANYWHERE in the string -> Match "30" re.match(r"\d+", text) # matches only at the BEGINNING -> None (starts with "U") re.fullmatch(r"\d+", "42") # whole string must match -> Match "42" reindall(r"\d+", text) # all non-overlapping matches -> ["30"] re.sub(r"\d+", "??", text) # replace all matches -> "User: alice, age ??" ``` **Why raw strings?** A backslash in a Python string has special meaning: `\n` = newline, `\t` = tab. But regex also uses backslashes heavily (`\d`, `\w`, `\s`). Without raw strings you'd have to double every backslash: ```python re.search("\\d+", text) # works but unreadable re.search(r"\d+", text) # same thing - use raw strings always ``` Rule of thumb: every regex pattern should be a raw string `r"..."`. **Reading a regex pattern character by character** Take `r"(\w+)@(\w+\.\w+)"` (a simple email pattern): ``` ( open capturing group 1 \w+ one or more word characters [a-zA-Z0-9_] ) close group 1 @ literal "@" ( open capturing group 2 \w+ one or more word characters \. literal dot (escaped - plain . means "any char") \w+ one or more word characters ) close group 2 ``` ```python m = re.search(r"(\w+)@(\w+\.\w+)", "Send to [email protected] please") m.group(0) # "[email protected]" - full match m.group(1) # "bob" - group 1 m.group(2) # "example.com" - group 2 ``` **Match object methods** ```python m = re.search(r"\d{4}", "Year: 2024 or 2025") if m: m.group() # "2024" - matched text m.start() # 6 - start index m.end() # 10 - end index m.span() # (6, 10) - both at once ``` Always check `if m:` before calling methods - `search` returns `None` when there's no match. **re.sub with a function** The replacement can be a callable, receiving each match object: ```python def double(m): return str(int(m.group()) * 2) re.sub(r"\d+", double, "a=3, b=7") # "a=6, b=14" ``` This is more powerful than a static string replacement.

Groups, non-capturing groups, flags, re.compile, finditer

#
**Capturing groups** let you extract parts of a match. Add parentheses around the part you want to capture. ```python import re log = "2024-03-15 ERROR: disk full" m = re.search(r"(\d{4}-\d{2}-\d{2}) (\w+): (.+)", log) m.group(1) # "2024-03-15" m.group(2) # "ERROR" m.group(3) # "disk full" ``` **Named groups** make code self-documenting: `(?P<name>...)` defines the group, `m.group('name')` retrieves it. ```python m = re.search(r"(?P<date>\d{4}-\d{2}-\d{2}) (?P<level>\w+): (?P<msg>.+)", log) m.group("date") # "2024-03-15" m.group("level") # "ERROR" m.group("msg") # "disk full" ``` **Non-capturing groups** `(?:...)` group without creating a numbered capture - useful when you need alternation or quantifiers but don't want to extract: ```python # Match 'colour' or 'color' but don't capture the optional 'u' re.findall(r"colou?r", "colour and color") # ['colour', 'color'] # Use non-capturing group for alternation inside a larger pattern re.findall(r"(?:cat|dog)s?", "cats and dogs") # ['cats', 'dogs'] ``` **Flags** change how the engine interprets the pattern: ```python re.search(r"hello", "Hello World", re.IGNORECASE) # case-insensitive re.findall(r"^\w+", text, re.MULTILINE) # ^ matches start of each line re.search(r"begin.+end", text, re.DOTALL) # . matches newlines too re.search(r"\d+", text, re.IGNORECASE | re.MULTILINE) # combine with | ``` Inline flag in the pattern: `(?i)hello` means case-insensitive (same as `re.IGNORECASE`). Useful when passing patterns as strings without access to the flags argument. **re.compile** - precompile when you use the same pattern many times: ```python phone_pat = re.compile(r"\+?\d[\d\s\-]{7,14}\d") for line in big_list: m = phone_pat.search(line) # faster than re.search(pattern, line) every time ``` **re.finditer vs re.findall** `findall` returns a list of strings. `finditer` returns an iterator of Match objects - more memory-efficient for large text, and gives you position information: ```python for m in re.finditer(r"\d+", "a1 b22 c333"): print(m.group(), m.start()) # "1" 1 / "22" 4 / "333" 7 ``` **re.split** splits on a pattern (not just a fixed delimiter): ```python re.split(r"[,;\s]+", "one, two; three") # ['one', 'two', 'three'] # Capturing group in the pattern -> delimiter is included in result re.split(r"(,)", "a,b,c") # ['a', ',', 'b', ',', 'c'] ```

Greedy vs lazy, re.VERBOSE, and pattern syntax reference

#
**Greedy vs lazy quantifiers** By default, quantifiers (`*`, `+`, `{n,m}`) are *greedy* - they match as much as possible. Add `?` to make them *lazy* - match as little as possible. ```python import re html = "<b>bold</b> and <i>italic</i>" re.findall(r"<.+>", html) # greedy -> ['<b>bold</b> and <i>italic</i>'] (one big match) re.findall(r"<.+?>", html) # lazy -> ['<b>', '</b>', '<i>', '</i>'] (each tag) ``` Common mistake: using `.+` inside angle brackets and matching too much. Always ask: should this stop at the first possible position? **re.VERBOSE - writing readable complex patterns** Verbose mode ignores whitespace and `#` comments inside the pattern. Use triple single-quotes so the pattern can span multiple lines cleanly: ```python date_pat = re.compile(r''' (?P<year> \d{4} ) # 4-digit year - (?P<month> \d{2} ) # 2-digit month - (?P<day> \d{2} ) # 2-digit day ''', re.VERBOSE) m = date_pat.search("Deadline: 2024-12-31") m.group("year") # "2024" m.group("month") # "12" m.group("day") # "31" ``` Use VERBOSE for any pattern longer than ~40 characters. **Quick pattern syntax reference** ``` . any character except newline (use re.DOTALL to include newline) ^ start of string (or line with re.MULTILINE) $ end of string (or line with re.MULTILINE) \d digit [0-9] \D non-digit \w word character [a-zA-Z0-9_] \W non-word character \s whitespace (space, tab, newline) \S non-whitespace * 0 or more (greedy) *? lazy + 1 or more (greedy) +? lazy ? 0 or 1 {n} exactly n {n,m} n to m (greedy) {n,m}? lazy [abc] character class: a, b, or c [^abc] negated class: anything except a, b, c [a-z] range (...) capturing group (?:...) non-capturing group (?P<name>...) named capturing group | alternation: a|b matches a or b \ escape next character ``` **Common pitfalls** - `.` matches any character - use `\.` for a literal dot - `^` and `$` behave differently with `re.MULTILINE` - test explicitly - `re.match` only checks the start; `re.fullmatch` checks the whole string - don't confuse them for validation - Backtracking in complex patterns can be slow - test with long strings if performance matters

Common regex patterns cookbook

#
A ready-to-use collection of patterns for the most common matching tasks. **Email address (simple)** ```python import re EMAIL = re.compile(r"[a-zA-Z0-9._%+\-]+@[a-zA-Z0-9.\-]+\.[a-zA-Z]{2,}") EMAIL.findall("Contact: [email protected] or [email protected]") # ['[email protected]', '[email protected]'] ``` **Phone numbers (international format)** ```python PHONE = re.compile(r"\+?\d[\d\s\-().]{6,14}\d") PHONE.findall("Call +1 (800) 555-1234 or +44 20 7946 0958") # ['+1 (800) 555-1234', '+44 20 7946 0958'] ``` **ISO date (YYYY-MM-DD)** ```python ISO_DATE = re.compile(r"\b(\d{4})-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01])\b") ISO_DATE.findall("Event on 2024-03-15 and deadline 2024-12-31") # [('2024', '03', '15'), ('2024', '12', '31')] ``` **URL (basic)** ```python URL = re.compile(r"https?://[^\s<>'"]+") URL.findall("See https://docs.python.org and http://example.com/path?q=1") # ['https://docs.python.org', 'http://example.com/path?q=1'] ``` **Extract numbers (int or float)** ```python NUMBER = re.compile(r"-?\d+(?:\.\d+)?") NUMBER.findall("Temp: -3.5 C, altitude: 1200 m") # ['-3.5', '1200'] ``` **Camel to snake case** ```python def camel_to_snake(name): s = re.sub(r"([A-Z]+)([A-Z][a-z])", r"\1_\2", name) return re.sub(r"([a-z\d])([A-Z])", r"\1_\2", s).lower() camel_to_snake("HTTPSResponse") # "https_response" camel_to_snake("myVariableName") # "my_variable_name" ``` **When NOT to use regex** - Parsing HTML/XML - use `html.parser`, `lxml`, or `BeautifulSoup` - Validating email for production - use a library like `email-validator` - Parsing JSON/CSV - use `json`, `csv` modules - Complex nested structure - regex cannot handle recursive grammar Regex is perfect for scanning, extracting, and replacing text patterns. It's the wrong tool when the format has hierarchy or nesting.
01

#

Use `re.search` to check if a string contains a sequence of 3 or more consecutive digits. Return True or False.

import re

def has_three_digits(s):
    # your code here
    pass

print(has_three_digits('abc123def'))  # True
print(has_three_digits('abc12def'))   # False
print(has_three_digits('1234'))       # True
Solution
import re

def has_three_digits(s):
    return bool(re.search(r'\d{3,}', s))
02

#

Use `re.match` to validate that a string is a valid Python identifier (letters, digits, underscores; must start with a letter or underscore).

import re

def is_identifier(s):
    # your code here
    pass

print(is_identifier('my_var'))    # True
print(is_identifier('_private'))  # True
print(is_identifier('2bad'))      # False
print(is_identifier('hello world')) # False
Solution
import re

def is_identifier(s):
    return bool(re.match(r'^[A-Za-z_][A-Za-z0-9_]*$', s))
04

#

Replace all occurrences of multiple spaces with a single space in a string.

import re

def normalize(s):
    # your code here
    pass

print(normalize('hello   world'))   # 'hello world'
print(normalize('  a  b  c  '))     # ' a b c '
Solution
import re

def normalize(s):
    return re.sub(r' {2,}', ' ', s)
05

#

Use capturing groups to extract the year, month, and day from a date string in 'DD/MM/YYYY' format.

import re

def parse_date(s):
    # Return (day, month, year) as integers, or None if format is wrong
    pass

print(parse_date('15/03/2024'))  # (15, 3, 2024)
print(parse_date('01/12/1999'))  # (1, 12, 1999)
print(parse_date('invalid'))     # None
Solution
import re

def parse_date(s):
    m = re.match(r'^(\d{2})/(\d{2})/(\d{4})$', s)
    if not m:
        return None
    return int(m.group(1)), int(m.group(2)), int(m.group(3))
06

#

Use `re.sub` with a function to double all numbers found in a string.

import re

def double_numbers(s):
    # your code here
    pass

print(double_numbers('I have 3 cats and 12 dogs'))  # 'I have 6 cats and 24 dogs'
print(double_numbers('price: 9.99'))                 # 'price: 19.98'
Solution
import re

def double_numbers(s):
    return re.sub(r'\d+\.?\d*', lambda m: str(float(m.group()) * 2), s)
07

#

Compile a regex pattern once and use it to filter a list of strings, keeping only those that match a phone number pattern: `+X-XXX-XXX-XXXX` (where X is a digit).

import re

def filter_phones(strings):
    # your code here
    pass

items = ['+1-800-555-1234', 'not a phone', '+44-020-555-9876', '123-456']
print(filter_phones(items))  # ['+1-800-555-1234', '+44-020-555-9876']
Solution
import re

def filter_phones(strings):
    pattern = re.compile(r'^\+\d+-\d{3}-\d{3}-\d{4}$')
    return [s for s in strings if pattern.match(s)]
08

#

Use named groups to parse a log line of the format `[LEVEL] YYYY-MM-DD: message`. Return a dict with 'level', 'date', and 'message' keys.

import re

def parse_log(line):
    # your code here
    pass

print(parse_log('[ERROR] 2024-03-15: Connection refused'))
# {'level': 'ERROR', 'date': '2024-03-15', 'message': 'Connection refused'}
print(parse_log('[INFO] 2024-01-01: Server started'))
# {'level': 'INFO', 'date': '2024-01-01', 'message': 'Server started'}
Solution
import re

def parse_log(line):
    m = re.match(r'^\[(?P<level>\w+)\] (?P<date>\d{4}-\d{2}-\d{2}): (?P<message>.+)$', line)
    return m.groupdict() if m else None
09

#

Use `re.split` to split a string on any punctuation mark (`.`, `,`, `!`, `?`, `;`, `:`). Filter out empty strings from the result.

import re

def split_on_punctuation(s):
    # your code here
    pass

print(split_on_punctuation('Hello, world! How are you?'))
# ['Hello', ' world', ' How are you']
print(split_on_punctuation('one.two;three:four'))
# ['one', 'two', 'three', 'four']
Solution
import re

def split_on_punctuation(s):
    return [part for part in re.split(r'[.,!?;:]', s) if part]
10

#

Use `re.VERBOSE` (or `re.X`) flag to write a readable, commented regex that validates a strong password: at least 8 chars, at least one uppercase, one lowercase, one digit, one special char (`!@#$%^&*`).

import re

def is_strong_password(s):
    # your code here
    pass

print(is_strong_password('Abc@1234'))    # True
print(is_strong_password('abcdefgh'))    # False (no uppercase/digit/special)
print(is_strong_password('Abc@12'))      # False (too short)
Solution
import re

def is_strong_password(s):
    pattern = re.compile(r'''
        ^                    # start of string
        (?=.*[A-Z])          # at least one uppercase
        (?=.*[a-z])          # at least one lowercase
        (?=.*\d)             # at least one digit
        (?=.*[!@#$%^&*])    # at least one special char
        .{8,}               # at least 8 characters total
        $                   # end of string
    ''', re.VERBOSE)
    return bool(pattern.match(s))