Python re Module

Python’s re module is a standard library module for handling regular expressions.

Regular expressions (regex or regexp) are powerful tools for matching, searching, and manipulating text.

Through the re module, you can use regular expressions to process strings in Python.

Why Use the re Module?

When processing text, we often need to find specific patterns or replace certain characters. For example, validating email addresses, extracting links from web pages, or formatting text. Writing code manually to accomplish these tasks can be very tedious, while regular expressions provide a concise and efficient way to solve these problems.

---

Basic Usage of the re Module

1. Importing the re Module

Before using the re module, you first need to import it:

import re

2. Common re Module Functions

2.1 re.match()

The re.match() function matches a regular expression from the beginning of a string. If the match is successful, it returns a match object; otherwise, it returns None.

Example

import re

pattern = r"hello"
text = "hello world"

match = re.match(pattern, text)
if match:
    print("Match successful:", match.group())
else:
    print("Match failed")

Output:

Match successful: hello

2.2 re.search()

The re.search() function searches for the first match of a regular expression in a string. Unlike re.match(), re.search() does not require the match to start from the beginning of the string.

Example

import re

pattern = r"world"
text = "hello world"

match = re.search(pattern, text)
if match:
    print("Match successful:", match.group())
else:
    print("Match failed")

Output:

Match successful: world

2.3 re.findall()

The re.findall() function finds all substrings in a string that match the regular expression and returns a list.

Example

import re

pattern = r"\d+"
text = "There are 3 apples and 5 oranges."

matches = re.findall(pattern, text)
print("Numbers found:", matches)

Output:

Numbers found: ['3', '5']

2.4 re.sub()

The re.sub() function replaces parts of a string that match the regular expression.

Example

import re

pattern = r"apple"
text = "I have an apple."

new_text = re.sub(pattern, "banana", text)
print("Replaced text:", new_text)

Output:

Replaced text: I have an banana.

---

Basic Syntax of Regular Expressions

1. Regular Characters

Regular characters (such as letters, numbers) match themselves directly.

Example

import re

pattern = r"cat"
text = "The cat is on the mat."

match = re.search(pattern, text)
if match:
    print("Match successful:", match.group())

Output:

Match successful: cat

2. Special Characters

Regular expressions have some special characters with special meanings. For example:

• .: Matches any single character (except newline).
• *: Matches the preceding character zero or more times.
• +: Matches the preceding character one or more times.
• ?: Matches the preceding character zero or one time.
• \d: Matches any digit character (equivalent to [0-9]).
• \w: Matches any letter, digit, or underscore character (equivalent to [a-zA-Z0-9_]).

Example

import re

pattern = r"\d+"
text = "The price is 100 dollars."

match = re.search(pattern, text)
if match:
    print("Match successful:", match.group())

Output:

Match successful: 100

3. Character Sets

Character sets are used to match any one character from a group of characters. For example, [abc] matches a, b, or c.

Example

import re

pattern = r"[aeiou]"
text = "Hello World!"

matches = re.findall(pattern, text)
print("Vowels found:", matches)

Output:

Vowels found: ['e', 'o', 'o']

4. Grouping

Grouping allows you to combine multiple characters together and perform operations on them. For example, (abc) matches abc.

Example

import re

pattern = r"(ab)+"
text = "ababab"

match = re.search(pattern, text)
if match:
    print("Match successful:", match.group())

Output:

Match successful: ababab

---

Practice Exercises

Exercise 1: Validate an Email Address

Write a regular expression to validate the format of an email address.

Example

import re

pattern = r"^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$"
email = "[email protected]"

if re.match(pattern, email):
    print("Valid email address")
else:
    print("Invalid email address")

Exercise 2: Extract a Phone Number

Write a regular expression to extract a phone number from text.

Example

import re

pattern = r"\d{3}-\d{3}-\d{4}"
text = "My phone number is 123-456-7890."

match = re.search(pattern, text)
if match:
    print("Phone number found:", match.group())

1. Core Functions

Method	Description	Example
re.compile(pattern)	Precompile a regular expression (improves reuse performance)	pat = re.compile(r'\d+')
re.search(pattern, string)	Search for the first match in a string	re.search(r'\d+', 'a1b2') → matches '1'
re.match(pattern, string)	Match from the beginning of the string	re.match(r'\d+', '123a') → matches '123'
re.fullmatch(pattern, string)	Match the entire string completely	re.fullmatch(r'\d+', '123') → matches '123'
re.findall(pattern, string)	Return a list of all non-overlapping matches	re.findall(r'\d+', 'a1b22c') → ['1', '22']
re.finditer(pattern, string)	Return an iterator of all matches (with position info)	for m in re.finditer(r'\d+', 'a1b2'): print(m.group())
re.sub(pattern, repl, string)	Replace matches	re.sub(r'\d+', 'X', 'a1b2') → 'aXbX'
re.split(pattern, string)	Split a string by matches	re.split(r'\d+', 'a1b2c') → ['a', 'b', 'c']
re.escape()	Escape special characters	re.escape("C:\\Users\\test.txt")
re.purge()	Clear the cache	re.purge()

2. Match Object (Match) Methods/Attributes

Method/Attribute	Description	Example
group()	Return the entire matched string	m.group() → 'abc'
group(n)	Return the content of the nth capture group	m = re.search(r'(\d)(\d)', '12'); m.group(1) → '1'
groups()	Return a tuple of all capture groups	m.groups() → ('1', '2')
start()/end()	Start/end position of the match	m.start() → 0
span()	Return the match range (start, end)	m.span() → (0, 2)

3. Regular Expression Metacharacters (Partial)

Metacharacter	Description	Example Match
.	Matches any character (except newline)	a.c → 'abc'
\d	Matches digits	\d+ → '123'
\D	Matches non-digits	\D+ → 'abc'
\w	Matches word characters (letters, digits, underscore)	\w+ → 'Ab_1'
\W	Matches non-word characters	\W+ → '!@#'
\s	Matches whitespace characters (space, tab, etc.)	\s+ → ' \t'
\S	Matches non-whitespace characters	\S+ → 'abc'
[]	Character set	[A-Za-z] → any letter
^	Matches the beginning of a string	^\d+ → digits at the start
$	Matches the end of a string	\d+$ → digits at the end
*	Matches the preceding character 0 or more times	a* → '', 'aaa'
+	Matches the preceding character 1 or more times	a+ → 'a', 'aaa'
?	Matches the preceding character 0 or 1 time	a? → '', 'a'
{m,n}	Matches the preceding character m to n times	a{2,3} → 'aa', 'aaa'
**`	`**	OR operation
()	Capture group	(\d+) → extract digits

4. Compilation Flags (flags Parameter)

Flag	Description	Example
re.IGNORECASE (re.I)	Ignore case	re.search(r'abc', 'ABC', re.I)
re.MULTILINE (re.M)	Multi-line mode (affects ^ and $)	re.findall(r'^\d+', '1\n2', re.M) → ['1', '2']
re.DOTALL (re.S)	Make . match all characters including newline	re.search(r'a.*b', 'a\nb', re.S)
re.ASCII	Make \w, \W, etc. match only ASCII characters	re.search(r'\w+', 'こん', re.ASCII) → no match
re.VERBOSE (re.X)	Allow comments and whitespace in regex	re.compile(r'''\d+ # match digits''', re.X)

Examples

1. Extracting Email Addresses

Example

import re

text = "Contact: [email protected], [email protected]"
emails = re.findall(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b', text)
print(emails)  # Output: ['[email protected]', '[email protected]']

2. Replacing Date Format

Example

date_str = "Today is 05-15-2023"
new_str = re.sub(r'(\d{2})-(\d{2})-(\d{4})', r'\3-\1-\2', date_str)
print(new_str)  # Output: "Today is 2023-05-15"

3. Multi-Condition Matching

Example

pattern = re.compile(r'''
    ^(?P<username>\w+)  # username
    :(?P<password>\S+)  # password
    @(?P<domain>\w+\.\w+)  # domain
$''', re.VERBOSE)

m = pattern.match("john:[email protected]")
if m:
    print(m.groupdict())  # Output: {'username': 'john', 'password': 'pass123', 'domain': 'example.com'}

4. Splitting Complex Strings

Example

text = "Apple1Banana2Cherry3Date"
parts = re.split(r'\d+', text)
print(parts)  # Output: ['Apple', 'Banana', 'Cherry', 'Date']

Important Notes

1. Raw Strings: It is recommended to use raw strings for regular expressions (e.g., r'\d') to avoid escape character conflicts.

2. Greedy Matching: By default, matching is greedy (e.g., .* matches the longest possible). Use .*? for non-greedy matching.

3. Performance Optimization: Frequently used regex should be precompiled with re.compile().

4. Backtracking Issues: Complex regex can lead to performance problems (e.g., nested quantifiers like (a+)+).