Working with CSV & JSON in Python

Exchanging tabular and structured data between applications relies heavily on standard formats like CSV (Comma-Separated Values) and JSON (JavaScript Object Notation). Python includes powerful built-in modules—csv and json—that let you parse, serialize, and transform structured files without relying on third-party dependencies.

1. Working with CSV Data (`csv` Module)

The csv module handles parsing tabular data while properly escaping delimeters, quotes, and newline characters across operating systems.

Class / Function	Input / Target	Primary Use Case
`csv.reader(file)`	File object	Iterates over CSV lines, returning each row as a list of strings.
`csv.writer(file)`	File object	Writes sequence objects (lists, tuples) as comma-separated rows.
`csv.DictReader(file)`	File object	Parses rows into native Python `dict` objects using the header row as keys.
`csv.DictWriter(file, fieldnames)`	File object + field headers	Writes dictionaries directly to CSV using defined header mappings.

Example 1: Reading and Writing CSV Files with DictReader & DictWriter

Using DictReader and DictWriter keeps your code explicit by treating each row as a dictionary mapped to column headers.

import csv # Sample structured dataset employees = [ {"Name": "Alice Vance", "Role": "Backend Developer", "Salary": "92000"}, {"Name": "Bob Smith", "Role": "Data Engineer", "Salary": "88000"}, {"Name": "Charlie Day", "Role": "DevOps Engineer", "Salary": "95000"} ] fieldnames = ["Name", "Role", "Salary"] # Writing to a CSV file (Always specify newline='' for cross-platform compatibility) with open("employees.csv", mode="w", newline="", encoding="utf-8") as csv_file: writer = csv.DictWriter(csv_file, fieldnames=fieldnames) writer.writeheader() writer.writerows(employees) print("CSV File 'employees.csv' successfully created!") # Reading from the generated CSV file with open("employees.csv", mode="r", encoding="utf-8") as csv_file: reader = csv.DictReader(csv_file) print("\n[PARSED CSV DATA]") for row in reader: print(f"Employee: {row['Name']} | Position: {row['Role']} | Compensation: ${row['Salary']}")

Important Note: Always set newline='' when calling open() in write mode for CSV files. On platforms like Windows, omitting this argument results in extra blank line breaks between every record.

2. Working with JSON Data (`json` Module)

JSON is the standard format for web APIs and configuration files. Python's json module maps standard JSON data types directly to native Python data types.

JSON Data Type	Python Data Type	Serialization / Deserialization Method
Object (`{}`)	`dict`	`json.dumps()` (String) / `json.dump()` (File Stream)
Array (`[]`)	`list`	`json.loads()` (String) / `json.load()` (File Stream)
String (`""`)	`str`	Maps automatically during conversion
Number (`10 / 3.14`)	`int` / `float`	Maps automatically during conversion
Boolean / Null (`true/false/null`)	`True` / `False` / `None`	Maps automatically during conversion

Example 2: JSON String & File Operations (dumps vs dump, loads vs load)

The s suffix in loads and dumps stands for String, while load and dump operate directly on file descriptors.

import json # Python dictionary with mixed primitive data types app_config = { "app_name": "DataPipelineServer", "version": 2.4, "debug_mode": False, "allowed_hosts": ["localhost", "127.0.0.1", "api.compillo.com"], "database": { "engine": "postgresql", "port": 5432 } } # 1. Serializing Python Dict to a formatted JSON String (indent=4 for pretty printing) json_string = json.dumps(app_config, indent=4) print("[SERIALIZED JSON STRING]") print(json_string) # 2. Writing JSON data directly to a physical file with open("config.json", "w", encoding="utf-8") as f: json.dump(app_config, f, indent=4) # 3. Deserializing back from the JSON file into a Python Dictionary with open("config.json", "r", encoding="utf-8") as f: loaded_config = json.load(f) print(f"\nSuccessfully loaded config for: {loaded_config['app_name']}") print(f"Primary Host: {loaded_config['allowed_hosts'][2]}")

Example 3: Handling Custom Objects using Custom JSONEncoders

By default, the json module throws a TypeError when encountering non-serializable objects like Python datetime instances or custom classes. Implement custom encoder subclasses to serialize complex objects gracefully.

import json from datetime import datetime class Transaction: def __init__(self, tx_id, amount, status): self.tx_id = tx_id self.amount = amount self.status = status self.timestamp = datetime.now() # Custom JSON Encoder subclassing json.JSONEncoder class CustomTransactionEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, datetime): return obj.isoformat() if isinstance(obj, Transaction): return { "transaction_id": obj.tx_id, "amount": obj.amount, "status": obj.status, "timestamp": obj.timestamp } return super().default(obj) # Create custom class instance tx = Transaction("TX-99021", 450.75, "COMPLETED") # Serialize using custom encoder class serialized_tx = json.dumps(tx, cls=CustomTransactionEncoder, indent=2) print("[CUSTOM OBJECT SERIALIZATION]") print(serialized_tx)

Best Practice: Use json.dumps(data, indent=4, sort_keys=True) during debugging or configuration dumping. Sorting keys produces clean, consistent diffs in git repositories.

🏋️ Try It Yourself: Practice Challenges

Challenge 1: CSV-to-JSON Converter

Write a script that reads a CSV file containing user details (id, username, email) using csv.DictReader and converts the dataset into a pretty-printed JSON file.

Challenge 2: Robust JSON Config Loader with Fallbacks

Build a function load_config(filepath) that reads a JSON configuration file. If the file is missing or contains invalid JSON syntax (json.JSONDecodeError), handle the error safely and return a default configuration dictionary.

Challenge 3: Custom Data Sanitizer

Create a script that reads a JSON payload containing user profiles, filters out profiles with active: false, and outputs the filtered dataset into a new CSV file with csv.DictWriter.

⚡ Interactive Sandbox (CSV & JSON Data Processing)

import csv
import json
import io

# 1. Prepare sample CSV payload using StringIO stream
csv_raw_data = """id,item,price,stock
101,Mechanical Keyboard,120.50,15
102,Wireless Mouse,45.00,42
103,4K Monitor,340.00,8"""

# Read CSV and transform into Python list of dictionaries
csv_file_stream = io.StringIO(csv_raw_data)
reader = csv.DictReader(csv_file_stream)

inventory = []
for row in reader:
    # Convert data types for json compliance
    row['id'] = int(row['id'])
    row['price'] = float(row['price'])
    row['stock'] = int(row['stock'])
    inventory.append(row)

# 2. Serialize transformed data into JSON output
json_result = json.dumps({"inventory_catalog": inventory}, indent=2)

print("--- TRANSFORMED CSV TO JSON RESULT ---")
print(json_result)

Console Output:

Click "Run Code" above to execute CSV & JSON conversion script...

📝 Knowledge Check Quiz

1. Why is setting `newline=''` recommended when opening a file for writing CSV data in Python?

To enable binary encoding for non-ASCII characters To prevent extra blank lines between rows across operating systems To automatically sort column headers alphabetically

2. What is the primary difference between `json.dumps()` and `json.dump()`?

`dumps()` serializes data to a JSON string; `dump()` writes JSON directly to a file object `dumps()` is used for dictionaries; `dump()` is used strictly for lists `dumps()` encrypts output; `dump()` writes plain unformatted text

3. Which class should you subclass to handle custom object serialization in the `json` module?

`csv.DictWriter` `json.JSONDecoder` `json.JSONEncoder`

Horizontal Ad Banner Slot (728x90 / Responsive)

Working with CSV and JSON Data in Python

1. Working with CSV Data (csv Module)