Python · API · Advanced

Async Requests with httpx

5 tasks

Use httpx with asyncio to make concurrent API requests and speed up multi-call scripts.

Async requests and httpx

#
When you make HTTP requests sequentially, each one blocks the script until the server responds. If you need to fetch 13 categories, you wait for response 1, then response 2, and so on — the total time is the sum of all individual response times. Async requests let you send all 13 requests at once and wait for all of them in parallel. If each request takes 200ms, sequential takes 2.6s. Parallel takes ~200ms. httpx is a modern HTTP client that mirrors the requests API but adds async support via Python's asyncio: ```python import httpx # Synchronous — same as requests response = httpx.get('https://apilearn.tukas.dev/api/products/') print(response.json()['count']) ``` For async, use AsyncClient with async/await: ```python import httpx import asyncio async def main(): async with httpx.AsyncClient() as client: response = await client.get('https://apilearn.tukas.dev/api/products/') print(response.json()['count']) asyncio.run(main()) ``` async with ensures the client is properly closed even if an error occurs. await pauses the coroutine until the response arrives, allowing other coroutines to run in the meantime. asyncio.gather() is the key to running multiple requests in parallel — it takes a list of coroutines and runs them all concurrently: ```python results = await asyncio.gather( client.get(url1), client.get(url2), client.get(url3), ) # results is a list of Response objects, in the same order as the calls ``` When to use async: - You have many independent requests (fetching all categories, all pages, all product details). - The requests do not depend on each other's results. - Each request has non-trivial latency (network calls almost always do). Async does not help when requests are sequential by nature (e.g. you need the result of request 1 to know what to ask in request 2).

Parallel requests with asyncio.gather

#
Fetch all category product counts in parallel: ```python import httpx import asyncio BASE_URL = 'https://apilearn.tukas.dev' async def fetch_category_count(client, slug): response = await client.get( f'{BASE_URL}/api/products/', params={'category': slug, 'page_size': 1}, ) return slug, response.json()['count'] async def main(): # Get all category slugs first async with httpx.AsyncClient() as client: r = await client.get(f'{BASE_URL}/api/categories/') categories = r.json()['results'] slugs = [cat['slug'] for cat in categories] # Fetch all counts concurrently tasks = [fetch_category_count(client, slug) for slug in slugs] results = await asyncio.gather(*tasks) for slug, count in results: print(f'{slug}: {count}') asyncio.run(main()) ``` Fetch 5 specific products by id in parallel: ```python import httpx import asyncio BASE_URL = 'https://apilearn.tukas.dev' async def fetch_product(client, product_id): response = await client.get(f'{BASE_URL}/api/products/{product_id}/') return response.json() async def main(): product_ids = [1, 2, 3, 4, 5] async with httpx.AsyncClient() as client: products = await asyncio.gather( *[fetch_product(client, pid) for pid in product_ids] ) for p in products: print(p['name'], p['price']) asyncio.run(main()) ``` The *[...] unpacks the list into positional arguments for asyncio.gather. gather(*tasks) is equivalent to gather(task1, task2, task3, ...). Note on httpx vs requests: httpx supports both sync and async, has a nearly identical API to requests, and handles HTTP/2. For sync-only scripts requests is fine. Use httpx when you need async.

httpx and asyncio reference

#
httpx sync (drop-in for requests): ```python import httpx response = httpx.get(url) response = httpx.post(url, json={...}) response.json() response.status_code response.raise_for_status() ``` httpx async pattern: ```python import httpx import asyncio async def main(): async with httpx.AsyncClient() as client: response = await client.get(url) data = response.json() asyncio.run(main()) ``` asyncio.gather for parallel requests: ```python results = await asyncio.gather( client.get(url1), client.get(url2), ) # or with a list: results = await asyncio.gather(*[client.get(u) for u in urls]) ``` httpx vs requests: ``` requests httpx Sync Yes Yes Async No Yes (AsyncClient) API Standard Nearly identical HTTP/2 No Yes ```
01

First async request

#

Write an async function using httpx.AsyncClient that fetches /api/products/ and prints the product count. Run it with asyncio.run().

import httpx
import asyncio

BASE_URL = 'https://apilearn.tukas.dev'

async def fetch_products():
    # Use httpx.AsyncClient to fetch /api/products/
    pass

asyncio.run(fetch_products())
Solution
import httpx
import asyncio

BASE_URL = 'https://apilearn.tukas.dev'

async def fetch_products():
    async with httpx.AsyncClient() as client:
        response = await client.get(f'{BASE_URL}/api/products/')
        data = response.json()
        print(data['count'])

asyncio.run(fetch_products())
02

Async fetch of one endpoint

#

Write an async function fetch_products() that uses httpx.AsyncClient to fetch /api/products/ and returns the count. Call it with asyncio.run() and print the result.

import httpx
import asyncio

BASE_URL = 'https://apilearn.tukas.dev'

async def fetch_products():
    # Use AsyncClient to fetch products and return count
    pass

print(asyncio.run(fetch_products()))
Solution
import httpx
import asyncio

BASE_URL = 'https://apilearn.tukas.dev'

async def fetch_products():
    async with httpx.AsyncClient() as client:
        response = await client.get(f'{BASE_URL}/api/products/')
        return response.json()['count']

print(asyncio.run(fetch_products()))
03

Fetch multiple products in parallel

#

Write a coroutine fetch_product(client, product_id) that fetches /api/products/{id}/. Use asyncio.gather() to fetch products with ids 1 to 5 simultaneously. Print the name and price of each.

import httpx
import asyncio

BASE_URL = 'https://apilearn.tukas.dev'

async def fetch_product(client, product_id):
    # GET /api/products/{product_id}/
    pass

async def main():
    async with httpx.AsyncClient() as client:
        # Gather and run 5 requests in parallel
        pass

asyncio.run(main())
Solution
import httpx
import asyncio

BASE_URL = 'https://apilearn.tukas.dev'

async def fetch_product(client, product_id):
    response = await client.get(f'{BASE_URL}/api/products/{product_id}/')
    return response.json()

async def main():
    async with httpx.AsyncClient() as client:
        products = await asyncio.gather(*[
            fetch_product(client, i) for i in range(1, 6)
        ])
    for p in products:
        print(p['name'], p['price'])

asyncio.run(main())
04

Fetch multiple products by id

#

Fetch products with ids 1, 2, 3, 4, and 5 concurrently using asyncio.gather. Print the name and price of each.

import httpx
import asyncio

BASE_URL = 'https://apilearn.tukas.dev'

async def fetch_product(client, product_id):
    # GET /api/products/{product_id}/ and return the JSON
    pass

async def main():
    product_ids = [1, 2, 3, 4, 5]
    # Fetch all in parallel and print name + price
    pass

asyncio.run(main())
Solution
import httpx
import asyncio

BASE_URL = 'https://apilearn.tukas.dev'

async def fetch_product(client, product_id):
    r = await client.get(f'{BASE_URL}/api/products/{product_id}/')
    return r.json()

async def main():
    product_ids = [1, 2, 3, 4, 5]
    async with httpx.AsyncClient() as client:
        products = await asyncio.gather(
            *[fetch_product(client, pid) for pid in product_ids]
        )
    for p in products:
        print(p['name'], p['price'])

asyncio.run(main())
05

Compare sequential vs parallel

#

Fetch 10 pages of products (/api/products/?page=N) both ways: first sequentially in a for loop (measuring time), then concurrently with asyncio.gather (measuring time). Print both elapsed times.

import httpx
import asyncio
import time

BASE_URL = 'https://apilearn.tukas.dev'

# Sequential: fetch pages 1-10 one by one
# Parallel: fetch pages 1-10 concurrently
# Print elapsed time for each approach
Solution
import httpx
import asyncio
import time

BASE_URL = 'https://apilearn.tukas.dev'

pages = list(range(1, 11))

# Sequential
start = time.time()
for page in pages:
    httpx.get(f'{BASE_URL}/api/products/', params={'page': page})
print(f'Sequential: {time.time() - start:.2f}s')

# Parallel
async def fetch_page(client, page):
    return await client.get(
        f'{BASE_URL}/api/products/', params={'page': page},
    )

async def run_parallel():
    async with httpx.AsyncClient() as client:
        await asyncio.gather(*[fetch_page(client, p) for p in pages])

start = time.time()
asyncio.run(run_parallel())
print(f'Parallel:   {time.time() - start:.2f}s')