Paging

Some Flare APIs use a common pattern for paging results. This pattern pairs the from request parameter to the next response field.

Using `from` and `next` paging

First Request

Perform the first request without specifying from. This indicates that you want the first page of results.

Subsequent Requests

Use the next response field from your last response as the value for your following request’s from parameter. This indicates that you want the next page of results.All parameters other than from must remain the same as the original request.If next is missing from the last response (or is null), this means that you viewed all pages.

Empty pages (no results) does not mean you viewed all pages. Some filters (like ignored events) will cause empty pages. This means that you should still request the next page even after receiving an empty page.

The `from` Request Parameter

from

string

Used to indicate where to resume paging.Should be empty on the first request.

The `next` Response field

string

The value that should be included in the next request’s from parameter.Most of the time, this is an opaque string.The value of the next response field can be saved for long periods of time. For example if you want to export all results for a given search and then resume to fetch new results the next day.

Page size vs number of returned items

Flare APIs can return less results than the requested page size. This is normal and you should assume that there are still results until you reach a page where next is null.

End-to-End Examples

These are end-to-end examples in various programming languages.

Python SDK Example

import os
import time

from flareio import FlareApiClient


api_key = os.environ.get("FLARE_API_KEY")
if not api_key:
    raise Exception("Please provide an API key")

api_client = FlareApiClient(api_key=api_key)

last_from: str | None = None
fetched_pages: int = 0

for resp in api_client.scroll(
    method="GET",
    url="/leaksdb/v2/sources",
    params={
        "from": last_from,
    },
):
    # Rate limiting (default).
    time.sleep(0.25)

    # Get results from the response
    resp_data = resp.json()
    items = resp_data.get("items")

    fetched_pages += 1
    print(f"Fetched page {fetched_pages} ({last_from=}) with {len(items)} items...")

    # Save the last "next" value.
    last_from = resp_data.get("next") or last_from

print("The last value for 'next' was", last_from)

Python Generic Example

import os
import time

from flareio import FlareApiClient


api_key = os.environ.get("FLARE_API_KEY")
if not api_key:
    raise Exception("Please provide an API key")

api_client = FlareApiClient(api_key=api_key)

from_: str | None = None
fetched_pages: int = 0

while True:
    # Rate limiting (default).
    time.sleep(0.25)

    params: dict = {}
    if from_:
        params["from"] = from_

    # Fetch the next page
    resp = api_client.get(
        "/leaksdb/v2/sources",
        params=params,
    )
    resp.raise_for_status()

    fetched_pages += 1
    print(f"Fetched {fetched_pages} pages...")

    # Stop if there is no "next" value.
    next_page = resp.json().get("next")
    if not next_page:
        print("The last value for 'next' was", from_)
        break
    else:
        print(f"Next page will be from {next_page}")
        from_ = next_page

Go SDK Example

package main

import (
	"fmt"
	"os"
	"time"

	"github.com/Flared/go-flareio"
)

func main() {
	client := flareio.NewApiClient(
		os.Getenv("FLARE_API_KEY"),
	)

	fetchedPages := 0

	for result, err := range client.IterGet(
		"/leaksdb/v2/sources", nil,
	) {
		// Rate Limiting
		time.Sleep(time.Second * 1)

		if err != nil {
			fmt.Printf("unexpected error: %s\n", err)
			os.Exit(1)
		}

		// Handle the response...
		result.Response.Body.Close()

		// Print the status
		fetchedPages = fetchedPages + 1
		fmt.Printf(
			"Fetched %d page(s) of LeaksDB Sources, next=%s\n",
			fetchedPages,
			result.Next,
		)
	}

}

Introduction

Base Concepts

SDK & MCP

Advanced

Guides

Using `from` and `next` paging

The `from` Request Parameter

The `next` Response field

Page size vs number of returned items

End-to-End Examples

Introduction

Base Concepts

SDK & MCP

Advanced

Guides

​Using from and next paging

​The from Request Parameter

​The next Response field

​Page size vs number of returned items

​End-to-End Examples

Using `from` and `next` paging

The `from` Request Parameter

The `next` Response field

Page size vs number of returned items

End-to-End Examples