Search in All of Flare's Events

Global searches in Flare is exposed through the global/_search endpoint. This guide details a typical search use case and provides an example for how to impement it.

Use Case

Export all chat messages collected by Flare in the last 6 hours that match the fraud keyword.
Save our last page cursor so that we can resume fetching results in a future execution.

Body Parameters

To achieve the desired results, the following body parameters will be used:

Parameter	Value	Justification
query.query_string	fraud	The keyword we are looking for.
filter.types	chat_message	Retrieve only the chat_message events.
order	asc	Retrieve results in ascending order so that we can resume fetching in the future.
filter.estimated_created_at.gte	`<from_date>`	Replace `<from_date>` with a timestamp that corresponds to 6 hours that we can fetch all results until the current time.

Query string examples

Global search supports the Lucene Query Syntax, which supports, among other things, boolean operators and regexes. Here are some common example queries.

Description	Example
Search for a telegram channel named “Best Telegram Source”	conversation_name:“Best Telegram Source”

Paging

The search endpoint uses parameters that match the Flare standard paging pattern .

Fetching new results in future executions

It is possible to save the next in a database and use it to resume fetching new results in the future. However, it is important that future requests use the same parameters for everything else but next. Even the time filter.

End-to-End Examples

These are end-to-end examples in various programming languages.

Python SDK Example

import datetime
import time

from flareio import FlareApiClient


api_client = FlareApiClient.from_env()

from_timestamp: str = (
    datetime.datetime.now(tz=datetime.timezone.utc) - datetime.timedelta(hours=1)
).isoformat()

last_from: str | None = None
fetched_pages: int = 0

for resp in api_client.scroll(
    method="POST",
    url="/firework/v4/events/global/_search",
    json={
        "size": 10,
        "order": "asc",
        "from": last_from,
        "filters": {
            "type": ["chat_message"],
            "estimated_created_at": {"gte": from_timestamp},
        },
        "query": {
            "type": "query_string",
            "query_string": "hello",
        },
    },
):
    # Rate limiting (search).
    time.sleep(1)

    resp_data: dict = resp.json()
    items: list[dict] = resp_data["items"]

    fetched_pages += 1
    print(f"Fetched page {fetched_pages} with {len(items)} results...")

    # Save the last "next" value.
    last_from = resp_data.get("next") or last_from

    # (Optional): Get the full data
    for item in items:
        # Rate limiting (default).
        time.sleep(0.25)

        event_response = api_client.get(
            url="/firework/v2/activities/",
            params={
                "uid": item["metadata"]["uid"],
            },
        )

        full_data = event_response.json()
        print(f"Here is the full data of the event: {full_data}")

print(f"The next execution could resume using {last_from=}.")

Introduction

Base Concepts

SDK

Advanced

Guides

Search in All of Flare's Events

Use Case

Body Parameters

Query string examples

Paging

Fetching new results in future executions

End-to-End Examples

Introduction

Base Concepts

SDK

Advanced

Guides

​Use Case

​Body Parameters

​Query string examples

​Paging

​Fetching new results in future executions

​End-to-End Examples

Use Case

Body Parameters

Query string examples

Paging

Fetching new results in future executions

End-to-End Examples