> For the complete documentation index, see [llms.txt](https://knowledgebase.fabricdata.com/insights/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://knowledgebase.fabricdata.com/insights/data-hub/structure-and-files.md).

# Structure and files

This page describes how the Data Hub is physically organized on S3: prefix layout, file format, and segmentation.

## Format

* **Files:** **JSONL** (JSON Lines) — one JSON object per line, UTF-8 encoded.
* Each line is an independent record; files are processed in streaming without loading everything into memory.

## General layout

Storage is **S3-compatible** (Amazon S3 or DigitalOcean Spaces). The layout is identical in both; only the endpoint changes when connecting (see [Data access](/insights/data-hub/data-access.md)).

Each dataset lives under its own prefix, usually with a snapshot in `latest/` and, depending on the dataset, history by date.

```
s3://<listing-bucket>/<Family>/<dataset>/latest/<files>.jsonl
```

Examples of path patterns:

```
# Streaming Availability — stats by countryxplatform
s3://<listing-bucket>/SA/Stats/latest/
    <country>_<platform>.jsonl        

# Release dates
s3://<listing-bucket>/Sources/Nexus/ReleaseDate/latest/release_date.jsonl
```

## Segmentation

Files are usually split by **countryxplatform**, so you can download only the subset you need by filtering on the prefix.

| Dataset                        | Typical segmentation            |
| ------------------------------ | ------------------------------- |
| Streaming Availability / Stats | one file per `countryxplatform` |
| Release dates (Nexus)          | single file in `latest/`        |
| Demand / Hits                  | by date (weekly)                |
| Top Ten                        | by country and date (weekly)    |

## Querying with Athena

JSONL can be queried with Athena by creating an external table with a **JSON SerDe** pointing to the dataset prefix:

```sql
CREATE EXTERNAL TABLE sa_stats (
  platform_id int,
  platform_country string,
  total_movies int,
  total_series int,
  total_episodes int
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://<listing-bucket>/SA/Stats/latest/';
```

For large volumes and repeated queries it's worth converting the JSONL to your own columnar format (CTAS to Parquet) in the client's lake — see [Consumption best practices](/insights/data-hub/consumption-best-practices.md).

## Update frequency

The cadence at which each `latest/` is refreshed depends on the dataset — see [Update frequency](/insights/origin-insights-dashboards/general-reference/frequency.md). In general:

* **Presence / catalog / stats:** daily and weekly.
* **Demand / Top Ten:** weekly.
* **Prices:** monthly / quarterly depending on the feed.

***

<details>

<summary>Related Articles</summary>

* [Getting started](/insights/data-hub/getting-started.md)
* [Data access](/insights/data-hub/data-access.md)
* [Schemas by dataset](/insights/data-hub/schemas-by-dataset.md)

</details>

> We hope you found this article helpful. If you have a question this article doesn't address, reach out on the [Service Desk](https://fabric.atlassian.net/servicedesk/customer/portal/336).