> For the complete documentation index, see [llms.txt](https://knowledgebase.fabricdata.com/origin/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://knowledgebase.fabricdata.com/origin/apis-all/entertainment-api-docs/initial-ingest-and-ongoing-maintenance.md).
# Initial Ingest and Ongoing Maintenance
This guide covers Fabric Entertainment catalog ingestion and maintenance. Parallel ID enumeration and hydration ingest about 5.1 million records in 3–4 hours.
### 1. Initial Ingest Phase (One-Time Burst)
The initial download is a bounded, one-time operation designed for maximum throughput.
#### Step 1: Enumerate IDs
Retrieve a stable, ordered list of record IDs using the specific /Ids/ endpoints for Movies, Shows, Seasons, and Episodes (e.g., Movies/Ids/).
* Use the Take parameter to specify the maximum number of IDs returned per request. The default is 10,000, and the limit is 100,000.
* Always read the live Total count from each endpoint before partitioning your data.
* The returned IDs are already prefixed (e.g., Movie/, Episode/) and ready for hydration.
#### Step 2: Hydrate in Batches
Use the SearchEntertainment endpoint to retrieve the full documents for the IDs you enumerated.
* Pass up to 10 IDs per request. This 10-document limit is a hard cap for the hydration endpoint.
* Control your payload size using the Includes parameter. Requesting fewer included sub-objects results in smaller payloads and significantly higher throughput.
* Every request must be authenticated by passing your API key via the Authorization header.\
### 2. Concurrency and Initial Rate-Limit Boost
To achieve the 4-hour ingest target, you must process the ID ranges in parallel across multiple agents.
* **Aggregate Limits**: Your account rate limit applies to the combined total of all active agents. Exceeding this limit will result in HTTP 429 (Too Many Requests) errors.
* **Throttling Handling**: Clients must respect the Retry-After header and implement exponential backoff when hitting limits.
* **Temporary Boost Coordination**: Because the initial ingest is a high-volume burst, you must coordinate an estimated time window with Fabric prior to execution. During this agreed-upon window, a temporary rate-limit boost (typically 40–45 requests per second) will be applied to your account to facilitate the rapid download.\
### 3. Ongoing Maintenance (Regular Data Updates)
Once the initial ingest is successfully completed, the temporary rate-limit boost will be removed, and your account will transition to standard operational limits. Ongoing catalog continuity and updates will be managed with a minimum rate limit of 5 calls per second, or the specific threshold defined in your contract.
For continuous updates, we recommend storing the items in their original form and retrieving changes regularly based on your application's needs (e.g., hourly, daily).
#### Delta Retrieval
To pull only the changed records, use the SearchEntertainment endpoint combined with the ModifiedRecordsSince parameter. Do not use the /Ids/ endpoints for ongoing delta maintenance.
* **Time Tracking**: Pass the datetime of the beginning of your last update process to the ModifiedRecordsSince parameter to receive only the items that have changed since that exact time. Always keep track of exactly when you started the current run to use in your next scheduled cycle.
* **Timestamp vs. Modified**: Note that the Modified field in the response only reflects changes to the top-level object, whereas the Timestamp field represents changes to any part of the program. The ModifiedRecordsSince parameter correctly filters against the Timestamp field.
#### Handling Deletions
To keep your cache accurately synced with the catalog, you must process removed records:
* Include Deleted Records: Set the parameter IncludeDeleted=true in your request. This will return removed records. You can check the deleted property in the returned record to determine if it should be purged from your cache.
* Cascading Deletes: Season and Episode records are child records of Show records. If a Show is deleted, all of the episode and season records under that Show should also be removed from your cache. Since Episode and Season records are hard-deleted (removed entirely) rather than soft-deleted, their absence combined with a deleted parent Show indicates they must be purged.
#### Pagination
Paginate through the returned dataset of changed items. If your delta update is exceptionally large, use the NextPageToken guidance for paging datasets greater than 10,000 items.
Repeat this delta retrieval process on your established schedule to ensure continuous catalog synchronization.\
\
*NOTE: In order to specify the response content type, you can use the "format" parameter in your query and pass "json", "xml", "csv", or "html".*
*Related API Guides:* [*Metadata Match Guide*](/origin/apis-all/entertainment-api-docs/metadata-match-guide.md)*,* [*Trending On Guide*](/origin/apis-all/entertainment-api-docs/trending-on-guide.md)*,* [*Celebrity API*](/origin/apis-all/celebrity-api-docs.md)*,* [*Video API*](/origin/apis-all/video-api-docs.md)*,* [*Image API*](/origin/apis-all/image-api-docs.md)*.*
*Open API/Swagger documentation can be found on the Entertainment API page.*