Skip to main content

Search Configuration

An index is configured by three things, all of which can be set when it is created and changed afterwards:

SettingPurpose
featuresWhat is searchable: which models run over the content, or which existing tag tracks are indexed.
aggregationHow tags are grouped into the documents that get searched.
search_configThe default behavior when searching the index.

Set on creation with Create an index:

POST /vector_search/indexes/<index_qid> HTTP/1.1
Host: ai-04.contentfabric.io
Authorization: Bearer <token>
Content-Type: application/json

{
  "qids": ["<qid1>", "<qid2>"],
  "features": ["video_description", "speech"],
  "aggregation": "shot",
  "search_config": {
    "rerank_level": "document",
    "clips_min_duration_seconds": 15,
    "clips_max_duration_seconds": 45
  }
}

Features​

A feature is one searchable field. In the common case it is just the name of a feature preset: a named bundle of one or more tagger models, set up for search. Adding it starts those models on every content in the index and indexes the tags they produce.

For advanced configuration you may pass a json object instead of a name.

"features": [
  "video_description",
  {"name": "ocr", "params": {"model_params": {"thres": 0.7}}},
  {"name": "captions", "tracks": ["subtitles_en", "subtitles_es"]}
]
FieldTypeDescription
namestringThe field name. A feature preset name, unless tracks is given.
paramsobjectTagger overrides for the preset's model, accepting whatever the tagger's per-job overrides section accepts. Mutually exclusive with tracks, and not allowed on presets that run more than one model.
tracksstring[]Existing tagstore tracks to index under name, instead of running a model. Nothing is tagged for you, so you are responsible for these tracks existing. In this case name must not be the name of a tagger model or feature preset.

Multiple tracks under one name are grouped into a single search field.

Feature presets​

GET /features lists the presets available on the server, along with the models each one runs:

GET /vector_search/features HTTP/1.1
Host: ai-04.contentfabric.io
Authorization: Bearer <token>
{
  "features": [
    {"name": "people", "description": "Face detection and identification ...", "models": ["face_vectors", "face_tagger"]},
    {"name": "speech", "description": "Speech Transcription", "models": ["asr"]},
    {"name": "ocr", "description": "Optical Character Recognition", "models": ["ocr"]},
    {"name": "video_description", "description": "Video Understanding", "models": ["scene_description"]}
  ]
}
note

The people preset requires some basic tenant-level setup before it can be used. Contact Eluvio to get your tenant configured.

Changing features​

Refreshing a feature​

A refresh resubmits a feature's tagging jobs for every content currently in the index. Use it when jobs have failed (see Check indexing progress) or when a model has been updated and you want new tags.

POST /vector_search/indexes/<index_qid>/features/<feature>/refresh?hard=true HTTP/1.1
Host: ai-04.contentfabric.io
Authorization: Bearer <token>
{"feature": "<feature>", "contents": ["<qid1>", "<qid2>"]}
Query paramDefaultDescription
hardfalseWhen false, each feature follows its own replace policy which may use cached output from previous runs. Set to true to rerun the feature entirely from scratch skipping any potential caching.

Only features that run models can be refreshed. A feature defined with tracks returns a 400, since there is no job to resubmit.

Changing contents​

The set of contents is managed the same way, with POST /indexes/{index_id}/contents and DELETE /indexes/{index_id}/contents/{qid}. Contents added to an index are tagged with its features automatically.

Aggregation​

Aggregation defines the video time-range segments over which we search. It defaults to "shot", which runs shot detection on the content and groups tags by shot boundary — for most content there is nothing to configure here.

When a segmentation model is used, the indexer creates aggregated documents for each of its segments: all configured tags overlapping that segment are grouped into a single textual document, along with the content's fabric-level field data.

What's the point?

Aggregating gives us rich contextual information for a scene that can be used to answer complex queries which might span across different fields. For example a query like: "Jennifer Lawrence talks with a mechanic in No Hard Feelings" could match 3 separate fields for cast/celebrity, visual scene description, and film title.

Fixed time buckets​

Instead of a segmentation model, aggregation may be set to a fixed interval in whole seconds like "10s" or "15s". Tags are then grouped into contiguous, fixed-width windows of that duration. This is useful when shot boundaries are not a meaningful unit for your content.

"aggregation": "10s"

Search Defaults​

search_config overrides the defaults for the clip search arguments, so callers don't have to specify them on every request. Update it with PATCH /indexes/{index_id}/search_config, which merges in only the options you supply.

PATCH /vector_search/indexes/<index_qid>/search_config HTTP/1.1
Host: ai-04.contentfabric.io
Authorization: Bearer <token>
Content-Type: application/json

{
  "rerank_level": "document",
  "rerank_user_query": true
}
OptionTypeDefaultDescription
clips_min_duration_secondsnumber—Minimum clip duration in seconds.
clips_max_duration_secondsnumber—Maximum clip duration in seconds.
rerank_levelstring"tag"How much context to give the reranker: "chunk", "tag", or "document", in increasing order.
rerank_user_queryboolfalseEmbed the user's original query for the semantic search instead of the decomposed semantic component.
disable_rerankingboolfalseSkip reranking entirely and leave results in order of vector similarity.
rewriter_promptstring—Instructions for an LLM that rewrites the query before the rest of the pipeline runs. Empty skips the rewrite step.
note

For detailed explanations of these arguments, consult the clip search API docs.