HTTP interface for arangosearch Views
The HTTP API for Views lets you manage arangosearch
Views, including handling the general View properties and View links
Create an arangosearch View
cleanupIntervalStep integer (default:
2
)Wait at least this many commits between removing unused files in the ArangoSearch data directory (
0
= disable). For the case where the consolidation policies merge segments often (i.e. a lot of commit+consolidate), a lower value causes a lot of disk space to be wasted. For the case where the consolidation policies rarely merge segments (i.e. few inserts/deletes), a higher value impacts performance without any added benefits.Background: With every “commit” or “consolidate” operation, a new state of the View’s internal data structures is created on disk. Old states/snapshots are released once there are no longer any users remaining. However, the files for the released states/snapshots are left on disk, and only removed by “cleanup” operation.
commitIntervalMsec integer (default:
1000
)Wait at least this many milliseconds between committing View data store changes and making documents visible to queries (
0
= disable). For the case where there are a lot of inserts/updates, a higher value causes the index not to account for them and memory usage continues to grow until the commit. A lower value impacts performance, including the case where there are no or only a few inserts/updates because of synchronous locking, and it wastes disk space for each commit call.Background: For data retrieval, ArangoSearch follows the concept of “eventually-consistent”, i.e. eventually all the data in ArangoDB will be matched by corresponding query expressions. The concept of ArangoSearch “commit” operations is introduced to control the upper-bound on the time until document addition/removals are actually reflected by corresponding query expressions. Once a “commit” operation is complete, all documents added/removed prior to the start of the “commit” operation will be reflected by queries invoked in subsequent ArangoDB transactions, in-progress ArangoDB transactions will still continue to return a repeatable-read state.
consolidationIntervalMsec integer (default:
10000
)Wait at least this many milliseconds between applying
consolidationPolicy
to consolidate the View data store and possibly release space on the filesystem (0
= disable). For the case where there are a lot of data modification operations, a higher value could potentially have the data store consume more space and file handles. For the case where there are a few data modification operations, a lower value impacts performance due to no segment candidates being available for consolidation.Background: For data modification, ArangoSearch follows the concept of a “versioned data store”. Thus old versions of data may be removed once there are no longer any users of the old data. The frequency of the cleanup and compaction operations are governed by
consolidationIntervalMsec
and the candidates for compaction are selected viaconsolidationPolicy
.consolidationPolicy object
The consolidation policy to apply for selecting which segments should be merged.
- If the
tier
type is used, then thesegments*
andminScore
properties are available. - If the
bytes_accum
type is used, then thethreshold
property is available.
Background: With each ArangoDB transaction that inserts documents, one or more ArangoSearch-internal segments get created. Similarly, for removed documents, the segments that contain such documents have these documents marked as ‘deleted’. Over time, this approach causes a lot of small and sparse segments to be created. A “consolidation” operation selects one or more segments and copies all of their valid documents into a single new segment, thereby allowing the search algorithm to perform more optimally and for extra file handles to be released once old segments are no longer used.
type string (default:
"tier"
)The segment candidates for the “consolidation” operation are selected based upon several possible configurable formulas as defined by their types. The currently supported types are:
"tier"
: consolidate based on segment byte size and live document count as dictated by the customization attributes."bytes_accum"
: consolidate if and only if{threshold} > (segment_bytes + sum_of_merge_candidate_segment_bytes) / all_segment_bytes
i.e. the sum of all candidate segment byte size is less than the total segment byte size multiplied by the{threshold}
.
Possible values:
"tier"
,"bytes_accum"
- If the
links object
Expects an object with the attribute keys being names of to be linked collections, and the link properties as attribute values. See
arangosearch
View Link Properties for details.optimizeTopK array of strings
An array of strings defining sort expressions that you want to optimize. This is also known as WAND optimization (introduced in v3.12.0).
This option is immutable.
If you query a View with the
SEARCH
operation in combination with aSORT
andLIMIT
operation, search results can be retrieved faster if theSORT
expression matches one of the optimized expressions.Only sorting by highest rank is supported, that is, sorting by the result of a scoring function in descending order (
DESC
). Use@doc
in the expression where you would normally pass the document variable emitted by theSEARCH
operation to the scoring function.You can define up to 64 expressions per View.
Example:
["BM25(@doc) DESC", "TFIDF(@doc, true) DESC"]
This property is available in the Enterprise Edition only.
primaryKeyCache boolean
If you enable this option, then the primary key columns are always cached in memory (introduced in v3.9.6, Enterprise Edition only). This can improve the performance of queries that return many documents. Otherwise, these values are memory-mapped and it is up to the operating system to load them from disk into memory and to evict them from memory.
This option is immutable.
See the
--arangosearch.columns-cache-limit
startup option to control the memory consumption of this cache. You can reduce the memory usage of the column cache in cluster deployments by only using the cache for leader shards, see the--arangosearch.columns-cache-only-leader
startup option (introduced in v3.10.6).primarySort array of objects
You can define a primary sort order to enable an AQL optimization. If a query iterates over all documents of a View, wants to sort them by attribute values and the (left-most) fields to sort by as well as their sorting direction match with the
primarySort
definition, then theSORT
operation is optimized away. This option is immutable.Expects an array of objects, each specifying a field (attribute path) and a sort direction:
[ { "field": "attr", "direction": "asc"}, … ]
primarySortCache boolean
If you enable this option, then the primary sort columns are always cached in memory (Enterprise Edition only). This can improve the performance of queries that utilize the primary sort order. Otherwise, these values are memory-mapped and it is up to the operating system to load them from disk into memory and to evict them from memory.
This option is immutable.
See the
--arangosearch.columns-cache-limit
startup option to control the memory consumption of this cache. You can reduce the memory usage of the column cache in cluster deployments by only using the cache for leader shards, see the--arangosearch.columns-cache-only-leader
startup option.storedValues array of objects
An array of objects to describe which document attributes to store in the View index. It can then cover search queries, which means the data can be taken from the index directly and accessing the storage engine can be avoided.
This option is immutable.
Each object is expected in the following form:
{ "fields": [ "attr1", "attr2", ... "attrN" ], "compression": "none", "cache": false }
You may use the following shorthand notations on View creation instead of an array of objects as described above. The default compression and cache settings are used in this case:
An array of strings, like
["attr1", "attr2"]
, to place each attribute into a separate column of the index.An array of arrays of strings, like
[["attr1", "attr2"]]
, to place the attributes into a single column of the index, or[["attr1"], ["attr2"]]
to place each attribute into a separate column. You can also mix it with the full form:[ ["attr1"], ["attr2", "attr3"], { "fields": ["attr4", "attr5"], "cache": true } ]
The
storedValues
option is not to be confused with thestoreValues
option, which allows to store meta data about attribute values in the View index.cache boolean (default:
false
)Whether to always cache stored values in memory (Enterprise Edition only). This can improve the query performance if stored values are involved. Otherwise, these values are memory-mapped and it is up to the operating system to load them from disk into memory and to evict them from memory.
See the
--arangosearch.columns-cache-limit
startup option to control the memory consumption of this cache. You can reduce the memory usage of the column cache in cluster deployments by only using the cache for leader shards, see the--arangosearch.columns-cache-only-leader
startup option.fields* array of strings
An array of strings with one or more document attribute paths. The specified attributes are placed into a single column of the index. A column with all fields that are involved in common search queries is ideal for performance. The column should not include too many unneeded fields, however.
writebufferSizeMax integer (default:
33554432
)Maximum memory byte size per writer (segment) before a writer (segment) flush is triggered. The value
0
turns off this limit for any writer (buffer) and data is flushed periodically based on the value defined for the flush thread (ArangoDB server startup option). This should be used carefully due to high potential memory consumption (immutable,0
= disable).
201 Created
The View has been created.
consolidationPolicy* object
The consolidation policy to apply for selecting which segments should be merged.
- If the
tier
type is used, then thesegments*
andminScore
properties are available. - If the
bytes_accum
type is used, then thethreshold
property is available.
type string
The segment candidates for the “consolidation” operation are selected based upon several possible configurable formulas as defined by their types. The currently supported types are:
"tier"
: consolidate based on segment byte size and live document count as dictated by the customization attributes."bytes_accum"
: consolidate if and only if{threshold} > (segment_bytes + sum_of_merge_candidate_segment_bytes) / all_segment_bytes
i.e. the sum of all candidate segment byte size is less than the total segment byte size multiplied by the{threshold}
.
Possible values:
"tier"
,"bytes_accum"
- If the
links* object
An object with the attribute keys being names of to be linked collections, and the link properties as attribute values. See
arangosearch
View Link Properties for details.
Response Body application/json object
Examples
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/view' <<'EOF'
{
"name": "products",
"type": "arangosearch"
}
EOF
Get information about a View
Examples
Using an identifier:
curl --header 'accept: application/json' --dump - 'http://localhost:8529/_api/view/72332'
Using a name:
curl --header 'accept: application/json' --dump - 'http://localhost:8529/_api/view/productsView'
Get the properties of a View
view-name
.200 OK
An object with a full description of the specified View, including
arangosearch
View type-dependent properties.consolidationPolicy* object
The consolidation policy to apply for selecting which segments should be merged.
- If the
tier
type is used, then thesegments*
andminScore
properties are available. - If the
bytes_accum
type is used, then thethreshold
property is available.
type string
The segment candidates for the “consolidation” operation are selected based upon several possible configurable formulas as defined by their types. The currently supported types are:
"tier"
: consolidate based on segment byte size and live document count as dictated by the customization attributes."bytes_accum"
: consolidate if and only if{threshold} > (segment_bytes + sum_of_merge_candidate_segment_bytes) / all_segment_bytes
i.e. the sum of all candidate segment byte size is less than the total segment byte size multiplied by the{threshold}
.
Possible values:
"tier"
,"bytes_accum"
- If the
links* object
An object with the attribute keys being names of to be linked collections, and the link properties as attribute values. See
arangosearch
View Link Properties for details.
Response Body application/json object
Examples
Using an identifier:
curl --header 'accept: application/json' --dump - 'http://localhost:8529/_api/view/72257/properties'
Using a name:
curl --header 'accept: application/json' --dump - 'http://localhost:8529/_api/view/productsView/properties'
List all Views
Examples
Return information about all Views:
curl --header 'accept: application/json' --dump - 'http://localhost:8529/_api/view'
Replace the properties of an arangosearch View
cleanupIntervalStep integer (default:
2
)Wait at least this many commits between removing unused files in the ArangoSearch data directory (
0
= disable). For the case where the consolidation policies merge segments often (i.e. a lot of commit+consolidate), a lower value causes a lot of disk space to be wasted. For the case where the consolidation policies rarely merge segments (i.e. few inserts/deletes), a higher value impacts performance without any added benefits.Background: With every “commit” or “consolidate” operation, a new state of the View’s internal data structures is created on disk. Old states/snapshots are released once there are no longer any users remaining. However, the files for the released states/snapshots are left on disk, and only removed by “cleanup” operation.
commitIntervalMsec integer (default:
1000
)Wait at least this many milliseconds between committing View data store changes and making documents visible to queries (
0
= disable). For the case where there are a lot of inserts/updates, a higher value causes the index not to account for them and memory usage continues to grow until the commit. A lower value impacts performance, including the case where there are no or only a few inserts/updates because of synchronous locking, and it wastes disk space for each commit call.Background: For data retrieval, ArangoSearch follows the concept of “eventually-consistent”, i.e. eventually all the data in ArangoDB will be matched by corresponding query expressions. The concept of ArangoSearch “commit” operations is introduced to control the upper-bound on the time until document addition/removals are actually reflected by corresponding query expressions. Once a “commit” operation is complete, all documents added/removed prior to the start of the “commit” operation will be reflected by queries invoked in subsequent ArangoDB transactions, in-progress ArangoDB transactions will still continue to return a repeatable-read state.
consolidationIntervalMsec integer (default:
10000
)Wait at least this many milliseconds between applying
consolidationPolicy
to consolidate the View data store and possibly release space on the filesystem (0
= disable). For the case where there are a lot of data modification operations, a higher value could potentially have the data store consume more space and file handles. For the case where there are a few data modification operations, a lower value impacts performance due to no segment candidates being available for consolidation.Background: For data modification, ArangoSearch follows the concept of a “versioned data store”. Thus old versions of data may be removed once there are no longer any users of the old data. The frequency of the cleanup and compaction operations are governed by
consolidationIntervalMsec
and the candidates for compaction are selected viaconsolidationPolicy
.consolidationPolicy object
The consolidation policy to apply for selecting which segments should be merged.
- If the
tier
type is used, then thesegments*
andminScore
properties are available. - If the
bytes_accum
type is used, then thethreshold
property is available.
Background: With each ArangoDB transaction that inserts documents, one or more ArangoSearch-internal segments get created. Similarly, for removed documents, the segments that contain such documents have these documents marked as ‘deleted’. Over time, this approach causes a lot of small and sparse segments to be created. A “consolidation” operation selects one or more segments and copies all of their valid documents into a single new segment, thereby allowing the search algorithm to perform more optimally and for extra file handles to be released once old segments are no longer used.
type string (default:
"tier"
)The segment candidates for the “consolidation” operation are selected based upon several possible configurable formulas as defined by their types. The currently supported types are:
"tier"
: consolidate based on segment byte size and live document count as dictated by the customization attributes."bytes_accum"
: consolidate if and only if{threshold} > (segment_bytes + sum_of_merge_candidate_segment_bytes) / all_segment_bytes
i.e. the sum of all candidate segment byte size is less than the total segment byte size multiplied by the{threshold}
.
Possible values:
"tier"
,"bytes_accum"
- If the
links object
Expects an object with the attribute keys being names of to be linked collections, and the link properties as attribute values. See
arangosearch
View Link Properties for details.
200 OK
The View has been updated successfully.
consolidationPolicy* object
The consolidation policy to apply for selecting which segments should be merged.
- If the
tier
type is used, then thesegments*
andminScore
properties are available. - If the
bytes_accum
type is used, then thethreshold
property is available.
type string
The segment candidates for the “consolidation” operation are selected based upon several possible configurable formulas as defined by their types. The currently supported types are:
"tier"
: consolidate based on segment byte size and live document count as dictated by the customization attributes."bytes_accum"
: consolidate if and only if{threshold} > (segment_bytes + sum_of_merge_candidate_segment_bytes) / all_segment_bytes
i.e. the sum of all candidate segment byte size is less than the total segment byte size multiplied by the{threshold}
.
Possible values:
"tier"
,"bytes_accum"
- If the
links* object
An object with the attribute keys being names of to be linked collections, and the link properties as attribute values. See
arangosearch
View Link Properties for details.
Response Body application/json object
Examples
curl -X PUT --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/view/productsView/properties' <<'EOF'
{
"locale": "en"
}
EOF
Update the properties of an arangosearch View
cleanupIntervalStep integer (default:
2
)Wait at least this many commits between removing unused files in the ArangoSearch data directory (
0
= disable). For the case where the consolidation policies merge segments often (i.e. a lot of commit+consolidate), a lower value causes a lot of disk space to be wasted. For the case where the consolidation policies rarely merge segments (i.e. few inserts/deletes), a higher value impacts performance without any added benefits.Background: With every “commit” or “consolidate” operation, a new state of the View’s internal data structures is created on disk. Old states/snapshots are released once there are no longer any users remaining. However, the files for the released states/snapshots are left on disk, and only removed by “cleanup” operation.
commitIntervalMsec integer (default:
1000
)Wait at least this many milliseconds between committing View data store changes and making documents visible to queries (
0
= disable). For the case where there are a lot of inserts/updates, a higher value causes the index not to account for them and memory usage continues to grow until the commit. A lower value impacts performance, including the case where there are no or only a few inserts/updates because of synchronous locking, and it wastes disk space for each commit call.Background: For data retrieval, ArangoSearch follows the concept of “eventually-consistent”, i.e. eventually all the data in ArangoDB will be matched by corresponding query expressions. The concept of ArangoSearch “commit” operations is introduced to control the upper-bound on the time until document addition/removals are actually reflected by corresponding query expressions. Once a “commit” operation is complete, all documents added/removed prior to the start of the “commit” operation will be reflected by queries invoked in subsequent ArangoDB transactions, in-progress ArangoDB transactions will still continue to return a repeatable-read state.
consolidationIntervalMsec integer (default:
10000
)Wait at least this many milliseconds between applying
consolidationPolicy
to consolidate the View data store and possibly release space on the filesystem (0
= disable). For the case where there are a lot of data modification operations, a higher value could potentially have the data store consume more space and file handles. For the case where there are a few data modification operations, a lower value impacts performance due to no segment candidates being available for consolidation.Background: For data modification, ArangoSearch follows the concept of a “versioned data store”. Thus old versions of data may be removed once there are no longer any users of the old data. The frequency of the cleanup and compaction operations are governed by
consolidationIntervalMsec
and the candidates for compaction are selected viaconsolidationPolicy
.consolidationPolicy object
The consolidation policy to apply for selecting which segments should be merged.
- If the
tier
type is used, then thesegments*
andminScore
properties are available. - If the
bytes_accum
type is used, then thethreshold
property is available.
Background: With each ArangoDB transaction that inserts documents, one or more ArangoSearch-internal segments get created. Similarly, for removed documents, the segments that contain such documents have these documents marked as ‘deleted’. Over time, this approach causes a lot of small and sparse segments to be created. A “consolidation” operation selects one or more segments and copies all of their valid documents into a single new segment, thereby allowing the search algorithm to perform more optimally and for extra file handles to be released once old segments are no longer used.
type string (default:
"tier"
)The segment candidates for the “consolidation” operation are selected based upon several possible configurable formulas as defined by their types. The currently supported types are:
"tier"
: consolidate based on segment byte size and live document count as dictated by the customization attributes."bytes_accum"
: consolidate if and only if{threshold} > (segment_bytes + sum_of_merge_candidate_segment_bytes) / all_segment_bytes
i.e. the sum of all candidate segment byte size is less than the total segment byte size multiplied by the{threshold}
.
Possible values:
"tier"
,"bytes_accum"
- If the
links object
Expects an object with the attribute keys being names of to be linked collections, and the link properties as attribute values. See
arangosearch
View Link Properties for details.
200 OK
The View has been updated successfully.
consolidationPolicy* object
The consolidation policy to apply for selecting which segments should be merged.
- If the
tier
type is used, then thesegments*
andminScore
properties are available. - If the
bytes_accum
type is used, then thethreshold
property is available.
type string
The segment candidates for the “consolidation” operation are selected based upon several possible configurable formulas as defined by their types. The currently supported types are:
"tier"
: consolidate based on segment byte size and live document count as dictated by the customization attributes."bytes_accum"
: consolidate if and only if{threshold} > (segment_bytes + sum_of_merge_candidate_segment_bytes) / all_segment_bytes
i.e. the sum of all candidate segment byte size is less than the total segment byte size multiplied by the{threshold}
.
Possible values:
"tier"
,"bytes_accum"
- If the
links* object
An object with the attribute keys being names of to be linked collections, and the link properties as attribute values. See
arangosearch
View Link Properties for details.
Response Body application/json object
Examples
curl -X PATCH --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/view/productsView/properties' <<'EOF'
{
"locale": "en"
}
EOF
Rename a View
Renames a View.
200 OK
The View has been renamed successfully.
consolidationPolicy* object
The consolidation policy to apply for selecting which segments should be merged.
- If the
tier
type is used, then thesegments*
andminScore
properties are available. - If the
bytes_accum
type is used, then thethreshold
property is available.
type string
The segment candidates for the “consolidation” operation are selected based upon several possible configurable formulas as defined by their types. The currently supported types are:
"tier"
: consolidate based on segment byte size and live document count as dictated by the customization attributes."bytes_accum"
: consolidate if and only if{threshold} > (segment_bytes + sum_of_merge_candidate_segment_bytes) / all_segment_bytes
i.e. the sum of all candidate segment byte size is less than the total segment byte size multiplied by the{threshold}
.
Possible values:
"tier"
,"bytes_accum"
- If the
links* object
An object with the attribute keys being names of to be linked collections, and the link properties as attribute values. See
arangosearch
View Link Properties for details.
Response Body application/json object
Examples
curl -X PUT --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/view/productsView/rename' <<'EOF'
{
"name": "catalogView"
}
EOF
Drop a View
view-name
.Examples
Using an identifier:
curl -X DELETE --header 'accept: application/json' --dump - 'http://localhost:8529/_api/view/72410'
Using a name:
curl -X DELETE --header 'accept: application/json' --dump - 'http://localhost:8529/_api/view/productsView'