HTTP interfaces for AQL queries
The HTTP API for AQL queries lets you to execute, track, kill, explain, and validate queries written in ArangoDB’s query language
Cursors
Results of AQL queries are returned as cursors in order to batch the communication between server and client. Each batch contains a number of documents and an indication if the current batch is the final batch. Depending on the query, the total number of documents in the result set may or may not be known in advance.
If the number of documents doesn’t exceed the batch size, the full query result is returned to the client in a single round-trip. If there are more documents, then the first batch is returned to the client and the client needs to use the cursor to retrieve the other batches.
In order to free up server resources, the client should delete a cursor as soon as it is no longer needed.
Single roundtrip
The server only transfers a certain number of result documents back to the
client in one roundtrip. This number is controllable by the client by setting
the batchSize
attribute when issuing the query.
If the complete result can be transferred to the client in one go, the client
does not need to issue any further request. The client can check whether it has
retrieved the complete result set by checking the hasMore
attribute of the
result set. If it is set to false
, then the client has fetched the complete
result set from the server. In this case, no server-side cursor is created.
> curl --data @- -X POST --dump - http://localhost:8529/_api/cursor
{ "query" : "FOR u IN users LIMIT 2 RETURN u", "count" : true, "batchSize" : 2 }
HTTP/1.1 201 Created
Content-type: application/json
{
"hasMore" : false,
"error" : false,
"result" : [
{
"name" : "user1",
"_rev" : "210304551",
"_key" : "210304551",
"_id" : "users/210304551"
},
{
"name" : "user2",
"_rev" : "210304552",
"_key" : "210304552",
"_id" : "users/210304552"
}
],
"code" : 201,
"count" : 2
}
Using a cursor
If the result set contains more documents than should be transferred in a single
roundtrip (i.e. as set via the batchSize
attribute), the server returns
the first few documents and creates a temporary cursor. The cursor identifier
is also returned to the client. The server puts the cursor identifier
in the id
attribute of the response object. Furthermore, the hasMore
attribute of the response object is set to true
. This is an indication
for the client that there are additional results to fetch from the server.
Examples
Create and extract first batch:
> curl --data @- -X POST --dump - http://localhost:8529/_api/cursor
{ "query" : "FOR u IN users LIMIT 5 RETURN u", "count" : true, "batchSize" : 2 }
HTTP/1.1 201 Created
Content-type: application/json
{
"hasMore" : true,
"error" : false,
"id" : "26011191",
"result" : [
{
"name" : "user1",
"_rev" : "258801191",
"_key" : "258801191",
"_id" : "users/258801191"
},
{
"name" : "user2",
"_rev" : "258801192",
"_key" : "258801192",
"_id" : "users/258801192"
}
],
"code" : 201,
"count" : 5
}
Extract next batch, still have more:
> curl -X POST --dump - http://localhost:8529/_api/cursor/26011191
HTTP/1.1 200 OK
Content-type: application/json
{
"hasMore" : true,
"error" : false,
"id" : "26011191",
"result": [
{
"name" : "user3",
"_rev" : "258801193",
"_key" : "258801193",
"_id" : "users/258801193"
},
{
"name" : "user4",
"_rev" : "258801194",
"_key" : "258801194",
"_id" : "users/258801194"
}
],
"code" : 200,
"count" : 5
}
Extract next batch, done:
> curl -X POST --dump - http://localhost:8529/_api/cursor/26011191
HTTP/1.1 200 OK
Content-type: application/json
{
"hasMore" : false,
"error" : false,
"result" : [
{
"name" : "user5",
"_rev" : "258801195",
"_key" : "258801195",
"_id" : "users/258801195"
}
],
"code" : 200,
"count" : 5
}
Do not do this because hasMore
now has a value of false:
> curl -X POST --dump - http://localhost:8529/_api/cursor/26011191
HTTP/1.1 404 Not Found
Content-type: application/json
{
"errorNum": 1600,
"errorMessage": "cursor not found: disposed or unknown cursor",
"error": true,
"code": 404
}
The response object contains a nextBatchId
attribute, except for the last batch
(when hasMore
is false
). If the allowRetry
query option is set to true
and if retrieving a result batch fails because of a connection issue, you
can ask for that batch again using the POST /_api/cursor/<cursor-id>/<batch-id>
endpoint. The first batch has an ID of 1
and the value is incremented by 1
with every batch. Every result response except the last one also includes a
nextBatchId
attribute, indicating the ID of the batch after the current.
You can remember and use this batch ID should retrieving the next batch fail.
> curl --data @- -X POST --dump - http://localhost:8529/_api/cursor
{ "query": "FOR i IN 1..5 RETURN i", "batchSize": 2, "options": { "allowRetry": true } }
HTTP/1.1 201 Created
Content-type: application/json
{
"result":[1,2],
"hasMore":true,
"id":"3517",
"nextBatchId":2,
"cached":false,
"error":false,
"code":201
}
Fetching the second batch would look like this:
> curl -X POST --dump - http://localhost:8529/_api/cursor/3517
Assuming the above request fails because of a connection issue but advances the
cursor nonetheless, you can retry fetching the batch using the nextBatchId
of
the first request (2
):
curl -X POST --dump http://localhost:8529/_api/cursor/3517/2
{
"result":[3,4],
"hasMore":true,
"id":"3517",
"nextBatchId":3,
"cached":false,
"error":false,
"code":200
}
To allow refetching of the last batch of the query, the server cannot
automatically delete the cursor. After the first attempt of fetching the last
batch, the server would normally delete the cursor to free up resources. As you
might need to reattempt the fetch, it needs to keep the final batch when the
allowRetry
option is enabled. Once you successfully received the last batch,
you should call the DELETE /_api/cursor/<cursor-id>
endpoint so that the
server doesn’t unnecessarily keep the batch until the cursor times out
(ttl
query option).
curl -X POST --dump http://localhost:8529/_api/cursor/3517/3
{
"result":[5],
"hasMore":false,
"id":"3517",
"cached":false,
"error":false,
"code":200
}
curl -X DELETE --dump http://localhost:8529/_api/cursor/3517
{
"id":"3517",
"error":false,
"code":202
}
If you are no longer interested in the results of a query, you can call the
DELETE /_api/cursor/<cursor-id>
endpoint as long as a cursor exists to discard
the cursor, even before you requested the last batch.
Execute AQL queries
Create a cursor
Submits an AQL query for execution in the current database. The server returns a result batch and may indicate that further batches need to be fetched using a cursor identifier.
The query details include the query string plus optional query options and bind parameters. These values need to be passed in a JSON representation in the body of the POST request.
x-arango-allow-dirty-read boolean
Set this header to
true
to allow the Coordinator to ask any shard replica for the data, not only the shard leader. This may result in “dirty reads”.The header is ignored if this operation is part of a Stream Transaction (
x-arango-trx-id
header). The header set when creating the transaction decides about dirty reads for the entire transaction, not the individual read operations.
bindVars object
An object with key/value pairs representing the bind parameters. For a bind variable
@var
in the query, specify the value using an attribute with the namevar
. For a collection bind variable@@coll
, use@coll
as the attribute name. For example:"bindVars": { "var": 42, "@coll": "products" }
.count boolean
indicates whether the number of documents in the result set should be returned in the “count” attribute of the result. Calculating the “count” attribute might have a performance impact for some queries in the future so this option is turned off by default, and “count” is only returned when requested.
options object
key/value object with extra options for the query.
allowDirtyReads boolean
If you set this option to
true
and execute the query against a cluster deployment, then the Coordinator is allowed to read from any shard replica and not only from the leader.You may observe data inconsistencies (dirty reads) when reading from followers, namely obsolete revisions of documents because changes have not yet been replicated to the follower, as well as changes to documents before they are officially committed on the leader.
This feature is only available in the Enterprise Edition.
allowRetry boolean
Set this option to
true
to make it possible to retry fetching the latest batch from a cursor. The default isfalse
.If retrieving a result batch fails because of a connection issue, you can ask for that batch again using the
POST /_api/cursor/<cursor-id>/<batch-id>
endpoint. The first batch has an ID of1
and the value is incremented by 1 with every batch. Every result response except the last one also includes anextBatchId
attribute, indicating the ID of the batch after the current. You can remember and use this batch ID should retrieving the next batch fail.You can only request the latest batch again (or the next batch). Earlier batches are not kept on the server-side. Requesting a batch again does not advance the cursor.
You can also call this endpoint with the next batch identifier, i.e. the value returned in the
nextBatchId
attribute of a previous request. This advances the cursor and returns the results of the next batch. This is only supported if there are more results in the cursor (i.e.hasMore
istrue
in the latest batch).From v3.11.1 onward, you may use the
POST /_api/cursor/<cursor-id>/<batch-id>
endpoint even if theallowRetry
attribute isfalse
to fetch the next batch, but you cannot request a batch again unless you set it totrue
.To allow refetching of the very last batch of the query, the server cannot automatically delete the cursor. After the first attempt of fetching the last batch, the server would normally delete the cursor to free up resources. As you might need to reattempt the fetch, it needs to keep the final batch when the
allowRetry
option is enabled. Once you successfully received the last batch, you should call theDELETE /_api/cursor/<cursor-id>
endpoint so that the server doesn’t unnecessarily keep the batch until the cursor times out (ttl
query option).failOnWarning boolean
When set to
true
, the query will throw an exception and abort instead of producing a warning. This option should be used during development to catch potential issues early. When the attribute is set tofalse
, warnings will not be propagated to exceptions and will be returned with the query result. There is also a server configuration option--query.fail-on-warning
for setting the default value forfailOnWarning
so it does not need to be set on a per-query level.fillBlockCache boolean
if set to
true
or not specified, this will make the query store the data it reads via the RocksDB storage engine in the RocksDB block cache. This is usually the desired behavior. The option can be set tofalse
for queries that are known to either read a lot of data which would thrash the block cache, or for queries that read data which are known to be outside of the hot set. By setting the option tofalse
, data read by the query will not make it into the RocksDB block cache if not already in there, thus leaving more room for the actual hot set.fullCount boolean
if set to
true
and the query contains aLIMIT
clause, then the result will have anextra
attribute with the sub-attributesstats
andfullCount
,{ ... , "extra": { "stats": { "fullCount": 123 } } }
. ThefullCount
attribute will contain the number of documents in the result before the last top-level LIMIT in the query was applied. It can be used to count the number of documents that match certain filter criteria, but only return a subset of them, in one go. It is thus similar to MySQL’s SQL_CALC_FOUND_ROWS hint. Note that setting the option will disable a few LIMIT optimizations and may lead to more documents being processed, and thus make queries run longer. Note that thefullCount
attribute may only be present in the result if the query has a top-level LIMIT clause and the LIMIT clause is actually used in the query.maxDNFConditionMembers integer
A threshold for the maximum number of
OR
sub-nodes in the internal representation of an AQLFILTER
condition.Yon can use this option to limit the computation time and memory usage when converting complex AQL
FILTER
conditions into the internal DNF (disjunctive normal form) format.FILTER
conditions with a lot of logical branches (AND
,OR
,NOT
) can take a large amount of processing time and memory. This query option limits the computation time and memory usage for such conditions.Once the threshold value is reached during the DNF conversion of a
FILTER
condition, the conversion is aborted, and the query continues with a simplified internal representation of the condition, which cannot be used for index lookups.You can set the threshold globally instead of per query with the
--query.max-dnf-condition-members
startup option.maxNodesPerCallstack integer
The number of execution nodes in the query plan after that stack splitting is performed to avoid a potential stack overflow. Defaults to the configured value of the startup option
--query.max-nodes-per-callstack
.This option is only useful for testing and debugging and normally does not need any adjustment.
optimizer object
Options related to the query optimizer.
rules array of strings
A list of to-be-included or to-be-excluded optimizer rules can be put into this attribute, telling the optimizer to include or exclude specific rules. To disable a rule, prefix its name with a
-
, to enable a rule, prefix it with a+
. There is also a pseudo-ruleall
, which matches all optimizer rules.-all
disables all rules.
profile integer
If set to
true
or1
, then the additional query profiling information is returned in theprofile
sub-attribute of theextra
return attribute, unless the query result is served from the query cache. If set to2
, the query includes execution stats per query plan node instats.nodes
sub-attribute of theextra
return attribute. Additionally, the query plan is returned in theextra.plan
sub-attribute.skipInaccessibleCollections boolean
Let AQL queries (especially graph traversals) treat collection to which a user has no access rights for as if these collections are empty. Instead of returning a forbidden access error, your queries execute normally. This is intended to help with certain use-cases: A graph contains several collections and different users execute AQL queries on that graph. You can naturally limit the accessible results by changing the access rights of users on collections.
This feature is only available in the Enterprise Edition.
spillOverThresholdMemoryUsage integer
This option allows queries to store intermediate and final results temporarily on disk if the amount of memory used (in bytes) exceeds the specified value. This is used for decreasing the memory usage during the query execution.
This option only has an effect on queries that use the
SORT
operation but without aLIMIT
, and if you enable the spillover feature by setting a path for the directory to store the temporary data in with the--temp.intermediate-results-path
startup option.Default value: 128MB.
Spilling data from RAM onto disk is an experimental feature and is turned off by default. The query results are still built up entirely in RAM on Coordinators and single servers for non-streaming queries. To avoid the buildup of the entire query result in RAM, use a streaming query (see thestream
option).spillOverThresholdNumRows integer
This option allows queries to store intermediate and final results temporarily on disk if the number of rows produced by the query exceeds the specified value. This is used for decreasing the memory usage during the query execution. In a query that iterates over a collection that contains documents, each row is a document, and in a query that iterates over temporary values (i.e.
FOR i IN 1..100
), each row is one of such temporary values.This option only has an effect on queries that use the
SORT
operation but without aLIMIT
, and if you enable the spillover feature by setting a path for the directory to store the temporary data in with the--temp.intermediate-results-path
startup option.Default value:
5000000
rows.Spilling data from RAM onto disk is an experimental feature and is turned off by default. The query results are still built up entirely in RAM on Coordinators and single servers for non-streaming queries. To avoid the buildup of the entire query result in RAM, use a streaming query (see thestream
option).stream boolean
Can be enabled to execute the query lazily. If set to
true
, then the query is executed as long as necessary to produce up tobatchSize
results. These results are returned immediately and the query is suspended until the client asks for the next batch (if there are more results). Depending on the query this can mean that the first results will be available much faster and that less memory is needed because the server only needs to store a subset of results at a time. Read-only queries can benefit the most, unlessSORT
without index orCOLLECT
are involved that make it necessary to process all documents before a partial result can be returned. It is advisable to only use this option for queries without exclusive locks.Remarks:
- The query will hold resources until it ends (such as RocksDB snapshots, which prevents compaction to some degree). Writes will be in memory until the query is committed.
- If existing documents are modified, then write locks are held on these documents and other queries trying to modify the same documents will fail because of this conflict.
- A streaming query may fail late because of a conflict or for other reasons after some batches were already returned successfully, possibly rendering the results up to that point meaningless.
- The query options
cache
,count
andfullCount
are not supported for streaming queries. - Query statistics, profiling data and warnings are delivered as part of the last batch.
If the
stream
option isfalse
(default), then the complete result of the query is calculated before any of it is returned to the client. The server stores the full result in memory (on the contacted Coordinator if in a cluster). All other resources are freed immediately (locks, RocksDB snapshots). The query will fail before it returns results in case of a conflict.
ttl integer
The time-to-live for the cursor (in seconds). If the result set is small enough (less than or equal to
batchSize
) then results are returned right away. Otherwise they are stored in memory and will be accessible via the cursor with respect to thettl
. The cursor will be removed on the server automatically after the specified amount of time. This is useful to ensure garbage collection of cursors that are not fully fetched by clients. If not set, a server-defined value will be used (default: 30 seconds). The time-to-live is renewed upon every access to the cursor.
201 Created
is returned if the result set can be created by the server.
extra object
An optional JSON object with extra information about the query result.
Only delivered as part of the first batch, or the last batch in case of a cursor with the
stream
option enabled.stats* object
An object with query statistics.
cacheHits* integer
The total number of index entries read from in-memory caches for indexes of type edge or persistent. This value is only non-zero when reading from indexes that have an in-memory cache enabled, and when the query allows using the in-memory cache (i.e. using equality lookups on all index attributes).
cacheMisses* integer
The total number of cache read attempts for index entries that could not be served from in-memory caches for indexes of type edge or persistent. This value is only non-zero when reading from indexes that have an in-memory cache enabled, the query allows using the in-memory cache (i.e. using equality lookups on all index attributes) and the looked up values are not present in the cache.
documentLookups* integer
The number of real document lookups caused by late materialization as well as
IndexNode
s that had to load document attributes not covered by the index. This is how many documents had to be fetched from storage after an index scan that initially covered the attribute access for these documents.filtered* integer
The total number of documents removed after executing a filter condition in a
FilterNode
or another node that post-filters data. Note that nodes of theIndexNode
type can also filter documents by selecting only the required index range from a collection, and thefiltered
value only indicates how much filtering was done by a post filter in theIndexNode
itself or followingFilterNode
nodes. Nodes of theEnumerateCollectionNode
andTraversalNode
types can also apply filter conditions and can report the number of filtered documents.fullCount integer
The total number of documents that matched the search condition if the query’s final top-level
LIMIT
operation were not present. This attribute may only be returned if thefullCount
option was set when starting the query and only contains a sensible value if the query contains aLIMIT
operation on the top level.intermediateCommits* integer
The number of intermediate commits performed by the query. This is only non-zero for write queries, and only for queries that reached either the
intermediateCommitSize
orintermediateCommitCount
thresholds. Note: in a cluster, intermediate commits can happen on each participating DB-Server.peakMemoryUsage* integer
The maximum memory usage of the query while it was running. In a cluster, the memory accounting is done per shard, and the memory usage reported is the peak memory usage value from the individual shards. Note that to keep things lightweight, the per-query memory usage is tracked on a relatively high level, not including any memory allocator overhead nor any memory used for temporary results calculations (e.g. memory allocated/deallocated inside AQL expressions and function calls).
nextBatchId string
Only set if the
allowRetry
query option is enabled in v3.11.0. From v3.11.1 onward, this attribute is always set, except in the last batch.The ID of the batch after the current one. The first batch has an ID of
1
and the value is incremented by 1 with every batch. You can remember and use this batch ID should retrieving the next batch fail. Use thePOST /_api/cursor/<cursor-id>/<batch-id>
endpoint to ask for the batch again. You can also request the next batch.
Response Body application/json object400 Bad Request
is returned if the JSON representation is malformed, the query specification is missing from the request, or if the query is invalid.
The body of the response contains a JSON object with additional error details. The object has the following attributes:
Response Body application/json object
Examples
Execute a query and extract the result in a single go
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR p IN products LIMIT 2 RETURN p",
"count": true,
"batchSize": 2
}
EOF
Execute a query and extract a part of the result
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR p IN products LIMIT 5 RETURN p",
"count": true,
"batchSize": 2
}
EOF
Using the query option “fullCount”
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR i IN 1..1000 FILTER i > 500 LIMIT 10 RETURN i",
"count": true,
"options": {
"fullCount": true
}
}
EOF
Enabling and disabling optimizer rules
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR i IN 1..10 LET a = 1 LET b = 2 FILTER a + b == 3 RETURN i",
"count": true,
"options": {
"maxPlans": 1,
"optimizer": {
"rules": [
"-all",
"+remove-unnecessary-filters"
]
}
}
}
EOF
Execute instrumented query and return result together with execution plan and profiling information
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "LET s = SLEEP(0.25) LET t = SLEEP(0.5) RETURN 1",
"count": true,
"options": {
"profile": 2
}
}
EOF
Execute a data-modification query and retrieve the number of modified documents
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR p IN products REMOVE p IN products"
}
EOF
Execute a data-modification query with option ignoreErrors
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "REMOVE 'bar' IN products OPTIONS { ignoreErrors: true }"
}
EOF
The following example appends a value to the array arr
of the document
with key test
in the collection documents
. The normal update behavior of the
UPDATE
operation is to replace the array attribute completely, but using the
PUSH()
function allows you to append to the array:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR doc IN documents FILTER doc._key == @myKey UPDATE doc._key WITH { arr: PUSH(doc.arr, @value) } IN documents RETURN NEW",
"bindVars": {
"myKey": "test",
"value": 42
}
}
EOF
To set a memory limit for the query, the memoryLimit
option can be passed to
the server.
The memory limit specifies the maximum number of bytes that the query is allowed to use. When a single AQL query reaches the specified limit value, the query is aborted with a resource limit exceeded exception. In a cluster, the memory accounting is done per server, so the limit value is effectively a memory limit per query per server.
If no memory limit is specified, then the server default value (controlled by
startup option --query.memory-limit
) is used for restricting the maximum amount
of memory the query can use. A memory limit value of 0
means that the maximum
amount of memory for the query is not restricted.
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR i IN 1..100000 SORT i RETURN i",
"memoryLimit": 100000
}
EOF
Bad query - Missing body
curl -X POST --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor'
Bad query - Unknown collection
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR u IN unknowncoll LIMIT 2 RETURN u",
"count": true,
"batchSize": 2
}
EOF
Bad query - Execute a data-modification query that attempts to remove a non-existing document
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "REMOVE 'foo' IN products"
}
EOF
Read the next batch from a cursor
If the cursor is still alive, returns an object with the next query result batch.
If the cursor is not fully consumed, the time-to-live for the cursor is renewed by this API call.
200 OK
The server will respond with HTTP 200 in case of success.
extra object
An optional JSON object with extra information about the query result.
Only delivered as part of the first batch, or the last batch in case of a cursor with the
stream
option enabled.stats* object
An object with query statistics.
cacheHits* integer
The total number of index entries read from in-memory caches for indexes of type edge or persistent. This value is only non-zero when reading from indexes that have an in-memory cache enabled, and when the query allows using the in-memory cache (i.e. using equality lookups on all index attributes).
cacheMisses* integer
The total number of cache read attempts for index entries that could not be served from in-memory caches for indexes of type edge or persistent. This value is only non-zero when reading from indexes that have an in-memory cache enabled, the query allows using the in-memory cache (i.e. using equality lookups on all index attributes) and the looked up values are not present in the cache.
documentLookups* integer
The number of real document lookups caused by late materialization as well as
IndexNode
s that had to load document attributes not covered by the index. This is how many documents had to be fetched from storage after an index scan that initially covered the attribute access for these documents.filtered* integer
The total number of documents removed after executing a filter condition in a
FilterNode
or another node that post-filters data. Note that nodes of theIndexNode
type can also filter documents by selecting only the required index range from a collection, and thefiltered
value only indicates how much filtering was done by a post filter in theIndexNode
itself or followingFilterNode
nodes. Nodes of theEnumerateCollectionNode
andTraversalNode
types can also apply filter conditions and can report the number of filtered documents.fullCount integer
The total number of documents that matched the search condition if the query’s final top-level
LIMIT
operation were not present. This attribute may only be returned if thefullCount
option was set when starting the query and only contains a sensible value if the query contains aLIMIT
operation on the top level.intermediateCommits* integer
The number of intermediate commits performed by the query. This is only non-zero for write queries, and only for queries that reached either the
intermediateCommitSize
orintermediateCommitCount
thresholds. Note: in a cluster, intermediate commits can happen on each participating DB-Server.peakMemoryUsage* integer
The maximum memory usage of the query while it was running. In a cluster, the memory accounting is done per shard, and the memory usage reported is the peak memory usage value from the individual shards. Note that to keep things lightweight, the per-query memory usage is tracked on a relatively high level, not including any memory allocator overhead nor any memory used for temporary results calculations (e.g. memory allocated/deallocated inside AQL expressions and function calls).
nextBatchId string
Only set if the
allowRetry
query option is enabled in v3.11.0. From v3.11.1 onward, this attribute is always set, except in the last batch.The ID of the batch after the current one. The first batch has an ID of
1
and the value is incremented by 1 with every batch. You can remember and use this batch ID should retrieving the next batch fail. Use thePOST /_api/cursor/<cursor-id>/<batch-id>
endpoint to ask for the batch again. You can also request the next batch.
Response Body application/json object
Examples
Valid request for next batch
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR p IN products LIMIT 5 RETURN p",
"count": true,
"batchSize": 2
}
EOF
curl -X POST --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor/71864'
Missing identifier
curl -X POST --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor'
Unknown identifier
curl -X POST --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor/123123'
Read the next batch from a cursor (deprecated)
If the cursor is still alive, returns an object with the following attributes:
id
: acursor-identifier
result
: a list of documents for the current batchhasMore
:false
if this was the last batchcount
: if present the total number of elementscode
: an HTTP status codeerror
: a boolean flag to indicate whether an error occurrederrorNum
: a server error number (iferror
istrue
)errorMessage
: a descriptive error message (iferror
istrue
)extra
: an object with additional information about the query result, with the nested objectsstats
andwarnings
. Only delivered as part of the last batch in case of a cursor with thestream
option enabled.
Note that even if hasMore
returns true
, the next call might
still return no documents. If, however, hasMore
is false
, then
the cursor is exhausted. Once the hasMore
attribute has a value of
false
, the client can stop.
If the cursor is not fully consumed, the time-to-live for the cursor is renewed by this API call.
Examples
Valid request for next batch
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR p IN products LIMIT 5 RETURN p",
"count": true,
"batchSize": 2
}
EOF
curl -X PUT --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor/71886'
Missing identifier
curl -X PUT --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor'
Unknown identifier
curl -X PUT --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor/123123'
Read a batch from the cursor again
You can use this endpoint to retry fetching the latest batch from a cursor.
The endpoint requires the allowRetry
query option to be enabled for the cursor.
Calling this endpoint with the last returned batch identifier returns the query results for that same batch again. This does not advance the cursor. Client applications can use this to re-transfer a batch once more in case of transfer errors.
You can also call this endpoint with the next batch identifier, i.e. the value
returned in the nextBatchId
attribute of a previous request. This advances the
cursor and returns the results of the next batch.
From v3.11.1 onward, you may use this endpoint even if the allowRetry
attribute is false
to fetch the next batch, but you cannot request a batch
again unless you set it to true
.
Note that it is only supported to query the last returned batch identifier or
the directly following batch identifier. The latter is only supported if there
are more results in the cursor (i.e. hasMore
is true
in the latest batch).
Note that when the last batch has been consumed successfully by a client application, it should explicitly delete the cursor to inform the server that it successfully received and processed the batch so that the server can free up resources.
The time-to-live for the cursor is renewed by this API call.
200 OK
The server responds with HTTP 200 in case of success.
extra object
An optional JSON object with extra information about the query result.
Only delivered as part of the first batch, or the last batch in case of a cursor with the
stream
option enabled.stats* object
An object with query statistics.
cacheHits* integer
The total number of index entries read from in-memory caches for indexes of type edge or persistent. This value is only non-zero when reading from indexes that have an in-memory cache enabled, and when the query allows using the in-memory cache (i.e. using equality lookups on all index attributes).
cacheMisses* integer
The total number of cache read attempts for index entries that could not be served from in-memory caches for indexes of type edge or persistent. This value is only non-zero when reading from indexes that have an in-memory cache enabled, the query allows using the in-memory cache (i.e. using equality lookups on all index attributes) and the looked up values are not present in the cache.
documentLookups* integer
The number of real document lookups caused by late materialization as well as
IndexNode
s that had to load document attributes not covered by the index. This is how many documents had to be fetched from storage after an index scan that initially covered the attribute access for these documents.filtered* integer
The total number of documents removed after executing a filter condition in a
FilterNode
or another node that post-filters data. Note that nodes of theIndexNode
type can also filter documents by selecting only the required index range from a collection, and thefiltered
value only indicates how much filtering was done by a post filter in theIndexNode
itself or followingFilterNode
nodes. Nodes of theEnumerateCollectionNode
andTraversalNode
types can also apply filter conditions and can report the number of filtered documents.fullCount integer
The total number of documents that matched the search condition if the query’s final top-level
LIMIT
operation were not present. This attribute may only be returned if thefullCount
option was set when starting the query and only contains a sensible value if the query contains aLIMIT
operation on the top level.intermediateCommits* integer
The number of intermediate commits performed by the query. This is only non-zero for write queries, and only for queries that reached either the
intermediateCommitSize
orintermediateCommitCount
thresholds. Note: in a cluster, intermediate commits can happen on each participating DB-Server.peakMemoryUsage* integer
The maximum memory usage of the query while it was running. In a cluster, the memory accounting is done per shard, and the memory usage reported is the peak memory usage value from the individual shards. Note that to keep things lightweight, the per-query memory usage is tracked on a relatively high level, not including any memory allocator overhead nor any memory used for temporary results calculations (e.g. memory allocated/deallocated inside AQL expressions and function calls).
nextBatchId string
Only set if the
allowRetry
query option is enabled in v3.11.0. From v3.11.1 onward, this attribute is always set, except in the last batch.The ID of the batch after the current one. The first batch has an ID of
1
and the value is incremented by 1 with every batch. You can remember and use this batch ID should retrieving the next batch fail. Use thePOST /_api/cursor/<cursor-id>/<batch-id>
endpoint to ask for the batch again. You can also request the next batch.
Response Body application/json object
Examples
Request the second batch (again):
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR i IN 1..5 RETURN i",
"count": true,
"batchSize": 2,
"options": {
"allowRetry": true
}
}
EOF
curl -X POST --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor/71895'
curl -X POST --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor/71895/2'
Delete a cursor
Deletes the cursor and frees the resources associated with it.
The cursor will automatically be destroyed on the server when the client has retrieved all documents from it. The client can also explicitly destroy the cursor at any earlier time using an HTTP DELETE request. The cursor id must be included as part of the URL.
Note: the server will also destroy abandoned cursors automatically after a certain server-controlled timeout to avoid resource leakage.
Examples
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/cursor' <<'EOF'
{
"query": "FOR p IN products LIMIT 5 RETURN p",
"count": true,
"batchSize": 2
}
EOF
curl -X DELETE --header 'accept: application/json' --dump - 'http://localhost:8529/_api/cursor/71913'
Track queries
You can track AQL queries by enabling query tracking. This allows you to list all currently executing queries. You can also list slow queries and clear this list.
Get the AQL query tracking configuration
Returns the current query tracking configuration. The configuration is a JSON object with the following properties:
enabled
: if set totrue
, then queries will be tracked. If set tofalse
, neither queries nor slow queries will be tracked.trackSlowQueries
: if set totrue
, then slow queries will be tracked in the list of slow queries if their runtime exceeds the value set inslowQueryThreshold
. In order for slow queries to be tracked, theenabled
property must also be set totrue
.trackBindVars
: if set totrue
, then bind variables used in queries will be tracked.maxSlowQueries
: the maximum number of slow queries to keep in the list of slow queries. If the list of slow queries is full, the oldest entry in it will be discarded when additional slow queries occur.slowQueryThreshold
: the threshold value for treating a query as slow. A query with a runtime greater or equal to this threshold value will be put into the list of slow queries when slow query tracking is enabled. The value forslowQueryThreshold
is specified in seconds.maxQueryStringLength
: the maximum query string length to keep in the list of queries. Query strings can have arbitrary lengths, and this property can be used to save memory in case very long query strings are used. The value is specified in bytes.
Update the AQL query tracking configuration
The properties need to be passed in the attribute properties
in the body
of the HTTP request. properties
needs to be a JSON object.
After the properties have been changed, the current set of properties will be returned in the HTTP response.
List the running AQL queries
Returns an array containing the AQL queries currently running in the selected database. Each query is a JSON object with the following attributes:
id
: the query’s iddatabase
: the name of the database the query runs inuser
: the name of the user that started the queryquery
: the query string (potentially truncated)bindVars
: the bind parameter values used by the querystarted
: the date and time when the query was startedrunTime
: the query’s run time up to the point the list of queries was queriedpeakMemoryUsage
: the query’s peak memory usage in bytes (in increments of 32KB)state
: the query’s current execution state (as a string). One of:"initializing"
"parsing"
"optimizing ast"
"loading collections"
"instantiating plan"
"optimizing plan"
"instantiating executors"
"executing"
"finalizing"
"finished"
"killed"
"invalid"
stream
: whether or not the query uses a streaming cursor
List the slow AQL queries
Returns an array containing the last AQL queries that are finished and
have exceeded the slow query threshold in the selected database.
The maximum amount of queries in the list can be controlled by setting
the query tracking property maxSlowQueries
. The threshold for treating
a query as slow can be adjusted by setting the query tracking property
slowQueryThreshold
.
Each query is a JSON object with the following attributes:
id
: the query’s iddatabase
: the name of the database the query runs inuser
: the name of the user that started the queryquery
: the query string (potentially truncated)bindVars
: the bind parameter values used by the querystarted
: the date and time when the query was startedrunTime
: the query’s total run timepeakMemoryUsage
: the query’s peak memory usage in bytes (in increments of 32KB)state
: the query’s current execution state (will always be “finished” for the list of slow queries)stream
: whether or not the query uses a streaming cursor
Clear the list of slow AQL queries
Kill queries
Running AQL queries can be killed on the server. To kill a running query, its ID (as returned for the query in the list of currently running queries) must be specified. The kill flag of the query is then set, and the query is aborted as soon as it reaches a cancelation point.
Kill a running AQL query
Explain and parse AQL queries
You can retrieve the execution plan for any valid AQL query, as well as syntactically validate AQL queries. Both functionalities don’t actually execute the supplied AQL query, but only inspect it and return meta information about it.
You can also retrieve a list of all query optimizer rules and their properties.
Explain an AQL query
To explain how an AQL query would be executed on the server, the query string can be sent to the server via an HTTP POST request. The server will then validate the query and create an execution plan for it. The execution plan will be returned, but the query will not be executed.
The execution plan that is returned by the server can be used to estimate the probable performance of the query. Though the actual performance will depend on many different factors, the execution plan normally can provide some rough estimates on the amount of work the server needs to do in order to actually run the query.
By default, the explain operation will return the optimal plan as chosen by
the query optimizer The optimal plan is the plan with the lowest total estimated
cost. The plan will be returned in the attribute plan
of the response object.
If the option allPlans
is specified in the request, the result will contain
all plans created by the optimizer. The plans will then be returned in the
attribute plans
.
The result will also contain an attribute warnings
, which is an array of
warnings that occurred during optimization or execution plan creation. Additionally,
a stats
attribute is contained in the result with some optimizer statistics.
If allPlans
is set to false
, the result will contain an attribute cacheable
that states whether the query results can be cached on the server if the query
result cache were used. The cacheable
attribute is not present when allPlans
is set to true
.
Each plan in the result is a JSON object with the following attributes:
nodes
: the array of execution nodes of the plan.estimatedCost
: the total estimated cost for the plan. If there are multiple plans, the optimizer will choose the plan with the lowest total cost.collections
: an array of collections used in the queryrules
: an array of rules the optimizer applied.variables
: array of variables used in the query (note: this may contain internal variables created by the optimizer)
bindVars object
An object with key/value pairs representing the bind parameters. For a bind variable
@var
in the query, specify the value using an attribute with the namevar
. For a collection bind variable@@coll
, use@coll
as the attribute name. For example:"bindVars": { "var": 42, "@coll": "products" }
.options object
Options for the query
optimizer object
Options related to the query optimizer.
rules array of strings
A list of to-be-included or to-be-excluded optimizer rules can be put into this attribute, telling the optimizer to include or exclude specific rules. To disable a rule, prefix its name with a
-
, to enable a rule, prefix it with a+
. There is also a pseudo-ruleall
, which matches all optimizer rules.-all
disables all rules.
Examples
Valid query
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/explain' <<'EOF'
{
"query": "FOR p IN products RETURN p"
}
EOF
A plan with some optimizer rules applied
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/explain' <<'EOF'
{
"query": "FOR p IN products LET a = p.id FILTER a == 4 LET name = p.name SORT p.id LIMIT 1 RETURN name"
}
EOF
Using some options
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/explain' <<'EOF'
{
"query": "FOR p IN products LET a = p.id FILTER a == 4 LET name = p.name SORT p.id LIMIT 1 RETURN name",
"options": {
"maxNumberOfPlans": 2,
"allPlans": true,
"optimizer": {
"rules": [
"-all",
"+use-index-for-sort",
"+use-index-range"
]
}
}
}
EOF
Returning all plans
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/explain' <<'EOF'
{
"query": "FOR p IN products FILTER p.id == 25 RETURN p",
"options": {
"allPlans": true
}
}
EOF
A query that produces a warning
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/explain' <<'EOF'
{
"query": "FOR i IN 1..10 RETURN 1 / 0"
}
EOF
Invalid query (missing bind parameter)
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/explain' <<'EOF'
{
"query": "FOR p IN products FILTER p.id == @id LIMIT 2 RETURN p.n"
}
EOF
The data returned in the plan attribute of the result contains one element per AQL top-level statement
(i.e. FOR
, RETURN
, FILTER
etc.). If the query optimizer removed some unnecessary statements,
the result might also contain less elements than there were top-level statements in the AQL query.
The following example shows a query with a non-sensible filter condition that the optimizer has removed so that there are less top-level statements.
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/explain' <<'EOF'
{ "query" : "FOR i IN [ 1, 2, 3 ] FILTER 1 == 2 RETURN i" }
EOF
Parse an AQL query
/api/cursor
.200 OK
If the query is valid, the server will respond with HTTP 200 and return the names of the bind parameters it found in the query (if any) in the
bindVars
attribute of the response. It will also return an array of the collections used in the query in thecollections
attribute. If a query can be parsed successfully, theast
attribute of the returned JSON will contain the abstract syntax tree representation of the query. The format of theast
is subject to change in future versions of ArangoDB, but it can be used to inspect how ArangoDB interprets a given query. Note that the abstract syntax tree will be returned without any optimizations applied to it.
Examples
a valid query
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/query' <<'EOF'
{ "query" : "FOR i IN 1..100 FILTER i > 10 LIMIT 2 RETURN i * 3" }
EOF
an invalid query
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/query' <<'EOF'
{ "query" : "FOR i IN 1..100 FILTER i = 1 LIMIT 2 RETURN i * 3" }
EOF
List all AQL optimizer rules
Examples
Retrieve the list of all query optimizer rules:
curl --header 'accept: application/json' --dump - 'http://localhost:8529/_api/query/rules'