HTTP interface for data import
The Import HTTP API allows you to load JSON data in bulk into ArangoDB
Import JSON data as documents
POST
/_db/{database-name}/_api/import
Load JSON data and store it as documents into the specified collection.
If you import documents into edge collections, all documents require a _from
and a _to
attribute.
Path Parameters
database-name*
string
The name of the database.
Query Parameters
collection*
string
The name of the target collection. The collection needs to exist already.
type
string (default: ""
)
Possible values: ""
, "documents"
, "array"
, "auto"
Determines how the body of the request is interpreted.
documents
: JSON Lines (JSONL) format. Each line is expected to be one
JSON object.
Example:
{"_key":"john","name":"John Smith","age":35}
{"_key":"katie","name":"Katie Foster","age":28}
array
(or list
): JSON format. The request body is expected to be a
JSON array of objects. This format requires ArangoDB to parse the complete
array and keep it in memory for the duration of the import. This is more
resource-intensive than the line-wise JSONL processing.
Any whitespace outside of strings is ignored, which means the JSON data can be
a single line or be formatted as multiple lines.
Example:
[
{"_key":"john","name":"John Smith","age":35},
{"_key":"katie","name":"Katie Foster","age":28}
]
auto
: automatically determines the type (either documents
or array
).
Omit the type
parameter entirely (or set it to an empty string)
to import JSON arrays of tabular data, similar to CSV.
The first line is an array of strings that defines the attribute keys. The
subsequent lines are arrays with the attribute values. The keys and values
are matched by the order of the array elements.
Example:
["_key","name","age"]
["john","John Smith",35]
["katie","Katie Foster",28]
ignoreMissing
boolean (default: false
)
When importing JSON arrays of tabular data (type
parameter is omitted),
the first line of the request body defines the attribute keys and the
subsequent lines the attribute values for each document. Subsequent lines
with a different number of elements than the first line are not imported
by default.
["attr1", "attr2"]
[1, 2] // matching number of elements
[1] // misses 2nd element
[1, 2, 3] // excess 3rd element
You can enable this option to import them anyway. For the missing elements,
the document attributes are omitted. Excess elements are ignored.
fromPrefix
string
The collection name prefix to prepend to all values in the _from
attribute that only specify a document key.
toPrefix
string
The collection name prefix to prepend to all values in the _to
attribute that only specify a document key.
overwriteCollectionPrefix
boolean (default: false
)
Force the fromPrefix
and toPrefix
, possibly replacing existing
collection name prefixes.
overwrite
boolean (default: false
)
If enabled, then all data in the collection is removed prior to the
import. Any existing index definitions are preserved.
waitForSync
boolean (default: false
)
Wait until documents have been synced to disk before returning.
onDuplicate
string (default: "error"
)
Possible values: "error"
, "update"
, "replace"
, "ignore"
Controls what action is carried out in case of a unique key constraint
violation.
error
: this will not import the current document because of the unique
key constraint violation. This is the default setting.update
: this will update an existing document in the database with the
data specified in the request. Attributes of the existing document that
are not present in the request will be preserved.replace
: this will replace an existing document in the database with the
data specified in the request.ignore
: this will not update an existing document and simply ignore the
error caused by a unique key constraint violation.
Note that update
, replace
and ignore
will only work when the
import document in the request contains the _key
attribute. update
and
replace
may also fail because of secondary unique key constraint violations.
complete
boolean (default: false
)
If set to true
, the whole import fails if any error occurs. Otherwise, the
import continues even if some documents are invalid and cannot be imported,
skipping the problematic documents.
details
boolean (default: false
)
If set to true
, the result includes a details
attribute with information
about documents that could not be imported.
Request Body text/plain; charset=utf-8
Responses
201
Created
is returned if all documents could be imported successfully.
The response is a JSON object with the following attributes:
400
Bad Request
The type
contains an invalid value, no collection
is
specified, the documents are incorrectly encoded, or the request
is malformed.
404
Not Found
The collection
parameter or the _from
or _to
attributes of an
imported edge refer to an unknown collection.
409
Conflict
The complete
option is enabled and the import triggers a
unique key violation.
500
Internal Server Error
The complete
option is enabled and the input is invalid,
or the server cannot auto-generate a document key (out of keys error)
for a document with no user-defined key.
Examples
Importing documents with heterogenous attributes from an array of JSON objects:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products&type=list' <<'EOF'
[
{
"_key": "abc",
"value1": 25,
"value2": "test",
"allowed": true
},
{
"_key": "foo",
"name": "baz"
},
{
"name": {
"detailed": "detailed name",
"short": "short name"
}
}
]
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 3,
"errors" : 0,
"empty" : 0,
"updated" : 0,
"ignored" : 0
}
Importing documents using JSON objects separated by new lines (JSONL):
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products&type=documents' <<'EOF'
{ "_key": "abc", "value1": 25, "value2": "test","allowed": true }
{ "_key": "foo", "name": "baz" }
{ "name": { "detailed": "detailed name", "short": "short name" } }
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 3,
"errors" : 0,
"empty" : 1,
"updated" : 0,
"ignored" : 0
}
Using the auto
type detection:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products&type=auto' <<'EOF'
[
{
"_key": "abc",
"value1": 25,
"value2": "test",
"allowed": true
},
{
"_key": "foo",
"name": "baz"
},
{
"name": {
"detailed": "detailed name",
"short": "short name"
}
}
]
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 3,
"errors" : 0,
"empty" : 0,
"updated" : 0,
"ignored" : 0
}
Importing JSONL into an edge collection, with _from
, _to
and name
attributes:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=links&type=documents' <<'EOF'
{ "_from": "products/123", "_to": "products/234" }
{"_from": "products/332", "_to": "products/abc", "name": "other name" }
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 2,
"errors" : 0,
"empty" : 0,
"updated" : 0,
"ignored" : 0
}
Importing an array of JSON objects into an edge collection,
omitting _from
or _to
:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=links&type=list&details=true' <<'EOF'
[
{
"name": "some name"
}
]
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 182
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 0,
"errors" : 1,
"empty" : 0,
"updated" : 0,
"ignored" : 0,
"details" : [
"at position 1: missing '_from' or '_to' attribute, offending document: {\"name\":\"some name\"}"
]
}
Violating a unique constraint, but allowing partial imports:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products&type=documents&details=true' <<'EOF'
{ "_key": "abc", "value1": 25, "value2": "test" }
{ "_key": "abc", "value1": "bar", "value2": "baz" }
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 244
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 1,
"errors" : 1,
"empty" : 0,
"updated" : 0,
"ignored" : 0,
"details" : [
"at position 1: creating document failed with error 'unique constraint violated', offending document: {\"_key\":\"abc\",\"value1\":\"bar\",\"value2\":\"baz\"}"
]
}
Violating a unique constraint, not allowing partial imports:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products&type=documents&complete=true' <<'EOF'
{ "_key": "abc", "value1": 25, "value2": "test" }
{ "_key": "abc", "value1": "bar", "value2": "baz" }
EOF
Show outputHTTP/1.1 409 Conflict
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 85
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"code" : 409,
"error" : true,
"errorMessage" : "unique constraint violated",
"errorNum" : 1210
}
Using a non-existing collection:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products&type=documents' <<'EOF'
{ "name": "test" }
EOF
Show outputHTTP/1.1 404 Not Found
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 97
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"code" : 404,
"error" : true,
"errorMessage" : "collection or view not found: products",
"errorNum" : 1203
}
Using a malformed body with an array of JSON objects being expected:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products&type=list' <<'EOF'
{ }
EOF
Show outputHTTP/1.1 400 Bad Request
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 95
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"code" : 400,
"error" : true,
"errorMessage" : "expecting a JSON array in the request",
"errorNum" : 400
}
Importing two documents using the JSON arrays format. The documents have a
_key
, value1
, and value2
attribute each. One line in the import data is
empty and skipped:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products' <<'EOF'
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]
[ "foo", "bar", "baz" ]
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 2,
"errors" : 0,
"empty" : 1,
"updated" : 0,
"ignored" : 0
}
Importing JSON arrays into an edge collection, with _from
, _to
, and name
attributes:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=links' <<'EOF'
[ "_from", "_to", "name" ]
[ "products/123","products/234", "some name" ]
[ "products/332", "products/abc", "other name" ]
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 72
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 2,
"errors" : 0,
"empty" : 0,
"updated" : 0,
"ignored" : 0
}
Importing JSON arrays into an edge collection, omitting _from
or _to
:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=links&details=true' <<'EOF'
[ "name" ]
[ "some name" ]
[ "other name" ]
EOF
Show outputHTTP/1.1 201 Created
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 281
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"error" : false,
"created" : 0,
"errors" : 2,
"empty" : 0,
"updated" : 0,
"ignored" : 0,
"details" : [
"at position 1: missing '_from' or '_to' attribute, offending document: {\"name\":\"some name\"}",
"at position 2: missing '_from' or '_to' attribute, offending document: {\"name\":\"other name\"}"
]
}
Using a malformed body with JSON arrays being expected:
curl -X POST --header 'accept: application/json' --data-binary @- --dump - 'http://localhost:8529/_api/import?collection=products' <<'EOF'
{ "_key": "foo", "value1": "bar" }
EOF
Show outputHTTP/1.1 400 Bad Request
content-type: application/json
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0, s-maxage=0
connection: Keep-Alive
content-length: 92
content-security-policy: frame-ancestors 'self'; form-action 'self';
expires: 0
pragma: no-cache
server: ArangoDB
strict-transport-security: max-age=31536000 ; includeSubDomains
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
{
"code" : 400,
"error" : true,
"errorMessage" : "no JSON array found in second line",
"errorNum" : 400
}