Bulk Insertions

Import

Note: Imports do not trigger callbacks automatically. See Running Callbacks.
Each model has an .import method that will save an array of models in one bulk insert statement.
1
models = [
2
Model.new(id: 1, name: "Fred", age: 14),
3
Model.new(id: 2, name: "Joe", age: 25),
4
Model.new(id: 3, name: "John", age: 30),
5
]
6
7
Model.import(models)
Copied!

update_on_duplicate

The import method has an optional update_on_duplicate + columns params that allows you to specify the columns (as an array of strings) that should be updated if primary constraint is violated.
1
models = [
2
Model.new(id: 1, name: "Fred", age: 14),
3
Model.new(id: 2, name: "Joe", age: 25),
4
Model.new(id: 3, name: "John", age: 30),
5
]
6
7
Model.import(models)
8
9
Model.find!(1).name # => Fred
10
11
models = [
12
Model.new(id: 1, name: "George", age: 14),
13
]
14
15
Model.import(models, update_on_duplicate: true, columns: %w(name))
16
17
Model.find!(1).name # => George
Copied!
NOTE: If using PostgreSQL you must have version 9.5+ to have the on_duplicate_key_update feature.

ignore_on_duplicate

The import method has an optional ignore_on_duplicate param, that takes a boolean, which will skip records if the primary constraint is violated.
1
models = [
2
Model.new(id: 1, name: "Fred", age: 14),
3
Model.new(id: 2, name: "Joe", age: 25),
4
Model.new(id: 3, name: "John", age: 30),
5
]
6
7
Model.import(models)
8
9
Model.find!(1).name # => Fred
10
11
models = [
12
Model.new(id: 1, name: "George", age: 14),
13
]
14
15
Model.import(models, ignore_on_duplicate: true)
16
17
Model.find!(1).name # => Fred
Copied!

batch_size

The import method has an optional batch_size param, that takes an integer. The batch_size determines the number of models to import in each INSERT statement. This defaults to the size of the models array, i.e. only 1 INSERT statement.
1
models = [
2
Model.new(id: 1, name: "Fred", age: 14),
3
Model.new(id: 2, name: "Joe", age: 25),
4
Model.new(id: 3, name: "John", age: 30),
5
Model.new(id: 3, name: "Bill", age: 66),
6
]
7
8
Model.import(models, batch_size: 2)
9
# => First SQL INSERT statement imports Fred and Joe
10
# => Second SQL INSERT statement imports John and Bill
Copied!

Running Callbacks

Since the import method runs on the class level, callbacks are not triggered automatically, they have to be triggered manually. For example, using the Item class with a UUID primary key:
1
require "uuid"
2
3
class Item < Granite::Base
4
connection mysql
5
table items
6
7
column item_id : String, primary: true, auto: false
8
column item_name : String
9
10
before_create :generate_uuid
11
12
def generate_uuid
13
@item_id = UUID.random.to_s
14
end
15
end
Copied!
1
items = [
2
Item.new(item_name: "item1"),
3
Item.new(item_name: "item2"),
4
Item.new(item_name: "item3"),
5
Item.new(item_name: "item4"),
6
]
7
8
# If we did `Item.import(items)` now, it would fail since the item_id wouldn't get set before saving the record, violating the primary key constraint.
9
10
# Manually run the callback on each model to generate the item_id.
11
items.each(&.before_create)
12
13
# Each model in the array now has a item_id set, so can be imported.
14
Item.import(items)
15
16
# This can also be used for a single record.
17
item = Item.new(item_name: "item5")
18
item.before_create
19
item.save
Copied!
Note: Manually running your callbacks is mainly aimed at bulk imports. Running them before a normal .save, for example, would run your callbacks twice.
Last modified 2yr ago