Auto-scaling example
Beginner
Tutorial
Overview
Projects deployed on ICP can use a variety of different architectures with regard to how services and assets are organized. Projects may use a single canister to hold the entire dapp's assets and functions, or they may separate services across different canisters, deploy different canisters on different subnets, or separate different data into different canisters.
This page demonstrates the concept of auto-scaling architectures using an older workflow with heap storage. For current projects, it is recommended to use either a single canister architecture with stable memory, as described in the write overview page, or create an auto-scaling architecture that creates new canisters on a different subnet.
One multi-canister architecture option is to configure a primary canister that has the ability to spawn other canisters based on certain criteria, such as when a new user is created or if the application's total data reaches a certain amount.
Auto-scaling example
This guide will provide an example of a multi-canister architecture where there are two types of canisters:
Primary canister: Spawns secondary canisters whenever the application's storage limit is reached.
Secondary canister: Stores data. In this example, each secondary data canister is storing text notes that are each limited to 1 MiB of storage. Whenever a secondary canister reaches 2 GiB of total used storage, the primary canister will create another secondary canister to store additional data.
This architecture is used to prevent the application from trapping or resulting in data loss due to running out of available heap storage in a single canister.
The full code example can be found in the sample's GitHub repo. This guide will walk through the code logic of the primary ('main') canister and the secondary ('datastore') canister.
First, this primary canister (stored in the Main.mo file) imports a variety of packages from the Motoko base library and data from the IC.mo, Types.mo, and Datastore.mo files:
import Cycles "mo:base/ExperimentalCycles";
import Debug "mo:base/Debug";
import Iter "mo:base/Iter";
import HashMap "mo:base/HashMap";
import Option "mo:base/Option";
import List "mo:base/List";
import Principal "mo:base/Principal";
import Result "mo:base/Result";
import Nat "mo:base/Nat";
import ICType "./IC";
import Types "./Types";
import Datastore "./Datastore";
Then, it creates a shared actor class containing type definitions:
shared ({ caller }) actor class Self() {
type UserId = Types.UserId;
type NoteId = Types.NoteId;
type User = Types.User;
type DatastoreCanisterId = Types.DatastoreCanisterId;
type Byte = Types.Byte;
type Note = Types.Note;
type DefiniteNote = Types.DefiniteNote;
Bind the caller principal to an _admin variable:
let _admin : Principal = caller;
Set a variable for the management canister:
// The IC management canister.
let IC : ICType.Self = actor "aaaaa-aa";
Currently, a single canister smart contract is limited to 4 GiB of heap storage. In this project, you will ensure that the datastore canister does not meet this limit by restricting the memory usage to 2 GiB:
let DATASTORE_CANISTER_CAPACITY : Byte = 2_000_000_000;
This application will store notes. Each note will be limited to 1 MiB:
let NOTE_DATA_SIZE = 1_000_000;
The number of notes on a single datastore canister can be calculated as follows:
// DATASTORE_CANISTER_CAPACITY / NOTE_DATA_SIZE
let _numberOfDataPerCanister : Nat = DATASTORE_CANISTER_CAPACITY / NOTE_DATA_SIZE;
Configure stable variables for the application:
stable var _count = 0;
stable var _currentDatastoreCanisterId : ?DatastoreCanisterId = null;
stable var _stableUsers : [(UserId, User)] = [];
stable var _stableDatastoreCanisterIds : [DatastoreCanisterId] = [];
let _users = HashMap.fromIter<UserId, User>(
_stableUsers.vals(), 10, Principal.equal, Principal.hash
);
var _datastoreCanisterIds = List.fromArray(_stableDatastoreCanisterIds);
var _dataStoreCanister : ?Types.Datastore = null;
Define a series of functions for different purposes:
// Returns the current number of notes.
public query func count() : async Nat {
_count;
};
// Returns the number of notes on a single datastore canister.
public query func numberOfDataPerCanister() : async Nat {
_numberOfDataPerCanister;
};
// Returns the number of secondary (datastore) canisters
public query func sizeOfDatastoreCanisterIds() : async Nat {
List.size(_datastoreCanisterIds);
};
// Returns the cycle balance. Useful for monitoring.
public query func balance() : async Nat {
Cycles.balance()
};
// Returns a canister id of the current secondary (datastore) canister.
// Traps if there is no secondary canister.
public query ({ caller }) func currentDatastoreCanisterId(): async Result.Result<DatastoreCanisterId,Text> {
switch _currentDatastoreCanisterId {
case null { #err "A datastore canister is currently null." };
case (?canisterId_) { #ok canisterId_ };
}
};
// Returns a canister id of the canister containing a note of [noteId]
// Traps if:
// - [caller] is not a registered user.
// - [noteId] exceeds [_count].
// - [index] exceeds the size of [_datastoreCanisterIds] list.
public query ({ caller }) func getCanisterIdByNoteId(noteId: NoteId) : async Result.Result<DatastoreCanisterId, Text> {
if (not (_isUserRegistered caller)) { return #err "You are not registered." };
if (noteId >= _count) { return #err "Out of bounds error."};
let index = noteId / _numberOfDataPerCanister;
switch (List.get(List.reverse(_datastoreCanisterIds), index)){
case null { #err ("Canister ID is not found. index: " # Nat.toText(index) # " size: " # Nat.toText(List.size(_datastoreCanisterIds))) };
case (?id) { #ok id };
}
};
// Returns [user_.id].
// Traps if [caller] is not a registered user.
public query ({ caller }) func userId(): async Result.Result<UserId,Text> {
switch (_users.get(caller)) {
case (?user_) { #ok (user_.id) };
case null { #err "You are not registered." };
}
};
// Returns [_datastoreCanisterIds] as a array.
// Traps if [caller] is not a registered user.
public query ({ caller }) func datastoreCanisterIds(): async Result.Result<[DatastoreCanisterId],Text> {
if (not (_isUserRegistered caller)) { return #err "You are not registered." };
#ok (List.toArray(_datastoreCanisterIds))
};
// Register [caller] as a new user.
// Returns [caller] if the registration process successfully finishes.
// Traps if:
// - [caller] is not a registered user.
// - [caller] is the anonymous identity.
public shared ({ caller }) func register(): async Result.Result<UserId,Text>{
if (Principal.isAnonymous(caller)) { return #err "You need to be authenticated." };
switch (_users.get(caller)) {
case (?_) {
#err "This principal id is already in use."
};
case null {
let user = {
id = caller;
var name = "";
};
_users.put(caller, user);
#ok caller
};
}
};
Define the logic to create a new note in the current datastore caller:
// Traps if:
// - [caller] is not a registered user.
// - there is no datastore canister.
// - it fails to generate a new datastore canister.
public shared ({ caller }) func createNote(title: Text, content: Text) : async Result.Result<DefiniteNote, Text> {
if (not (_isUserRegistered caller)) { return #err "You are not registered." };
// Check the current datastore has reached its limit.
if (_count % _numberOfDataPerCanister == 0){
// Generate a new datastore canister
switch (await _generateDataStoreCanister()){
case (#err m) { return #err m };
case (#ok canisterId_){
_currentDatastoreCanisterId := ?canisterId_;
};
}
};
let noteId = _count;
_count += 1;
switch _currentDatastoreCanisterId {
case null { #err "A datastore canister is currently null." };
case (?canisterId_){
let dataStoreCanister = _getDatastoreCanister(canisterId_);
await dataStoreCanister.createNote(caller, canisterId_, noteId, title, content);
}
}
};
// Returns `true` if [caller] is a registered user.
public shared query ({ caller }) func isRegistered(): async Bool {
_isUserRegistered(caller)
};
// Sets [_dataStoreCanister] to an actor of a datastore canister.
// Returns an actor of a datastore canister.
private func _getDatastoreCanister(canisterId: Principal) : Types.Datastore {
switch _dataStoreCanister {
case null {
let canister = actor (Principal.toText(canisterId)) : Types.Datastore;
_dataStoreCanister := ?canister;
canister
};
case (?d) { return d }
}
};
Below is the logic that spawns a new canister from this master canister:
// Generates a new datastore canister.
// Returns a canister id of the generated canister.
// Traps if it fails to generate a new canister.
private func _generateDataStoreCanister(): async Result.Result<DatastoreCanisterId,Text>{
try {
Cycles.add(4_000_000_000_000);
let noteStoreCanister = await Datastore.Self(NOTE_DATA_SIZE);
let canisterId = Principal.fromActor(noteStoreCanister);
_currentDatastoreCanisterId := ?canisterId;
_dataStoreCanister := ?noteStoreCanister;
_datastoreCanisterIds := List.push(canisterId, _datastoreCanisterIds);
let settings: ICType.CanisterSettings = {
controllers = [_admin];
};
let params: ICType.UpdateSettings = {
canister_id = canisterId;
settings = settings;
};
await IC.update_settings(params);
#ok (canisterId)
} catch (e) {
#err "An error occurred in generating a datastore canister."
}
};
// Returns `true` if [principal] is a registered user.
private func _isUserRegistered(principal: Principal): Bool {
Option.isSome(_users.get(principal))
};
Lastly, the code defines the upgrade logic for the canister:
// The work required before a canister upgrade begins.
system func preupgrade() {
Debug.print("Starting pre-upgrade hook...");
_stableUsers := Iter.toArray(_users.entries());
_stableDatastoreCanisterIds := List.toArray(_datastoreCanisterIds);
Debug.print("pre-upgrade finished.");
};
// The work required after a canister upgrade ends.
system func postupgrade() {
Debug.print("Starting post-upgrade hook...");
_stableUsers := [];
_stableDatastoreCanisterIds := [];
Debug.print("post-upgrade finished.");
};
};
Then, each secondary canister created by the primary canister will implement the following code (in this example, stored in the Datastore.mo file) :
import Array "mo:base/Array";
import Buffer "mo:base/Buffer";
import Debug "mo:base/Debug";
import Result "mo:base/Result";
import Nat "mo:base/Nat";
import Text "mo:base/Text";
import Iter "mo:base/Iter";
import Time "mo:base/Time";
import List "mo:base/List";
import HashMap "mo:base/HashMap";
import Hash "mo:base/Hash";
import Option "mo:base/Option";
import Principal "mo:base/Principal";
import Types "./Types";
import Note "./Note";
shared ({ caller }) actor class Self(_noteDataSize: Types.Byte): async Types.Datastore {
type UserId = Types.UserId;
type NoteId = Types.NoteId;
type Note = Types.Note;
type DefiniteNote = Types.DefiniteNote;
type Byte = Types.Byte;
// Stable variables
stable var _stableDatastores : [(NoteId, Note)] = [];
// Bind [caller] and [_main]
let _main : Principal = caller;
let _datastores = HashMap.fromIter<NoteId, Note>(_stableDatastores.vals(), 10, Nat.equal, Hash.hash);
// Creates a new note in a canister.
// Returns a created note.
// Traps if:
// - [caller] is not [_main]. (Design choice: one cannot directly access a secondary (datastore) canister.)
// - the data size exceeds the limit.
public shared ({ caller }) func createNote(userId: UserId, canisterId: Principal, noteId: NoteId, title: Text, content: Text) : async Result.Result<DefiniteNote,Text> {
if (_main != caller) { return #err "You can only create a note by calling the main canister." };
if (_isLimit (title # content)) { return #err "The data size exceeded the limit." };
let note = Note.create(noteId, canisterId, userId, title, content);
_datastores.put(noteId, note);
#ok (Note.freeze(note))
};
// Returns a note of [noteId].
// Traps if:
// - a note does not exist for a given [noteId].
// - [caller] is not the owner of a note.
public query ({ caller }) func getNoteById(noteId: NoteId) : async Result.Result<DefiniteNote,Text> {
switch (_datastores.get(noteId)) {
case null {
#err ("A note does not exist for ID: " # Nat.toText(noteId))
};
case (?note_){
if (not _isAuthenticated(note_, caller)) { return #err "You are not authenticated." };
#ok (Note.freeze(note_))
};
}
};
// Returns all notes of [caller].
// Returns an empty array if [caller] does not have any note in a canister.
public query ({ caller }) func getAllNotes() : async [DefiniteNote] {
let notes: HashMap.HashMap<NoteId, DefiniteNote> = HashMap.mapFilter<NoteId, Note, DefiniteNote>(_datastores, Nat.equal, Hash.hash, func (_: NoteId, note: Note) {
if (note.userId == caller){
return ?Note.freeze(note);
} else {
return null;
};
});
Iter.toArray(notes.vals())
};
// Updates a note of the [caller].
// Returns an updated note.
// Traps if:
// - there is no note associated with a given [noteId].
// - [caller] is not the owner of a note.
// - the data size exceeds the limit.
public shared ({ caller }) func updateNote(noteId: NoteId, title: ?Text, content: ?Text) : async Result.Result<DefiniteNote, Text> {
switch (_datastores.get(noteId)) {
case null {
#err ("A note does not exist for ID: " # Nat.toText(noteId))
};
case (?note_){
if (not _isAuthenticated(note_, caller)) { return #err "You are not authenticated." };
if (_hasUpdateReachedLimit(note_, title, content)) { return #err "The data size exceeded the limit." };
let updatedNote = Note.update(note_, title, content);
_datastores.put(noteId, updatedNote);
#ok (Note.freeze(updatedNote))
};
}
};
// Deletes a note of [noteId]
// Returns [noteId] of a deleted note.
// Traps if:
// - there is no note associated with a given [noteId].
// - [caller] is not the owner of a note.
public shared ({ caller }) func deleteNote(noteId: NoteId) : async Result.Result<NoteId, Text> {
switch (_datastores.get(noteId)) {
case null {
#err ("A note does not exist for ID: " # Nat.toText(noteId))
};
case (?note_){
if (not _isAuthenticated(note_, caller)) { return #err "You are not authenticated." };
_datastores.delete(noteId);
#ok (note_.id)
};
}
};
// Returns `true` if the owner of [note] is [userId].
private func _isAuthenticated(note: Note, userId: UserId) : Bool {
note.userId == userId
};
// Returns `true` if the data size exceeds the limit.
private func _isLimit(t: Text) : Bool {
Text.encodeUtf8(t).size() > _noteDataSize
};
// Returns `true` if the data size of an updated note exceeds the limit.
private func _hasUpdateReachedLimit(note: Note, title: ?Text, content: ?Text) : Bool {
switch(title, content){
case (?t, ?c){
_isLimit(t # c)
};
case (?t, null){
_isLimit(t # note.content)
};
case (null, ?c){
_isLimit(note.title # c)
};
case (null, null){
false
};
}
};
// Below, we implement the upgrade hooks for our canister.
// See https://smartcontracts.org/docs/language-guide/upgrades.html
// The work required before a canister upgrade begins.
system func preupgrade() {
Debug.print("Starting pre-upgrade hook...");
_stableDatastores := Iter.toArray(_datastores.entries());
Debug.print("pre-upgrade finished.");
};
// The work required after a canister upgrade ends.
system func postupgrade() {
Debug.print("Starting post-upgrade hook...");
_stableDatastores := [];
Debug.print("post-upgrade finished.");
};
}