Image classification sample
Overview
ICP's unique ability to run compute at scale allows AI and neural networks to run directly onchain within a canister smart contract.
To showcase this capability, this demo example displays how an AI that identifies an image can be deployed as a smart contract with a frontend and backend, both running onchain.
You can find the source code for this demo on GitHub.
How this example works
In this example, the ICP smart contract accepts an image as input from the user and runs an image classification inference. The smart contract has two canisters:
The frontend canister that contains HTML, JS, and CSS assets that are served in the web browser.
The backend canister that embeds the Tract ONNX inference engine with the MobileNet v2-7 model. This canister provides a
classify()
endpoint that the frontend canister calls.
ICP features that make it possible
To make running AI in a canister possible, two key ICP features are utilized:
- WebAssembly virtual machine: Canister code is compiled into Wasm modules to be deployed on ICP. The WebAssembly virtual machine supports standards such as the WebAssembly System Interface, which can be supported through a community tool called wasi2ic.
- Deterministic time slicing (DTS): DTS splits the execution of very large messages that require billions of Wasm instructions across multiple execution rounds.
Important notes
The ICP mainnet subnets and dfx
running a replica version older than 463296
may fail with an instruction-limit-exceeded
error.
Currently, Wasm execution is not optimized for this workload. A single call executes about 24B instructions (~10s).
Deploying the demo
Prerequisites
Download and install the Rust programming language and Cargo as described in the Rust installation instructions for your operating system.
Download and install the IC SDK package as described in the installing the IC SDK page.
Download and install git.
Install
wasi-skd-21.0
.Export
CC_wasm32_wasi
in your shell such that it points to WASI clang and sysroot:export CC_wasm32_wasi="/path/to/wasi-sdk-21.0/bin/clang --sysroot=/path/to/wasi-sdk-21.0/share/wasi-sysroot"
Install
wasi2ic
and make sure thatwasi2ic
binary is in your$PATH
.
Downloading the example
You can clone the GitHub repo for this example with the command:
git clone https://github.com/dfinity/examples.git
Then navigate into the directory for the AI demo:
cd examples/rust/image-classification
Download MobileNet v2-7 to src/backend/assets/mobilenetv2-7.onnx
by running the script:
./download_model.sh
Install Node.js dependencies for the frontend:
npm install
Add the following Rust target:
rustup target add wasm32-wasi
Deploying the code
To deploy the example, first start dfx
:
dfx start --clean --background
Then to deploy the canisters, run the command:
dfx deploy // Deploy locally
dfx deploy --network ic // Deploy to the mainnet
Using the demo
Once deployed, open the frontend canister's URL in your browser. You'll see the demo's user interface:
Click on the Internet Computer logo. You'll be prompted to select an image file from your local files. In this example, we'll use an astronaut image. Then, select 'Go':
The smart contract will do some computation to infer what the image is. This process may take about 10 seconds.
The image inference results will be returned: