Developer's Commentary

An exclusive behind-the-scenes look at writing apps.


Thu Apr 28 08:16:54 PM PDT 2022

I added notes on cross-compiling quill to aarch64.

Work on Organic Apps continues. I’m rather proud of an ECDSA walkthrough in a few lines of Haskell in one of the articles.


Wed Apr 6 08:32:51 PM PDT 2022

I’ve been reorganizing and expanding my notes into a more cohesive form:

The name of the guide is strange, but I like to pick names ending with "IC"!


Fri Mar 4 23:58:37 PST 2022

Added an in-depth look at principals and accounts.


Wed Mar 2 10:47:22 PM PST 2022

I fixed the echo server which a recent change in the network broke. In the old days, the boundary nodes were less strict about checking the Candid HTTP response, and I could get away with a vec null instead of a vec empty.

Another change since the network is initially released: Candid buffers now support fields of type principal. I finally got around to supporting them in the Candid explainer.


Thu Feb 17 09:54:24 AM PST 2022

Note to self. I wasted hours because I failed to realize ic0.msg_caller_size traps when called in the reply or reject callback function provided to ic0.call_new. (For no good reason, I had thought the caller might be set to the canister invoking the callback.)

I wasted several more because I failed to realize ic0.msg_reply also traps when called in these callbacks.

It took me a long time to figure out that I should use ic0.call_on_cleanup to print a debug message to help diagnose the problem, after confirming that ic0.call_perform has returned 0. Even so, as far as I could tell, this only gave me a bit of information, namely, that the reply or reject callback trapped.

It’s possible the spec explains all the reasons why these callbacks may trap, but my attention was drawn to statements about traps being caused by insufficient cycles, so I was barking up the wrong tree for a while.


Sun Nov 14 02:56:53 PM PST 2021

I moved the NetWalk page here to free up a canister for other experiments.


Fri Sep 10 11:26:02 AM PDT 2021

For an internal hackathon I cannibalized some canister IDs.

  • https://ffgig-jyaaa-aaaae-aaaoa-cai.raw.ic0.app/ is now a Haskell compiler. It was once a walkthrough of a minimal webpage canister. The Monic page has roughly the same content anyway.

  • The canister aq6z7-uyaaa-aaaae-aaaqa-cai now hosts a simple persistent store used by a demo of a PuzzleScript-like engine. It used to serve the Monic tool in a tarball. My IC homepage serves the Monic git repo, so this tarball is no longer needed. It was outdated anyway.

I created a new canister 7ha3m-qaaaa-aaaae-aaffa-cai for the backend of my Rock Paper Scissors demo. Its source is one of the compiler examples.


Wed Aug 25 09:01:29 AM PDT 2021

For my rock-paper-scissors demo, I needed to call the IC from the web page. DFINITY’s agent-js can do this, but I wanted to avoid npm and also learn what was happening under the hood.

So I wrote my own web IC agent, which turned out to be more work than I had anticipated. I wound up relying on a standalone JavaScript library to decode CBOR. I also compiled a tiny wasm file to handle the expiry time because JavaScript is no good with 64-bit ints.

I’ve cleaned it up and replaced the library with my own CBOR routines. I used Haskell to simplify, and also to exercise my Haskell to wasm compiler. The result is a self-contained web agent for the IC.


Mon Jun 28 09:26:09 PM PDT 2021

I released a browser game I implemented years ago as a static web app on the IC, at the same time confirming the most recent dfx uses fewer cycles to create a new canister:


Thu Jun 24 07:02:29 PM PDT 2021

Almost a week ago I released a multiplayer rock-paper-scissors server that uses the api/v2 interface so games can be played in the browser. I posted it to #team, and soon had a few testers:

I hope to write about it soon. For now, I want to report another success: I finally figured out certified HTTP assets!

I added an IC-Certificate header but I still saw the dreaded: Body does not pass verification. So I spent a while decoding the headers of a successfully certified IC app (I picked the identity canister rdmx6-jaaaa-aaaaa-aaadq-cai), until I found the issue.

The trouble is the leaves of the tree. I had thought each leaf was supposed to hold the contents of an HTML page. Instead, each leaf is supposed to hold the SHA-256 hash of the page it represents. Thus if we run HashTree from https://github.com/dfinity/ic-hs in GHCi, the following is wrong:

Labeled (f "http_assets") $ Labeled (f "/") $ Leaf (f "Hi, all!")

where f = BS.pack . map (toEnum . fromEnum).

Instead, we want:

Labeled (f "http_assets") $ Labeled (f "/") $ Leaf (h (f "Hi, all!"))

We compute the hash of this tree step by step:

$ printf "Hi, all!" | sha256sum
8f08f084b1d8663ec3b64cb14e299626e3922d6656ff7c77f9cd3df8c27ed02f  -
$ (printf "\x10ic-hashtree-leaf"; echo 8f08f084b1d8663ec3b64cb14e299626e3922d6656ff7c77f9cd3df8c27ed02f | xxd -r -p) | sha256sum
6a9ff90089840036a7c0dad5cde5d8fe9fb763e34141353b3c05c8736452c97c  -
$ (printf "\x13ic-hashtree-labeled/"; echo 6a9ff90089840036a7c0dad5cde5d8fe9fb763e34141353b3c05c8736452c97c | xxd -r -p) | sha256sum
3a16c380fee7f25a7c7a625c855601e44e6030ab02fbff57a386802bd64a6063  -
$ (printf "\x13ic-hashtree-labeledhttp_assets"; echo 3a16c380fee7f25a7c7a625c855601e44e6030ab02fbff57a386802bd64a6063 | xxd -r -p) | sha256sum
defa8a6d9aef6f0a3273769b343af9403d49316d0995c21c83a7209cc725e0e8  -

Our canister should set its certified data to the hash defa....

The rest is straightforward, though tedious. We create an IC-Certificate header whose certificate field is the base-64 encoding of the buffer we obtain via ic0.data_certificate_copy(), and tree is the base-64 encoding of the CBOR encoding of the tree we just defined. As mentioned in the spec, we use CBOR arrays where the first element is an integer tag indicating how to interpret the rest of the array:

d9d9f7      # CBOR Self-Describe.
  8302      # Labeled.
    48 "http_assets"
    8302    # Labeled.
      41 "/"
      8203  # Leaf. SHA-256("Hi, all!").
        5820 8f08f084b1d8663ec3b64cb14e299626e3922d6656ff7c77f9cd3df8c27ed02f

Of course, we must encode the header itself with Candid before replying.


Wed Jun 16 02:46:56 PM PDT 2021

I found out from Joachim why my certifed data experiment fails: I’m neglecting to send an IC-Certificate header containing a tree and actual certificate. It looks like my hash computation is correct, but our design requires the canister to deliberately disclose its certificate. One does not simply request its certificate.

Certification sounds a little more complex than I had anticipated. I’ll put it on hold for now.

I had thought there was no way for a web app to send an update call. I knew about the api/v2 endpoint, but the official specification notes "This document does not yet explain how to find the location and port of the Internet Computer."

It turns out the location is ic0.app and the port is the default HTTPS port! For example:

$ curl -X POST https://ic0.app/api/v2/canister/fqbzl-iqaaa-aaaae-aaanq-cai/query

This expects a CBOR-encoded query.

Norton’s ic.rocks tool is indispensable. Type in a URL such as:

and it’ll show data such as the controller tree, which includes the canister ID of the wallet running the app.

Speaking of which, I had started from scratch and copied over a private key, but ran into trouble deploying my apps because my wallet was missing. Running deploy-wallet would probably blow away my existing cycles balance, and set-wallet appeared to do nothing. Prithvi had the answer: I had to force it through:

$ dfx identity --network ic set-wallet --force WALLET_ID

This indeed creates the wallets.json file, and deployment worked again.

Crafting a request to api/v2 by hand is tedious but possible. First we type:

{ "content":
  { "request_type":"query"
  , "canister_id": h'000000000080001b0101'
  , "ingress_expiry" : 1234567800000000000
  , "sender" : h'04'
  , "method_name":"go"
  , "arg": h'4449444c00017103416263'
  }
}

where the ingress_expiry field is obtained by adding 300 to the result of date +%s then appending nine 0s to convert to nanoseconds. We obtain the canister ID by decoding the base-32 ID of the canister, then discarding the initial 4-byte checksum. The sender of 04 indicates an anonymous sender, and saves us from having to sign the request.

We feed this to cbor.me, then convert the output hex to raw bytes by piping through:

cat hex | sed 's/#.*$//' | tr -d ' \n' | xxd -r -p > dat

Then:

curl -H "Content-Type: application/cbor" --data-binary @/tmp/dat \
  https://ic0.app/api/v2/canister/fqbzl-iqaaa-aaaae-aaanq-cai/query

Thu Jun 10 03:25:45 PM PDT 2021

Today’s academy session was about certified data. Afterwards, I tried to certify my minimal "Hi, All!" app.

Loading the former in GHCi:

> a = SubTrees $ M.singleton "http_assets" $ SubTrees $ M.singleton "/" $ Value "Hi, all!"
> BS.writeFile "/tmp/hash" $ reconstruct $ construct a

Hex dumping the resulting hash:

33112304660d37e55058f3734684a80773d75286459fae5698667a18c4033624

I tried calling ic0.certified_data_set on this 32-byte string but I still saw "Body does not pass verification".

Ran into a subtlety involving dfx deploy. It took me a long time to figure out canister_init only gets called the first time an app is deployed. The second time, the code is overwritten and the memory wiped, but canister_init is skipped.


Wed Jun 9 09:45:55 AM PDT 2021

The first app I deployed is now a homepage of sorts, containing these notes, and various tools I’ve written. There’s even a working git repo.


Tue Jun 8 14:49:42 UTC 2021

I realized that updating an existing canister is cheap. I lack sufficient cycles to create new canisters, but I can afford to overwrite my old ones.

I started by overwriting my first main-net canister with my Candid explainer.

Alexa explained why: dfx spends 1T cycles to create then auto-fills with 10T cycles. This will change in today’s release.

I worked on the "Epic" script to turn a bunch of files into a website, so I can now throw lots of content in the same canister. I repeatedly stomped on my first canister to test it out.


Fri Jun 4 18:02:38 PDT 2021

I’m losing track of my work. I better start taking notes.

On Tuesday, I asked Alexa for cycles in a testing account. I think I got 1 ICP, which turned into a ludicrous number of cycles. Soon I had "Hello, World!" web server working on main net!

It actually does more: it’s based on my old string-reversing example so it also exports a canister_update reverse function. I stripped down the code to make a smaller example:

which I followed up with a page explaining what I did:

How about looking at the incoming request for once? Decoding the raw Candid message is non-trivial so I settled for a simple echo server:

Coworkers gave invaluable feedback. My workaround of renaming the Motoko compiler to stop dfx deploy stomping on my wasm is in fact unnecessary, as was manually copying wasm binaries to the correct subdirectories. Instead, simply write a dfx.json that specifies a custom build process.

It also turns out declaring headers type to be Vec Null is wrong and will be rejected after a future upgrade. I want to experience this first-hand so I left the above apps alone. However, going forward I’ll use Vec Empty instead.

Over the next few days, I wrote Monic, a script that converts a single HTML file to an IC app:

Monic is French for "my IC". Also, the monic arrows in the category of sets are injective functions, and my script maps one file to one app. Because my webserver ignores the incoming request, I had to create a second app to serve the tarball:

After sprucing up my toy Haskell compiler, I wrote a Candid decoder:

I ran out of cycles so I was forced to deploy it on my homepage. This surprises me: I thought I had many left. Perhaps certain operations such as canister creation are far more expensive than I’d expect.