summaryrefslogtreecommitdiff
path: root/ocaml
diff options
context:
space:
mode:
authorpolwex <polwex@sortug.com>2025-10-06 10:39:39 +0700
committerpolwex <polwex@sortug.com>2025-10-06 10:39:39 +0700
commit2d6c3bab18cf5063246fcdb869ae36132bbfe3fc (patch)
treecf99623ab085e21ba47697faf8c7cd60eca94cfe /ocaml
parent68da09fc6fcc4745010cd61c52573da7cc18c527 (diff)
moar docsarvo
Diffstat (limited to 'ocaml')
-rw-r--r--ocaml/PILLS.md274
1 files changed, 203 insertions, 71 deletions
diff --git a/ocaml/PILLS.md b/ocaml/PILLS.md
index 41df920..7775611 100644
--- a/ocaml/PILLS.md
+++ b/ocaml/PILLS.md
@@ -1,71 +1,203 @@
-# Urbit Pills
-What are they huh
-
-## Ivory
-Ivory Pill Performance Comparison:
-
- C Vere: 0.21 seconds (6.88 MB/s)
- OCaml: 0.44 seconds (3.41 MB/s)
- Ratio: ~2x slower
-
- So for the 1.5MB ivory pill, we're only 2x slower - totally reasonable! This confirms the problem grows non-linearly with size/complexity.
-
- What Does the Ivory Pill Do?
-
- From the docs and C Vere source:
-
- Ivory Pill = The %zuse core (Hoon standard library) compiled into a pill
-
- Purpose
-
- C Vere's I/O drivers (Ames, Behn, Clay, Dill, Eyre, Iris) need access to Hoon standard library functions like:
- - Text parsing/formatting
- - Data structure manipulation
- - Cryptographic operations
- - HTTP parsing
-
- Rather than reimplement these in C, C Vere:
- 1. Loads the ivory pill at startup
- 2. Runs u3v_life() lifecycle on it
- 3. Keeps the %zuse core available
- 4. Calls Hoon functions from C when needed via u3v_wish()
-
- What It Contains
-
- - %zuse library: All Hoon stdlib (++ enjs, ++ dejs, ++ by, ++ so, etc.)
- - NOT full Arvo: No vanes, no poke interface, no event processing
- - Structure: ["ivory" core] - just the tagged core
-
- How C Vere Uses It
-
- // At startup (pkg/vere/king.c)
- _king_boot_ivory(); // Load ivory pill
- u3v_boot_lite(pil); // Run u3v_life() on it
- u3A->roc = result; // Store %zuse core
-
- // Later, when I/O driver needs a Hoon function:
- u3v_wish("(rap 3 'hello' ' ' 'world')"); // Call into %zuse
-
- Why It Exists
-
- Without ivory, every Hoon stdlib function would need to be:
- 1. Reimplemented in C (huge effort)
- 2. Kept in sync with Hoon version (maintenance nightmare)
- 3. Tested separately (double the work)
-
- Instead, ivory lets C Vere reuse the canonical Hoon stdlib by calling into it as needed.
-
- Our OCaml Implementation
-
- We successfully:
- - ✅ Load ivory pill (0.44s)
- - ✅ Validate "ivory" tag
- - ✅ Implement u3v_life() lifecycle formula
- - ❌ Lifecycle execution fails (still debugging)
-
- For OCaml runtime, ivory would let us:
- - Call Hoon formatters from OCaml I/O drivers
- - Parse/serialize Hoon data structures
- - Use crypto functions without reimplementing
-
- Bottom line: Ivory is a clever way to avoid duplicating the Hoon stdlib in the runtime. It's fast to load (2x C is fine), but we need to debug why the lifecycle formula fails.
+# Urbit Pills: When They're Used
+
+## The Two-Pill System
+
+C Vere uses **two different pills** for different purposes:
+
+### 1. Ivory Pill (Always Loaded)
+
+**When**: At **every startup**, before any ship operations
+**Where**: Embedded in the binary as `u3_Ivory_pill[]` byte array
+**Size**: ~1.5 MB (jammed)
+**Load Time**: ~0.2 seconds (C Vere), ~0.44 seconds (OCaml)
+
+**Boot Sequence**:
+```c
+u3_king_commence() // King process entry point
+ → _king_boot_ivory() // Load ivory pill
+ → u3s_cue_xeno() // Deserialize from embedded bytes
+ → u3v_boot_lite() // Run lifecycle formula
+ → u3v_life() // Execute [2 [0 3] [0 2]]
+ → [start event loop] // Continue with normal operations
+```
+
+**What It Contains**:
+- The **%zuse core** (Hoon standard library)
+- Structure: `["ivory" <core>]` (tagged pair)
+- All stdlib functions: `++by`, `++so`, `++enjs`, `++dejs`, etc.
+
+**Why It's Needed**:
+C Vere's I/O drivers (Ames, Behn, Clay, Dill, Eyre, Iris) need to call Hoon functions:
+- Parse HTTP requests
+- Format JSON responses
+- Hash/sign packets
+- Serialize data structures
+
+Rather than reimplementing these in C, Vere loads ivory once and calls into it via `u3v_wish()`:
+
+```c
+// C code in Eyre driver needs to format JSON
+u3_noun json = u3v_wish("(enjs:format some-data)");
+```
+
+This keeps the runtime in sync with Hoon and avoids duplicating the entire stdlib.
+
+### 2. Brass/Solid Pill (Only for New Ships)
+
+**When**: Only when **booting a new ship**
+**Where**: Downloaded from `https://bootstrap.urbit.org/git-<hash>.pill`
+**Size**: ~8-170 MB (varies by version)
+**Load Time**: ~1.5 seconds for solid (C Vere)
+
+**Boot Sequence**:
+```c
+urbit -F zod // User boots new fake ship
+ → _king_boot() // Handle boot command
+ → _boothack_pill() // Get pill (download or --pill arg)
+ → _king_get_atom("https://bootstrap.urbit.org/...")
+ → u3v_boot() // Process boot events
+ → Process 5 events: +aeon, +boot, +fate, +hoon, +arvo
+ → Result: Full Arvo kernel in u3A->roc
+ → [continue with ship operations]
+```
+
+**What It Contains**:
+- Structure: `[tag [event1 event2 event3 event4 event5]]`
+- **Boot events** that build full Arvo when processed:
+ 1. `+aeon`: Start event loop
+ 2. `+boot`: Bootstrap Arvo kernel
+ 3. `+fate`: Produce Hoon bootstrap compiler
+ 4. `+hoon`: Produce compiler source
+ 5. `+arvo`: Produce kernel source
+
+**Result**: A complete Arvo kernel with:
+- All vanes (Ames, Behn, Clay, Dill, Eyre, Gall, Iris, Jael, Khan)
+- Full poke/peek interface (slot 23)
+- Event processing capability
+- Persistent state
+
+**Pill Types**:
+- **Brass**: Full recompilation of vanes (~170 MB for prod.pill)
+- **Solid**: Pre-compiled vanes, faster boot (~8.7 MB)
+- **Baby**: Minimal pill for testing
+
+## Summary Table
+
+| Aspect | Ivory | Brass/Solid |
+|--------|-------|-------------|
+| **When** | Every startup | New ship boot only |
+| **Where from** | Embedded in binary | Downloaded/provided |
+| **Size** | ~1.5 MB | ~8-170 MB |
+| **Contains** | %zuse stdlib | Boot events |
+| **Purpose** | I/O driver support | Create Arvo kernel |
+| **Result** | Runtime utilities | Full operating system |
+| **Frequency** | Always | Once per ship lifetime |
+| **Stored in** | `u3A->roc` (lite) | `u3A->roc` (full) |
+
+## Key Insight
+
+**Ivory is not Arvo** - it's just the Hoon stdlib compiled into a pill.
+
+You cannot:
+- ❌ Poke ivory with events
+- ❌ Run vanes on ivory
+- ❌ Use ivory as a ship OS
+
+You can:
+- ✅ Call Hoon stdlib functions from C/OCaml
+- ✅ Parse/format data using Hoon code
+- ✅ Avoid reimplementing stdlib in runtime language
+
+**Brass/Solid creates Arvo** - processing the boot events builds the full OS.
+
+Once processed:
+- ✅ Full event processing
+- ✅ Poke interface at slot 23
+- ✅ All vanes running
+- ✅ Persistent state
+
+## Performance Comparison
+
+### Ivory Pill (1.5 MB)
+```
+C Vere: 0.21 seconds (6.88 MB/s)
+OCaml: 0.44 seconds (3.41 MB/s)
+Ratio: 2x slower ✅ Acceptable!
+```
+
+### Solid Pill (8.7 MB)
+```
+C Vere: 1.45 seconds (6.0 MB/s)
+OCaml: >300 seconds (0.03 MB/s)
+Ratio: ~200x slower ❌ Blocker
+```
+
+The performance gap grows non-linearly with size/complexity, suggesting algorithmic issues in the OCaml cue implementation (see `ARVO_CHALLENGE.md`).
+
+## For OCaml Implementation
+
+### What We Have ✅
+
+Ivory pill loading:
+- Load ivory pill (0.44s vs C's 0.2s = 2x slower, acceptable)
+- Validate "ivory" tag (`0x79726f7669` = "ivory")
+- Implement `u3v_life()` lifecycle formula `[2 [0 3] [0 2]]`
+- ❌ Lifecycle execution fails (Nock exits - still debugging)
+
+### What We Need ⏳
+
+For full Arvo:
+1. **Fast cue** for solid pill (currently ~200x slower, blocker)
+2. **Process 5 boot events** to build Arvo
+3. **Result**: Working Arvo kernel with poke interface
+
+### What Ivory Enables 🎯
+
+Even without full Arvo, ivory pill lets us:
+- Test I/O drivers with real Hoon code
+- Implement `wish()` to call into Hoon
+- Validate jam/cue correctness on real data
+- Benchmark Nock performance on stdlib code
+
+But for **full ship operation** with vanes and event processing, we need solid pill boot.
+
+## Bootstrap Flow for New Ship
+
+```
+User runs: urbit -F zod
+
+1. King process starts
+ ├─ Load embedded ivory pill (0.2s)
+ ├─ Run u3v_life() on ivory
+ └─ u3A->roc = %zuse core
+
+2. User command triggers new ship boot
+ ├─ Download brass pill from bootstrap.urbit.org
+ ├─ Cue pill (1.5s for solid, 30s+ for brass)
+ ├─ Extract 5 boot events from pill
+ └─ Process events sequentially:
+ ├─ +aeon: Initialize event loop
+ ├─ +boot: Bootstrap Arvo
+ ├─ +fate: Create Hoon compiler
+ ├─ +hoon: Load compiler source
+ └─ +arvo: Load kernel source
+
+3. Result: Full Arvo kernel
+ ├─ u3A->roc now contains complete Arvo
+ ├─ All vanes loaded and running
+ ├─ Poke interface active at slot 23
+ └─ Ship ready for normal operation
+
+4. Subsequent startups
+ ├─ Load ivory (still needed for I/O)
+ ├─ Load ship state from snapshot/eventlog
+ └─ No pill download needed
+```
+
+## Source Code References
+
+- Ivory embedded: `pkg/vere/ivory/ivory.c:2` (`u3_Ivory_pill[]`)
+- Ivory boot: `pkg/vere/king.c:962` (`_king_boot_ivory()`)
+- Lifecycle: `pkg/noun/vortex.c:26` (`u3v_life()`)
+- Pill download: `pkg/vere/king.c:600` (bootstrap URL generation)
+- Boot events: `pkg/vere/king.c:606` (`_boothack_pill()`)