Instruction Set Architecture¶

This NPU targets edge SoC integration, so the ISA is designed to be compact and tile-friendly. We tile GEMM+activation work across multiple pipeline stages inside the core, inspired by OpenTPU and systolic tiling literature. The FP32/BF16/BF8 conversion and VALU families enable post-GEMM quantization entirely in hardware without round-tripping through host memory.

Notation¶

These three symbols appear throughout all ISA, VALU, register-file, and backend code. Confusing them is the primary source of errors.

Physical register file total = L × K × N/8 bytes = 256 B (test) / 2 KiB (top).

Instruction Word Layout¶

All instructions are 32-bit words. Three formats are defined.

R-type — register-register operations¶

Bit 31 is the MSB. Fields are shown MSB → LSB (bit 31 on the left).

I-type — register + immediate (e.g. bcast.imm, ld)¶

The 12-bit immediate is sign-extended to the lane width.

S-type — three-source FMA (VALU_FP_FMA only)¶

rnd is the rounding mode for this instruction only. rs3 is the addend for FMA.

Field definitions¶

funct7 Attribute Field (R-type)¶

funct7 for VALU_CVT (different layout)¶

CVT repurposes funct7 to carry source format and BF8 variant:

CVT Format Codes¶

Used in both funct3 (destination) and funct7[2:0] (source):

Opcode Family Map¶

opcode selects one of 13 functional families. Reserved codes (0x04–0x0F, 0x19–0x7F) are detected by the decoder as illegal instructions.

graph TD
    ISA["7-bit Opcode Space<br/>13 active families"]

    ISA --> MEM["Memory &amp; Control<br/>NOP 0x00 · LD 0x01 · ST 0x02"]
    ISA --> MMA["Matrix Multiply-Accumulate<br/>MMA 0x03"]
    ISA --> INT["Integer Vector<br/>ARITH 0x10 · LOGIC 0x11<br/>REDUCE 0x12 · LUT 0x13 (vlut/vsetlut)"]
    ISA --> CVT["Type Conversion<br/>CVT 0x14"]
    ISA --> BCAST["Broadcast<br/>BCAST 0x15"]
    ISA --> FP["Floating-Point<br/>FP 0x16 · FP_FMA 0x17"]
    ISA --> MOV["Move (proposed)<br/>MOV 0x18 ⚠"]

Family Reference¶

VALU_ARITH — Elementwise arithmetic on VX / VE / VR¶

Width selected by funct7[1:0]. Saturation controlled by funct7[4]. Applies to all integer lane widths (N, 2N, 4N bits).

VALU_LOGIC — Bitwise and shift on VX / VE / VR¶

Operates on raw bit patterns. sat and round are ignored. Shift amount = rs2[i][ log2(lane_width)−1 : 0 ] (low bits of each lane of rs2).

VALU_REDUCE — Horizontal reduction, broadcast result¶

Result is broadcast to all K lanes of out_vr. Operates on VX lanes (sign-extended to 4N bits for the accumulation tree).

VALU_LUT — Programmable two-bank 256-entry lookup table (VX only)¶

The LUT family provides two independently programmable 256-byte banks (bank A and bank B). Each bank holds 256 one-byte entries. The LUT is not a fixed ROM: entries must be written before use with vsetlut, then queried per-lane with vlut.

vlut — per-lane lookup (R-type, 1 tick)¶

rd[i] = lut_bank[in_a_vx[i].asUInt]

rs2 is unused. The raw 8-bit unsigned value of each in_a_vx lane is the LUT index. Output goes to out_vx. Bank selected by funct3[0] (propagated as round[0] in the decoded bundle).

vsetlut — write LUT segment (I-type, no RF write)¶

Writes one segment of K×4 bytes from VR[rs1] into the selected bank. imm = segment index; segment s covers LUT entries [s×K×4 .. (s+1)×K×4 − 1].

At K=8: one VR holds 32 bytes → 8 vsetlut calls fill a full 256-byte bank. At K=64: one VR holds 256 bytes → 1 vsetlut call fills a full 256-byte bank.

No register-file write occurs; the operation is a side-effect on VALU-internal state only.

VALU_CVT — Type conversion¶

funct3 = destination format code. funct7[2:0] = source format code. Illegal if src == dst. Width of output register class is determined by the destination format.

VALU_BCAST — Scalar broadcast to all K lanes¶

bcast.reg broadcasts lane 0 of rs1 to all K output lanes. Used to splat a scale or zero-point constant prior to vfma.

VALU_FP — FP32 arithmetic (VR only)¶

Operands and result are always in VR (K lanes of 32-bit FP). Width implicit (always VR), dtype implicit (always FP). Tier-2 FP32 constraints: RNE rounding; NaN/Inf inputs treated as zero; overflow saturates to max finite normal; subnormals flushed to zero.

VALU_FP_FMA — Fused multiply-add (VR, S-format, 2 ticks)¶

rd = rs1 × rs2 + rs3 (and variants). S-format encodes rs3 at bits [31:27] and round mode at [26:25].

VALU_MOV — Register copy and immediate load¶

MMA — Matrix Multiply-Accumulate¶

rd = destination VR base index; rs1 = A operand VX base; rs2 = B operand VX base. The keep signal (from funct7[4], i.e. the sat bit) controls accumulation.

MMALU output goes directly to VR — no INT8 truncation. The 4N-bit accumulator is preserved intact.

Instruction Timing¶

1-tick ops (all VALU except FMA)¶

All VALU instructions (ARITH, LOGIC, REDUCE, LUT, CVT, BCAST, FP, MOV) have a 1-tick output register stage. The result appears on out_vx / out_ve / out_vr one clock edge after issue.

2-tick: vfma and rounding CVT ops¶

vfma performs a multiply then add, requiring two clock edges. CVT ops that require rounding logic (e.g. vcvt_s32_f32, vcvt_s8_f32) also take 2 ticks.

Reduction ops (1-tick, broadcast to all K lanes)¶

vsum and vrmax reduce all K input lanes combinationally, then broadcast the scalar result to every lane of out_vr.

MMA: 3K−2 tick pipeline¶

For a K×K systolic array. Input vectors are consumed for the first K ticks; the first output column appears at tick 2K−1; the last at tick 3K−2.

Activation Function Software Sequences¶

Activation functions are not hardware opcodes; they are composed from VALU primitives. See Quantization Pipeline for the full worked example.

Softmax flow¶

flowchart LR
    X["x[K] in VX\n(SQ1.6)"]
    X --> A["vrmax\nmax=max(x)\n1 tick\n→ out_vr"]
    X --> B["vsub.sat\nx'=x-max\n1 tick\n→ out_vx"]
    A --> B
    B --> C["vlut (exp, bank A)\ne=exp(x')\n1 tick\n→ out_vx"]
    C --> D["vsum\nΣ=sum(e)\n1 tick\n→ out_vr"]
    D --> E["vlut (recip, bank B)\nr=1/Σ\n1 tick\n→ out_vx"]
    C --> F["vmul\np=e×r\n1 tick\n→ out_vx"]
    E --> F
    F --> G["softmax(x)\n6 ticks total"]

GELU flow¶

flowchart LR
    X["x[K] in VX\n(SQ1.6)"]
    X --> A["vsra by 1\n≈ x/√2\n1 tick"]
    A --> B["vlut (erf, bank A)\nerf(x/√2)\n1 tick → out_vx"]
    B --> C["vadd 64\n1+erf(·)\n(bcast.imm 64)\n1 tick"]
    X --> D["vmul\nx·(1+erf)\n1 tick → out_vr"]
    C --> D
    D --> E["vsra by 7\n÷128 ≈ ×0.5/scale²\n1 tick"]
    E --> F["GELU(x)\n5 ticks total"]

Assembler¶

src/main/scala/isa/NpuAssembler.scala provides a Scala-side assembler. All methods return a Scala Int (the 32-bit bit pattern).

import isa.NpuAssembler._

// Arithmetic
val i1 = vadd(rd=0, rs1=1, rs2=2, width=VX, sat=false)  // VX add
val i2 = vmul(rd=4, rs1=4, rs2=5, width=VR, sat=true)   // VR mul saturated

// FP32
val i3 = vfma(rd=2, rs1=2, rs2=0, rs3=1)                // VR fused multiply-add

// Conversion
val i4 = vcvt_f32_s32(rd=2, rs1=2)                      // INT32 → FP32
val i5 = vcvt_s8_f32(rd=31, rs1=2, sat=true)            // FP32 → INT8 saturated

// Broadcast
val i6 = vbcast(rd=0, rs1=0, width=VR)                  // splat VR[0] lane 0

// Programmable LUT
val i_set = vsetlut(rs1=4, segment=0, bank=0)           // write VR[4] → LUT bank A seg 0
val i_lut = vlut(rd=2, rs1=1, bank=0)                   // rd[i] = lut_A[VX[1][i]]

// MMA
val i7 = mma(rd=2, rs1=0, rs2=8, keep=true)
val i8 = mmaLast(rd=2, rs1=0, rs2=8)

// Poke in simulation (convert Scala Int → Chisel UInt safely):
dut.io.instr.poke((i1.toLong & 0xFFFFFFFFL).U)