Piano roll

A piano roll for writing a phrase: a keyboard down the side, notes on a grid, and a velocity bar under each note. It sits inside a Timeline, so notes use the timeline's units, snap to its grid and follow its zoom.

Overview

• Notes: press on the grid to draw a note or to start a selection rectangle, depending on the mode. Drag a note to move it, drag an edge to resize it, and double-click to delete it. Selected notes move and resize together.
• Keys: pressing a key on the side plays its note. The rows light up for the pitches playing.
• Velocity: each note has a bar in its own lane, drawn from the bottom.
• Keyboard: every edit works without a pointer, including copy, paste, duplicate and quantize.
• Accessibility: each note is a toggle button named after its pitch and span. Each velocity bar holds a native <input type="range">.

Installation

bunnpmpnpmyarn
Copy
pnpm dlx shadcn@latest add @audio-ui/piano-roll

Anatomy

Import the registry components for the ready-made look, or compose the primitive's parts yourself.

Copy
import { PianoRoll, Timeline } from "@audio-ui/react";

<Timeline.Root duration={16} snapStep={1}>
  <PianoRoll.Root>
    <PianoRoll.Keys>
      <PianoRoll.Rows>
        {(pitch) => (
          <PianoRoll.Key pitch={pitch}>
            <PianoRoll.KeyLabel />
          </PianoRoll.Key>
        )}
      </PianoRoll.Rows>
    </PianoRoll.Keys>
    <PianoRoll.Grid aria-label="Notes">
      <PianoRoll.Rows>{(pitch) => <PianoRoll.Row pitch={pitch} />}</PianoRoll.Rows>
      <PianoRoll.SelectionSpan>
        <PianoRoll.StretchHandle edge="start" />
        <PianoRoll.StretchHandle edge="end" />
      </PianoRoll.SelectionSpan>
      <PianoRoll.Notes>
        {(note) => (
          <PianoRoll.Note key={note.id} note={note}>
            <PianoRoll.NoteHandle edge="start" />
            <PianoRoll.NoteHandle edge="end" />
          </PianoRoll.Note>
        )}
      </PianoRoll.Notes>
      <PianoRoll.Marquee />
    </PianoRoll.Grid>
    <PianoRoll.VelocityLane aria-label="Velocity">
      <PianoRoll.Notes>
        {(note) => <PianoRoll.Velocity key={note.id} note={note} />}
      </PianoRoll.Notes>
    </PianoRoll.VelocityLane>
  </PianoRoll.Root>
</Timeline.Root>;

Usage

Copy
import { Timeline, TimelineNavigator } from "@/components/audio-ui/timeline";
import {
  PianoRoll,
  PianoRollGrid,
  PianoRollKeys,
  PianoRollStrip,
  PianoRollVelocity,
  PianoRollVelocityScale,
  PianoRollViewport,
} from "@/components/audio-ui/piano-roll";

<Timeline duration={16} snapStep={1}>
  <PianoRoll
    className="h-96"
    maxPitch={72}
    minPitch={48}
    onNoteOn={(pitch, details) => play(pitch, details.velocity)}
    onValueChange={setNotes}
    value={notes}
  >
    <PianoRollViewport>
      <PianoRollStrip className="flex-1" lead={<PianoRollKeys />}>
        <PianoRollGrid aria-label="Notes" gridStep={1} />
      </PianoRollStrip>
      <PianoRollStrip lead={<PianoRollVelocityScale />} sticky="bottom">
        <PianoRollVelocity gridStep={1} />
      </PianoRollStrip>
    </PianoRollViewport>
  </PianoRoll>
</Timeline>;

A note is { id, pitch, start, length, velocity?, deviation?, muted?, chance? }: pitch is a MIDI number (60 is middle C), start and length are in the timeline's units, velocity is 0 to 1 (full when left out), deviation is how far the velocity may stray each time it plays (-1 to 1, up or down), muted deactivates it (it stays on the roll, silent) and chance is how likely it is to play each time round, 0 to 1. The roll draws them; playing them is yours, so a sequencer skips a muted note, rolls the dice for its chance and strays its velocity by up to its deviation. value and selection are controlled; defaultValue and defaultSelection leave them to the roll. The roll reports every edit through onValueChange and, when a gesture or a key press ends, onValueCommitted; eventDetails.edit tells what happened ("add", "move", "resize", "remove", "duplicate", "paste", "quantize", "stretch", "velocity", "chance" or "deviation") and eventDetails.cancel() keeps the notes as they were.

onNoteOn and onNoteOff say when a pitch should sound or stop: a key pressed, a note drawn, picked up or moved to a new pitch.

scale marks the rows of a scale (data-in-scale on each PianoRoll.Row), and folded="scale" keeps only those rows and the ones carrying notes. @audio-ui/core/pitch holds the scales (SCALES, inScale, nearestInScale), and @audio-ui/core/piano-roll the edits a toolbar or a menu applies to the selection, each returning the new notes: splitNotes, chopNotes, joinNotes, legatoNotes, reverseNotes, invertNotes, stretchNotes, fitNotesToScale, rampVelocity, humanizeNotes, toggleMuted, setChance, setDeviation and changeLevel, and the time commands deleteTime, insertTime and duplicateTime; selectionSpan gives the time a selection covers and findNotes the notes a query matches, by pitch, velocity, length, chance, scale, state or count.

The toolbar of the preview above is three menus that install on their own: the Scale menu picks the scale the roll marks, Note find selects notes by what they are, and the Note edit menu runs the commands above on the selection.

Playing the notes

usePianoRollPlayback plays the notes you hold through a synth: play, pause and stop, a loop or once through, a metronome, and the notes the roll auditions. npx shadcn@latest add @audio-ui/use-piano-roll-playback.

Copy
import { useMasterBus } from "@/hooks/use-master-bus";
import { meterOf, usePianoRollPlayback } from "@/hooks/use-piano-roll-playback";
import { useSynthVoices } from "@/hooks/use-synth-voices";

const tempo = { beatsPerBar: 4, beatUnit: 4, bpm: 120 };
const meter = meterOf(tempo, 4); // four bars, in steps

const output = useMasterBus(0.7);
const voices = useSynthVoices({ destination: output });
const playback = usePianoRollPlayback({ meter, notes, tempo, voices });

<Timeline
  duration={meter.duration}
  onPositionChange={playback.seek}
  snapStep={1}
  source={playback.position}
>
  <PianoRoll
    activePitches={playback.playingPitches}
    onNoteOff={playback.onNoteOff}
    onNoteOn={playback.onNoteOn}
    onValueChange={setNotes}
    value={notes}
  >
    {/* … */}
  </PianoRoll>
</Timeline>;

The voices and the bus come from Synthesis and Output, and the clock under the hook from Clock and time. The hook plays the notes; it does not hold them, so an undo, a file import or an agent can change them and the next pass plays the change. Time is in steps, four to a quarter note whatever the signature; meterOf(tempo, bars) gives a clip's length in steps and where its bars and beats fall. Notes are scheduled ahead on the audio clock, so they keep time when the page is busy, and each plays as the roll draws it: a deactivated note stays silent, a chance plays that often, a deviation strays its velocity each time round.

It returns play (on from the playhead), pause (stops where it is), stop (back to the start of the loop), seek, isPlaying, playingPitches, onNoteOn and onNoteOff for the roll, position (in steps) for the timeline's source, and clock (in seconds) for a Transport's source. It also takes looping (on by default), a loopRegion of steps, whose commitLoopRegion makes a moved loop heard at once, metronome with the destination it clicks into, and the envelope the notes play with.

Undo and redo

useUndoHistory undoes and redoes any value you hold. npx shadcn@latest add @audio-ui/use-undo-history.

Copy
import { useUndoHistory } from "@/hooks/use-undo-history";

const [notes, setNotes] = useState<Note[]>([]);
const history = useUndoHistory(notes, setNotes);

<PianoRoll
  onValueChange={history.change}
  onValueCommitted={history.commit}
  value={notes}
/>
<Button disabled={!history.canUndo} onClick={history.undo}>Undo</Button>
<Button disabled={!history.canRedo} onClick={history.redo}>Redo</Button>

change is each step of a gesture and commit ends it, so a whole drag undoes in one step. edit records a change made in one go, such as a menu command. A new change after an undo drops the redo, and the history keeps the last 100 changes. It works the same for an envelope's points or a sequencer's pattern.

MIDI files

The midi-file library reads a .mid file onto the roll's steps and writes one back. npx shadcn@latest add @audio-ui/midi-file.

Copy
import { downloadMidiFile, MIDI_FILE_ACCEPT, readMidiFile } from "@/lib/midi-file";

<input
  accept={MIDI_FILE_ACCEPT}
  onChange={async (event) => {
    const file = event.target.files?.[0];
    if (file) {
      const { bpm, notes } = await readMidiFile(file, {
        duration: 64,
        maxPitch: 84,
        minPitch: 36,
      });
      setNotes(notes);
      setBpm(bpm);
    }
  }}
  type="file"
/>
<Button onClick={() => downloadMidiFile(notes, { bpm, filename: "clip.mid" })}>
  Export
</Button>

readMidiFile puts each note on the nearest step, at least one step long, drops the notes starting past duration, keeps the pitches between minPitch and maxPitch and returns the file's tempo, kept between minBpm and maxBpm (20 and 200 by default). It takes any File or Blob, so a drop works the same as an input. writeMidiFile goes the other way, at 480 ticks a quarter note, and leaves deactivated notes out; downloadMidiFile writes the file and saves it. A step is a sixteenth note: set stepsPerQuarter on both sides for another grid.

Driving it from an agent

createPianoRollTools gives a language model the tools to write on the roll from what a user asks ("a walking bassline in C minor", "up an octave"): setNotes, addNotes, removeNotes, transpose, clear and setTempo, each a description, a zod schema and an apply that returns the next state. npx shadcn@latest add @audio-ui/piano-roll-tools.

Copy
import { tool } from "ai";
import { createPianoRollTools } from "@/lib/piano-roll-tools";

const roll = createPianoRollTools({ maxPitch: 72, minPitch: 48, stepCount: 16 });

const tools = Object.fromEntries(
  Object.entries(roll).map(([name, t]) => [
    name,
    tool({
      description: t.description,
      execute: (args) => {
        setState((state) => t.apply(state, args as never));
        return `applied ${name}`;
      },
      inputSchema: t.schema,
    }),
  ])
);

The pitch range and the grid are in the schemas, and every note a model writes is clamped into them, so whatever it answers lands on the roll's rows and steps. The tools read and write { notes, bpm } and depend on no agent library: the example wires them to the AI SDK, and any tool-calling API takes the same description and schema.

Interaction

Pointer

Keyboard

API reference

Root

Groups the parts of a piano roll and owns its notes and selection. Must be used within a Timeline. Renders a <div> element.

Keys

The keyboard down the side: a Piano laid out on the roll's rows, low end at the bottom. Pressing a key sounds it through the roll's onNoteOn; with Shift, pressing it (or Shift + Enter) selects every note on its row, or deselects them when they all are. Put a PianoRoll.Key per row inside, with PianoRoll.Rows. Renders a <div> element.

Rows

Renders children once per row, from the highest pitch down: the place to put a PianoRoll.Key or a PianoRoll.Row. Renders no element of its own.

Key and KeyLabel

The same parts as Piano: a key of the keyboard, and its label.

Grid

The surface notes are drawn on: a timeline lane, with the roll's rows stacked down it. A press on empty grid draws a note (mode="draw"), whose end then follows the pointer, or starts a selection rectangle, Shift adding to what is selected; in that mode a double-click draws. Focused, Enter adds a note at the playback position, Escape clears the selection and Cmd/Ctrl+A selects every note. From the grid or a note, Cmd/Ctrl+D repeats the selection right after itself, Cmd/Ctrl+C, X and V copy, cut and paste at the playback position, and Q snaps the starts of the selection (every note, with none selected) to the grid. Renders a <div> element.

Row

The band a pitch takes across the grid, for shading black-key rows, the rows of the roll's scale, or ruling where an octave starts. It lets presses through to the grid and is hidden from assistive technology. Renders a <div> element.

Notes

Renders children once per note, in the order they are read: by start, then from the highest pitch down. That order is also the tab order. Renders no element of its own.

Note

One note, placed on its row from its start, as long as its length. A toggle button named after its pitch and span (getNoteLabel on Root). Its --piano-roll-velocity and --piano-roll-chance CSS variables are its velocity and how likely it is to play. Renders a <div> element.

NoteHandle

One edge of the note it sits in, dragged to change the note's length: the end makes it longer or shorter, the start trims it from the front while its end stays put. On the grid unless Cmd/Ctrl is held, and for the whole selection when the note is part of it. Shift + Left and Right on the note do the same from the keyboard, so the handle is hidden from assistive technology. Renders a <div> element.

Marquee

The selection rectangle being dragged across the grid; renders nothing the rest of the time. Hidden from assistive technology. Renders a <div> element.

SelectionSpan

The time the selected notes cover, from the earliest start to the latest end, as a band from top to bottom of what it sits in: the grid, the velocity lane or the ruler. Renders nothing with no note selected. It lets presses through, all but its PianoRoll.StretchHandles, and is hidden from assistive technology. Renders a <div> element.

StretchHandle

One edge of the selection's span, for its SelectionSpan: a drag stretches the selected notes in time from the other edge, on the grid unless Cmd/Ctrl is held. The Notes edits that double and halve lengths do the same from the keyboard, so it is hidden from assistive technology. Renders a <div> element.

VelocityLane

A strip under the grid where each note's velocity, or with level="chance" its chance to play, stands as a bar placed at the note's start: a timeline lane, like the grid. In draw mode it is a brush: a drag across it sets every bar under the pointer, in the grid cell it is over, to the pointer's height. Put a PianoRoll.Velocity per note inside, with PianoRoll.Notes. Renders a <div> element.

Velocity

One note's velocity, or its chance in a level="chance" lane, as a bar from the bottom of the lane: its height is the value, through the --piano-roll-level CSS variable. Dragging it up or down changes the value of the selection by the same amount when the note is part of it, so a phrase keeps its dynamics; Shift + click toggles the note in the selection. On a velocity bar, Cmd/Ctrl + drag sets how far the velocity may stray each time the note plays, its --piano-roll-deviation. A native <input type="range"> inside takes the arrow keys (Page Up/Down for larger steps, Home/End for none and full) and carries the accessible name, which defaults to the note's. The next drawn note gets the velocity set last. Renders a <div> element.

Registry components

PianoRoll takes the Root props. The rest lay the editor out on two columns, the keys' and the timeline's: