Keyboard Piano

Name: Keyboard Piano
Author: @rebind

Your keyboard becomes a piano with chords, sustain and octave shift. A lessons page in your browser teaches five songs.

@rebindv0.1.0free

Install in Rebind

More install options

View in Rebind Get the app

$rebind install @rebind/keyboard-piano

Gallery

Readme

Your keyboard plays piano notes. Hold several keys for a chord; a note rings while its key is down and fades when you let go.

Keys

The bottom two letter rows are the lower octave, the top two are the upper octave. White keys sit on the Z and Q rows, black keys on the row above each.

Notes	White keys	Black keys
C3 to E4	Z X C V B N M , . /	S D G H J L ;
C4 to G5	Q W E R T Y U I O P [ ]	2 3 5 6 7 9 0 =

Key	Does
Space	Sustain pedal. Notes released while it is held keep ringing until you let it go.
Left, Right	Shift every key down or up an octave, one octave either way.
F9	Plays the chosen song by itself, both hands. Press it again to stop. The keys stay live, so you can play along.
F8	Piano off or on. While it is off the keys type normally.

Those five keys, the note volume and the damper time are configurable. Restart the script after changing a key.

Chords that include Ctrl, Alt or Cmd/Win are left alone, so shortcuts keep working while the piano is on.

The game

Starting the script opens http://localhost:47800 in your default browser. Blocks fall toward a piano, each printed with the key that plays it. Press that key as the block lands on the line.

Hits within 70 ms score Perfect, within 160 ms Early or Late, and anything else is a miss. A combo multiplies the score and a wrong key breaks it. Each run ends with a grade, and the page keeps your best score per song and speed.
The piano plays the left hand while you play the melody. Watch it played has it play both hands.
Three speeds: slow, medium and full.

Five songs, easiest first: Twinkle, Twinkle, Little Star; Ode to Joy; Jingle Bells; Minuet in G; Für Elise. All are in the public domain, and every melody fits the upper key row without an octave shift. The song you pick on the page is the one F9 plays.

The package declares two permissions. net serves the page and makes no outbound connection. exec is used once, to open the page in your browser; turn off "Open the game page when the script starts" under Game to skip it. The page is served on localhost only. The WebSocket that feeds it, port 47801, accepts connections from your local network; it sends the notes being played and accepts only a song choice, play and stop. Turn off "Serve the game page" and restart the script to open no ports at all.

Sound

The notes are synthesized, not recorded from an instrument: one Ogg Vorbis file per note under samples/, from C2 to G6, at 48 kHz. The model stacks the stretched partials of stiff strings, shapes them by where and how hard a felt hammer strikes, lets each one decay in two stages, and gives every note two or three slightly mistuned strings and a short hammer knock. tools/synth.py regenerates the samples with Python's standard library and ffmpeg.

Limits

The note keys are captured system wide while the piano is on. Press the on or off key before you type.
Every key plays at the same loudness. A computer keyboard cannot tell how hard you press.
Many keyboards cannot report some combinations of three or more keys at once. A chord that drops a note is the keyboard's limit, not the script's.

Reviews

No reviews yet. Write a review in the app.

Source

Version 0.1.0 (current)

main.luau

-- Keyboard Piano: two rows of keys play notes, chords included. One
-- synthesized sample per note lives in samples/, named by MIDI number. A lessons page on
-- localhost follows what you play over a WebSocket and teaches the songs.

local SONGS = require("songs")

-- semitones above the base C. Z row is the lower octave with its sharps on
-- the A row; Q row is the upper octave with its sharps on the number row.
-- The two rows overlap from Comma to Slash, as tracker keyboards do.
local KEYS = {
  Z = 0,
  S = 1,
  X = 2,
  D = 3,
  C = 4,
  V = 5,
  G = 6,
  B = 7,
  H = 8,
  N = 9,
  J = 10,
  M = 11,
  Comma = 12,
  L = 13,
  Period = 14,
  Semicolon = 15,
  Slash = 16,
  Q = 12,
  ["2"] = 13,
  W = 14,
  ["3"] = 15,
  E = 16,
  R = 17,
  ["5"] = 18,
  T = 19,
  ["6"] = 20,
  Y = 21,
  ["7"] = 22,
  U = 23,
  I = 24,
  ["9"] = 25,
  O = 26,
  ["0"] = 27,
  P = 28,
  LeftBrace = 29,
  Equal = 30,
  RightBrace = 31,
}

local BASE = 48 -- Z plays C3 at octave shift 0
local MAX_SHIFT = 1 -- samples cover the key map one octave either way
local FADE_STEP = 15
-- Net.Listen answers on localhost only. Net.WSListen binds every interface, so
-- the socket accepts nothing but song choices and play or stop.
local HTTP_PORT = 47800
local WS_PORT = 47801

local cfg = UI.Schema({
  toggleKey = UI.Keybind("F8", { label = "Piano on or off", tab = "Keys" }),
  songKey = UI.Keybind(
    "F9",
    { label = "Play or stop the chosen song", tab = "Keys" }
  ),
  sustainKey = UI.Keybind("Space", { label = "Sustain pedal", tab = "Keys" }),
  octaveDownKey = UI.Keybind("Left", { label = "Octave down", tab = "Keys" }),
  octaveUpKey = UI.Keybind("Right", { label = "Octave up", tab = "Keys" }),
  volume = UI.Slider(60, {
    min = 10,
    max = 100,
    step = 5,
    suffix = "%",
    label = "Note volume",
    tab = "Sound",
  }),
  damperTime = UI.Slider(1800, {
    min = 100,
    max = 3000,
    step = 50,
    suffix = " ms",
    label = "Damper: how long a released note takes to die away",
    tab = "Sound",
  }),
  lessons = UI.Toggle(true, {
    label = `Serve the game page at http://localhost:{HTTP_PORT}`,
    tab = "Game",
  }),
  openPage = UI.Toggle(true, {
    label = "Open the game page when the script starts",
    tab = "Game",
  }),
})

local on = true
local octave = 0
local pedal = false
-- key name -> { sound, note } for every note key currently down. A sound
-- nobody holds a reference to is stopped by the collector, so every live sound
-- stays in one of these tables, or in `fading`, until it should end.
local held = {}
-- sounds whose key came up while the pedal was down
local sustained = {}
local socket = nil

-- tell the lessons page what just happened. a no-op until a page connects.
local function emit(message)
  if socket and socket:ClientCount() > 0 then
    socket:Broadcast(JSON.Stringify(message))
  end
end

-- There is no fade in the SDK, and a hard Stop clicks. A released note dies
-- away the way a damper settles on a string: the volume falls by the same
-- ratio every step, reaching -60 dB after the damper time, then the sound
-- stops. A straight-line fade to zero is heard as the note being cut off. One timer fades every sound: the
-- relay keeps each timer callback referenced for the life of the script, so a
-- timer per note would pin that note's sound forever, and a sound that is
-- never released keeps loading the audio thread until playback distorts.
local fading = {} -- sound -> { volume, floor, ratio }

local function damp(sound)
  local volume = sound:GetVolume()
  fading[sound] = {
    volume = volume,
    floor = volume / 1000,
    ratio = 0.001 ^ (FADE_STEP / cfg.damperTime),
  }
end

Timer.Every(FADE_STEP, function()
  for sound, fade in fading do
    fade.volume *= fade.ratio
    if fade.volume <= fade.floor or not sound:IsPlaying() then
      sound:Stop()
      fading[sound] = nil
    else
      sound:SetVolume(fade.volume)
    end
  end
end)

local function dampSustained()
  for _, sound in sustained do
    damp(sound)
  end
  sustained = {}
end

-- Every piano bind is guarded by this, so a key passes through untouched when
-- the piano is off or the chord is a shortcut (Ctrl, Alt, Cmd/Win). The on/off
-- switch is this flag and not bind:disable(): a bind handle method called from
-- inside a bind callback deadlocks the relay's input thread.
local function playing()
  if not on then
    return false
  end
  local m = Input.GetModifiers()
  return not (m.ctrl or m.alt or m.win)
end

local function play(note, source)
  local ok, sound =
    pcall(Audio.Play, `samples/{note}.ogg`, { volume = cfg.volume / 100 })
  if not ok then
    Log.Error(`note {note}: {sound}`)
    return nil
  end
  emit({ t = "down", note = note, src = source })
  return sound
end

for key, semitone in KEYS do
  Bind(key, {
    when = playing,
    action = function()
      local note = BASE + semitone + 12 * octave
      local sound = play(note, "key")
      held[key] = if sound then { sound = sound, note = note } else nil
      return nil
    end,
    release = function()
      local press = held[key]
      held[key] = nil
      if press == nil then
        return
      end
      emit({ t = "up", note = press.note, src = "key" })
      if pedal then
        table.insert(sustained, press.sound)
      else
        damp(press.sound)
      end
    end,
  })
end

Bind(cfg.sustainKey, {
  when = playing,
  action = function()
    pedal = true
    return nil
  end,
  release = function()
    pedal = false
    dampSustained()
  end,
})

local function shift(by)
  octave = math.max(-MAX_SHIFT, math.min(MAX_SHIFT, octave + by))
  UI.Notify(`Octave {if octave > 0 then "+" else ""}{octave}`, "info")
  emit({ t = "octave", octave = octave })
end

Bind(cfg.octaveDownKey, {
  when = playing,
  action = function()
    shift(-1)
    return nil
  end,
})
Bind(cfg.octaveUpKey, {
  when = playing,
  action = function()
    shift(1)
    return nil
  end,
})

-- A song plays itself, one coroutine per hand, while the keys stay live so you
-- can play along. F9 plays both hands of the song chosen on the game page. The
-- page's game asks for the left hand alone, at its own speed, and leaves the
-- melody to the player.
local selected = SONGS[#SONGS]
local hands = {}
-- sound -> note for every song note still ringing. a cancelled hand never
-- reaches its own damp, so stopSong ends these itself: restarting a song must
-- not stack the old run's notes under the new one.
local ringing = {}

local function songPlaying()
  for _, hand in hands do
    if hand:IsRunning() then
      return true
    end
  end
  return false
end

local function stopSong()
  for _, hand in hands do
    hand:Cancel()
  end
  hands = {}
  for sound, note in ringing do
    emit({ t = "up", note = note, src = "song" })
    damp(sound)
  end
  ringing = {}
end

local function playHand(notes, step)
  for i = 1, #notes, 2 do
    local note = notes[i]
    local sound = if note ~= 0 then play(note, "song") else nil
    if sound then
      ringing[sound] = note
    end
    Sleep(notes[i + 1] * step)
    if sound then
      ringing[sound] = nil
      emit({ t = "up", note = note, src = "song" })
      damp(sound)
    end
  end
end

local function startSong(leftOnly, speed)
  stopSong()
  local song = selected
  local step = song.step / speed
  hands = {
    Run(function()
      playHand(song.left, step)
    end),
  }
  if not leftOnly then
    table.insert(
      hands,
      Run(function()
        playHand(song.right, step)
      end)
    )
  end
end

Bind(cfg.songKey, {
  when = playing,
  action = function()
    if songPlaying() then
      stopSong()
    else
      startSong(false, 1)
    end
    return nil
  end,
})

local function silence()
  stopSong()
  Audio.StopAll()
  fading = {}
  held = {}
  sustained = {}
  pedal = false
end

Bind(cfg.toggleKey, function()
  on = not on
  if not on then
    silence()
  end
  UI.Notify(if on then "Piano on" else "Piano off, keys type", "info")
  return nil
end)

local function songById(id)
  for _, song in SONGS do
    if song.id == id then
      return song
    end
  end
  return nil
end

-- the page may choose a song, start it (left hand alone or both, at a speed)
-- and stop it. nothing else is accepted.
local function onPageMessage(_, payload, isBinary)
  if isBinary or #payload > 200 then
    return
  end
  local ok, message = pcall(JSON.Parse, payload)
  if not ok or type(message) ~= "table" then
    return
  end
  if message.t == "stop" then
    stopSong()
    return
  end
  local song = songById(message.id)
  if song == nil then
    return
  end
  if message.t == "select" then
    stopSong()
    selected = song
  elseif message.t == "play" and on then
    selected = song
    local speed = if type(message.speed) == "number" then message.speed else 1
    startSong(message.hands == "left", math.max(0.4, math.min(1.5, speed)))
  end
end

local function servePage(request)
  if request.method ~= "GET" then
    return { status = 405, body = "GET only" }
  end
  if request.path == "/" then
    return {
      body = File.Read("site/index.html"),
      headers = { ["Content-Type"] = "text/html; charset=utf-8" },
    }
  end
  if request.path == "/piano.json" then
    return {
      body = JSON.Stringify({
        keys = KEYS,
        base = BASE,
        wsPort = WS_PORT,
        songs = SONGS,
      }),
      headers = { ["Content-Type"] = "application/json" },
    }
  end
  return { status = 404, body = "not found" }
end

function OnStart()
  Log.Info(
    `Z and Q rows play. {cfg.sustainKey} sustains, {cfg.octaveDownKey} and {cfg.octaveUpKey} shift octaves, {cfg.songKey} plays a song, {cfg.toggleKey} turns the piano off so keys type again.`
  )
  if not cfg.lessons then
    return
  end
  Net.Listen(HTTP_PORT, servePage)
  -- on a restart the previous run's socket takes up to 100 ms to let go of
  -- the port, and Net.WSListen reports success even when its bind fails.
  Timer.After(500, function()
    socket = Net.WSListen(WS_PORT, {
      OnConnect = function(client)
        client:Send(JSON.Stringify({
          t = "hello",
          octave = octave,
          selected = selected.id,
        }))
      end,
      OnMessage = onPageMessage,
    })
  end)
  local url = `http://localhost:{HTTP_PORT}`
  Log.Info(`Game page: {url}`)
  if not cfg.openPage then
    return
  end
  -- a page left open from the last run reconnects by itself within two
  -- seconds. open the default browser only when none did.
  Timer.After(2500, function()
    if socket:ClientCount() > 0 then
      return
    end
    local mac = _REBIND.platform == "macos"
    local ok, err = pcall(
      System.ExecDetached,
      if mac then "open" else "rundll32",
      if mac then { url } else { "url.dll,FileProtocolHandler", url }
    )
    if not ok then
      Log.Warn(`could not open {url}: {err}`)
    end
  end)
end

function OnStop()
  silence()
end

function OnError(message)
  Log.Error(message)
  silence()
end

56 files without text previews