HN Debrief

Show HN: I trained a 125M model to autocomplete piano on-device

  • AI
  • Music
  • Mobile
  • Developer Tools

The post showed a piano autocomplete system that takes live MIDI input, feeds it into a 125M parameter transformer, and generates a continuation on an iPhone 15 at roughly 108 notes per second. The author framed it as a music version of Copilot, with the model running entirely on-device through Core ML, trained on a few hundred thousand MIDI files totaling about 300 million note events. Follow-up answers filled in some of the engineering details. Pretraining the 125M model took about half a day, preference tuning used only about 700 DPO examples and finished in minutes on a single GPU, and the biggest speed gain came from switching to compound note events, which cut the number of autoregressive passes per note by about 5x.

If you build creative AI tools, the opportunity here is not full replacement but tight, low-latency co-creation inside existing workflows. The technical lesson is just as important: representation choices and product integration may matter more than scaling the model when you need real-time usefulness on consumer hardware.

Discussion mood

Strongly positive and curious. People were impressed by the fact that a 125M transformer runs in real time on an iPhone, and they saw the rough musical output as acceptable for an early co-creation tool. Criticism focused on weak rhythm and larger-scale structure, plus a smaller pocket of resistance to AI-assisted creation on artistic grounds.

Key insights

  1. 01

    Representation beat kernel tuning

    Changing the event format did more for performance than low-level optimization. The switch to compound note events cut autoregressive passes per note by about 5x, which is why a 125M model can feel real time on a phone without heroic inference work. That is a useful reminder that in sequence models, the tokenization scheme can decide whether a product is viable at all.

    If you are chasing on-device latency, revisit the event representation before spending weeks on runtime optimization. Measure tokens per meaningful user action, not just parameters or FLOPS.

      Attribution:
    • simedw #1
    • Naitik88 #1
  2. 02

    The missing piece is musical planning

    The weak output is not mainly about raw speed or model size. It is about the model lacking explicit structure for bars, rhythm, and longer-range intent. The author's own next steps, adding measure-aware tokens and generating several continuations before picking one, point toward a two-stage system where fast local continuation is guided by a lighter planning layer.

    For creative generators, separate instant reaction from longer-horizon structure. A hybrid design can preserve responsiveness while improving coherence enough to matter to real users.

      Attribution:
    • evalystai #1
    • simedw #1 #2
  3. 03

    MIDI routing is the real product boundary

    People did not just want to hear output from a phone speaker. They wanted MIDI notes routed back into hardware, DAWs, Max for Live, VST hosts, and even player pianos. That changes the project from a neat demo into infrastructure for existing music workflows, where the model becomes another controllable instrument instead of a toy app.

    If you are productizing a creative model, invest early in interoperability. Shipping plugin formats and MIDI I/O can matter more than another small jump in model quality.

      Attribution:
    • cillianpower #1
    • senbrow #1
    • h2aichat #1
    • ryanschneider #1
    • axoltl #1
  4. 04

    Autocomplete fits older music practice

    Classical musicians pointed out that continuing a phrase from a prompt is not a new artistic idea at all. Partimento, extemporization, and phrase-continuation games trained composers to internalize patterns and produce stylistically correct continuations from sparse cues. That framing makes the tool easier to understand. It is mechanizing one slice of musicianship rather than inventing a foreign relationship to music.

    When pitching AI creative tools, tie them to established human workflows and traditions. Users adopt faster when the tool feels like a new instrument for an old practice.

      Attribution:
    • tom_vidal #1 #2
    • vunderba #1
  5. 05

    The value is faster exploration, not final authorship

    One musician-designer cut through the usual AI-art fight by focusing on taste. If generation is cheap, the scarce skill is choosing, rejecting, and steering possibilities. That maps well to how composers and producers actually work. They do not need every suggestion to be great. They need more plausible branches to audition and kill quickly.

    Design these systems around branching, previewing, and discard speed. The win is compressing exploration time, not pretending first-pass output is the finished work.

      Attribution:
    • joshuamerrill #1
    • stickfigure #1

Against the grain

  1. 01

    A transformer alone may be the wrong level

    The harshest technical critique was that these continuations do not obviously outperform far simpler sequence models, and that better music systems may need explicit harmonic or structural layers before token-level generation. One correction mattered here. The project is not an LLM pipeline, so the useful objection is architectural, not about language models spilling into music by default. It presses on whether next-note modeling is enough to produce convincing composition.

    Do not assume scaling an autoregressive transformer will automatically buy musicality. Benchmark against simpler baselines and test whether adding explicit theory-level representations improves output more than more parameters do.

      Attribution:
    • rajivayyangar #1
    • Computer0 #1
  2. 02

    Some musicians hear creative erosion, not augmentation

    A small but intense pushback treated the whole idea as dodging the hard work of learning harmony, voicing, and improvisation. That view did not persuade many people, because the demo was mostly read as a playful instrument and engineering exercise. Still, it points to a real adoption barrier. In creative markets, some users will see assistance itself as the product defect.

    Expect a segment of expert users to reject AI help on principle. Position the tool for experimentation and collaboration, and avoid marketing language that implies the machine deserves artistic credit.

      Attribution:
    • bubblegumcrisis #1
    • jamilton #1

In plain english

autoregressive ↩
A model setup where the system predicts the next token or event one step at a time based on what came before.
bar ↩
A segment of musical time, also called a measure, that groups beats into repeated units.
beam-style selection ↩
A decoding strategy that generates several candidate continuations in parallel and keeps the most promising ones instead of following only one path.
compound note events ↩
A representation that groups multiple musical attributes or actions into a single model event so the model needs fewer prediction steps.
Core ML ↩
Apple’s framework for running machine learning models efficiently on iPhone, iPad, Mac, and other Apple devices.
DAW ↩
Digital audio workstation, software used for recording, editing, and producing music or other audio.
DPO ↩
Direct Preference Optimization, a training method that adjusts a model using examples of outputs people preferred over alternatives.
extemporization ↩
Improvising music in real time, often within a learned style or form.
GPU ↩
Graphics Processing Unit, a processor specialized for rendering graphics and often used for AI and other compute-heavy workloads.
LLM ↩
Large Language Model, a machine learning model trained to generate and analyze human-like text.
Max for Live ↩
A platform inside Ableton Live for building custom music devices, instruments, and interactive tools.
MIDI ↩
Musical Instrument Digital Interface, a standard way for electronic instruments and software to send note, timing, and control information rather than recorded audio.
partimento ↩
A historical method of music training where students learned composition and improvisation from bass lines, patterns, and stylistic rules.
tokenization ↩
A security method that replaces a real card number with a substitute token so the merchant or device does not store the actual number.
transformer ↩
A neural network architecture commonly used for sequences like text, audio events, or notes, where each output depends on the surrounding context.
VST ↩
Virtual Studio Technology, a common plugin format for audio effects and instruments.

Reference links

Historical music practice and improvisation

Earlier and related AI music systems

Music examples and listening references

Broader music generation context

  • All the Music
    Mentioned as a project that algorithmically enumerates melodies to challenge copyright claims, adding context on generative music and ownership.