Koto: dictation for Windows in Indian languages

I have been working on Koto, a dictation app for Windows that types what you say into whichever app has focus. You hold Ctrl + Win, speak in English, Hindi, Marathi or any of 22 Indian languages (or a mix of them), and release. The text appears at the cursor, whether that is a chat window, a code editor, a browser field or Notepad.

Speech is transcribed by Saaras v4[1], a speech model from Sarvam AI built for Indian languages. Koto reaches it through a small server that each person deploys to their own Cloudflare account, with their own API keys. Koto is software only: there is the Windows app and the server code, and every running server belongs to the person who deployed it.

Why I made it

I speak a mix of English, Hindi and Marathi through most of a working day, often within a single sentence, and I wanted dictation that treats that mix as ordinary input. Saaras v4 detects the language by itself and handles mixed Hindi and English, so the remaining work was everything around the model: capturing audio on Windows, sending it somewhere the person controls, cleaning up the transcript, and placing the text where the cursor is.

I also wanted each person to keep their own data. A hosted service would route every user's audio through a server I run. With Koto, the server is a Cloudflare Worker in the person's own account, and the only data it stores is the optional backup in their own R2 bucket.

Using it from the keyboard

While Koto records, a small orb appears on the monitor you are working on, and its halo grows with your voice. Clicking it finishes a hands-free dictation and keeps focus in the app you are dictating into. Koto keeps running from the tray after its window closes (Open Koto, Writing style, Microphone, Quit Koto), and it starts when you sign in to Windows, which you can switch off in Settings.

How a dictation moves through the system

There are three parts to it. The Windows app records, segments and formats; the Worker in your Cloudflare account passes audio to Sarvam and text to Gemini; and native C++ handles the hotkey, the orb and pasting. The diagrams below show the pipeline, the requests behind one dictation, and the hotkey states.

Recording in parts while you speak

Saaras accepts segments under 30 seconds, and a Koto dictation can run for up to 20 minutes. The app records 16 kHz mono 16-bit PCM through WASAPI, which converts from the microphone's own format, and a speech segmenter splits the stream as it arrives. Once 15 seconds are buffered, it checks every half second for a pause by measuring the loudness of 200 ms windows. It cuts at the first pause below the threshold, or at 28 seconds whatever the audio is doing, which keeps every segment inside the limit with room to spare.

Each segment is sent as soon as it is cut, so by the time you release the hotkey most of a long dictation is already transcribed and only the last segment is still on its way. A segment that holds only room sound is skipped instead of sent.

1export const SEGMENT_SEARCH_START_SECONDS = 15;
2export const SEGMENT_FORCE_SECONDS = 28;
3const WINDOW_SECONDS = 0.2;
4const PAUSE_THRESHOLD = 0.012;

Raising soft speech before transcription

On a laptop microphone at arm's length, loud speech measured 0.04 to 0.08 RMS, and the laptop's own noise suppression gated silence to digital zero. A fixed speech threshold of 0.02 dropped a real one-second dictation as silence, so the threshold now adapts to the room.

Koto estimates the background level from the softest 10 percent of 50 ms windows and treats anything three times louder as speech, within a range of 0.008 to 0.02 RMS. Before sending audio, it applies gain so loud speech lands near 0.18 RMS (about minus 15 dBFS), capped at 6 times (about 15.6 dB) so an empty room stays an empty room. Peaks above 0.85 are rounded off with a tanh curve instead of clipping. The saved voice note stays exactly as recorded, and only the copy sent to Sarvam changes.

1export function speechGain(samples: Int16Array): number {
2  const threshold = speechThreshold(samples);
3  const voiced = windowLevels(samples)
4    .filter(level => level > threshold)
5    .sort((a, b) => a - b);
6  const loudSpeech = percentile(voiced, 0.9);
7  if (loudSpeech <= 0) {
8    return 1;
9  }
10  return Math.min(MAXIMUM_GAIN, Math.max(1, TARGET_SPEECH_RMS / loudSpeech));
11}

What the server does

The server is a Cloudflare Worker with separate modules for transcription, the AI tidy-up and backups. It holds the person's Sarvam and Gemini keys only for the length of a request: the app keeps them in Windows Credential Manager and sends them over HTTPS with each call, and the Worker uses them for that call alone.

Access is controlled by one key of at least 16 characters[2], chosen when the Worker is deployed. The Worker hashes the given key and the expected key with SHA-256 and compares the hashes byte by byte, so the response time stays the same however much of the key matches.

1export async function isSameSecret(given: string, expected: string): Promise<boolean> {
2  const [a, b] = await Promise.all([digest(given), digest(expected)]);
3  let difference = 0;
4  for (let index = 0; index < a.length; index += 1) {
5    difference |= a[index] ^ b[index];
6  }
7  return difference === 0;
8}

The transcription route accepts only the format the app produces (PCM WAV, mono, 16 kHz, 16-bit) and answers a segment over 30 seconds with a 413. For script, the Worker first asks Saaras for its default mode, which punctuates English best. Only when the result contains an Indian script and you asked for English letters or a mixed script does it ask again in translit or codemix mode, so English dictation costs one request.

Formatting rules that run on your PC

Saaras already removes most “um” and “uh” and adds punctuation. Koto then applies its own rules locally, and each rule can be switched off in Settings.

Talking about punctuation, as in “the question mark is missing”, stays as words. Hindi and Marathi filler (mhanje, matlab, toh, arre, yaar) is removed only where it fills a pause, because the same words often carry meaning, as “mhanje” does in the last row. Words you add to the Dictionary are sent to Saaras as key terms so names come out spelled correctly, and replacements rewrite phrases after transcription.

There are three writing styles: Neutral, Casual (“by the way” becomes BTW) and Formal (contractions written out, slang and chat abbreviations removed). You can switch between them in Settings, from the tray menu or by right-clicking the orb.

When the AI tidy-up runs

Rules handle most dictation. For the rest, Koto decides on your PC, instantly, whether the text still reads like speech. Dictation under 6 words stays with the rules. Between 6 and 12 words, a request goes out only for a strong signal, and from 12 words the softer signals count as well.

The check below is a reduced port of polish-need.ts that runs in your browser. Edit the text to see which signals it finds.

These examples come from the instructions the Worker sends to Gemini, and show the kind of text that reaches the tidy-up and what comes back.

HindiMarathiAsidesRepetition

kal ki meeting hum log, nahi nahi, parso rakhte hai

Koto

parso ki meeting hum log rakhte hai

Checking what Gemini sends back

The tidy-up uses Gemini 3.5 Flash-Lite at temperature 0 with minimal thinking. The dictation goes inside <dictation> tags, so a question or instruction you speak is treated as text to tidy. The answer then has to pass these checks on the Worker before it replaces the rule-based text:

• It keeps at least half of the meaningful words that were said, where meaningful means four letters or more and outside a short list of filler.
• At most 20 percent of its meaningful words are new, after allowing for word forms (“moving” and “moved”), the spelled-out forms the Formal style writes (“gonna” to “going”) and the abbreviations the Casual style adds.
• It is at most 1.4 times the length of the input.
• It keeps every Hindi or Marathi “na”, since “aao na” softens a request and “na aana” reverses it.
• It starts with the text itself, as opposed to “Here is” or “Sure”.

1const said = new Set(contentWords(original));
2const written = contentWords(text);
3if (said.size === 0) {
4  return text;
5}
6const stem = (word: string) => word.replace(/['’]s$/, '').replace(/(ing|ed|es|s|ly)$/, '');
7const saidStems = new Set([...said].map(stem));
8const writtenStems = new Set(written.map(stem));
9const kept = [...saidStems].filter(word => writtenStems.has(word)).length / saidStems.size;
10const invented = written.filter(
11  word => !saidStems.has(stem(word)) && !spelledOut.has(stem(word)) && !STYLE_ABBREVIATIONS.has(word),
12).length;
13const inventedShare = written.length === 0 ? 0 : invented / written.length;
14if (kept < 0.5 || inventedShare > 0.2) {
15  return undefined;
16}
17return text;

A failed check, an error or a response slower than 2.5 seconds returns the rule-based text, so the tidy-up can delay a dictation by 2.5 seconds at most and every dictation still arrives. History keeps the text from before the tidy-up, with a button to copy that version.

Long dictation is tidied in parts

Tidying a long dictation only after you stop would leave you waiting on the whole text at once, so Koto tidies it in parts while you are still speaking. Once the text that is still untidied reaches 40 words and ends a sentence, it goes off as one part, and each part becomes its own paragraph.

A part closes only once the next segment's text is known. It stays open when that next segment starts in lowercase, starts with a correction such as “sorry” or “scratch that”, or when either side contains a spoken list, because the rules need a list whole. Parts get 7 seconds to come back and the final part gets 2.5 seconds; a part still out after that uses its rule-based text. If you switch writing style partway through, every paragraph follows the style in use when you finish.

Pasting into the right place

Text insertion is native C++. Koto takes a snapshot of every format on the clipboard (text, rich text, images and files), places the dictation on it, sends Ctrl + V and then restores the snapshot, as long as the clipboard is unchanged since the paste. The restored copy stays out of Win + V history, which already holds the original. Line breaks become CRLF for classic Win32 edit controls such as Notepad, and Koto waits for Win, Alt and Shift to be released first, because Ctrl + V with Win held would open clipboard history instead.

The hotkey uses a low-level keyboard hook. Releasing Win by itself opens the Start menu, so Koto injects an unassigned virtual key between Win down and Win up. If you press another shortcut such as Ctrl + Win + D, the hook lets it through and abandons the dictation.

Voice notes and backup

Every recording is saved as a WAV file in Documents\Koto\Voice notes, at about 2 MB a minute, and kept until you choose a shorter period in Settings (30, 7 or 1 days).

Back up now copies voice notes, history and preferences to the R2 bucket on your server in parts. It runs in the background, waits while you dictate, and resumes where it stopped. Download backup creates a link that lasts 30 minutes, signed by the Worker with HMAC using the access key, and a large backup arrives as several zips listed on one page. Each server holds up to 3 GB of backups, and server settings and keys stay out of every backup.

Privacy

Koto collects zero analytics and has zero accounts.

What it costs

A minute of dictation makes a few requests to the Worker, well inside the free allowance. Koto is free and open source under the MIT licence, and you pay the providers directly for what you use.

Setting it up

1. Get a Sarvam API key from the Sarvam dashboard.
2. If you want the tidy-up, get a Gemini API key from Google AI Studio.
3. Deploy the server with Deploy to Cloudflare. Cloudflare copies the worker folder into a repository on your GitHub account, creates an R2 bucket for backups and deploys the Worker. When it asks for KOTO_ACCESS_KEY, enter a random password of at least 16 characters.
4. Note the server address Cloudflare shows, such as https://koto-server.<your-subdomain>.workers.dev. Opening it in a browser shows “Your Koto server is running”.
5. Install Koto from the latest release. Once per PC, open Koto-signing.cer, choose Install Certificate, Local Machine, and place it in Trusted People. Then open Koto-x64.msix, or Koto-ARM64.msix on ARM PCs.
6. On first launch, enter the server address, the access key and your Sarvam key (and the Gemini key if you have one), then select Save and connect.

Updates, releases and tests

The app is built on React Native for Windows 0.83, with native C++ for the hotkey, audio capture, the orb, pasting and updates. Releases are signed MSIX packages for x64 and ARM64, built by a GitHub Actions workflow that publishes v1.0.<run number>. Koto checks the latest release when it starts and every 6 hours, and Update and restart hands the install to Windows, which closes Koto, installs the package and reopens it. Every build is signed with the same certificate, because Windows accepts an update only from the same publisher.

The tests run on Linux in CI, so Visual Studio is needed only to build the app itself. The app has 17 test files with about 160 cases covering formatting, audio and storage logic, and the Worker has its own tests and typecheck. Updating your server means copying the latest worker folder into the repository Cloudflare created for you, which Cloudflare redeploys on each push.

1cd app && npm install && npx jest && npx tsc --noEmit && npx eslint src __tests__ index.js
2cd worker && npm install && npm test && npm run typecheck

Brand

The logomark is drawn from 言, the character read as koto, meaning word or speech. The orange dot above it is the recording orb.