How to build a meeting notetaker without a bot

Short answer: skip the meeting platform and record the computer. Capture two local streams, the microphone and the system audio the call plays (ScreenCaptureKit or Core Audio taps on macOS, WASAPI loopback on Windows, a PulseAudio or PipeWire monitor on Linux), send them to a speech-to-text API, then summarise the transcript with an LLM. No participant joins the call.

This guide is for developers who want to build one, or who want to understand what the notetaker they pay for actually does. It covers the two architectures, how local capture works on each operating system and which permission each one asks for, then transcription, summaries, storage and consent. Every API named here comes from the vendor's own documentation, listed at the end and checked on 2026-10-21.

There are two ways to get meeting audio. A bot is a server-side client that joins the call as a participant and receives the media from the platform. Local capture runs on the user's computer and records what that computer already hears: the microphone for the user's voice, and the system output for everyone else. A newer middle option is a platform media API that streams a meeting to your server, but each one belongs to a single platform.

• Zoom: the Meeting SDK joins meetings and gives direct access to raw audio and video on native platforms. Zoom's docs say that from March 2, 2026, apps joining meetings outside their own account must be authorized with ZAK or OBF tokens, or use RTMS.
• Zoom RTMS (Realtime Media Streams) sends live audio, video and transcript data to your app without a participant bot, and runs on credits bought from Zoom. It only covers Zoom.
• Microsoft Teams: real-time audio needs an application-hosted media bot written in C# or .NET, deployed in production on Windows Server in Azure, with a public IP address per VM instance. Microsoft still describes its real-time media platform as a developer preview.
• Google Meet: the Meet Media API gives real-time audio, video and participant data, but it is in Developer Preview and, per Google's docs, no longer accepting new signups. Someone in the call has to consent, and encrypted or watermarked meetings cannot connect.

None of that is wrong. It is three integrations with three approval paths, and participants see a bot or a consent prompt. If you want one capture path that works the same for every platform, plus in-person meetings and phone calls on a laptop, you capture locally.

A local notetaker records two inputs at once: the microphone (the user) and a loopback of the system output (the other people, as the meeting app plays them). Keep them as two channels for as long as you can. The split is free speaker separation, because everything on the mic channel is the user. It also matters for echo: Microsoft's docs say WASAPI loopback exists primarily to support acoustic echo cancellation. If the user is on laptop speakers, the mic hears the remote voices too, so run echo cancellation on the mic (in a browser, the echoCancellation constraint of getUserMedia) or every remote sentence lands in the transcript twice.

• ScreenCaptureKit streams screen content and audio. Set capturesAudio to true on SCStreamConfiguration (macOS 13 and later); captureMicrophone adds the mic to the same stream on macOS 15 and later. It needs screen recording permission and an NSScreenCaptureUsageDescription key in Info.plist.
• Core Audio taps (macOS 14.2 and later) capture outgoing audio from one process, a group of processes or the whole system. Describe the tap with CATapDescription, create it with AudioHardwareCreateProcessTap, add it to an aggregate device created with AudioHardwareCreateAggregateDevice, then read that device like a microphone.
• Taps need an NSAudioCaptureUsageDescription key. The first time you record from an aggregate device that contains a tap, macOS asks the user for system audio recording permission. Users manage it under System Settings, Privacy & Security, Screen & System Audio Recording, where an app can be allowed to record audio only.
• The microphone is a separate permission with its own usage string. Design onboarding for both prompts, and for the user who clicked Deny once and now hears nothing.

• Open the render endpoint (eRender instead of eCapture in GetDefaultAudioEndpoint) and initialize a shared-mode capture stream with AUDCLNT_STREAMFLAGS_LOOPBACK. Exclusive-mode streams cannot run in loopback mode.
• The loopback carries the whole mix played on that endpoint, not just the meeting app, so notification sounds and music end up in the recording too.
• Event-driven loopback works on Windows 10 version 1703 and later; older builds need a render-stream workaround that Microsoft documents.
• Process loopback (AUDIOCLIENT_ACTIVATION_TYPE_PROCESS_LOOPBACK through ActivateAudioInterfaceAsync) includes or excludes the audio of one process and its children, for example to keep your own app's sounds out. Its minimum client is Windows 10 build 20348, so feature-detect it and fall back to endpoint loopback.
• A trusted audio driver does not let loopback capture DRM-protected streams. Microsoft's loopback docs describe no user prompt like the one on macOS, which makes your own recording indicator and consent flow more important, not less.

• In PulseAudio, each output device (sink) has a monitor source that carries what the sink plays. Record the monitor of the default sink for the call audio, and the default source for the mic.
• Most current distros run PipeWire with pipewire-pulse, which PipeWire's docs call a drop-in replacement for the PulseAudio daemon, so code written against the PulseAudio API keeps working.
• Natively in PipeWire, a capture stream reads a sink's monitor ports when you set stream.capture.sink = true and point target.object at the sink's name, as the example config of the loopback module shows.
• Two problems you will meet in the field: a host with bare ALSA and no sound server has no monitor at all, and a muted output can hand you a monitor full of silence. Meter the level and warn the user instead of transcribing nothing.

Send audio to a streaming speech-to-text API over a WebSocket while the meeting runs, so the transcript is ready when it ends. Batch APIs are simpler and fine if nobody needs live text. These choices matter more than which vendor you pick:

• Two channels in, two transcripts out. Transcribe the mic and the system audio separately (or as one multichannel stream, if your API supports it) and merge them by timestamp. It is the cheapest speaker separation you will ever get.
• Speech-to-text APIs that tell speakers apart give numbers, not names. Deepgram's docs, for example, return a numeric speaker per word (speaker 0, speaker 1) and leave mapping them to people to your app; in streaming mode there is no per-word speaker confidence. Names need a second signal: the mic channel is the user, the calendar invite lists the attendees, and the meeting app's interface knows who is talking.
• Telling remote voices apart happens on the system channel, which is one mixed signal of every remote voice after the meeting codec. Expect merged speakers when two people sound alike or talk over each other.
• Languages: check the language list and whether auto-detection works mid-meeting. A wrong language hint breaks a transcript more than a slightly weaker model does.
• Reconnects: networks drop. Buffer audio locally and resend it on reconnect, or you lose the minute where the decision was made.

• Send the full transcript with timestamps and speaker labels, not a trimmed excerpt. Current models have large context windows; split only when a meeting is longer than the window.
• Ask for structure people can check: decisions, action items with an owner and a date, open questions. Each item should be traceable to a timestamp.
• Keep the transcript and the audio next to the summary. An LLM summary can state something nobody said; the fix is making the check fast, not trusting the model.
• Generate the summary after the meeting ends, and mark it stale when someone edits the transcript.

• Audio and transcripts are personal data. Encrypt them at rest, make notes private by default, and decide retention before launch, including how a user deletes a recording.
• If you have users in the EU, know where audio and transcripts are stored and which subprocessors (speech-to-text, LLM) see them, and say so in your privacy policy.
• No bot means nobody is announced. Show a clear recording indicator, remind the user to tell participants, and offer a one-line disclosure they can paste into the meeting chat. Several US states require all-party consent and many countries have their own rules. This is not legal advice.
• Do not describe local capture as local processing. If audio goes to a cloud speech-to-text API, say so.

1. Detect a meeting (a calendar event starting, or a meeting app using the mic), or let the user press record.
2. Ask for mic and system audio permission the first time, and explain why in one sentence.
3. Start two captures: mic and system loopback. Meter both, and warn when either stays silent for more than a few seconds.
4. Convert the audio to the format your speech-to-text provider expects (check its docs for sample rate and encoding) and stream it, buffering locally for reconnects.
5. Show the live transcript: merge the two channels by timestamp and label the mic channel as the user.
6. On stop, flush the buffers, close the stream, and store the audio with the final transcript.
7. Send the whole transcript to the LLM for the summary and action items, and store them with pointers back to timestamps.
8. Expose the result: a UI, an API, webhooks, or an MCP server so AI assistants can read it.

• Permission UX on macOS, and users who said no once.
• Device changes mid-call: headphones plugged in, Bluetooth switching to a headset profile, the default output changing.
• Code signing and notarization for macOS and Windows installers, and auto-updates.
• Sleep, a closed lid and two-hour meetings, plus disk and memory use over a long call.
• Echo on laptop speakers, and silence when the output is muted.

If you need meeting notes rather than a notetaker project, Reline (reline.so) already ships this architecture. Its desktop app for macOS, Windows and Linux (beta) records the microphone and system audio locally, with no bot in the call, and the web app captures call audio in Chrome or Edge on desktop through screen-share audio. Transcription (60+ languages), AI summaries and storage run in the cloud, so it is bot-free, not on-device.

For developers who want the data rather than the app: Reline runs an MCP server at https://mcp.reline.so/mcp that lets Claude, ChatGPT, Cursor and other clients search notes, read transcripts and action items, and create notes, signing in with OAuth or a workspace API key. Signed webhooks fire on six events, such as note.created, note.summarized and action_items.extracted, with an HMAC-SHA256 signature over the timestamp and the raw body.

• Apple, Capturing system audio with Core Audio taps: https://developer.apple.com/documentation/coreaudio/capturing-system-audio-with-core-audio-taps
• Apple, ScreenCaptureKit: https://developer.apple.com/documentation/screencapturekit
• Apple Support, Control access to screen and system audio recording on Mac: https://support.apple.com/guide/mac-help/mchld6aa7d23/mac
• Microsoft, Loopback Recording (WASAPI): https://learn.microsoft.com/windows/win32/coreaudio/loopback-recording
• Microsoft, AUDIOCLIENT_ACTIVATION_TYPE: https://learn.microsoft.com/windows/win32/api/audioclientactivationparams/ne-audioclientactivationparams-audioclient_activation_type
• Microsoft, Build application-hosted media bots (Teams): https://learn.microsoft.com/microsoftteams/platform/bots/calls-and-meetings/requirements-considerations-application-hosted-media-bots
• MDN, MediaDevices.getDisplayMedia(): https://developer.mozilla.org/docs/Web/API/MediaDevices/getDisplayMedia
• PulseAudio, pa_sink_info (monitor_source): https://freedesktop.org/software/pulseaudio/doxygen/structpa__sink__info.html
• PipeWire, pipewire-pulse: https://docs.pipewire.org/page_man_pipewire-pulse_1.html
• PipeWire, Loopback module: https://docs.pipewire.org/page_module_loopback.html
• Zoom, Meeting SDK: https://developers.zoom.us/docs/meeting-sdk/
• Zoom, Realtime Media Streams (RTMS): https://developers.zoom.us/docs/rtms/
• Google, Meet Media API overview: https://developers.google.com/workspace/meet/media-api/guides/overview
• Deepgram documentation (speaker labels per word): https://developers.deepgram.com/docs