Arcadium Data Scrapers
Home / Tutorials / YouTube transcripts for RAG (LangChain)

YouTube transcripts for RAG with LangChain

A channel’s videos hold answers your users ask about, but a vector store needs text. Downloading captions by hand does not scale, and YouTube often asks scripts from cloud servers to confirm they are not bots.

arcadium/youtube-transcripts returns the caption text of each video with its metadata and, optionally, every caption line with its start time. The code below turns that into LangChain documents whose metadata links back to the exact second in the video.

Cost of this example on the Apify Free plan: 20 × $0.00248 = $0.0496 ($0.036 on the Apify Gold plan and higher), plus $0.001 per GB of memory when the run starts.

Open YouTube Transcript & Subtitles Scraper on the Apify StoreActor overview

What you need

Step 1: Run it in the Apify Store (no code)

  1. Open arcadium/youtube-transcripts on the Apify Store and sign in (a free Apify account works).
  2. Paste a channel, playlist or video link into YouTube URLs or video IDs. A channel link gives its newest videos first.
  3. Set Max videos per channel or playlist (20 in this example) and Preferred caption languages to en.
  4. Keep Include timed segments on: segments carry the start time of every caption line.
  5. Optional: set a maximum cost per run in the run options as a hard cap.
  6. Click Start. Rows appear in the Output tab; export them as JSON, CSV or Excel.

The same input as JSON (paste it in the JSON tab of the input form):

{
  "urls": [
    "https://www.youtube.com/@kurzgesagt"
  ],
  "maxVideosPerSource": 20,
  "languages": [
    "en"
  ],
  "includeSegments": true
}

Step 2: Call it over the Apify API

One HTTP call runs the actor and returns its rows. It suits runs up to about five minutes; for longer runs start the run asynchronously and read the dataset when it finishes.

curl -X POST "https://api.apify.com/v2/acts/arcadium~youtube-transcripts/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://www.youtube.com/@kurzgesagt"],"maxVideosPerSource":20,"languages":["en"],"includeSegments":true}'

Step 3: Run it from Python

With apify-client 3.x, call() returns a Run object: read the dataset id as run.default_dataset_id (older examples use run["defaultDatasetId"], which no longer works).

# pip install "apify-client>=3" langchain-core langchain-text-splitters langchain-openai
import os

from apify_client import ApifyClient
from langchain_core.documents import Document
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_openai import OpenAIEmbeddings  # any LangChain embeddings class works

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("arcadium/youtube-transcripts").call(
    run_input={
        "urls": ["https://www.youtube.com/@kurzgesagt"],
        "maxVideosPerSource": 20,
        "languages": ["en"],
        "includeSegments": True,
    }
)

WORDS_PER_CHUNK = 250
docs = []
# apify-client 3.x: the run is an object, the dataset id is run.default_dataset_id
for row in client.dataset(run.default_dataset_id).iterate_items():
    if row.get("rowType") != "transcript":
        continue  # free rows: no captions, private or missing videos
    chunk, start = [], None
    for seg in row.get("segments") or []:
        start = seg["start"] if start is None else start
        chunk.append(seg["text"])
        if len(" ".join(chunk).split()) >= WORDS_PER_CHUNK:
            docs.append(Document(page_content=" ".join(chunk), metadata={
                "title": row["title"], "videoId": row["videoId"], "publishDate": row.get("publishDate"),
                "source": f"https://www.youtube.com/watch?v={row['videoId']}&t={int(start)}s",
            }))
            chunk, start = [], None
    if chunk:
        docs.append(Document(page_content=" ".join(chunk), metadata={
            "title": row["title"], "videoId": row["videoId"], "publishDate": row.get("publishDate"),
            "source": f"https://www.youtube.com/watch?v={row['videoId']}&t={int(start or 0)}s",
        }))

store = InMemoryVectorStore.from_documents(docs, OpenAIEmbeddings())
for hit in store.similarity_search("how big is the observable universe?", k=3):
    print(hit.metadata["source"], hit.page_content[:120])

Step 4: Run it from JavaScript

// npm install apify-client
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('arcadium/youtube-transcripts').call({
  "urls": [
    "https://www.youtube.com/@kurzgesagt"
  ],
  "maxVideosPerSource": 20,
  "languages": [
    "en"
  ],
  "includeSegments": true
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Step 5: Ask an AI assistant (MCP)

Apify’s hosted MCP server turns the actor into one tool. Add it to Claude, Cursor, VS Code or any MCP client:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=arcadium/youtube-transcripts",
      "headers": {
        "Authorization": "Bearer <APIFY_TOKEN>"
      }
    }
  }
}

Then ask, for example:

Use arcadium/youtube-transcripts to get the English transcript of https://www.youtube.com/watch?v=dQw4w9WgXcQ and summarise it in five bullet points with timestamps.

More clients and the local setup on the MCP page.

Use it as a LangChain tool instead

When an agent should decide which videos to read, give it the actor as a tool. langchain-apify reads APIFY_API_TOKEN from the environment.

# pip install langchain-apify
from langchain_apify import ApifyActorsTool

tool = ApifyActorsTool("arcadium/youtube-transcripts")
rows = tool.invoke({"run_input": {"urls": ["https://www.youtube.com/watch?v=dQw4w9WgXcQ"], "includeSegments": False}})

Keep the index fresh

Run the same input on a schedule with Published on or after (publishedAfter, a YYYY-MM-DD date) set to your last run. On a channel the listing stops at the first older video, and skipped videos are free.

Output fields

Every row has a rowType; filter on it first. Failed inputs come back as free rows with a code and message. The fields this tutorial uses, from the actor’s dataset schema:

FieldWhat it holds
rowTypeKind of row. Filter on this first. Values: 'transcript': one video's transcript (charged). 'error': a video or URL gave no transcript; see code and message (free).
videoIdYouTube video id, 11 characters, e.g. dQw4w9WgXcQ.
urlWatch page link of the video (Shorts included).
titleVideo title.
channelNameName of the channel that published the video (the publisher, not a viewer).
publishDatePublish date and time as YouTube gives it, ISO 8601 with offset, e.g. 2009-10-24T23:57:33-07:00.
languageLanguage code of the returned transcript, e.g. en, de-DE, pt-BR.
isAutoGeneratedTrue when the transcript is YouTube's speech recognition (no punctuation, may misspell names); false when the creator uploaded it.
transcriptThe whole transcript as one text: caption lines joined with spaces. Emails and phone numbers are removed.
wordCountWords in transcript, for chunking and token estimates.
segmentsCaption lines in order: start (seconds from the start), duration (seconds), text. Present when includeSegments is on.

All fields: YouTube Transcript & Subtitles Scraper guide.

Cost per run

Pay per event: you pay for the rows delivered, at the actor’s live Apify price on 2026-10-31. Error, coverage and summary rows are free.

WhatRowsPrice per row (Free plan)Subtotal
Transcript: 20 videos with captions20$0.00248$0.0496

Cost of this example on the Apify Free plan: 20 × $0.00248 = $0.0496 ($0.036 on the Apify Gold plan and higher), plus $0.001 per GB of memory when the run starts. Higher Apify plans pay less per row. The Store page always has the current price.

Limitations

Links