Streaming not working

[mynona]

I set up a docker compose

kokoro:
    container_name: kokoro-tts
    image: ghcr.io/remsky/kokoro-fastapi-cpu:latest
    ports:
      - 8880:8880
    networks:
      - tts
    restart: unless-stopped

kokoro runs behind nginx:

server {
	http2 on;
    listen 443;    
    server_name tts.yourdomain.com;
   
  	add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Content-Type-Options "nosniff";
    add_header X-XSS-Protection "1; mode=block"; 
    
    proxy_next_upstream error timeout http_502 http_503 http_504;
    
    client_body_buffer_size 10M;
	client_max_body_size 10M;
	proxy_buffers 16 4k;
	proxy_buffer_size 8k;
	proxy_read_timeout 60s;
	proxy_connect_timeout 60s;
	
	try_files $uri $uri/ =404;

    gzip on;
    gzip_disable "msie6";
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_proxied expired no-cache no-store private auth;
    gzip_types 
      text/plain 
      text/css 
      application/json 
      application/x-javascript 
      application/javascript
      text/xml 
      application/xml 
      application/xml+rss 
      text/javascript
      audio/mpeg 
      audio/wav 
      audio/L16;
    gzip_min_length 1000;
    
    location / {
    	# mime types
    	types {
        	audio/mpeg mp3;
        	audio/wav wav;
        	audio/L16 pcm;
    	}
    	# allow streaming
    	add_header Accept-Ranges bytes;
    	
    	expires max;
    	add_header Cache-Control "public";
    	
    	access_log off;
    
        proxy_pass http://kokoro-tts:8880/;
        
    }
    
}

As I have a smaller digital ocean droplet it works in the web interface but when I hit the api I cannot manage to download files.

In the web interface the download of mp3 is not working but wav works. I also see a javascript error in the console:

TypeError: null is not an object (evaluating 'this.audio.error') AudioService.js:85

Can you help me and check if the nginx configuration is correctly set up for kokoro or if I missed something?
I am also wondering where the created audio files are stored and if I should link from the host system in docker compose (volumes)

[mynona]

debug storage

{
  "storage_info": [
    {
      "device": "/dev/vda1",
      "mountpoint": "/etc/resolv.conf",
      "fstype": "ext4",
      "total_gb": 48.27519989013672,
      "used_gb": 12.058238983154297,
      "free_gb": 36.20133590698242,
      "percent_used": 25
    },
    {
      "device": "/dev/vda1",
      "mountpoint": "/etc/hostname",
      "fstype": "ext4",
      "total_gb": 48.27519989013672,
      "used_gb": 12.058238983154297,
      "free_gb": 36.20133590698242,
      "percent_used": 25
    },
    {
      "device": "/dev/vda1",
      "mountpoint": "/etc/hosts",
      "fstype": "ext4",
      "total_gb": 48.27519989013672,
      "used_gb": 12.058238983154297,
      "free_gb": 36.20133590698242,
      "percent_used": 25
    }
  ]
}

[mynona]

debug system

{
  "cpu": {
    "cpu_count": 1,
    "cpu_percent": 28.7,
    "per_cpu_percent": [
      30
    ],
    "load_avg": [
      0.7568359375,
      1.0400390625,
      0.95166015625
    ]
  },
  "memory": {
    "virtual": {
      "total_gb": 1.9219779968261719,
      "available_gb": 0.3273124694824219,
      "used_gb": 1.4171104431152344,
      "percent": 83
    },
    "swap": {
      "total_gb": 0.9609870910644531,
      "used_gb": 0.568115234375,
      "free_gb": 0.3928718566894531,
      "percent": 59.1
    }
  },
  "process": {
    "pid": 11,
    "status": "running",
    "create_time": "2025-03-09T19:12:38.740000",
    "cpu_percent": 0,
    "memory_percent": 53.055663064296844
  },
  "network": {
    "connections": 2,
    "network_io": {
      "bytes_sent": 793884,
      "bytes_recv": 83760,
      "packets_sent": 425,
      "packets_recv": 517,
      "errin": 0,
      "errout": 0,
      "dropin": 0,
      "dropout": 0
    }
  },
  "gpu": null
}

[mynona]

debug session pools

Internal Server Error

[mynona]

When I call it via api

07:34:17 PM | INFO     | text_processor:265 | Split completed in 6.33ms, produced 1 chunks
07:34:17 PM | ERROR    | tts_service:355 | Error in audio generation: list index out of range
07:34:17 PM | ERROR    | openai_compatible:342 | Unexpected error in speech generation: list index out of range
INFO:     172.20.0.3:50620 - "POST /v1/audio/speech HTTP/1.1" 500 Internal Server Error
07:34:24 PM | DEBUG    | paths:153 | Scanning for voices in path: /app/api/src/voices/v1_0
07:34:24 PM | DEBUG    | paths:131 | Searching for voice in path: /app/api/src/voices/v1_0
07:34:24 PM | DEBUG    | tts_service:235 | Using single voice path: /app/api/src/voices/v1_0/af_heart.pt
07:34:24 PM | DEBUG    | tts_service:261 | Using voice path: /app/api/src/voices/v1_0/af_heart.pt
07:34:24 PM | INFO     | tts_service:265 | Using lang_code 'en' for voice 'af_heart' in audio stream
07:34:24 PM | INFO     | text_processor:131 | Starting smart split for 6 chars
07:34:24 PM | INFO     | text_processor:141 | Skipping text normalization as it is only supported for english
07:34:24 PM | DEBUG    | text_processor:54 | Total processing took 0.22ms for chunk: 'string'
07:34:24 PM | INFO     | text_processor:259 | Yielding final chunk 1: 'string' (6 tokens)
07:34:24 PM | INFO     | kokoro_v1:73 | Creating new pipeline for language code: en
07:34:24 PM | ERROR    | kokoro_v1:305 | Generation failed: ('en', {'a': 'American English', 'b': 'British English', 'e': 'es', 'f': 'fr-fr', 'h': 'hi', 'i': 'it', 'p': 'pt-br', 'j': 'Japanese', 'z': 'Mandarin Chinese'})
07:34:24 PM | ERROR    | tts_service:164 | Failed to process tokens: Generation failed: ('en', {'a': 'American English', 'b': 'British English', 'e': 'es', 'f': 'fr-fr', 'h': 'hi', 'i': 'it', 'p': 'pt-br', 'j': 'Japanese', 'z': 'Mandarin Chinese'})
07:34:24 PM | INFO     | text_processor:265 | Split completed in 4.32ms, produced 1 chunks
07:34:24 PM | ERROR    | tts_service:355 | Error in audio generation: list index out of range
07:34:24 PM | ERROR    | openai_compatible:342 | Unexpected error in speech generation: list index out of range
INFO:     172.20.0.3:50632 - "POST /v1/audio/speech HTTP/1.1" 500 Internal Server Error

called via postman

{
  "model": "kokoro",
  "input": "hello world",
  "voice": "af_heart",
  "response_format": "mp3",
  "download_format": "mp3",
  "speed": 1,
  "stream": false,
  "return_download_link": false,
  "lang_code": "en",
  "normalization_options": {
    "normalize": true,
    "unit_normalization": false,
    "url_normalization": true,
    "email_normalization": true,
    "optional_pluralization_normalization": true,
    "phone_normalization": true
  }
}

Model warmed up on cpu: kokoro_v1CUDA: False
67 voice packs loaded

Beta Web Player: http://0.0.0.0:8880/web/
or http://localhost:8880/web/
░░░░░░░░░░░░░░░░░░░░░░░░

INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8880 (Press CTRL+C to quit)
07:41:10 PM | INFO     | openai_compatible:70 | Created global TTSService instance
07:41:10 PM | DEBUG    | paths:153 | Scanning for voices in path: /app/api/src/voices/v1_0
07:41:10 PM | DEBUG    | paths:131 | Searching for voice in path: /app/api/src/voices/v1_0
07:41:10 PM | DEBUG    | tts_service:235 | Using single voice path: /app/api/src/voices/v1_0/af_heart.pt
07:41:10 PM | DEBUG    | tts_service:261 | Using voice path: /app/api/src/voices/v1_0/af_heart.pt
07:41:10 PM | INFO     | tts_service:265 | Using lang_code 'en' for voice 'af_heart' in audio stream
07:41:10 PM | INFO     | text_processor:131 | Starting smart split for 11 chars
07:41:10 PM | INFO     | text_processor:141 | Skipping text normalization as it is only supported for english
07:41:10 PM | DEBUG    | text_processor:54 | Total processing took 175.66ms for chunk: 'hello world'
07:41:10 PM | INFO     | text_processor:259 | Yielding final chunk 1: 'hello world' (13 tokens)
07:41:10 PM | INFO     | kokoro_v1:73 | Creating new pipeline for language code: en
07:41:10 PM | ERROR    | kokoro_v1:305 | Generation failed: ('en', {'a': 'American English', 'b': 'British English', 'e': 'es', 'f': 'fr-fr', 'h': 'hi', 'i': 'it', 'p': 'pt-br', 'j': 'Japanese', 'z': 'Mandarin Chinese'})
07:41:10 PM | ERROR    | tts_service:164 | Failed to process tokens: Generation failed: ('en', {'a': 'American English', 'b': 'British English', 'e': 'es', 'f': 'fr-fr', 'h': 'hi', 'i': 'it', 'p': 'pt-br', 'j': 'Japanese', 'z': 'Mandarin Chinese'})
07:41:10 PM | INFO     | text_processor:265 | Split completed in 219.83ms, produced 1 chunks
07:41:10 PM | ERROR    | tts_service:355 | Error in audio generation: list index out of range
07:41:10 PM | ERROR    | openai_compatible:342 | Unexpected error in speech generation: list index out of range
INFO:     172.20.0.3:40722 - "POST /v1/audio/speech?model=kokoro&input=hello%20world&voice=af-heart&response_format=mp3&download_format=mp3&speed=1&stream=false&return_download_link=true&lang_code=en HTTP/1.1" 500 Internal Server Error

[michael-chipmates]

Short version: drop lang_code from your request body (or set it to "a"). Passing "lang_code": "en" is what triggers the 500. nginx is not the cause here.

Your log shows the real failure:

kokoro_v1 | Generation failed: ('en', {'a': 'American English', 'b': 'British English', 'e': 'es', 'f': 'fr-fr', 'h': 'hi', 'i': 'it', 'p': 'pt-br', 'j': 'Japanese', 'z': 'Mandarin Chinese'})
... "POST /v1/audio/speech HTTP/1.1" 500 Internal Server Error

lang_code has to be a single letter, one of a b e f h i p j z. There is no "en". Under the hood the kokoro package does this when it builds a pipeline:

lang_code = ALIASES.get(lang_code, lang_code)
assert lang_code in LANG_CODES, (lang_code, LANG_CODES)

ALIASES knows en-us -> a and en-gb -> b, but plain en is in neither the alias table nor the code table, so that assert fails. That tuple in your log (('en', {...})) is the failed assert. Kokoro-FastAPI catches it and you end up with the generic "list index out of range" / 500.

The field is optional. When you leave it out, the server uses the first letter of the voice name, so af_heart becomes a (American English). That is why the web player works and your API call does not. The web player only sends lang_code when you pick something in its language dropdown, otherwise it sends nothing.

Fix, just remove the line:

{
  "model": "kokoro",
  "input": "hello world",
  "voice": "af_heart",
  "response_format": "mp3",
  "speed": 1,
  "stream": false
}

Or if you want to be explicit, "lang_code": "a". Both work. (You're also tacking the params on as a query string. That part is ignored, the server reads the JSON body, so the body is the only thing that matters here.)

On the other two questions:

Storage: for normal API use you do not need a host volume. The audio comes back in the HTTP response, and response_format controls mp3/wav/etc. return_download_link: true makes the server expose a path via the X-Download-Path response header, but you do not need that just to receive the bytes. So no volumes: entry is required for this.

nginx: it is not involved in the 500. That error happens inside the app before any of your proxy settings matter. Two notes if you do streaming later though. You are gzipping audio/mpeg audio/wav audio/L16. Gzipping mp3 is wasted work and gzipping any of them can break range requests, so drop those from gzip_types. And expires max plus try_files belong on a static file server, not in front of a live TTS endpoint, so I would not apply them to the /v1/audio/speech route. You do not need to add proxy_buffering off by hand either, the app already sends X-Accel-Buffering: no on its streaming responses and nginx honors that.

The AudioService.js TypeError you see in the browser is a separate web-player thing, unrelated to the API 500. Get the API call working first with the change above.