Skip to content

Failed to work with https://docs.zeroclaw.com/v0.8.5/en/ #117

Description

@luisimasd

Basically it starts relatively ok grabbing pages from https://docs.zeroclaw.com/v0.8.5/en/ but it changes to https://docs.zeroclaw.com/v0.8.5/... building the rest of the links based on that last url, leading to multiple 404 errors.

Noy familiar with python but playing with an IA it mentioned....

Using... docker exec oikb sh -c 'sed -n "1,240p" /app/src/oikb/connectors/web.py'
It mentioned that the problem is around "self.url = url.rstrip("/")" and it should be "self.url = url if url.endswith("/") else url + "/""

Since I am not familiar with python I am not sure if that is correct or not, I leave that to the talented people here who can find out what is going and eventually, if a fix is required.

Environment is:

# docker exec oikb oikb --version
oikb, version 0.4.0

Config file (as of now):

defaults:
  interval: 8h
  concurrency: 2
  filter:
    max-size: 50mb
#  notify:
#    url: https://hooks.slack.com/services/T.../B.../xxx
#    on: error

sources:
  - name: zeroclaw-docs
    source: web:https://docs.zeroclaw.com/v0.8.5/en
    kb-id: 3a7b0abe-3924-4d13-915b-4d3b6b80563a

Example of the issue observed:

root@auto3:/opt/appdata# docker compose up  --force-recreate oikb
[+] Running 1/1
 ✔ Container oikb  Recreated                                                                                                    0.9s
Attaching to oikb
oikb  | oikb daemon listening on port 8080
oikb  |   1 source(s) configured
oikb  |   Auth: disabled (set OIKB_API_KEY to enable)
oikb  |   GET  http://localhost:8080/health
oikb  |   GET  http://localhost:8080/metrics
oikb  |   GET  http://localhost:8080/history
oikb  |   POST http://localhost:8080/sync/{kb_id}
oikb  |
oikb  |   OpenAPI spec: http://localhost:8080/openapi.json
oikb  |   Add as Tool Server in Open WebUI → Settings → Connections
oikb  | 2026-09-07 18:18:15,360 INFO     oikb.daemon  Scheduling web:https://docs.zeroclaw.com/v0.8.5/en every 28800s
oikb  | 2026-09-07 18:18:16,771 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/sitemap.xml "HTTP/1.1 404 Not Found"
oikb  | 2026-09-07 18:18:16,863 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/en "HTTP/1.1 301 Moved Permanently                                   "
oikb  | 2026-09-07 18:18:16,956 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/en/ "HTTP/1.1 200 OK"
oikb  | 2026-09-07 18:18:17,544 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/favicon.svg "HTTP/1.1 200 OK"
oikb  | 2026-09-07 18:18:18,139 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/favicon.png "HTTP/1.1 200 OK"
**oikb  | 2026-09-07 18:18:18,646 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/css/variables-8adf115d.css "HTTP/1                                   .1 404 Not Found"
oikb  | 2026-09-07 18:18:18,651 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/css/general-e96d0476.css "HTTP/1.1                                    404 Not Found"
oikb  | 2026-09-07 18:18:18,739 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/css/chrome.css "HTTP/1.1 200 OK"
oikb  | 2026-09-07 18:18:19,244 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/fonts/fonts-9644e21d.css "HTTP/1.1                                    404 Not Found"
oikb  | 2026-09-07 18:18:19,249 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/highlight-493f70e1.css "HTTP/1.1 4                                   04 Not Found"
oikb  | 2026-09-07 18:18:19,253 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/tomorrow-night-4c0ae647.css "HTTP/                                   1.1 404 Not Found"
oikb  | 2026-09-07 18:18:19,256 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/ayu-highlight-3fdfc3ac.css "HTTP/1                                   .1 404 Not Found"**
oikb  | 2026-09-07 18:18:19,358 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/theme/pc-themes.css "HTTP/1.1 200                                    OK"
oikb  | 2026-09-07 18:18:19,950 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/_shared/theme/custom.css "HTTP/1.1 200 OK                                   "
**oikb  | 2026-09-07 18:18:20,456 INFO     httpx  HTTP Request: GET https://docs.zeroclaw.com/v0.8.5/philosophy/index.html "HTTP/1.1 40                                   4 Not Found"**

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions