Runtime error diagnosed: startup downloads ~16 models at import on a 10-second timeout, and one timeout kills the Space

#70
by nootxlm - opened

This Space is in RUNTIME_ERROR. The recorded error points at a specific startup weakness, and the fix is small. Sharing the diagnosis and a patch.

Symptom

The runtime log ends in a network failure, not a model error:

ReadTimeout: HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)

raised through the requests library while the app is starting.

Mechanism

Startup loads the local models before the app exists:

  1. app.py imports awesome_chat, which imports models_server at module level.
  2. models_server.py runs pipes = load_pipes(local_deployment) at module level. With config.gradio.yaml set to inference_mode: hybrid and local_deployment: standard, that eagerly builds 8 standard pipelines plus the ControlNet set (the sd-controlnet models and the HED, Openpose, MLSD and Midas detectors) โ€” roughly 16 from_pretrained downloads, in series, before Gradio launches.
  3. Every download goes to huggingface.co through huggingface_hub. requirements.txt pins huggingface_hub==0.25.2, and in that version the download and etag timeouts both default to 10 seconds (HF_HUB_DOWNLOAD_TIMEOUT, HF_HUB_ETAG_TIMEOUT).
  4. There is no retry, no per-model isolation, and no lazy loading on this path. One download that takes longer than 10 seconds to respond aborts the import, demo.launch() is never reached, and the Space records RUNTIME_ERROR. The recorded error is exactly that event: host huggingface.co, read timeout=10.

Evidence

  • The error text above, from the Space's current runtime record (host and timeout value are the load-bearing parts).
  • huggingface_hub 0.25.2 wheel, huggingface_hub/constants.py: DEFAULT_DOWNLOAD_TIMEOUT = 10, DEFAULT_ETAG_TIMEOUT = 10.
  • The call graph is in the repo as quoted: app.py โ†’ awesome_chat.py (from models_server import models, status) โ†’ models_server.py module-level load_pipes call, with the standard and ControlNet from_pretrained lists in that file.

One caution about PR #68, which replaces the git pins for diffusers, transformers and controlnet_aux with unpinned PyPI releases: that does not touch this timeout path, and current releases of those libraries need a much newer huggingface_hub than 0.25.2. That matters here, because this code imports symbols the newer Hub removes โ€” verified in the published wheels: cached_download (imported in models_server.py, used only in a commented-out line) is gone in huggingface_hub 0.26.0, and InferenceApi and Repository (imported and used in awesome_chat.py) are gone in 1.0.0. Upgrading the Hub first would trade a timeout for an import error.

Fix

Three small changes, all in models_server.py:

  1. Before any Hub import, raise the timeouts:
import os
os.environ.setdefault("HF_HUB_DOWNLOAD_TIMEOUT", "120")
os.environ.setdefault("HF_HUB_ETAG_TIMEOUT", "60")
  1. Isolate the eager load, so a failed model download degrades the Space instead of killing it. Model selection already checks status() per model and falls back to hosted inference for models that are not loaded:
try:
    pipes = load_pipes(local_deployment)
except Exception:
    traceback.print_exc()
    print('[ startup ] local model loading failed; continuing with no local pipes')
    pipes = {}
  1. Drop the dead import:
from huggingface_hub import hf_hub_url

A ready-to-apply patch with exactly these changes accompanies this post.

Prove-wrong check

Restart the Space with only change 1 applied and watch the startup log: if the failure was a single slow download, startup now gets past the model that timed out; if it still dies at the same 10-second mark, the timeout is being set somewhere else and this diagnosis is wrong. Equally, run the import line from models_server.py against huggingface_hub 0.26.0 in a clean environment: if from huggingface_hub import cached_download succeeds there, the secondary warning above is wrong.

Sign up or log in to comment