top of page

vLLM con ray a mano

  • Foto del escritor: Alejandro Rivero
    Alejandro Rivero
  • hace 7 días
  • 1 Min. de lectura

#necesarioexport SSL_CERT_FILE=/fs/agustina/arivero/supercomplex/.local/lib/python3.11/site-packages/certifi/cacert.pem export RAY_NODE_MANAGER_HEARTBEAT_TIMEOUT_MILLISECONDS=20000 # 20 seconds export RAY_GCS_HEARTBEAT_TIMEOUT_SECONDS=60 1064 export RAY_DISABLE_PLACEMENT_GROUP_BUNDLING=1#opcionales por lios de debug, igual ni existen. 1116 export RAY_NODE_IP_ADDRESS=172.22.5.1 1134 export NCCL_DEBUG=INFO 1135 export NCCL_DEBUG_SUBSYS=ALL 1136 export NCCL_P2P_DISABLE=0 # or 1 if P2P is not supported 1137 export RAY_DEDUP_LOGS=0 1154 export NCCL_SOCKET_IFNAME=ib1

module load python-math/3.11.4ray start --head --node-ip-address=172.22.5.1 --num-gpus=4ray list placement-groups --detailvllm serve deepseek-ai/DeepSeek-R1 --distributed-executor-backend=ray --trust-remote-code --pipeline-parallel-size 2 --tensor-parallel-size 4

Entradas recientes

Ver todo
Mass Gap from Kaluza Klein

This is just a series of proposed blog posts from chatGPT, each in separate markdown format See also https://chatgpt.com/c/6953f699-3088-832d-8e4f-9104a9264251

 
 
 
Circular keypads and similar devices

Algunos links que he encontrado sobre el tema de teclados numericos, pensando en las aplicaciones de calculadora de las paginas anteriores https://www.doc.cc/articles/a-brief-history-of-the-numeric-ke

 
 
 

Comentarios


Never Miss a Post. Subscribe Now!

I'm a paragraph. Click here to add your own text and edit me. It's easy.

Thanks for submitting!

© 2035 by Kathy Schulders. Powered and secured by Wix

  • Grey Twitter Icon
bottom of page