obj_size <= remaining_buffer_size

[qzq-123]

Does anyone know what obj_size and remaining_buffer_size refer to? Where can I adjust them?

Container startup parameters
docker run --rm -it --net host --shm-size=20g \ --ulimit memlock=-1 --ulimit stack=67108864 --gpus all

I have an A5000 gpu, running the Qwen2.5-3B-Instruct model,
python3 ../run.py --input_text "Hello, what is your name?" --max_output_len=50 --tokenizer_dir ./tmp/Qwen/3B/ --engine_dir=./tmp/Qwen/3B/trt_engines/int4_weight_only/1-gpu/
I can get normal results.

But starting the backend service reported an error
python3 /tensorrtllm_backend/scripts/launch_triton_server.py --world_size=1 --model_repo=${MODEL_FOLDER}

[mutkach]

I've encountered the same problem with MLlama.