how to implement batch_chat for videochat2 benchmark?

[dragen1860]

Dear author:
Thnaks for the really solid work of videochat2.
Recently i finetunning a individual model on my custom dataset and want to test on my benchmark samples, which as up to 30K samples. It is extreme time-consuming for chat one by one and then compute the accuracy. Does anyone know how to implement a batch_chat like sytle parallel inference function, so that i could inference several samples meanwhile. thanks.

[Andy1621]

Hi! We will release the vllm verision for batch_chat recently.