About continuous_training

[Wintoplay]

With continuous_training on bunny VLM, do we still need to specify vision_tower path?

If we do point to siglip path, will it use that untrained weight instead of vision_tower that comes with bunny vlm I downloaded from huggingface?

What should I specify?

[Isaachhh]

Yes, it's still needed to specify --vision_tower to the path to huggingface/siglip-so400m-patch14-384.

But the vision tower of Bunny models would be used because "continuous_training": true in /path/to/merged_model/config.json.

https://github.com/BAAI-DCAI/Bunny?tab=readme-ov-file#continuous--fine-tuning

[Wintoplay]

do it work fine with this model too?
https://huggingface.co/scb10x/llama-3-typhoon-v1.5-8b-vision-preview
since it is trained by Bunny method

[Isaachhh]

I think so.

But also check #130.