MM-LLMs

Paper Name: MM-LLMs: Recent Advances in MultiModal Large Language Models

Summary

3+ Most Important Things

1+ Deficiencies

3+ New Ideas

  1. Use my current work on ARL and develop a dataset for Supervised fine tuning that can be used on the stage 2 of RLHF

Important Figures

Components of usual MM-LLMs

Advancement in 2023

Mainstream MM-LLMs

Multimodal Pre-Training Dataset

Multimodal SFT or IT dataset

Multimodal Benchmark