
CS PhD student at UC San Diego @ucsd_cse Computer Vision + Medical Imaging
San Diego, USA
Joined September 2015
- Tweets338
- Following700
- Followers193
- Likes1.3K
Pinned Tweet
Sharing a life milestone... Graduated from Jadavpur University (Bachelor of Engineering, Information Technology) batch of 2019-2023!
These past 4 years have undoubtedly been the best time of my life... it has been an absolute pleasure to be a part of the JU family!
#graJUated
1
12
681
Soumitri Chattopadhyay retweeted
I'm an AC for NeurIPS (first time). I've pinged reviewers 4 times to respond to reviews. Less than 40% of the reviewers have. Why serve, when one can't follow terms of service?
Just now, a reviewer responds with a request for more experiments. On a Sunday! WTH!!
Soumitri Chattopadhyay retweeted
BMVC 2024 proceedings are now online! bmvc2024.org/
Soumitri Chattopadhyay retweeted
Appalled at @qatarairways Baggage services. Flew from Kolkata to New York baggages didnt arrive. Flew back from Dallas to Kolkata, Baggage did not arrive. We had concerts and intruments did not come. Disgusting and very very dissapointing. #AnjanDutt #AmytDatta
Soumitri Chattopadhyay retweeted
The culture of “sports” does not require a world champion. It requires awareness and enthusiasm.
Growing up in 80s/90s in Kolkata, there were specialized chess clubs like Alekhine Chess Club, chess tournaments at school level, and this wasn’t just confined to Kolkata, but to mofussil towns.
This wasn’t a 80s phenomenon either.
Chess was played through generations by kings and nobles, and yes that’s what “sporting culture” means.
As a point of reference, India had cricketing culture for decades before it won the World Cup in 83.
Soumitri Chattopadhyay retweeted
#Hindus were silent when #Halal prasadam offered in HRCE controlled temple in TN, reports came that Sabarimala prasadam had halal jaggery. Now we face the most disgusting case of beef tallow, lard and veg oil found in #TirumalaLaddu.
Only solution - #ScrapHRCE. Free our faith😠
Soumitri Chattopadhyay retweeted
Unfortunately won't be attending #CVPR24 but please check out our papers
1. MeLFusion: schowdhury671.github.io/melf…
2. VistaLLM: shramanpramanick.github.io/V…
Both of these are Poster Highlights!
#CVPR24 #GenerativeAI
Soumitri Chattopadhyay retweeted
6th January 2008, India loose to Australia in a very controversial test match at Sydney . Harbhajan Singh was almost banned from 2 tests (due to Monkey Gate) and India were on the verge of pulling out of the series. Amidst of all these things India were scheduled to play Australia in the third test in Perth, this was India’s chance for redemption not only terms of levelling the series but in terms getting back the pride. (Thread)
Soumitri Chattopadhyay retweeted
এটা চলচ্চিত্রের গল্প, এটা মৃণালের গল্প।
Thank you অঞ্জন দত্ত & @neelinc for trusting us with this gem.
Presenting the official poster of #ChaalchitraEkhon, Anjan Dutt's tribute to #MrinalSen in his centenary year | Film in cinemas 10th May, only in selected theatres and simultaneously @hoichoitv @SVFsocial
Soumitri Chattopadhyay retweeted
Join the T4V 2024 Workshop for the latest research developments in Transformers, multimodal learning and related areas!
The 3rd Transformers for Vision workshop will be back at #CVPR2024! We have a great speaker lineup covering diverse Transformer topics! Papers Due: Apr 15.
Website: sites.google.com/view/t4v-cv…
Organized w/ @_rohitgirdhar_, @ZhidingYu, @giffmana, @gulvarol, @mohitban47 and others!
Soumitri Chattopadhyay retweeted
Come check out our workshop this year at CVPR 2024, and also submit any interesting papers you might care to share to a very large and interested audience! Deadline is 4/15, just enough time if you start soon. :-)
The 3rd Transformers for Vision workshop will be back at #CVPR2024! We have a great speaker lineup covering diverse Transformer topics! Papers Due: Apr 15.
Website: sites.google.com/view/t4v-cv…
Organized w/ @_rohitgirdhar_, @ZhidingYu, @giffmana, @gulvarol, @mohitban47 and others!
Soumitri Chattopadhyay retweeted
The 3rd Transformers for Vision workshop will be back at #CVPR2024! We have a great speaker lineup covering diverse Transformer topics! Papers Due: Apr 15.
Website: sites.google.com/view/t4v-cv…
Organized w/ @_rohitgirdhar_, @ZhidingYu, @giffmana, @gulvarol, @mohitban47 and others!
Soumitri Chattopadhyay retweeted
🔊Check out our preprint, AVSiam 🤝. We use a single shared ViT to process audio and visual inputs, improving its parameter efficiency⚡, reducing the GPU memory footprint💻, and allowing us to scale📈 to larger datasets and model sizes.
arxiv.org/abs/2403.19638
w. @gberta227
👇
``Siamese Vision Transformers are Scalable Audio-visual Learners,'' Yan-Bo Lin, Gedas Bertasius, ift.tt/feFJkwQ
Soumitri Chattopadhyay retweeted
🚨 #BMVC2024 Update 🚨
Attention researchers and innovators! We heard you and acted on it! The paper submission deadline has been extended to May 10th, 2024! The submission portal will open from April 1st!💡📚
#BMVC2024 #MachineLearning #ComputerVision #DeadlineExtended – at Glasgow, Scotland
Soumitri Chattopadhyay retweeted
🚨 Can a VidQA model train on continually arriving datasets effectively and efficiently adapt to time distribution shifts?
We propose DAM, which shows impressing adaptability to various VidQA domains (9.1% better Acc) while mitigating catastrophic forgetting (1.9% less forget).
Modern VidQA models are static, operating on fixed training datasets. In contrast, real-world applications demand adaptability to continually changing training domains.
We present a parameter-efficient method for continual VidQA learning.
arxiv.org/abs/2403.08755
🧵
Soumitri Chattopadhyay retweeted
Modern VidQA models are static, operating on fixed training datasets. In contrast, real-world applications demand adaptability to continually changing training domains.
We present a parameter-efficient method for continual VidQA learning.
arxiv.org/abs/2403.08755
🧵
Soumitri Chattopadhyay retweeted
ML research is following the exact same trend now, tweak fast and preprint just to beat others!! The number of preprints are skyrocketing while we still feel like we aren’t learning anything new while every question seems to be answered. Can we slow down on publications and try to build expertise?
(On the other hand, companies don’t even interview unless you trained a large-scale diffusion model or an LLM right now! Can’t blame the grad students🤷♀️)
#rant #phdlife #MachineLearning
"I wonder for people in their 20s if they shouldn't go to San Francisco.
The entrepreneurs are held in excessively high regard in my view.
I think that San Francisco doesn't really encourage the pursuit of really deep technical depth."
- @patrickc
Full episode out tomorrow
Soumitri Chattopadhyay retweeted
Video ReCap
Recursive Captioning of Hour-Long Videos
Most video captioning models are designed to process short video clips of few seconds and output text describing low-level visual concepts (e.g., objects, scenes, atomic actions). However, most real-world videos last for minutes or hours and have a complex hierarchical structure spanning different temporal granularities. We propose Video ReCap, a recursive video captioning model that can process video inputs of dramatically different lengths (from 1 second to 2 hours) and output video captions at multiple hierarchy levels. The recursive video-language architecture exploits the synergy between different video hierarchies and can process hour-long videos efficiently. We utilize a curriculum learning training scheme to learn the hierarchical structure of videos, starting from clip-level captions describing atomic actions, then focusing on segment-level descriptions, and concluding with generating summaries for hour-long videos. Furthermore, we introduce Ego4D-HCap dataset by augmenting Ego4D with 8,267 manually collected long-range video summaries. Our recursive model can flexibly generate captions at different hierarchy levels while also being useful for other complex video understanding tasks, such as VideoQA on EgoSchema.
Soumitri Chattopadhyay retweeted
Standard video captioning models excel on short clips but real-world videos are long & have complex structures.🎥Introducing Video ReCap🚀A recursive model that can process videos of dramatically different lengths (1 sec-2 hours) and output captions at multiple hierarchies. (1/n)
Video ReCap
Recursive Captioning of Hour-Long Videos
Most video captioning models are designed to process short video clips of few seconds and output text describing low-level visual concepts (e.g., objects, scenes, atomic actions). However, most real-world videos last for minutes or hours and have a complex hierarchical structure spanning different temporal granularities. We propose Video ReCap, a recursive video captioning model that can process video inputs of dramatically different lengths (from 1 second to 2 hours) and output video captions at multiple hierarchy levels. The recursive video-language architecture exploits the synergy between different video hierarchies and can process hour-long videos efficiently. We utilize a curriculum learning training scheme to learn the hierarchical structure of videos, starting from clip-level captions describing atomic actions, then focusing on segment-level descriptions, and concluding with generating summaries for hour-long videos. Furthermore, we introduce Ego4D-HCap dataset by augmenting Ego4D with 8,267 manually collected long-range video summaries. Our recursive model can flexibly generate captions at different hierarchy levels while also being useful for other complex video understanding tasks, such as VideoQA on EgoSchema.
Soumitri Chattopadhyay retweeted
Thanks, @_akhaliq, for including Video ReCap in your daily selection of papers. Our recursive video captioning model can process video inputs of dramatically different lengths (from 1 second to 2 hours) and output video captions at multiple hierarchy levels. More details in a separate tweet coming soon!
Project page: sites.google.com/view/vidrec…
Code: github.com/md-mohaiminul/Vid…
Dataset: github.com/md-mohaiminul/Vid…
Here is my selection of papers for today (21 Feb) on Hugging Face
FlashTex: Fast Relightable Mesh Texturing with LightControlNet
Instruction-tuned Language Models are Better Knowledge Learners
Improving Robustness for Joint Optimization of Camera
Poses and Decomposed Low-Rank Tensorial Radiance Fields
VideoPrism: A Foundational Visual Encoder for Video Understanding
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models
Video ReCap: Recursive Captioning of Hour-Long Videos
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization
A Touch, Vision, and Language Dataset for Multimodal Alignment
How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts
The FinBen: An Holistic Financial Benchmark for Large Language Models
MVDiffusion++: A Dense High-resolution Multi-view
Diffusion Model for Single or Sparse-view 3D Object Reconstruction
RealCompo: Dynamic Equilibrium between Realism and Compositionality Improves Text-to-Image Diffusion Models
Neural Network Diffusion

































