Mostofa Patwary

基本信息

Career Trajectory

Bio

My research interests span in the areas of Natural Language Processing, Large Scale Deep Learning, and High Performance Computing. I lead the large foundation language model training team that is resposible for data curation, model training, evaluation and the uses in real world applications.

Research Interests

Papers共 88 篇Author StatisticsCo-AuthorSimilar Experts

By YearBy Citation主题筛选期刊级别筛选合作者筛选合作机构筛选

Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models

Jupinder Parmar, Sanjev Satheesh,Mostofa Patwary,Mohammad Shoeybi,Bryan Catanzaro

CoRR (2024)

Cited0Views0EIBibtex

LLM Pruning and Distillation in Practice: the Minitron Approach

Sharath Turuvekere Sreenivas,Saurav Muralidharan,Raviraj Joshi,Marcin Chochowski, Ameya Sunil Mahabaleshwarkar,Gerald Shen,Jiaqi Zeng, Zijia Chen, Yoshi Suhara,Shizhe Diao, Chenhan Yu, Wei-Chun Chen, Hayley Ross, Oluwatobi Olabiyi,Ashwath Aithal,Oleksii Kuchaiev, Daniel Korzekwa,Pavlo Molchanov,Mostofa Patwary,Mohammad Shoeybi,Jan Kautz,Bryan Catanzaro

CoRR (2024)

Cited0Views0EIBibtex

Data, Data Everywhere: A Guide for Pretraining Dataset Construction

Jupinder Parmar,Shrimai Prabhumoye,Joseph Jennings, Bo Liu,Aastha Jhunjhunwala,Zhilin Wang,Mostofa Patwary,Mohammad Shoeybi,Bryan Catanzaro

EMNLP 2024 (2024): 10671-10695

Cited0Views0EIBibtex

StarCoder 2 and the Stack V2: the Next Generation

Anton Lozhkov,Raymond Li,Loubna Ben Allal,Federico Cassano,Joel Lamy-Poirier,Nouamane Tazi, Ao Tang, Dmytro Pykhtar,Jiawei Liu,Yuxiang Wei,Tianyang Liu, Max Tian,Denis Kocetkov,Arthur Zucker,Younes Belkada,Zijian Wang,Qian Liu,Dmitry Abulkhanov,Indraneil Paul, Zhuang Li,Wen-Ding Li, Megan Risdal,Jia Li,Jian Zhu,Terry Yue Zhuo,Evgenii Zheltonozhskii, Osae,Wenhao Yu, Lucas Krauß,Naman Jain,Yixuan Su,Xuanli He,Manan Dey, Edoardo Abati,Yekun Chai,Niklas Muennighoff,Xiangru Tang, Muhtasham Oblokulov,Christopher Akiki,Marc Marone,Chenghao Mou,Mayank Mishra,Alex Gu,Binyuan Hui,Tri Dao,Armel Zebaze,Olivier Dehaene,Nicolas Patry,Canwen Xu,Julian McAuley,Han Hu,Torsten Scholak,Sebastien Paquet, Jennifer Robinson,Carolyn Jane Anderson,Nicolas Chapados,Mostofa Patwary,Nima Tajbakhsh,Yacine Jernite,Carlos Muñoz Ferrandis,Lingming Zhang,Sean Hughes,Thomas Wolf,Arjun Guha,Leandro von Werra,Harm de Vries

CoRR (2024)

Cited29Views0EIBibtex

Compact Language Models Via Pruning and Knowledge Distillation

Saurav Muralidharan,Sharath Turuvekere Sreenivas,Raviraj Bhuminand Joshi,Marcin Chochowski,Mostofa Patwary,Mohammad Shoeybi,Bryan Catanzaro,Jan Kautz,Pavlo Molchanov

NeurIPS 2024 (2024)

Cited15Views0EIBibtex

LLM-Evolve: Evaluation for LLM’s Evolving Capability on Benchmarks

Jiaxuan You, Mingjie Liu,Shrimai Prabhumoye,Mostofa Patwary,Mohammad Shoeybi,Bryan Catanzaro

EMNLP 2024 (2024): 16937-16942

Cited0Views0EIBibtex

Nemotron-4 340B Technical Report

Nvidia, :, Bo Adler,Niket Agarwal,Ashwath Aithal, Dong H. Anh, Pallab Bhattacharya, Annika Brundyn,Jared Casper,Bryan Catanzaro, Sharon Clay, Jonathan Cohen, Sirshak Das, Ayush Dattagupta,Olivier Delalleau, Leon Derczynski, Yi Dong,Daniel Egert, Ellie Evans, Aleksander Ficek, Denys Fridman,Shaona Ghosh,Boris Ginsburg,Igor Gitman, Tomasz Grzegorzek, Robert Hero, Jining Huang, Vibhu Jawa,Joseph Jennings,Aastha Jhunjhunwala,John Kamalu, Sadaf Khan,Oleksii Kuchaiev,Patrick LeGresley, Hui Li, Jiwei Liu,Zihan Liu, Eileen Long, Ameya Sunil Mahabaleshwarkar,Somshubra Majumdar, James Maki, Miguel Martinez, Maer Rodrigues de Melo, Ivan Moshkov,Deepak Narayanan, Sean Narenthiran, Jesus Navarro, Phong Nguyen,Osvald Nitski,Vahid Noroozi, Guruprasad Nutheti,Christopher Parisien,Jupinder Parmar,Mostofa Patwary, Krzysztof Pawelec, Wei Ping,Shrimai Prabhumoye,Rajarshi Roy, Trisha Saar, Vasanth Rao Naik Sabavat,Sanjeev Satheesh, Jane Polak Scowcroft,Jason Sewall,Pavel Shamis,Gerald Shen,Mohammad Shoeybi, Dave Sizer,Misha Smelyanskiy, Felipe Soares,Makesh Narsimhan Sreedhar, Dan Su,Sandeep Subramanian, Shengyang Sun,Shubham Toshniwal, Hao Wang,Zhilin Wang,Jiaxuan You,Jiaqi Zeng, Jimmy Zhang, Jing Zhang, Vivienne Zhang, Yian Zhang, Chen Zhu

CoRR (2024)

Cited0Views0EIBibtex

Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset

Dan Su, Kezhi Kong, Ying Lin,Joseph Jennings,Brandon Norick,Markus Kliegl,Mostofa Patwary,Mohammad Shoeybi,Bryan Catanzaro

CoRR (2024)

Cited0Views0EIBibtex

Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining

Steven Feng,Shrimai Prabhumoye, Kezhi Kong,Dan Su,Mostofa Patwary,Mohammad Shoeybi,Bryan Catanzaro

CoRR (2024)

Cited0Views0EIBibtex

MIND: Math Informed Synthetic Dialogues for Pretraining LLMs

Syeda Nahida Akter,Shrimai Prabhumoye,John Kamalu,Sanjeev Satheesh,Eric Nyberg,Mostofa Patwary,Mohammad Shoeybi,Bryan Catanzaro

CoRR (2024)

Cited0Views0EIBibtex

Author Statistics

Co-Author

Co-Institution

D-Core

合作者
学生
导师

No result temporarily, you can search similar Scholar with the same research interest

Data Disclaimer

The page data are from open Internet sources, cooperative publishers and automatic analysis results through AI technology. We do not make any commitments and guarantees for the validity, accuracy, correctness, reliability, completeness and timeliness of the page data. If you have any questions, please contact us by email: report@aminer.cn

Principal Research Scientist

Papers共 88 篇Author StatisticsCo-AuthorSimilar Experts