profile picture

David Herel

PhD student
Czech Institute of Informatics, Robotics and
Cybernetics
Jugoslávských partyzánů 1580/3
160 00 Prague 6, Dejvice, Czech Republic
david.herel@seznam.cz
I am a PhD student in the Computer Science doctoral study program at the Department of Cybernetics, Faculty of Electrical Engineering (FEE), Czech Technical University (CTU) in Prague. I am researching Novel Methods in Natural Language Processing under the supervision of Tomas Mikolov.
I am the third winner of prestigious Alexa Prize Socialbot Grand Challenge 5. I regularly contribute to top-tier conferences in the AI&NLP field (ICLR, ECAI, Amazon Science). My diploma thesis has been honored with Dean's award nominations, and my research has garnered esteemed recognitions (grants, interviews).

News

1/4/2024 Our paper Collapse of self-trained langauge models got accepted to ICLR 2024 as notable top-5% (Oral). (HackerNews discussion, arxiv)
5/3/2024 National interview for various news media about my language modeling research. (Forbes, CTK, Novinky, AI institute)
12/9/2023 We have won 3rd place in prestigious Alexa Prize Socialbot Grand Challenge 5 with team Alquist AI.
04/8/2023 Our short paper Thinking Tokens for Language Modeling got accepted to AITP 2023.
25/7/2023 Our paper on Preserving Semantics in Textual Adversarial Attacks got accepted to ECAI 2023. Notable-top-5% (Oral).

Notable publications

Collapse of self-trained language models David Herel and Tomas Mikolov In International Conference on Learning Representations, 2024 (notable).
ICLR 2024 (notable)
Alquist 5.0: Dialogue Trees Meet Generative Models. A Novel Approach for Enhancing SocialBot Conversations Ondrej Kobza, Jan Cuhel, Tommaso Gargiani, David Herel, Petr Marek, Jan Pichl, Jakub Konrad, Petr Lorenc and Jan Sedivy In Amazon Science, 2023.
Amazon Science 2023
On Difficulties of Attention Factorization through Shared Memory Uladzislau Yorsh, Martin Holeňa, Ondřej Bojar, David Herel In International Conference on Learning Representations, 2023.
ICLR 2023
Preserving Semantics in Textual Adversarial Attacks David Herel and Hugo Cisneros and Tomas Mikolov In European Conference on Artificial Intelligence Proceedings, 2023.
ECAI 2023
Thinking Tokens for Language Modeling David Herel and Tomas Mikolov In Artificial Intelligence and Theorem Proving Proceedings, 2023.
AITP 2023
Emergence of Novelty in Evolutionary Algorithms David Herel and Dominika Zogatova and Matej Kripner and Tomas Mikolov In Artificial Life Conference Proceedings, 2022.
ALIFE 2022