Johannes Hoffart @johannes@qoto.org

Here are the slides for my #PyDataLondon keynote on LLMs from prototype to production ✨

Including:
◾ visions for NLP in the age of LLMS
◾ a case for LLM pragmatism
◾ solutions for structured data
◾ spaCy LLM + https://prodi.gy

https://speakerdeck.com/inesmontani/large-language-models-from-prototype-to-production

A new Ahead of AI issue is out, where I am covering the latest research highlights concerning LLM tuning and dataset efficiency:

https://magazine.sebastianraschka.com/p/ahead-of-ai-9-llm-tuning-and-dataset/

New Newsletter 📥 Get the inside scoop on GPT-4 & PaLM 2. Unpack the intricacies of these foundation models and understand the evolution of #LLMs
#NLproc #MachineLearning #DataScience
https://gradientflow.substack.com/p/what-you-need-to-know-about-gpt-4

#TheDataExchangePod: Are you looking for dependable, trustworthy #AI solutions for your company? 🏦🏥⚖️ Jonas Andrulis of Aleph Alpha explains what it takes to build/deploy reliable, source-cited responses, specializing in the legal, healthcare, and financial sectors.
https://thedataexchange.media/building-ai-for-enterprises/

New blog entry: Normalizing company names (and more) with SPARQL and Wikidata. As a service! https://www.bobdc.com/blog/wikidatanormalizing/

🎊🎁Big release of dirty-cat
https://dirty-cat.github.io/stable/

Broader focus: simplifying preparing non-curated dataframes for machine learning.
🔸Encoding of messy dataframes: a strong baseline for easy machine learning
🔸fuzzy_join: joining dataframes (pd.merge) despite typos
🔸Deduplication: matching categories with typos
🔸Feature augmentation: joining on an external data source to enrich tabular data
🔸Embedding of cites, companies, locations...

Tabular data can benefit from merging external sources of information.

The FeatureAugmenter is a sklearn transformer to augment a given dataframe by joins on reference tables.
https://dirty-cat.github.io/stable/generated/dirty_cat.FeatureAugmenter.html

fuzzy_join makes it robust to mismatch in vocabulary. Hyperparameter optimization can tune matches for prediction

For such external information,
diry-cat can download embeddings of wikipedia data on millions of entities: companies, cities, geographic locations...
https://dirty-cat.github.io/stable/auto_examples/07_ken_embeddings_example.html

Productive weekend! Just added 4 new Q&A's!

- Multi-GPU Training Paradigms
- The Distributional Hypothesis
- "Self"-Attention
- Training & Test Set Discordance

And "Machine Learning Q and AI" just crossed the 50% milestone! 🎉

PS: I included the Multi-GPU Training Paradigms section is in the free preview at
https://leanpub.com/machine-learning-q-and-ai/

This Politico article argues that #NMT (Neural Machine Translation) was one of the major drivers of European unity against the Russian invasion of Ukraine.

Not sure I fully buy the argument, but MT is probably one of the best examples of #NLProc / #AI that benefits society / #AIforGood.

https://www.politico.com/news/magazine/2023/02/03/europe-putin-ukraine-google-translate-00079301

h/t @daanvanesch

CALL FOR PAPERS: Research and Innovation Track

We welcome papers on novel scientific research and/or innovations relevant to #SemanticWeb, #KnowledgeGraphs, #AI, #ML, #NLP and more

Deadlines:
🗓️Abstracts: May 09
🗓️Papers: May 16

For more info: 🌐https://2023-eu.semantics.cc/page/cfp_rev_rep

Machbarkeitsstudie: KI-Leuchtturmprojekt in Deutschland möglich

ChatGPT kommt aus den USA. KI-Experten in Deutschland sehen darin ein Problem und fordern einen Kraftakt zur Wahrung der digitalen Souveränität.

https://www.heise.de/news/Machbarkeitsstudie-KI-Leuchtturmprojekt-in-Deutschland-moeglich-7468862.html?wt_mc=sm.red.ho.mastodon.mastodon.md_beitraege.md_beitraege

#Bundeswirtschaftsministerium #ChatGPT #Deutschland #Europa #KünstlicheIntelligenz #LEAM #Sprachmodelle #Supercomputer #USA #digitaleAssistenten

In eigener Sache: heise online zieht auf eigene Mastodon-Instanz

Das Chaos bei Twitter hält an und die Mastodon profitiert weiter. Heise Medien betreibt in dem Fediverse-Netzwerk nun eine eigene Instanz.

https://www.heise.de/news/Twitter-Alternative-Mastodon-heise-online-zieht-auf-eigene-Instanz-7465214.html?wt_mc=sm.red.ho.mastodon.mastodon.md_beitraege.md_beitraege

#Fediverse #Heise #Mastodon #SocialMedia #Twitter #TwitterÜbernahme #heiseonline

Do you love #selfhosting? What about providing service to the public via #Codeberg?

We are looking for maintainers that take on adding code search features to our #Forgejo instance to reduce the load on the existing infrastructure team and bring this project forward.

Please see https://codeberg.org/Codeberg/Community/issues/904 if you are interested.

We are looking forward to your contributions. Thank you a lot!

The most interesting thing about #ChatGPT that no one is talking about is how the future will be systems talking to each other with imprecise protocols but they’re still able to understand

Recent events have demonstrated how crucial resilience (e.g. of supply chains) is for our society. Semantic technologies can play a crucial here.

We will organize the #D2R2 (Linked Data-driven Resilience Research) #workshop at @eswc_conf@twitter.com
in May 2023 in Crete. We are looking forward to your contribution. Submission deadline is March 9. Check more details on our event page! https://d2r2.aksw.org #ESWC23 #CoyPu_Project #Resilience #LinkedData #cfp

The artificial-intelligence chatbot ChatGPT can write fake abstracts that scientists have trouble distinguishing from those written by humans. Increasing sophistication of chatbots could undermine research integrity and accuracy, researchers fear.

https://www.nature.com/articles/d41586-023-00056-7

#chatbots #ChatGPT #research #AI #ArtificialIntelligence

via @Nature

Hello NLP researchers around the globe! All ACL major conferences (@aclmeeting, @eaclmeeting, @aaclmeeting, and @emnlpmeeting) now have an account here. Please spread it word! #NLPRoc

I found the papers "Scaling Laws for Neural Language Models" (OpenAI, 2020) and "Training Compute-Optimal Large Language Models" (DeepMind, 2022) interesting:
https://arxiv.org/pdf/2001.08361.pdf
https://arxiv.org/pdf/2203.15556.pdf
They do a LOT of experiments training large language models (causal transformers) with varying hyperparameters, in particular model size, shape, batch size, and training data set size over many orders of magnitude. 1/?