AI Research

Sepp Hochreiter: "In AI, basic research and practice are very close to each other!"

AI researcher Sepp Hochreiter has made another remarkable breakthrough. A language model that works more efficiently and faster than ChatGPT and similar systems. However, the research lead painstakingly earned in Austria is in danger of fizzling out.
Table of contents: Sepp Hochreiter: "In AI, basic research and practice are very close to each other!"

An algorithm that has the potential to put the currently heavily courted ChatGPT out of business. According to Sepp Hochreiter, director of the Institute for Machine Learning and the Linz Institute of Technology AI Lab, it is currently located on the servers of Johannes Keppler University (JKU) in Linz.

The German-born scientist, who has been conducting machine learning research in Austria for more than a decade, laid the foundation for the development of today's AI systems with his work on Long Short-Term Memory (LSTM) in the late 1990s.

So far, however, Sepp Hochreiter is holding back the algorithm, mainly due to a lack of funds for the computing power needed to train the AI. But this could change in the foreseeable future, according to Hochreiter at the second Deep Dive Meet Up of the Industry Magazine stated.

Enormous computing capacities required

But back to the beginning: The development of the algorithm was associated with some difficulties. Above all, with a problem that very many researchers in Austria find fault with: For Basic research often lack the necessary small change.

This is not only indispensable in the field of AI, as Prof. Sepp Hochreiter explains in a Interview confirms: "The basic research and the practical application go hand in hand in the field of artificial intelligence . A prime example of this are the so-called 'self-normalized networks'which we developed in Linz and which were implemented at Amazon just three months later, resulting in significant increases in sales. The time span from the theoretical model that emerges from basic research to practical implementation can be extraordinarily short in this field."

Hochreiter adds: "This is also evident in Open AI, where researchers have hardly had time to bring developers onto the team because of the speed of progress. Another striking feature of this field is that the newly developed tools are immediately adopted by large companies. In Austria, people are more used to a product being extensively perfected before it is launched on the market. But in the AI field, things are different, as the rapid market launch of ChatGPT shows."

Hochreiter's latest achievement is based on its LSTM principle from the 1990s. In view of the increasing text length that programs like ChatGPT have to deal with, Sepp Hochreiter again resorts to the LSTM concept back. However, this was combined with Transformer technology, which is currently used in almost all large AI models. The result is an AI that is faster and can analyze much longer sentences.

However, the training of artificial intelligence requires enormous computing capacities and a lot of personnel in the form of software developers - neither of which is feasible in Austria due to the limited budget, according to Hochreiter. For example, two billion euros are invested in AI research and development in the Netherlands. The University of Tübingen, where Hochreiter is a member of a committee, receives about 30 million euros for its AI center.

Compared to other countries' budgets for AI research and development, Austria is on par with Uganda and Mexico, according to Hochreiter.

Funding requirements of around 500 million euros

For Basic AI Research Prof. Sepp Hochreiter sees a need for funding in Austria of around 500 million euros: "I have taken another detailed look at the Austrian funding measures and come up with an annual funding sum of around twelve million euros that is really earmarked for the field of AI - so we are a long way from the alleged 150 million euros that State Secretary Tursky mentioned. This annoys me because I see what is currently being done in Germany in this area. There are AI centers there that deserve that name."

Teaching the AI in particular requires a lot of computing power and, according to Hochreiter, is the most expensive factor in rolling out the algorithm. However, he now has investors from all over the world who want to invest in his technology. This would not be the first time that an intelligence developed by Hochreiter and his team has learned to run at U.S. companies: cutting-edge research by Team Hochreiter was already tested at Amazon a few years ago. At the time, this brought the U.S. company two billion more in sales. Nevertheless, Prof. Sepp Hochreiter does not rule out the possibility that his language model will be further developed in Austria and made ready for the market.

However, the AI pioneer is very skeptical about the location: "It's a disaster here! For six years, I have been criticizing the fact that as a researcher, but also as an entrepreneur, there is nowhere to dock in Austria. The industrial companies don't know what's possible with AI, and the researchers don't know what the industry needs."

Matthias Heschl

AI will play a central role

In his keynote address at the Deep Dive Meet Up of the Industry Magazine Sepp Hochreiter focused on various possible applications of artificial intelligence (AI) in industry. He emphasized that AI will play a supporting role in numerous key areas of human challenges, such as energy, climate change, food production, medicine and mobility.

Hochreiter presented the future picture of the energy sector as an example - an intricate, cross-border network of energy producers and consumers constantly interacting and improving each other through AI.

According to the AI expert, the future energy production continuously adapt to current situations, predict weather conditions and autonomously negotiate with consumers to ensure optimal distribution in the grid. Hochreiter can imagine a similar system being used in the transportation sector to regulate traffic flow and thus reduce CO2-emissions significantly.

Hochreiter's JKU technology in Canada

The global demand for the expertise from Linz was confirmed by another example from Hochreiter's keynote. The Canadian government, for example, is using the Linz expertise for the simulation of a Nuclear Fusion Reactor Technologywhich was developed at JKU Linz. AI is also being increasingly integrated in the automotive industry. At Audi, for example, AI is being used to maximize comfort in the interior.

Hochreiter ended his keynote with a comment on the call by some IT companies for a pause in AI systems research and development. He noted, "You just have to look at which companies support this demand. They are precisely the companies that already have an AI system in place. It doesn't seem credible that those of all companies that already have an advantage are calling for a moratorium. That is simply absurd!"

Sepp Hochreiter is one Regulation of AI not averse, however, this must be based on state level and must not be left in the hands of the companies.

Also of interest

AI in practice
AI in practice
It is both an economic necessity and a strategic goal: AI should give European industry a competitive edge. And indeed, more and more companies are turning to the technology of the future. Examples of AI in practice show the possible applications.
Play Video
AI Insights from Fraunhofer Austria
Welcome to the third part of our series "AI Rap with Eva Eggeling"! In this video we dive deep into the world of generative AI and learn how creative algorithms work.
AI Test_Header
AI test by Alan Turing
In this first part of our series 'AI Rap with Eva Eggeling', Eva Eggeling, Center Leader for Data Driven Design at Fraunhofer Austria and Head of the Innovation Center for Digitization and Artificial Intelligence KI4Life, dives deep into the topic of the Turing Test.
How AI works
AI is everywhere - but where is the journey heading for industrial applications? We highlight how different players - from global tech giants to small startups - are using artificial intelligence to drive innovation and optimize work processes.
Play Video
Experts talking about AI
Klaus Paukovits, Head of Business Development at WEKA Industrie Medien, talks to Sebastian Schlund, Center Manager for Sustainable Production and Logistics, and Eva Eggeling, Center Manager for Data Driven Design, at Fraunhofer Austria about innovations in AI in industry.
KI Regulation
In mid-June 2023, the EU Parliament settled on a system for regulating AI. What exactly is in the AI regulation, how companies are affected in the development and use of AI systems, who is in favor of it and what criticisms are being voiced, you can read here.



Read here about strategies, possible applications and technologies in the manufacturing industry.


We bring experts and personalities from science, research and industry together at one table and discuss ...

Powered by TranslatePress
WordPress Cookie Plugin by Real Cookie Banner