Artificial intelligence in image production

Written by Manuel Ruiz Dupont: Real-Time Development Process Consultant and Trainer at Pixelacademia

This article discusses the disruption that artificial intelligence will bring to the field of image production. Even if you are not in business, you can easily understand it because I have made my point with many visual examples (and I advise you to first watch the video of Mr. Villani’s speech in the Senate, in which he discusses the limits of artificial intelligence).

What is artificial intelligence?

There are several types of AI (ANI, AGI, ASI…), each with its own definition[1], but I will only deal with those who use a database that is managed by complex algorithms. There are also AI systems without already created databases: they are created at the moment.tWhen a request is triggered or incrementally generated (machine learning).

What do we do with artificial intelligence?

Today, there are already many applications that work well with artificial intelligence.

Create high quality images, created from simple text or graphic

Image created with Google Colab (left) and Nvidia (right)

The ability to compose music with a very simple interface or from text

The interface of Aiva, music creation software with AI
The interface of Aiva, music creation software with AI

Automatic story creation (Tales, in particular)

playabilitypreliminaryPlayer behavior in video games

The “Nervous state machinePresented at the Siggraph exhibition in 2019, able to learn andforecastInteractions between character (avatar) and scene from real-time motion capture data.

There are also a number of interesting apps that exclusively use artificial intelligence in the market as well. You can also find them for free on the Internet (Google Colab). They are usually programmed in Python, but you can use them without knowing how to code.

See your face instead of an actor in a movie scene

Example of deepfakes with actor Tom Cruise (right)
Example of deepfakes with actor Tom Cruise (right)

The possibility of seeing your face after 50 years, or if you are a woman, a man, or of another origin

Image generated from the Artbreeder artificial intelligence software
Image generated from the Artbreeder artificial intelligence software

Added to this are applications that are still in an embryonic state for search only

Generate 3D characters, with their animation from text

Generate 3D characters, with their animation from text
Introduction to HumanML3D Animation”textual

Create 3D folders from a single image

And of course, there are all those applications we wish AI would make possible today, such as an automatic muscle machine, or even creating hyper-realistic clothes that could be animated according to a simple pattern or image, but we’ll have to wait a little longer.

Artificial intelligence in image production

Current AI programs are not positioned as solutions to development issues. It took Unreal Engine over 6 years – despite its massive potential – to create a spin-off that was relatively passable with 3DSMax, Maya and Houdini, but it’s still too weak for Nuke. Thus, if we arbitrarily consider 2022 as year zero for AI, we will still have to wait before AI is truly applied in the image production chain, and this will necessarily be subject to the activation of plug-ins in the production program and – at the same time – by the birth of the AI ​​program Which takes into account production software needs: AI will truly assert itself when it is accessible on production software already installed.

Until now, if you were in the video game business, all you had to do was master a few technical concepts (UV, bones, norms, polygons, shaders) to develop them, but the increase in the power of machines has created bridges with other sectors of activity (cinema on in particular) that manages other concepts (fluids, poetry, renderings, etc.). I believe that artificial intelligence will initially make it possible to create tools with technical concepts that are very powerful and easy to use thanks to simple interfaces, which will be used for cinema and real-life professions.

Second, it will also enable AI to quickly produce visual effects or tricks that require a lot of development time. This includes creating graphic patterns or famous deepfakes (this is indeed possible, but the result is not accurate or qualitative enough to be implemented in production).

Portland, in the style of Van Gogh's Starry Night
Using a variety of mathematical models to convey the artistic style, here is the Portland city skyline, similar to Van Gogh’s Starry Night

Is this the end of some professions? Nobody thinks so, but Quixel, or Kitbash3d, or even fabric banks have greatly reduced the human needs of production, but at the same time, visual production has exploded and human needs have only increased.

For example, I tried to create 3D animation with as little human intervention as possible.

Here’s how I did it:

  • I generated a script using an artificial intelligence program that generates poems
  • I created the image using AI image creation software from text (I copied the text generated by the poem creation software)
  • I have made “2D sizeusing AI software
2D size
  • I created the 3D volume using photogrammetry software
  • I exported the volume in Maya and then made a cinematic with camera movement
  • I exported the rendering in AI software to make the animation more smooth

What is the future of artificial intelligence?

Currently, the biggest obstacle for artificial intelligence is the time required to generate and process the database through algorithms.

If we can reduce this time, we can imagine that artificial intelligence will help us in real time – during development – to improve our creations or correct our mistakes. If we combine this real-time artificial intelligence with other technologies (assets library, avatar creation, etc.) Elaborate, where the AI ​​simply fills in the gaps according to its own databases. The creator of this movie will only have to adjust certain details of the score, according to his vision, using a fairly simple interface. I know this may sound like science fiction, now…

Of course, many steps need to be taken before a viable solution is reached, but the closer we get to automated generation with very few human needs, the problem of originality of work will arise, because AI needs databases. However, at present, many of them have been created without the consent of the authors.

Can we really imagine creating new companies that will commercialize the image databases of which they will be the authors? And what if, after all, one of the possible futures for development companies is simply creating images intended to form databases?

When will you turn to artificial intelligence?

It reminds me of that period, 4 or 5 years ago, when real time started to be used in sectors other than video games. It was initially seen as the ultimate solution to many development issues. In some cases this was true, but in others it was not. And real time often provides more creative convenience. For example, real-time gives real added value to video mapping, and in the field of cinema, it provides greater staging convenience. In the field of animation, it makes it possible to create a more fluid development process, but the final rendering loses in quality.

Today, artificial intelligence is already in pre-production stages (concept artists use it). In production it’s obviously not developed or remains narrative, but I think it will be around soon in the last phase (post-production), especially for color adjustments (DVinchi), since there are thousands of databases that allow you to use it quickly .

So when do you turn to AI? Should we wait for the software to finally be ready, knowing that the next version will always be better? In the end, if you are careful and put some time into research and development and watch the technology, there is no bad option. But in certain sectors of activity, the implementation of artificial intelligence will be faster, especially in the field of photo or video recovery (and certainly, indeed, in the field of music).

Photo recovery
Photo recovery using Photoshop and AI software (stable spread).

How did you learn it and why?

Teaching AI to a graphic designer like Maya or Blender seems impossible to me because at the end of the day, it’s all about math. But we wouldn’t ask a graphic designer to program an artificial intelligence. On the other hand, we can explain to him the technical concepts that artificial intelligence uses that are very far from ultraviolet radiation or liquids.

What is certain is that the following trend appeared several years ago: companies are looking for profiles that have mastered aesthetics and technology.

New professions will appear, and therefore new features that will have qualities other than aesthetics or technology but rather a strong tendency to abstraction in order to be able to manipulate the technical concepts of artificial intelligence, without being gifted in mathematics.

We should already be teaching AI in schools and each school should determine how to do it because, in the short term, companies will require adequate profiles of AI concepts.

To summarize

I believe that in the medium term, AI will be a real vector of disruption within our production chains, because development processes will evolve as they are implemented in the software we use. The teaching of artificial intelligence will take a full place in video game schools and we will see it appear in the new courses.

Of course, new professions and companies will appear. In the short term, the real hurdle lies in the level of ethics and legal aspects. Indeed, AI allows the creation of high-quality content that may be morally reprehensible. Moreover, nowadays, databases may be created without the consent of the authors, which poses a legal problem.

If you are interested in the legal and ethical side of artificial intelligence, I advise you to read the interview given by Mr. Emad Mostaki, founder of Stable Diffusion (AI software) to The Times.

Manuel Ruiz Dupont
Real-time development consultant and trainer at Pixelacademia

#Artificial #intelligence #image #production



Leave a Reply

Your email address will not be published. Required fields are marked *