[Commentary on this post.]
Ok, so no one seems to like the videos. But I think that’s rather uncritical. Lets look at this more carefully.
The first important thing to notice is that it is the robot making this art. It is making aesthetic choices about the material and integrating those choices in novel ways to createthe final product. Its decisions are its– no one determines which decisions it will make, and its even incorrect to say this is a decision procedure: neural nets are trained, in this case on impressionists paintings, but no one has any priviledged access to the internal structure on the net, except the robot itself. The robot is in this special position because it can use the network.
We don’t know the internal structure, but that doesn’t mean we are entirely blind to its evaluative criteria. In particular, we know the input/output dimensions, and what features or properties those dimensions code for. We can call this the machine’s understanding of the art work. Notice that what it understands about the images is radically unlike our own understanding. It doesn’t see cars or roads or traffic, like we do. It sees colors at places. It sees composition. I’m inclined to say that we can’t really evaluate the art here, because we lack the machine’s understanding of its film. I’m not claiming that we need to know the artist’s intentions and understanding in order to evaluate a piece of art, but just that what the machine sees is so radically different from what we see, that our gut reactions to the work doesn’t say much about its merit.
Its important, then, to describe the machine’s relation to the art as a kind of understanding. Notice that this is different from attributing mental states to the neural net. We don’t have to attribute mental states to the machine, or talk about its beliefs and intentions, in order to distinguish its understanding of the images from ours. But it is that difference of understanding that makes the evaluative difference. This difference in understanding follows directly from the claim that the machine is making the art, and the machine alone is using its evaluative criteria. This is different from saying, for instance, that the machine is merely collaborating in the project. It is the artist, and to the machine the credit goes. Now that doesn’t make the art especially interesting, or give the machine that much credit. It is severely limited, for instance, by the fact that it is strapped on a bus and can only really encounter cars and roads and street lights. Once I’ve seen the kinds of films this bot makes, I’m not really interested in seeing more examples of the project. But its still a great example for my purposes.
Why would someone find this a more convincing example of computer understanding than, say, computer chess playing?
Because chess is an artificially circumscribed domain, so its easy to say the computer isn’t really playing chess, its ‘merely calculating’. This was Carman’s argument, after all. Listen, my claim is simple: the machines are doing things, in the normative sense of ‘doing’, in a way that implies understanding. Google uses language, etc. But its easy to say that Deep Blue isn’t really playing chess, because it isn’t really doing anything. Its just an automatic inference engine, whose calculations happen to represent a chess game.
But art isn’t nearly so formalizable or narrow an activity, and it is in the robot’s doing that the art is created. The ‘merely calculating’ move is a much harder to pull off when you have the tangible product (as it were) of a finished film in front of you.
I am just trying to clarify, but why isn’t what the video-making computer part of an artificailly circumscribed domain? It assigns numbers to flashes of color (or whatnot) and then manipulates the numbers to create a movie. Am I misdescribing what it does?
It doesn’t manipulate the numbers. It evaluates them according to an aesthetic criteria garndered from its training set. Because this is basically a qualitative judgement, as opposed to a formal heuristic calculation, its operating domain is essentially unlimited- it can evaluate any imagistic domain that can be coded into its input vector. Deep Blue, in contrast, can’t do much more than play chess. Sure, you can map just about anything onto chess positions and feed them into a chess playing machine, but what it does with those inputs can’t really be understood except as chess moves on that mapping.
The bot takes images, evaluated them according to its aesthetic preferences, and constructs a film based on those preferences. Its storage of the images is manipulated in various ways to produce false memories, but this only drives home the fact that the machine has no priviledged access to the images in its memory. Its only access here is its ability to use its evaluative criteria. As its designer says,
“Some experimental filmmakers are tempted to see their footage as a changing field of colour, and not as an indication of objects moving about in space. This is quite hard to do, and can result in many wasted years of studied dis-observation. I used to be such a filmmaker, until I realised that a machine would be better suited to the task.”
Now, maybe you are arguing that the bot is restricted to the domain of ‘changing fields of color’, but if you know anything about the early modern philosophers, you know thats a rather large domain.
Well, I am not “arguing” anything. Maybe you are so used to people being so implacably resistance to your project you can’t recognize a purely clarificatory question. All I am asking is why this is different from a chess playing computer.
But I couldn’t gleam from the webpage you linked to what is meant by training. I gather the machine breaks an image down into numerical values and compares it to the numerical values it was “trained” to “like” by the programmer. It then links together various criteria in evaluating the numbers (by plotting them on 20 dimensional space and evaluating their closeness and how frequently it has used those particular points in the past).
Similarly, the chess computer takes certain inputs which correspond to board position and assigns them a numerical value based on certain characteristics (which are also broken down into numbers)) that it is “trained” to “like” by the programmer. That is, it assigns high numerical value to positions and (and potential positions coming from that position) based upon a program that deploys certain heuristics. The fact that it can apply these heuristics to so much more data is what allows it to surpass the programmers’ abilities (and most computer chess programmers are fairly mediocre chess players when compared to the people the top computers normally play).
The difference you seemed to point to is that this computer can be used to evaluate a broader set of inputs because it can break ANY image down and apply its evaluations while a chess computer can only evaluate chess inputs. If I designed a computer that could play any board game, would that close the gap somewhat? Is that the only difference? Are you suggesting that neural nets being trained is qualitatively different when compared to programs?
Okay, fair enough. Now, a new question: What precisely are you suggesting this leads us to?
Are you saying that, as a result, we would be justified in employing reactive aesthetic judgments that we reserve to agents towards this computer? Should it be in the running for an Oscar, assuming Oscars only go to people or groups of people?
I can’t argue for or against your position until I know what precisely your position is. Maybe I am unfair to ask this in a short blog post, but you clearly think that this robot’s ability has something quite important to tell us. I would like to know what it is.
I didn’t think you were attacking me; I actually appreciated the chance to clarify my arguments.
“Are you suggesting that neural nets being trained is qualitatively different when compared to programs?”
Yes. It would probably help if you understood a bit about neural nets. Training in this case doesn’t require scare quotes- it is literally trained from a system with no discriminatory abilities to one with a sophisticated pallet, and this training is performed not by programming, or under the intentional influence of its designers, but merely by exposing it to input and correcting its output. Whether or not that amounts to genuine ‘liking’ is somewhat besides the point, and your scare quotes there are a bit pedantic.
The chess system ranks positions, true, but it does it according to formula specific to the chess domain- which is my point; those evaluations are only good evaluations when we are considering chess. They don’t have much use elsewhere. Aesthetic evaluations, one might think, have a much wider scope and import.
I may be abandoning my pomo heritage, but I think we judge the art work, not the artist; and I am arguing that we need to consider the machine the artist in these cases. Art films don’t go up for oscars, and even if these definitely don’t have a shot, but yeah, I see no reason why an robot artist can’t win awards. That doesn’t mean this will ever actually happen, but then again we make distinctions between men and women’s sports as well, and thats not because the women aren’t ‘really playing’.
I dont think the machine has anything to tell us. I simply think the machine understands the work it creates in a novel way, and its understanding is not something we have a very good grip on, and thus it is difficult to evaluate the art works ourselves. Maybe someone should build some robot art critics to settle the matter.