Notebooks

Visual Grammars (Shape, Image, Picture Grammars)

Last update: 02 Jan 2025 14:23
First version: 2 January 2025

Yet Another Inadequate Placeholder

In formal language theory, "grammars" are rules describing which sequences of symbols are allowed in a language, and so, implicitly, which sequences are not allowed. Such grammars correspond to abstract automata which can classify such sequences as belonging to the language or not, or which can generate all, and only, the allowed sequences. This is closely related to approaches to the complexity of time series on which I did my dissertation research. I was then very interested in various stabs people have made over the decades in extending these ideas to two and three dimensions, to visual grammars. (People draw sometimes very subtle shades of distinction between terms like "image grammar" and "picture grammar", which I am going to run together here.) I ended up being able to handle the spatio-temporal cases that most concerned me without using such machinery, but I still find the ideas interesting, and want to come back to them, eventually.

The "obvious" approach is to take a 2D image composed of pixels, chose a path which will go through every pixel, and apply 1D, sequential ideas. These is deeply unsatisfying for more reasons than I have time to list right now. I will just mention that the results would seem to depend crucially on the arbitrary choice of path. I am really only interested in approaches that actually use the 2D or 3D structure.

Issues: "Parsing" an image according to a grammar. Do human beings (or other animals?) use anything like a visual grammar in perception? Grammatical inference.

Grenander's "pattern theory" goes here, though it might really merit its own notebook someday.

permanent link for this note   RSS feed for this note

Notebooks: