@haverholm I experimented a bit with 2D -> 1D transforms in FScape, e.g. Hilbert curve and I think Gram–Schmidt, then overlap-adding the results per frame. Although I used some of that, in general it was not very successful. Probably better to use more high-level methods like blob detection or motion estimation.
@sciss I'm frantically taking notes of terms to look up 🙂 As I said I'm only starting out, so this is highly appreciated.
@haverholm during the first lockdown, Sean from Autechre was streaming some experiments on Twitch with a Max patch that was controlled in part by a video https://www.youtube.com/watch?v=EP37LslDAvo - clues on how it might work on blank screens and statics screens. I assume it's using the value (brightness?) of selected pixels to control parameters of the sound generation
@haverholm on Linux, Puredata + Gem could probably do something similar, Gem has an object that allows you to get pixels values https://puredata.info/downloads/gem/documentation/manual/manual/list-of-gem-objects
Le réseau social de l'avenir : pas de publicité, pas de surveillance institutionnelle, conception éthique et décentralisation ! Gardez le contrôle de vos données avec Mastodon !