Calculator to Meta-Medium

Notes

Multi-Media

Q: What do you think of when you hear "multimedia?"




- Web pages of 2000s
- Powerpoint
- Email, MMS


Media: "inserted" or "attached"

Powerpoint:
Notes

Hybrid

Mediums merge together to offer a coherent new experience different from experiencing all the elements separately (p167). .... In hybrid media the languages of previously distinct media come together. They exchange properties, create new structures, and interact on the deepest levels.

Motion graphics




learn motion graphics
what is motion graphics



Acrobat:





New way of navigating representation


New ways of representing the world
Notes

Programming


"While this logical differentiation is clear and useful for the person who understands programming..." "In the world of application software, media data and interfaces/tools never exist in isolation from each other. Unless you know how to program, you never encounter media content types - digital photos, digital videos, maps, etc. - by themselves. Instead you encounter media content through particular software applications, or the custom interfaces defined by the designers of a particular project."


Q: Media format for image?:







A pixel grid: 
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000
Either a red image, or a picture with columns of white and black lines. (Or it can be text, sound, 3D data...etc)



Algorithm: how would you cut/paste a selected area?





Q: how would you "draw" a line?




Q: How would you make your image brighter?




New hybrids by extending media format/data structure 
Image:
255,000,000,255,000,000,255,000,000,255,000,000,    
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000,
255,000,000,255,000,000,255,000,000,255,000,000


Lattitude/Longitude:
45.480978, -122.630372


There are certian things that make your life easier when dealing with media files. That is one of the strengths of "Creative Coding" environments/languages


  • Processing
  • Openframeowrks (C++)
  • Max/MSP/Jitter, PD
Notes

Coding Hybrid

Imagemap: clickable image


Programming clickable image:
PImage photo;         // create a variable of data type PImage


PShape shape1;

shape1 holds shape connecting (10,30) (40,20) (20,90) (4,11)


image(photo, 0, 0);   // display photo

if location of mouseclick (x, y) is inside shape1 {
  display another image   
}

Thought experiments:

- Programming clickable sound/video. (media format: sound, interface: hypertext)
- Programming playable text. (media format: text, interface: audio player)
Notes

Hybrid

Photos + geolocation:


Google Earth:


Examples:



Notes

Before softwarization

Before softwarization, the techniques available in a particular medium were part of its "hardware".







Hardware for writing: paper, writing instruments (fountain pen, typewriter)...etc
Hardware for photography: photosensitive paper, enlarger, camera....
Hardware for Filmmaking: film stock, recording instrument...etc


Q: What does it mean when Manovich says the hardware is incompatible?

Techniques in handling medium were specific to hardware. Hardware incompatibilities prevented techniques to be hybridized


Q: How is it different in software?
Notes

Simulation in software

Simulation of media



Materials:

Techniques:

Notes

Medium: Data structures + Algorithms

Data structure: bitmap image, vector image, text, video, 3D data/model, text, audio

- bitmap image: "A grid of discrete pixel" --> An array of integers (PImage, Movie)
- 3D model: an sequence/array of floating numbers (1.33, 5.43, 1.40)
- text: a sequence of numbers (ASCII code)


Algorithms: procedures that work on the data structure

Algorithms on bitmap image:
  • go through the array and display it as an image --> change the color at coordinate (x,y)
  • manipulate the numbers in the array. (adding makes the image brighter)



Hybrids: Why programming?


Data structure ------- representation, media/file formats
Algorithms ------- interface, tools, software
Exposing the data structure/algorthim pairing and creating new combinations: [pixel_as_array][pixel_as_grid][pixel_brightness], [BackgroundSubtraction_MotionDetect]

Notes