Hybrid & Processing Libraries

Hybrids again

Data structure 
-video
-sound
-image
-text

Technique
-dragging
-clicking
-typing
-image processing (pixel as raw data)
-sound processing (volume, )
-visualizing

Some ideas:

Ones that focus on new techniques:
- image as clickable (Image map)
- data visualization 


- What does it mean for sound to be draggable/clickable?
- Can a text be played?

- Can I present data in a new way? Video example - based on speed. Add subjectivity.  (See superbike, fast_real_train_corrido in class folder)

Can I think of new data structures?
- photo + location
- sound + image
- Sound + space (noise example)

Notes

Programming Language and Features

Programming language:

Fundamental programming constructs (grammar):

basic math operators (+,-,*,/%,), logical operators (&&, ||)
conditionals: if statements
for, while loops
functions
objects

Environment:

Features an environment (processing in this case) provides:

size(), ellipse(), bezier(), image()...etc

Language + environment:
- Javascript + web browser
- Swift/Objective-C + iOS
- C + UNIX

Libraries: extending features

New data structures
- video
- sound
- 3D (obj)
- vector (svg, dxf, PDF)
New protocols, systems
- networking (TCP/IP, http...etc)
- hardware (eg. Arduino)
New algorithms/infrastructure/services
- games (levels, interface, first person views)
- animation (interpreting frames, sprites/characters)
- weather
- computer vision (openCV: facial recognition...etc)
3rd party developers

Notes

Libraries

Download library (package).
- Sketch -> Import Libary -> Add Library
  Search for "video"
Import library into sketch.
- Sketch -> Import Libary -> Video
- This tells processing to look for the library and import it. Know the objects/functions that are provided are available to use.
Use it.

Evaluate feature, look at documentation and samples first. If sample files are not working, it may not work in your environment
- Reference: Video library reference
- Examples: File -> Examples
If you are using a library, make sure you tell instructor or you should export it as an application. The computer we critique on may not have the library you installed.

Notes

Video

Capture: from live camera

Examples -> Libraries -> Video -> Capture->GettingStartedCapture
Examples -> Libraries -> Video -> Capture->Ascii Video
Examples -> Libraries -> Video -> Capture-> Background Subtraction -> modify to detect presence
Examples -> Libraries -> Video -> Capture->TimeDisplacement
Examples -> Libraries -> Video -> Capture-> Slitscan
Examples -> Libraries -> Video -> Capture -> HsvSpace (hybrid: visualization)

Movie: from file

Examples -> Libraries -> Video -> Movie -> Scratch -- video + new interface

Notes

Sound

Sound (in Examples -> Libraries -> Sound)
- Analysis/AmplitudeRMS -- visualizing amplitude of an aiff file
- Analysis/FFTSpectrum -- visualization
- Demos/Keyboard -- instrument?
- Demos/Sampler -- generating sound
- IO/AudioInput -- microphone to screen
- Noise/Brown -- sound and space
Minim| Audio library that proides easy to use classes (and probably has more advanced features/examples)

(in Examples -> Contributed Libraries -> Mimim)
- Basics -> PlayAFile (hybrid: sound + visualization)
- Basics -> Monitorinput (visalize live input)
- Basics -> DrawWaveformAndLevel (hybrid: sound + visualization)
- Basics -> AnalyzeSound(hybrid: sound + visualization)
- Basics -> TriggerASample (hybrid: as instrument? Triggered as respons to [xyz]? )
- Basics -> SynthesizeSound, SequenceSound (See more inside Synthesis Folder). Sound as speach, use as background music, part of instrument, source of visualzation
- Advanced-> DrumMachine

There is (usually) nothing preventing you from using multiple libraries (eg. video & sound)

Notes

Other Processing Libraries

GifAnimation (see instruction from previous note)
Exporting to PDF (see textbook)
"Temboo": Looks like a startup service (Free for 30 days - Free for educational use?). Simplifies many APIs (web services) into one framework (Twitter, Yahoo, Gmail...etc) For those of you who are ambitious or know what they want.
1. You need to make an account with temboo. Then based on other web services you want to use, there are specific things like getting a key from the service you want to use (eg. twitter) you need to do.

Notes