Create your own polished presentation in minutes. Just describe what you need.
Free to try, no credit card requiredAn Analogous, Information Preserving, Spatial
Transformation From Audio To Image
A Presentation by: Beller, Chu, Iwuchukwu, Lokinendi
Background
Motivation
File Formats Used
Table from BMP Specification
BMP File Format
Table from WAVE Specification
WAVE File Format
Isn't there already a way to do this?
Not really...
Figure from JPEG Specification
JPEG File Format
These are graphs of digital audio, but the images themselves cannot be manipulated to get a resulting waveform. They're representations, but they're not domain-preserving.
The Idea
amplitude = 8096
R:132 G: 205 B:63
References
[1] Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Gold-man. PatchMatch: A randomized correspondence algorithm for struc-tural image editing. ACM Transactions on Graphics (Proc. SIG-GRAPH), 28(3), August 2009.
[2] Eric Lee and Jan Borchers. Dimaß:: a technique for audio scrub-bing and skimming using direct manipulation. In Proceedings of the 1stACM workshop on Audio and music computing multimedia, AMCMM'06, pages 107{114, New York, NY, USA, 2006. ACM.
[3] Y. Wexler, E. Shechtman, and M. Irani. Space-time completion ofvideo. Pattern Analysis and Machine Intelligence, IEEE Transactionson, 29(3):463 {476, march 2007.
[4] Richard Boulanger and Victor Lazzarini. The Audio Programming Book. The MIT Press. 2011.
[5] Multimedia Programming Interface and Data Specifications 1.0. IBM & Microsoft. August 1991.
Questions?
What's next?
Experimenting with Audio Processes
Experiments
Transformed BMP image
Transformed Frequency Equalized Image
Transformed BMP image
What is Audio Equalization?
Original Audio File
Conclusions
Audio Frequency Equalization
Experimenting with Compression
Example
Final output WAV File
WAV BMP JPEG BMP WAV
Reconvert to a BMP
Converted to JPEG at PSNR = 95
BMP File After Running Algorithm
Original Song (WAV)
Conclusions
varying compression levels in GIMP (Ranging from 0-100)
As expected, more distortion was observed as we lowered the JPEG quality. For song tracks with vocals, even at lower quality compression levels, song can be distinguished.
Experimenting with Texture Recognition
Naive Attempt at Audio Fingerprinting through Texture Recognition
Segment unidentified audio
Convert to Grayscale
Perform Fourier
Segment Audio
Calculate Euclidean Distance
Convert to Grayscale
Return the audio that has the lowest distance
Perform Fourier
Store coefficients in database
Synthetic Textures (produced by our algorithms)
Natural Textures
Algorithms
Beller's Algorithm
Chu's Algorithm
32767
-32767
1111111111111111
1111111111111111
Idea: Distribute empty 8 bits across all three color planes so that images are more visually representative.
R = 11111 = 31, G = 11111 = 31, B = 111111 = 63
R = 111111 = 63, G = 11111 = 63, B = 111111 = 63
10110100||11010001||11001011
Idea: Think of the amplitude as a 16-bit binary string and parse the string into three 8-bit regions and assign one to each color plane.
R: 1101001
G: 11010001
B: 11001011
Iwuchukwu's Algorithm
Lokinendi's Algorithm
R
G
B
Amplitude
0
255
1
-32768
0
254
1
-32767
0
253
1
-32766
...
...
...
...
Take in a signed 16-bit integer from the WAV file
Add 32768 to shift all values to be positive
Take the first 8 bits and represent them as the green value, and take the last 8 bits and represent them as the blue value
The red value is represented as the absolute difference between the two other values. This is strictly for visual purposes.
Cast the three values as unsigned 8-bit integers
1
253
0
32765
1
254
0
32766
1
255
0
32767
Create your own polished presentation in minutes. Just describe what you need.
Free to try, no credit card required