Teacher's Guide: Musical Objects Interlude

Overview

The goal of this lesson is to provide students with additional, authentic opportunities to interact with simple user-defined objects.

More about the lesson

Additional Piano sprite settings

The default Piano sprite can be customized using a variety of settings. The simplest customization is accomplished by passing a single numeric argument to the constructor, which sets the number of keys, e.g., p = new Piano(88).

Greater customization of the Piano sprite is achieved by passing an "instrument settings object" with one or more of the following properties to the Piano constructor, all but the last of which are self-explanatory:

In music, timbre refers to the the character or quality of a sound or voice—as distinct from its pitch (e.g., low or high) and intensity (quiet or loud). All sprites have a default timbre, which is a piano-like sound. In the case of a Piano sprite, timbre can be set via the timbre property of the instrument settings object when the sprite is instantiated (the focus of this section). It is also possible to override a sprite's initial timbre settings, either temporarily for the duration of a song (by passing an instrument settings object to the play method) or permanently (by accessing relevant CSS hooks). Both of these options are described below.

As the lesson makes clear, the value of the timbre property is itself an object. The basic building block of any such "timbre object" is its waveform, which is set using the timbre object's wave property, as described in the lesson. With the exception of "piano" (which is a custom Pencil Code option), these wave property values refer to the shape of the oscillator wave used to generate the sound.

The details behind how sounds are mathematically modeled and synthesized is complex. A great starting place for individuals interested in delving deeper into how sounds are modeled and how these models are used to generate musical sounds is provided in this 2018 interactive article by Josh Comeau. Fortunately, students don't need a deep understanding to explore this topic and get some practice using objects (and hopefully having some fun along the way). How each generated sound changes over time is determined by a wide range of additional timbre-object properties. To help keep the presentation manageable, the lesson focuses on aspects of sound relating to a sound's envelope (attack, decay, sustain, and release).

A closer look at the Pencil Code sound model

Human perception of sound derives from vibrations of air molecules. Musical sounds in computer systems (and electronic synthesizers more generally) are modeled (and generated) based on several components relating to the nature of these vibrations.

The waveform describes the underlying basic pattern of vibrations of air molecules. These fundamental building blocks of sound can be modeled in a variety of ways. Industry standards for describing waves are "sine", "triangle", "square", and "sawtooth". A waveform can be repeated to produce a constant tone. Waveforms can also be combined to generate new wave forms, with potentially richer sounds. For example, Pencil Code provides a custom waveform, "piano", designed to mimic the sounds generated by that familiar instrument.

The terms pitch and amplitude describe two other key features of sounds. Pitch describes how low or high a sound is. Pitch is determined by the number of times a particular waveform repeats per unit time, a concept refered to as a wave's frequency. Frequency is measured in units called Hertz (Hz). The higher the frequency, the higher the pitch. For example, the left-most key on a standard 88-key piano has a frequency of 27.5Hz. The right-most key has frequency 4186.1Hz.

Amplitude is the relative strength of sound waves, which we perceive through our sense of hearing as loudness or volume, i.e., the intensity of the sound. The greater the amplitude, the louder the sound.

When synthesizing sounds, we must pay attention not only to waveform, pitch, and amplitude, but how these values changes over time. When synthesizing sound in Pencil Code, we work with individual notes, and thus maintain a constant waveform and pitch, and we specify how amplitude will change over time. For example, a piano key, when struck and held, creates a near-immediate initial sound which then gradually decreases in amplitude to zero. On many electronic synthesizers, in contrast, a comparable key press might create a sound that after a modest decay is sustained at a constant amplitude until the key is released, at which point the amplitude immediately drops to zero

In sound and music, the term envelope describes how the amplitude of a sound changes over time. Computer systems and electronic synthesizers more generally typically describe the envelope as consisting of four consecutive stages: attack, decay, sustain, and release (ADSR).

The attack phase determines the time taken for the signal to grow to its maximum amplitude (called the gain), beginning when the note is first sounded. The envelope then enters the decay phase, during which time the signal level gradually reduces until it reaches the sustain level. This ends the decay phase. The signal remains at the sustain level until the key is released, at which point the release phase is entered and the signal level reduces back to zero.

In Summary:

• attack: the time it takes for the sound to rise from an amplitude of 0 to 100% of the gain (the max amplitude for that note)
• decay: the time it takes for the signal to fall from 100% amplitude to the designated sustain level.
• sustain: a steady minimum amplitude level maintained so long as the key is held down. If a key is released during the attack or decay stage, the sustain phase is usually skipped.
• release: the time it takes for the sound to diminish to an amplitude of 0 after the key is released.

Attack, decay, and release are specified amounts of time. Sustain, in contrast, is a minimum amplitude that is maintained until the key is released. A piano-like envelope, with no continuous steady level, is produced by setting a sustain level of 0.

Additional play options

In the Code Music! lesson, students learned to play songs by translating a melody into ABC notation and passing this as a string argument to the play function. In this lesson, students gain greater control over the sounds generated by passing "song objects" to play rather than a single, ABC-notation string. For example, using song-object properties volume and tempo, students can control the loudness and speed at which notes are played. The default values for volume and tempo are 1 and 120, respectively.

As is so often the case, Pencil Code provides additional flexibility in the number and order of arguments that can be passed to play. The song object's song property can be used to include the ABC-notation-melody string in the object. Alternatively, when using a song object, the melody can also be passed as a second argument to play, as illustrated here:

play "[CEG]2"

play {volume:0.25, tempo:240}, "[CEG]2"

play {song: "[CEG]2", volume:0.25, tempo:240}

The song object's wait property controls whether or not the played song blocks subsequent animations in the calling sprite's animation queue. By default this property has a value of true. Specifying wait:false frees up the sprite to continue with the animation queue while it plays its song. The wait property is illustrated in this script.

Sound objects can also specify a timbre value, which will override the sprite's timbre settings for the duration of that song.

Notes to activities

The first activities start with modifications to the appearance of piano sprite. These are the most straightforward. The third activity, MaestroTurtle, is complicated by an anomaly of the Piano constructor. When Piano sprites are created, their convex hull's don't match their image. As a consequence, touches won't work as expected. Remedy this shortcoming by calling clip on each Piano sprite subsequent to instantiating it.

The WaveSampler program is the only activity that directs students to explore timbre. Students will have succeeded at the objects-related aspect of the activity if they can successfully use relevant (nested) objects for the value of the timbre property, e.g., timber:{wave:"sawtooth"}. However, a simple solution will likely provide students a distorted view of the difference between waveforms. The issue is that, when specifying any single timbre property, all other unspecified values are set to default settings. Moreover, the default set for wave:"piano" differs from all others; as a result, the sounds generated for other waves will sound dramatically different from the default "piano" timbre.

This alternative WaveSampler solution explicity sets all of the timbre object properties, as follows:

  timbre: 
    wave: "sine"    # could be square, sawtooth, etc
    gain:0.5; 
    attack:0.001;   # near instantaneous increase to gain
    decay:0;        # no decay
    decayfollow:0; 
    sustain:0.5;    # sustain equals gain, for constant amplitude
    release:0;      # cut off the note immedately
    cutoff:0; 
    cutfollow:0; 
    resonance:0; 
    detune:0;

Additional activities

Encourage students who move on to further exploration of timbre to always begin by specifying a base set of values for every timbre property, such as was illustrated in the preceding section. Also encourage them to focus on aspects of the envelope addressed in the lesson: attack, gain, decay, sustain, and release. As with most explorations, a wise strategy is to focus on changing one timbre property at a time. However, even when adhering to this approach, there remain substantial pitfalls. Refer to the notes in the What can go wrong section for details. Of particular importance is to keep in mind that most timbre properties (attack, decay, and release) are based on clock time, which is the same regarless of tempo. When exploring the envelope, some helpful strategies are to slow down the tempo and to use rests between generated sounds, so that effects on individual notes are not obscured by other notes.

• The Rhythm and Clapping from the Pencil Code Jam pages illustrate that the "noise" waveform is well-suited to generating percussive sounds, such as for snare drums and symbols. Experiment with these properties to derive instruments to use in creating your own Drumline program.


• In MorseCode, various combinations of short and long notes (referred to in the communications business as dits and dahs) are used to represent english letters and numbers. Do a web search (or just go to the relevant page on Wikipedia) to find a translation table and other relevant information, such as the standard durations of dits and dahs and of the gaps between letters and words. Create an appropriate custom timbre for a telegraph, and then use it in a program to communicate a word or message in Morse Code. As an added challenge, add a (potentially animated) visual representation of the dits and dahs as well.

Beyond the lesson

The tone method

As noted above, pitch is a musical term describing how low or high a note is. The pitch of a sound is specified indirectly when using play via the choice of ABC-notation note in the song. Alternatively, a note with an equivalent pitch can be generated by passing a frequency (a number value representing the number of oscillations per second) when calling the tone function. For example, for a given instrument, the call tone 440 produces the same sound as play 'A'. The higher the frequency, the higher the pitch of the note played.

Frequencies are related to notes via the somewhat complicated expression,

Math.round(49 + Math.log(freq / 440) * 12 / Math.LN2)

For a given frequency (freq), this expression identifies the n-th key on a standard 88-key keyboard (with the left-most key numbered as 1 and so on) that captures it. For example, substituting a freq value of 40 yields 49, which corresponds to the "A" key to the right of "middle C".

A useful reference table of notes and frequencies is provided by InspiredAcoustics.com.

The tone method accepts five comma-delimited arguments:

tone(
  pitch,     # at the given pitch
  secs,      # for the given duration
  v,         # with the given volume
  delay,     # starting at the proper time (in secs)
  timbre     # with the selected timbre 
) 

Note that tone does not participate in the turtle animation queue. A single tone can be played immediately or it can be scheduled to be played later by specifying the delay argument. Thus, a chord equivalent to play "[A ^C' E']" can be generated with successive calls to tone:

tone 440, 1
tone 554.37, 1
tone 659.26, 1

Silence

The sprite-specific silence method Immediately silences sound from play or tone.

Default timbre settings

When a user makes use of a custom timbre, Pencil Code assigns values for all unspecified properties of the timbre object. When wave:"piano" is specified, the following Pencil Code piano sound defaults values are assigned:

{
wave:"piano"; 
gain:0.5; 
attack:0.002; 
decay:0.25; 
decayfollow:0.7; 
sustain:0.03; 
release:0.1; 
cutoff:800; 
cutfollow:0.1; 
resonance:1; 
detune:0.9994;
}

Whenever a wave value other than "piano" is specified—including if wave itself is unspecified when one or more other timbre object properties are set—the following default values are used:

{
wave:"square"; 
gain:0.1; 
attack:0.002; 
decay:0.4; 
decayfollow:0; 
sustain:0; 
release:0.1; 
cutoff:0; 
cutfollow:0; 
resonance:0; 
detune:0;
}

timbre shortcuts

Pencil Code provides a shortcut for setting up default timbres, allowing users to simply set the value of the timbre property to one of the waveform strings directly, rather than make use of a nested object—such as timbre:"sawtooth".

(Note that the lesson eschews this shortcut as it can muddle understanding of both the underlying music and coding concepts. It also sidesteps the use of nested objects, which is a specific goal of the lesson.)

What can go wrong

touches doesn't work

As noted in the Notes on Activities section, Piano sprites don't automatically get created with an appropriately-set hit-testing region. Thus, students need to explicitly set the convex hull with a (sprite-specific) call to the clip method.

Changes to ADSR seem to have little or no effect

The attack, decay, and release components of the envelope are defined in terms of time (in seconds). In contrast, calls to play generate sounds based on a combination of tempo and the base note-length. For example, by default, Pencil Code plays 1/4 notes at the rate of 120 beats per minute. Thus, play "A" will sound for roughly 1/2 of a second, and play "A2" will sound a note that will last approximately twice as long.

When exploring the envelope, care must be taken to ensure that played notes don't have a duration less than the values set for the ADSR properties in the timbre object, or else the student won't be able to discern differences between alternative settings. To remedy this, encourage students to begin by slowing down the tempo; perhaps an ideal setting is 60 beats per second. Also, incorporate silence withing songs, via musical rests (e.g., play "A2 Z2"), so that subsequent notes don't obscure previous ones.

Technicalities

CSS hooks: turtleVolume and turtleTimbre

A CSS hook is a custom CSS property which is defined to facilite accessing and modifying style settings for one or more HTML page elements. Pencil Code defines two sprite-specific CSS hooks related to sound: turtleVolume and turtleTimbre. As the name suggests, these hooks provide access to a sprite's volume and timbre properties. The statement p.css("turtleVolume") will return the current volume setting for that sprite; the statement p.css("turtleVolume":0.5) updates the volume setting to 0.5 of the computer systems current volume level. The turtleTimbre hook provides analagous access.

The jQuery css function will be introduced to students in Label Recycling!.

Web Audio API

Pencil Code sound features are based on the JavaScript Web Audio API. MDN web docs provide a comprehensive discussion of this resource, Using the Web Audio API.