HTML5 GUIs for Android Csound6:
A Tutorial in 14 Examples

Art Hunkins
abhunkin@uncg.edu
http://www.arthunkins.com
3/26/2016
Revised 4/7/2020


In June 2015, Michael Gogins introduced a version of Csound6 for Android[1] that included HTML5, a development that permitted Csound to include its own performance GUI. Shortly thereafter, he followed with a tutorial chapter in the Csound FLOSS manual[2], detailing this new facility, and documenting it with 5 examples[3]. This article is based on Michael's excellent introductory tutorial, and is meant to supplement it.[4]

The value of HTML5's inclusion in Android Csound, for both portability and live performance, is clear.[5] It is now possible to perform live anywhere, with only an Android device and a set of headphones. No electricity, no MIDI or other controller, no cords. The only software is one of the free Csound apps and your .csd file. The sole dedicated expense is the Android device; and your performance interface is customized to your individual composition. (No more limitation to 5 generic sliders and buttons.) These benefits accrue to performer as well as composer; there is no need (or expense) for anything other than a low-cost Android device for any musician to perform your piece. The only current proviso is that the Android OS must be at least 5.0.

This article, along with its accompanying examples, is a kind of "tutorial for the rest of us" - one designed for the more conventional Csounder. The amount of required HTML - and JavaScript, and CSS (style sheets) - is formidable, and necessitates a lot of additional code. The examples attempt to simplify and standardize to the degree possible, eliminating all bells and whistles.[6] Thankfully, many segments of this code, can be copy/pasted directly into user .csd's. (This is particularly relevant to our first example.)

My goals for this tutorial were: 1) to transfer .csd code from original "canonical" Android Csound directly into HTML5 GUIs with little or no change to the original; 2) to offer simple and functional HTML design, while basically mirroring the "canonical" GUI; 3) to be detailed and specific in GUI code, illustrating differing numbers/collections of widgets; 4) to use the full screen of the Android device, or most of it, for maximum "playability." - which in turn entails, importantly, 5) to auto-scale to device size. This last criterion turned out to be the most challenging.

Another difficult task was to come up with workable trackpad code. Michael had good examples of both slider and button code in his introductory tutorial, but had not implemented an HTML trackpad. Addressing both the auto-scaling and trackpad challenges, my son Dave Hunkins (www.zatchu.com), an experienced Android programmer, came to the rescue. (I am indebted to his extensive help on these and related technical coding issues.) His solutions are incorporated in the script found at the end of each tutorial example.[7]

Csound for Android with HTML5 GUI option requires Android OS 5.0+, and is not available for earlier versions of the OS. It is downloadable (free) from: https://play.google.com/store/apps/details?id=com.csounds.Csound6 .


GENERAL REMARKS ABOUT THESE GUIs

1) Android devices have many screen sizes, and HTML GUIs need to adjust for vertical size in particular. This can be done even on an individual device basis, if desired. The code to edit is at the very beginning of the <script> section. An example of editable variables from example 8 below (Clone with Trackpad) follows:

/* adjust widget dimensions (% of screen) here */
var slider_height_fraction = 0.08;
var button_height_fraction = 0.06;
var trackpad_height_fraction = 0.3;
var trackpad_width_fraction = 0.9;

Experiment with the given values until you get a good fit for your screen. (And don't forget the semicolon at line end!)

2) Note that also within the <script>, the code below requires the color and background-color variables *only for sliders*, not for buttons and/or trackpads:

table td {
	text-align: center;
	color: white;
	background-color: teal; /* user choice */
}

3) You also may want to adjust the slider *track height* (thickness). This is done toward the top of the <html> section, within the "actual slider" code:

height: 15px; /* user choice - suggested max: 20px */

4) Note finally that you can move the slider only when you first touch it *within the track* (i.e.," a thicker track is a safer track"). At the same time, be aware that a value greater than 20px will create a track thicker than its thumb, which may obscure the slider's current value.


It is crucial to test your HTML GUI on a smartphone or very small tablet early in your work. If the display looks right on a small screen, it will look fine on a larger one. Less text will fit per line on a small screen, or put another way, text - including button labels - take up relatively more space on a small screen. Button labels are especially problemmatic; longer labels (or more properly, a longer single string in a label) force a wider button, and will push the row of buttons partially off-screen. If you need more characters in a narrow button, you can often split a single longer string into two strings (into two lines) - with a space, forward slash or dash. Keep in mind that buttons will only remain the same width if their longest string is the same length; also that an *opening and/or trailing space* (perhaps alternating with &nbsp; characters) will add to string length. If you need more characters per label than you have space, you can always reduce the relative size of a label by using the <small> tag. (Of course, this should then be done for all the labels in a row.)

Here is a basic guide for maximum characters per button label:[8]
  For a row of buttons using the entire width of the screen (normal size text): 6 buttons will accommodate 4 characters; 5 buttons, 5 characters; 4 buttons, 7 characters. (As might be expected, 4 buttons encompassing 80% of screen width, also accommodates 5 characters [e.g., Simply Noise].) With the HTML <small> tag, 5- and 6-button rows allow one more character per label; and a 4-button row allows two additional characters. Note that a single <small> does not make a noticeable difference in appearance or readability of button labels.

With regard to *slider* label/text: most of the same comments apply as well to buttons. All sliders within a table autosize in width to the longest single string in the slider group. It is not necessary for the longest string in each label to be the same length (unlike buttons). If you wish to keep two strings from potentially separating into two lines of text, you can either omit the space or substitute a dot for it. (On a smartphone with lots of sliders, there may not be enough vertical space for two text lines - e.g., example 14.)

With regard to trackpad: all text must be external to the widget itself.


THE EXAMPLES

Though all these examples stand on their own, they are listed in a suggested order of study. They deal uniquely with sliders and buttons - both with text labels, and an x/y trackpad. Buttons are implemented both as simple triggers and as more complex "touch down/touch up" mechanisms. Sliders are illustrated both with and without displayed values. Both sliders and buttons, as in Gogins' examples, are implemented in tables for neat and user-friendly appearance.

For consistency and easy comparison, the following procedures are followed in the examples:
1) All HTML is placed at the bottom of the file. It could just as easily be placed at the top. Either way, messages regarding HTML (thankfully) do not appear in Csound console output.
2) CSS (style sheet data) appears within the HTML <head> section.
3) JavaScript code also appears within the HTML, but at the bottom of the <body> section.
4) With both style sheets and JavaScript, slider data appear above button data. (Any trackpad appears at the bottom.)
5) Within the <body> section, following any title text, slider table(s) appear above button table(s), with trackpad below. This parallels the layout of "canonical" Android app GUIs.

The example .csd's are found, along with this file, in the following zip archive:
http://www.arthunkins.com/Android_HTML5_GUIs.zip

1 - Csound6 HTML5 GUI Clone.csd
  This file simply "clones" the "canonical" Android Csound5/6 GUIs, except that it includes no x/y trackpad. Any .csd for "canonical" Android Csound5/6 without trackpad may be prepended to this file as is. Try it with your Android .csd on any recent device; or append the included code to the bottom of your Android .csd, and run it. Note that the five buttons accommodate a 5-character label maximum. (This parallels the "canonical" Android GUI label - e.g., "Butt1".)
  There is little inline explanation here.
  IMPORTANT NOTE: This example implements a simple trigger for buttons; it deals only with "touch down," not with "touch up." The coding for this kind of button is substantially less complex than for a "touch down/touch up" mechanism, which is useful for "notes" and other "gate" events. Surprisingly, however, there is substantially *less* latency for the full down/up arrangement. (Both types of button exhibit some inherent delay, which seems to be a disadvantage to using HTML widgets in contrast to "canonical" Android GUIs.) Except for the three examples marked "Enhanced Button", however, the tutorials that follow incorporate the simpler trigger mechanism, as most do not require "touch up", nor are they particularly sensitive to latency.

2 - Csound6 HTML5 GUI Clone Enhanced Button.csd
  Identical to #1 except for enhanced button coding that responds to "touch up." (This single feature lengthens the amount of javascript extensively, though it is essentially boilerplate.) Like example #1, this is an excellent candidate for appending "canonical" Android Csound5/6 .csd code - in this case, code that requires full "touch down/touch up" button response.

The next five tutorials are the most informative and worthy of study. Their .csd's are all heavily commented.

3 - HTML5 Slider GUI Demo.csd
  Basic demo for sliders only. Sliders all have labels; some display values, others not. Slider height is specified as a percentage of viewport height at the beginning of the <script> (bottom of .csd). While "canonical" Android sliders range only between 0 and 1, HTML5 sliders can range between any two values (though default values remain 0 to 1). Special steps must be taken, as shown in this demo, to constrain displayed values to a limited number of characters (here, 3). In short, the displayed values of all sliders should show the same number of characters (as should the initial display). Also, if the start value of the slider is not 0 (it need not be in HTML5), the .csd must be initialized to this value (as is again explained and illustrated by the .csd.)

4 - HTML5 Button GUI Demo.csd
  Basic demo for buttons only. Buttons all have labels. Button height is specified as a percentage of viewport height at the beginning of the <script> (bottom of .csd). Whereas slider labels auto-size to the longest "word" (string) in any label within a table, button labels are more difficult, forcing individual button width to its longest "word" (string). In all buttons, "clickable area" is restricted to the text label itself. If button height permits (and it may well *not* permit), a long label string may be shortened by introducing spaces, forward slashes or dashes as string dividers, thus creating additional but shorter lines of text.

5 - HTML5 Enhanced Button GUI Demo.csd
 Identical to #4 except for enhanced button coding that responds to "touch up."

6 - SimplyNoise.csd
  A button-only "drumset" simulation. 2 tables: 1 of two rows of 4 "drums" each, 1 of a single row of 4 amplitudes. Button height is again specified at the beginning of the <script>. Demos larger title and interpolated text. Shows how to center tables (when filling less than full width of screen). The second table of buttons (Amp) is extended with spaces to enhance the clickable area, and to match spacing of the previous table (Noise).

7 - SimplyNoise Enhanced Button.csd
 Similar to #6 except for enhanced button coding that responds to "touch up." In addition, .csd instrument coding is expanded to include control of percussive "note" duration. (These simple modifications increase the length of the .csd by 50%.) A basic template suggesting how to use HTML5 buttons as a simple "keyboard." Note that the final four "amp" buttons might just as well be coded as trigger-only widgets, as they don't make use of "touch up." 

These next three build on those above, and are more lightly commented.

8 - Csound6 HTML5 GUI Clone with Trackpad.csd
Example 1 with added trackpad. The commentary describes how touchmove outside the trackpad can be handled. The code as it stands, constrains values to the stated trackpad limits, even when the touch moves outside the pad. (Options are shown that allow x and/or y values to exceed these limits when touch moves outside.) In contrast, note that all "canonical" Android GUIs allow touchmove out-of-range values on the y axis. (Since x-axis width is fixed at 100%, there is no issue of out-of-range on this axis.) Our trackpad implementation differs from "canonical" only in that both width and height percentages (of screen, or more properly "viewport") are set by the user (at the top of the <script>), and that the two axes can be constrained or not, independently. As in "canonical" Android, the trackpad turns a different color when touched, returning to the default color when the finger raises. This gives important visual confirmation of when a message is sent, and is especially important when a touch is near a trackpad boundary. Note that the "trackpad-touched" color may be set by the user toward the top of the <script>.

9 - Csound6 HTML5 GUI Clone with Trackpad and Spiritus2.csd
Example 8 with added composition, SPIRITUS SANCTUS 2 (2015) for realtime Android Csound. This is the same composition as example 11, but adapted to the "canonical" Android collection of 5 sliders, 5 buttons and trackpad (the trackpad acting as a dual slider, and 5 buttons substituting for an 8th slider). Trackpad axes x and y are both constrained when touchmove exceeds stated limits (thus emulating dual sliders). Background and performance info about Spiritus2 are located here: http://arthunkins.com/SpiritusSanctus2altAndroid.txt .

10 - NoisePad.csd
Similar to example 6, but implemented with a single trackpad. Because touchmove is not involved here (only discrete clicks within trackpad bounderies), no constriction of out-ot-bounds values is needed - so the code is somewhat simplified. This is an excellent template for a .csd that uses only a trackpad.


The remaining four examples are modified HTML5 GUI versions of my recent works for "regular" Android Csound. Since all of them require more than 5 buttons and/or sliders, they were originally designed for performance by my "alternative" Csound6 apps, Csound6a, Csound6b and Csound6c - which are described and linked to here: http://arthunkins.com/Android_Csound_Apps.htm . (The *standard* 9-slider, 5-button GUI of the Csound for Android app can perform the original version of example 11 [SpiritusSanctus2altAndroid] however.) Toward the middle of this webpage, links to the original Android versions of these works are listed. Their titles all have an "alt" suffix, indicating that these .csd's originally required my "alternative" Android apps.[9]

These .csd's contain minimal commentary inline. (As mentioned above, these examples implement simple trigger buttons only.)

11 - Spiritus2AndroidGUI.csd
  8 sliders, no buttons. No required changes in original .csd to port to HTML5. This is the same composition as in example 6, but with a different and unique set of widgets. (The original .csd, minimally adapted here, was only performable on one of the "alternative" Android apps (see URL immediately above - especially Csound5a.apk and Csound6a.apk; but now also on the Csound for Android app). It is a good template for a multiple-sliders-only GUI. An explanation of what the various controls do (as adapted from the "alternative" Android version) can be found here: http://arthunkins.com/SpiritusSanctus2altAndroid.txt .

12 - SpiritusAndroidGUI.csd
  9 sliders, 12 buttons. No required changes in original .csd to port to HTML5. Note the 3-character limitation for button labels to accommodate, for smartphones, the 6 buttons per row. (As with example 11, this was originally intended for the Csound5a and Csound6a "alternative apps.") A good template for multiple sliders and several rows of buttons.  An explanation of what the various controls do (again from the "alternative" Android version) can be found here: http://arthunkins.com/SpiritusSanctusaltAndroid.txt .

13 - PeaceAndroidGUI.csd
  12 sliders, 9 buttons. Probably the largest cohort of widgets that should be attempted on smartphones. The 4-character limitation on button labels here accommodates, for smartphones, 5 buttons per row. Considerable recoding of original Android .csd, especially involving display of values for sliders 2, 5 and 8 (not possible in "canonical" Android, and dealt with by the one-line Csound console display in the alternative Csound6c app).[10] Demonstrates a mix of sliders displaying their values in a third column, and simpler ones that that do not (and so involve no compositional recoding) . An explanation of the various controls is found here: http://arthunkins.com/PeacealtAndroid.txt . For unknown reasons, this .csd tends to crash after multiple runs on some devices.

14 - Peace2AndroidGUI.csd
  16 sliders, 9 buttons. Probably the largest cohort of widgets that should be attempted (or not?) even on large tablets. Largely unworkable on smartphones due to extreme narrowness of sliders. (This also limits the sliders to single-line labels.) Peace2 is identical to Peace except for three additional sliders (= one additional "voice"). An explanation of the various controls is also found here: http://arthunkins.com/PeacealtAndroid.txt .


FOOTNOTES

[1] His initial HTML-enabled release can be found at:
https://github.com/gogins/csound-extended/releases/tag/v0.1.0-beta
(To download, click on Assets, then CsoundApplication-release.apk.) Note that this release is limited to Android OS 5.0+ only, as is the case with all later Csound for Android apps. 

[2] http://floss.booktype.pro/csound/h-csound-and-html/

[3] The complete Floss Manual Examples are found in the /CsoundQt/bin/Examples folder of the complete Windows installer for Csound.

[4] Michael's tutorial is fairly bare-bones; and his composing methods are both advanced and somewhat unique. This leads to some rather complex examples.

[5] The basic benefit is that of customization: layout and selection/number of widgets, availability of text and labels, and color. (For the advanced programmer, this is just the beginning.) There is also an additional *drawback*: increased latency (in varying degrees), seen especially in button and trackpad clicks. Latency is, on the other hand, *primarily* determined by the power of the Android device itself.

[6] There is no doubt about it; the learning curve for these technologies can be steep. Many potential users may not find the effort worth the trouble, and opt to stay with the "canonical" or "alternative" Csound6 apps. (The lines of added HTML vastly exceed those of an original .csd, as these examples amply demonstrate.) It should be noted, however, that the "alternative" Csound6 apps are built on Csound6.01. Thus they lack certain features of the more recent "canonical" Csound for Android app, currently built on at least Csound6.14.

[7] His trackpad widget is realized via the HTML <canvas> element.

[8] Smartphones exhibit a wide variety of pixel widths. The recommended guidelines are generally for the smallest pixel width you might expect. Some smartphones allow another character or so per label. On a related note, it is wise not to try to squeeze in too much material vertically; leave a little extra space at the bottom of the GUI. Otherwise the lowest portion of the GUI runs the risk of being cut off on devices - especially more recent ones - that take up a portion of the viewport bottom with a row of software buttons. (Hardware buttons, in contrast, do not affect the viewport.) Examples 9 and 14 run this risk, and are thus perhaps "unwise." 

[9] Csound6c includes a single-line Csound console output to feed back to the performer needed data from Csound. Both "canonical" Android Csound and its HTML5-enabled version currently lack implementation of chnset and/or chnexport for on-screen data display purposes. Both Peace alt and Peace2 alt make good use of this feature of Csound6c. Thus significant .csd code had to be rewritten in these two works to cope with this "canonical" limitation. Compare especially the code for sliders 2, 5, and 8 in parallel versions of Peace (PeaceAndroidGUI.csd and PeacealtAndroid.csd), to see how this issue was dealt with in HTML (at least for the time being). I find the current solution acceptable, though not ideal.

[10]  Hopefully, in upcoming builds of Android Csound, chnset and/or chnexport will be implemented.