Conventional musical instruments are provided as either stationary objects or portable devices carried by a user. A user may play a conventional instrument, for example, by pressing keys, plucking strings, etc. Musical instruments have well-established utility in entertainment and artistic pursuits. There is a need for a new type of musical instruments that can provide enhanced entertainment opportunities for individuals and groups of people.
Described herein are embodiments of systems and methods providing a location-aware musical instrument. In some embodiments, individual persons or groups of people can interact with the instrument by physically moving objects (or “nodes”) within a space to change timing, pitch, and/or texture of music generated by the instrument. In certain embodiments, the movable nodes produce visible light (or provide some other sensory feedback) in synchronization with the generated music, resulting in an immersive, entertainment experience for the users.
According to one aspect of the disclosure, a method comprises: receiving, from each of a plurality of moveable nodes, the position of the moveable node within a coordinate space; generating a graph of the moveable nodes based on the received positions; generating an audio-visual composition based on a sweep of the graph over time; and outputting the audio-visual composition.
In some embodiments, generating an audio-visual composition comprises generating a digital music composition. In certain embodiments, generating an audio-visual composition comprises generating light at each of the moveable nodes, wherein the generated light is synchronized to the digital music composition. In particular embodiments, generating an audio-visual composition based on a sweep of the graph over time comprises: sweeping a line across the graph; detecting when the line intersects with points on the graph corresponding to the moveable nodes; generating musical events in response to detecting the intersects. In some embodiments, generating an audio-visual composition based on a sweep of the graph over time comprises sweeping two or more lines across the graph simultaneously to generate musical events.
In particular embodiments, generating the audio-visual composition based on a sweep of the graph over time comprises dividing the coordinate space into a plurality of bins, and assigning, to each of the moveable nodes, a bin selected from the plurality of bins using a quantization process based on the received positions.
According to another aspect of the disclosure, a system comprises: a processor; at least one non-transitory computer-readable memory communicatively coupled to the processor; and processing instructions for a computer program, the processing instructions encoded in the computer-readable memory, the processing instructions, when executed by the processor, operable to perform one or more embodiments of the method disclosed herein.
The foregoing features may be more fully understood from the following description of the drawings in which:
The drawings are not necessarily to scale, or inclusive of all elements of a system, emphasis instead generally being placed upon illustrating the concepts, structures, and techniques sought to be protected herein.
The anchors 102 and movable nodes 104 each have a position within a two-dimensional (2D) coordinate system defined by x-axis 110x and y-axis 110y, as shown. In certain embodiments, the coordinate system (referred to herein as the “active area” 110) may correspond to a floor surface within a building, a ground surface outdoors, or another substantially horizontal planar surface. The positions of the anchors 102 and moveable nodes 104 within the active area 110 may be defined as (x, y) coordinate pairs. For example, anchor 102a may have position (xa, ya) and moveable node 104h may have position (xh, yh), as shown. The position of a given anchor/node within the active area 110 may be defined relative to some fixed point on the body of the anchor/node.
In the example of
The anchors 102 have known, fixed positions within the active area 110, whereas positions of the moveable nodes 104 can change. For example, the anchors 102 may be fixedly attached to mounts, while the moveable nodes 104 may have physical characteristics that allow persons to easily relocate the nodes within the active area 110.
In some embodiments, the positions of the anchors 102 may be determined automatically using a calibration process. In other embodiments, they may be programmed or otherwise configured with the anchors. In certain embodiments, the anchors 102 may be positioned along, or near, the perimeter of the active area 110. Each anchor 102 may broadcast (or “push”) its known position over a wireless channel such that it can be received by movable nodes 104 within the active space 110. In some embodiments, the anchor positions are transmitted over an ultra-wideband (UWB) communication channel provided between the anchors 102 and movable nodes 104.
A movable node 104 can use information transmitted from a plurality of anchors 102 (e.g., two anchors, three anchors, or a greater number of anchors) to calculate its own position within the active area 110. In many embodiments, a movable node 104 uses trilateration of signals based on Time Difference of Arrival (TDOA) to determine its position. In particular, each anchor 102 may broadcast a wireless signal that encodes timing information along with the anchor's position. A moveable node 104 can decode signals received from at least three distinct anchors 102 to determine the node's position in two dimensions by triangulating the signals using TDOA. In some embodiments, a node 104 can determine its position in three dimensions using signals received from at least four distinct anchors 102. Using the aforementioned techniques, a moveable node 104 can calculate its position a continuous or periodic basis.
The moveable nodes 104 can transmit (or “report”) their calculated positions to the coordinator 106 over, for example, the UWB communication channel. The nodes 104 may also communicate with the coordinator 106 via Wi-Fi. For example, a wireless local area network (WLAN) may be formed among the coordinator 106 and moveable nodes 104. In certain embodiments, a moveable node 104 may include components shown in
The coordinator 106 can receive the positions of the moveable nodes 104 and plot the positions on a 2D (or 3D) graph. The coordinator 106 may perform a sweep of the graph over time and, based on the positions of the nodes 104, may generate digital music events that are sent to the DAW 108. In turn, the DAW generates digital music composition which can be converted to audible sound output. Thus, the coordinator 106 and the DAW 108 cooperate to generate a location-based audible music composition. In many embodiments, the generated music events are Musical Instrument Digital Interface (MIDI) events, which are sometimes referred to as “bangs” or “triggers.” The DAW 108 receives the MIDI event data from the coordinator 106 and may use various control mechanisms to vary the timing, pitch, and/or texture of music based on the MIDI event data.
The audible sound output may be output via speakers 112 such that it can be heard by persons within and about the active area 110. In some embodiments, the speakers 112 are coupled to the DAW 108. In other embodiments, the speakers 112 may be coupled to the coordinator 106. Although two speakers 112 are shown in
In some embodiments, the DAW 108 may be incorporated into the coordinator 106. For example, the DAW 108 may correspond to MIDI-capable software running on the coordinator computer. In particular embodiments, the coordinator 106 may be provided as a laptop computer.
The physical position of the moveable nodes 104 within the active space 110 determines the timing, pitch, or texture, etc. of discrete “musical incidents” within the generated composition. The term “musical incident” may refer to an individual musical note, to a combination of notes (i.e., a chord), or to a digital music sample. In some embodiments, moving a node 104 to a higher y-axis value may raise the pitch of a musical incident within the musical composition, whereas moving the node 104 to a higher x-axis value may cause the musical incident to occur at a later point in time within the composition. Thus, the system 100 can function as a location-aware musical instrument, where the nodes 104 can be rearranged along multiple physical axes to change the musical composition. One or more persons can interact with the system 100 to “play” the instrument by changing the physical arrangement and organization of the nodes 104 in physical space.
In some embodiments, the coordinator 106 transmits (e.g., via the WLAN) sensory feedback control information to the moveable nodes 104 based on the position of individual nodes 104 and/or the overall arrangement of nodes 104. In response, the nodes 104 may generate sensory feedback, such as sound, light, or haptic feedback. In one example, the coordinator 106 directs each node 104 to produce light, sound, or other sensory feedback at the point in time when the corresponding musical incident occurs within the audible musical composition. In this way, a person can see and hear “time” moving sequentially across the active space 110. In some embodiments, the color or duration of light produced by a node 104 may be varied based on some quantitative aspect of the digital music composition.
In certain embodiments, coordinator 106 may include components shown in
The UWB transceiver 202 is configured to receive signals transmitted by anchors (e.g., anchors 102 in
The WLAN transceiver 206 is configured for wireless networking with a coordinator (e.g., coordinator 106 in
The sensory feedback module 208 controls the light source 210 and/or other sensory feedback mechanisms 112 based on the control information received from the coordinator. For example, the coordinator may communicate a LED program data (e.g., a sequence of commands such as blink, turn red, pulse blue, slow fade, etc.) to the node 202, which in turn sends this data to LED control hardware within the node 200. The LED control hardware may receive the LED program data and translate it into electronic pulses causing individual LEDs to produce light.
In some embodiments, the moveable node 200 is provided within a housing formed of plastic (e.g., high density polyethylene) or other rigid material. In particular embodiments, the housing is cube-shaped with the length of each side being approximately 17″.
The UWB transceiver 301 receives the positions of moveable nodes (e.g., nodes 104 in
The generated music events may be sent to a digital audio workstation (e.g., DAW 108 in
In the embodiment shown, the x-axis 402x may represent time and the y-axis 402 may be represent pitch, texture, or some other musical quality. A line (sometimes referred to as a “transport”) 404 may be swept across the graph 400 over time, i.e., from left to right starting at x=0. The sweep may stop when the transport 400 reaches some maximum position along the x-axis 402x (e.g., a maximum position defined by the physical size of the active area). In many embodiments, the sweep repeats (or “loops”) when the transport reaches the maximum x-axis value.
As the transport 404 intersects (or “collides”) with a moveable node position 406, a music event (or “bang”) may be triggered. The music event may include information about pitch, texture, etc. based on the node position 406 along the y-axis 402y. In the example of
Although a 2D graph is shown in the example in
Referring to
Referring to
All references cited herein are hereby incorporated herein by reference in their entirety.
Having described certain embodiments, which serve to illustrate various concepts, structures, and techniques sought to be protected herein, it will be apparent to those of ordinary skill in the art that other embodiments incorporating these concepts, structures, and techniques may be used. Elements of different embodiments described hereinabove may be combined to form other embodiments not specifically set forth above and, further, elements described in the context of a single embodiment may be provided separately or in any suitable sub-combination. Accordingly, it is submitted that the scope of protection sought herein should not be limited to the described embodiments but rather should be limited only by the spirit and scope of the following claims.
Number | Name | Date | Kind |
---|---|---|---|
4801141 | Rumsey | Jan 1989 | A |
4836075 | Armstrong | Jun 1989 | A |
5541358 | Wheaton et al. | Jul 1996 | A |
6990453 | Wang | Jan 2006 | B2 |
7750224 | Rav-Niv | Jul 2010 | B1 |
8539368 | Nam | Sep 2013 | B2 |
8686272 | Bonet | Apr 2014 | B2 |
20060075885 | Bailey | Apr 2006 | A1 |
20110167988 | Berkovitz | Jul 2011 | A1 |
20110191674 | Rawley et al. | Aug 2011 | A1 |
20130305905 | Barkley | Nov 2013 | A1 |
20160203805 | Strachan | Jul 2016 | A1 |
Number | Date | Country |
---|---|---|
0264782 | Apr 1988 | EP |
Entry |
---|
Chafe, “Case Studies of Physical Models in Music Composition”, In Proc. 18th Intl. Cong. Acoustics (ICA), (Apr. 2004) (5 pages). |
Hays + Ryan Holladay, Webpage, Retrieved from: https://www.hrholladay.com/, Printed Mar. 7, 2018. |
Ballston, Hays + Ryan Holladay, “Site:WA+FC(Ballston),” http://www.ballstonbid.com/art-projects/site-wa-fc-ballston, Copyright 2018. |
Pozyx Accurate Positioning Documentation, Webpage, Retrieved from: https://www.pozyx.io/Documentation, Printed Mar. 7, 2018. |
Hays + Ryan Holladay Location Aware Music, Webpage, Retrieved from: http://www.hrholladay.com/location-aware-music/, Printed Mar. 7, 2018. |