Design for the Real World

Design for the Real World: BeatRockers @ the Lavelle School

In this multidisciplinary course, students will work collaboratively to research, design, and develop a system of client-centered data-management, musical interfaces and interactive learning tools for the Beat Rocker beat boxing program at the Lavelle School for the Blind. The Beat Rocker Program incorporates a unique beat-boxing/speech therapy curriculum and children that are engaged in the program excel in both areas. Students in the class are expected to gain practical experience in user research/testing, human-centered design and the prototyping process. The class will be split into three sections: 1) Client/User Research, 2) Design and Prototyping 3) Testing

012 Final Presentation

https://docs.google.com/a/nyu.edu/presentation/d/1KCP1MYXuvhBztsGdAXbiOp77Amqf5Ruuk2K2o5ddGd0/edit?usp=sharing

User Testing 2

Date: Dec 7th

Time: 11:00 ~ 12:30

We tested our second prototype in the class. Unfortunately, we missed the first class, therefore we are only able to test with the second class.

There are six students in the class, but only 2 are actively participating the class. We have several findings listed below:

  1. When they want to clear the recordings, they often accidentally press the 2 buttons together, which initiates the first track
  2. Sometimes, when a person record a loooong track, the mic will only get seconds of it (longer than the audio bar in the program)
  3. Sometimes they press the button and record a very short sound by accident
  4. Even though we have bumps, they still don’t seems to recognize the orientation.

After the class, we tested the mic with the instructors, and they gave us some very useful feedback:

1. There is a small delay between tracks. For example, even if when you are adding another track and the beats match, the result from the program has a delay
2. After 3 or 4 tracks, the track is full of sound and cannot be listened very clearly. Some sounds are distorted. They suggested us having a compressor or limiter on each track. Or maybe lower the volume of previous recorded track after a new one is recorded.

3. We can have a thumb anchor on it to haptically show how they should hold it

4. Mark suggests that when press the second button, we only delete 1 track but not the whole thing so that they won’t need to start it from beginning. And when you hold it for more than 3s, it will clear everything. Because start over again is frustrating for kids

5. This tool is very useful when teaching, because we don’t need another person to take care of the loop machine in the front of classroom

6. This time the mic isn’t picking up a lot of noise around. It’s good.

Version 2.0 Demo

011 pre-Final Presentation and Storyboard

Presentation

The link to presentation

Storyboard

15253491_754831031334985_3164142021752770867_n

img_0920

***********************************************************************

010 User Testing II

Summary of User Testing I

No. of users tested: 4

Environmental implications

  • Very noisy environment
  • Students require 1 on one attention – usually instructor works with one student and the others are meant to listen (they sometimes don’t do this)
  • Feedback from the speakers was a slight issue – we had to move to the back of the room
    • Need to ensure cables are longer
  • Things need to be cleared out the way for the students to be able to walk freely – we need to minimize clutter.

Observations

  • Student 1:
    • Initially really excited to use it
    • Pressed both button repeatedly initially to figure out the shape
    • We didn’t give instructions clearly, took over 5mins and several attempts from both instructors and ourselves
    • She held the button to record – understanding the click-once was not intuitive
    • Managed to record a first loop but then would struggle to understand
    • Did not want to keep using the mic after 8 attempts
    • Struggled to orient the mic initially – need to have some reference for direction
    • Buttons were close together she sometimes pressed the wrong one
    • When she got to later recordings there was a lot of feedback as we were close to the speaker
  • Student 2:
    • She understood the functionality immediately
    • Very excited and delighted when she heard her own voice
    • Wanted to use it again – and take it home
    • Explained the button functionality perfectly
  • Student 3:
    • Very excited and says he wants to use it again
    • He was able to tell the different color buttons
    • The way Kayla talked to her is different
    • Explained the button functionality quite well – understood the colours on the buttons best
  • Student 4:
    • She didn’t want to use it again
    • But she has a mic at home
    • She doesn’t want to hold the mic with 2 hands so the instructor press the button for her (so we don’t know if she understand the logic)
    • Mark suggests that we should design it to be used by 1 or 2 hands
  • Student 5:
    • very excited about the device but did not get the chance to use it due to technical issues
    • Expressed desire to use to it – and wanted one for home

Next steps for user testing II

  • Re-design layout of buttons
    • Make main button bigger
    • Put reset button on back
  • Re-program the logic
    • Hold to record, then hold to overdub
    • Hold reset button for 3 sec to reset
  • Create more stable housing
  • Extend wires
  • Add physical marker to mic to help low vision students to orient it

After the redesign we intend to repeat our user testing procedure on December 5th to validate it.

***********************************************************************

009 Final Documentation for Continuation Draft

  1. Research

■ Initial research/secondary research

In class, first we learn how to conduct a research, and by the readings, we also gain knowledge about melody intonation therapy and other things related to blind and low vision people.

From the talk of beatrockers, we get some basic knowledge of how beatboxing can help blind people with not only verbal but different aspects, we learned what their initial expectation was, and what we needed to know about the class they teach at Lavelle School.

From the lecture of Leona, we are able to actually see a blind person using accessible software in front of us, thus we can gain insights from observation and talking to her. We also learned some helpful hints to design an accessible system.

In the research paper, we explored topics about user interface for low vision people, and also haptic musical interface. Haptic interface can avoid the problem of dimension loss of  designing a visual interface at first and translate it into accessibility.

In short, we learned different techniques to design a good interface for blind or low vision people.

■ User Testing

There are 2 main things we want to test in the user testing:

  1. Will they want to use our design?
  2. Is the button logic clear and easy enough for them?

During the first user testing, we found out that the students are really excited about our microphone, and the button logic is a little bit too hard for students to learn. We originally designed it as press once to start recording, and then press again to stop and play. But it is not very intuitive for them, even the instructor said it’s not very intuitive. Also, the wire we built was too short for the classroom, therefore they need to change seats during the class, which is very interrupting. We fix all the these problem in our second version of design by changing the button logic to press and hold for recording, and making longer wires.

  1. Technical Specs

■ Materials Used

There are two parts of our system design, hardware part and software part. For hardware, we use a dynamic microphone for base, and a modified mic holder with buttons and wires connected to an Arduino on it.

15319549_10210055940132282_884832941_n

For the software part, we use Max to receive data from Arduino and function as a loop machine.

14940134_10157689221245331_2752301006101383945_o

■ Stack

Arduino, Max/MSP, and Maya

■ Bills of Materials

Dynamic Microphone $20
3D Printing Material $10
Buttons $5
Arduino $25
Miscellaneous $5
  1. Process Outline

■  Beginning idea

At the beginning, we came up with 2 ideas: one microphone for all students and a lap interface for higher functioning students. The mic would be a handheld loop machine itself, with simple sound fx buttons on it.

The lap interface would be a modular thing for kids; They can choose to use it or only use the mic without it. We thought that the mic itself would not be enough of music production for higher functioning kids.

■ Midterm presentation

We did a first low-fidelity prototype for midterm presentation. We further designed  about how the buttons on mic will work as a loop machine, and how the modular lap interface will function. The first part of lap interface will only be 4 pads working like a drum pad or midi controller, but users can record their own sample. If user is not satisfied with it, one can plug into another modular add-on part for it: a 4-channel slider for each pad.

The prototype does not really work, but we demo the logic of loop machine with a Max patch, and the look of lap interface by making what it would look like when we’re done.

The guest critiques doubted whether our loop button logic is clear and easy enough to learn, and if we really have time and effort to build both the mic and lap interface.

■ User testing 1

We build an actually working prototype before the first user testing. We did not have time to do both, so we focused on the mic loop machine. Although the prototype is working fine, the wires are not well placed, and we need to plug the mic into our computer with the Max code running on it, so it’s still not in a good shape yet.

■ Prototype 2

Gotten the feedback from user testing 1, we changed a lot of things on our prototype. The most significant change the button position, and the button logic for loop machine.

008 Final Public Documentation Draft

15310486_754897891328299_328705193_n

 

physical-interface-pre-final

Project Pitch

Our goal was to design and create a physical interface that helps enhance the Beat Rockers beat boxing program at the Lavelle School for the Blind. Our solution is a hand help mic looper station we call “Loopi”. It allows students to record themselves and overdub allowing for independent beat making on the fly. The interface is designed for the visually impaired but can be used by anyone. The interface has 2 buttons – one that records and overdubs when held down and a second button that resets everything. The current version of the interface uses software coded in MAX MSP and Arduino to operate.

 

Goals

Our main goal is to ensure that our interface helps to reinforce the current aim of the beat rockers program to help the students of Lavelle School. After several discussions with the beat rockers, and after reviewing their curriculum we identified the following as the core goals of their program:

  • Help the students with their speech development
    • Loopi allows students to practice their beat boxing. It requires verbal input which forces students to practice
  • Help the students explore their musical expression
    • Loopi allows for infinite overdubbing so students can play and layer to their hearts content
  • Help the students to actively listen, to stay mentally engaged and communicating
    • Loopi plugs into any audio system so users can listen on headphones or on class room speakers. Students can use it privately or communally. If there is too much sound in the room it degrades the quality of the recording which encourages other students to be quiet while one is using the device
  • Help the students improve their emotional and social intelligence (group activities)
    • Loopi can be easily used for group activities and passed around a room.
  • Help foster a sense of leadership, and autonomy
    • Loopi is very simple so most students will be able to operate it without the help of an instructor.

Process

Our process is outlined below:

  1. Conduct User Research
    1. Understand current state of beat rockers program
    2. Understand the ideal state of where they want the program to be
    3. Understand needs of all stakeholders
  2. Define Goals
  3. Create alternatives
  4. Compare alternatives against goals to pick best one
  5. Build prototype
  6. Test Prototype, get user feedback
  7. Iterate on design
  8. Repeat steps 5-7 as necessary
  9. Provide recommendation

***********************************************************************

Week of Nov 10th:

To do list:

Button and USB extension -> Design Model -> print -> fit -> test -> make it stable

___________________________________________________________________________

Lavelle School User Testing (raw notes)

Date: Nov 7.

Time: 10:45 ~ 12:30

We tested our “Loopy mic” prototype with the students.

In the first class, one of the students who has steady beats volunteered to try out our mic. She had some trouble learning how to use it because she misunderstood the function of buttons.

But in the second class, 3 of the students tried out the mic and was happy about it. We told them to feel the 2 buttons first, and explained how to use it slowly, so that the students are able to pick up very quickly. Students love it. When they hear the sound of their own coming from the speaker in a loop, they were crazy. They all think it’s fun and will definitely want to use it again.

Notes:

  • They hold mic very close to their mouth
  • Usually the instructors setup the mic for them and pass it to them
  • Holding the button while recording seems to be more natural for most of them (even for the instructors)
  • They are confused when they are recording and did not hear the sound from speaker instantly just like normal mic
  • When explaining, they are confused with the wording of “stop the loop” and “stop the whole thing”
  • Mark brought up that we shouldn’t describe the function of the bump button as record and stop, but come up with a way to explain over dubbing because saying stop is confusing to the students.
  • They don’t care about precise timing  (maybe the higher function kids will be better?)
  • They can feel the difference between 2 buttons (either by texture or position, or color)
  • They often forget to press the button again when they finish recording
  • Need a signal whether it’s recording or not?! (it was suggested to be a hapic or vibration notifier)
  • They are excited to hear their own voice in playback
  • They need assistance in counting rhythm
  • feedback noise: maybe we should try cutting off 3k Hz to avoid feedback
  • Some students want to use it by one hand but could not do it ( Mark suggests that we design it to be used by both 1 hand and 2 hands)
  • When we use loop machine, we usually record a short period sample and loop it again and again, but they are not used to it. They sing like normal for a long time before they want to stop the recording.
  • Need to be reminded to press the bump button to pause the recording and to start another for overdubbing.
  • Contrasting colors worked well for the low vision students in the class
  • Hand on hand touch is important in teaching the students button placement and how to use the buttons.
  • Most students were prompted beforehand about what they will be doing by Kayla. She talked to them about looping and by using the mic they get to do it on their own.
  • Loopy mic, allows for multiple points of learning according to the teach guy in the classroom (counting, button function, coordination, timing, etc)
  • Average tries it took for the student to get an idea of the buttons was 3-4, but no one was 100% sure of it.

 

Feedback from Claire:

-Use a directional mic.

-Incorporate record monitoring so users know what they are recording.

-go wireless ASAP

 

Week of Nov 3rd:

For the past week, we have made a working prototype.
We 3D print a mic handle with 2 buttons on it, connect it to Arduino and computer.
(mic)
The 2 buttons on the mic handle are responsible for recording loops,  they are positioned a bit lower than usual, so that the students can hold it with one hand and press the buttons with the other hand.
(buttons)
 Not only they have different orientations, but they also have different texture because the green one has a dot on it.
The logic of loop machine:
upper button: start recording/ stop recording and play/ start recording … etc
lower button: stop everything
(Arduino)
Arduino is responsible for detecting whether the 2 buttons are pressed.
(Computer)

We use MaxMSP to build the loop machine.This patch recieves data from the arduino via the serial port. It unpacks the data and the uses the following logic:

Button 1

1st press -> start recording
2nd press -> set loop length and loop recorded audio
3rd press -> overdub
4th press -> stop overdub and continue to play
5th press -> same as 3rd press
6th press -> same as 4th press
etc

Button 2

1st press -> Reset everything

These trigger a looper station in max that stores and creates the files.

11/2/16

From the midterm to now, we have figured out how to do a double press on a button (one button press will record and second press will over dub), 3d print a prototype of the microphone sleeve which will house the 2 buttons, tried out the max patch with the buttons, and figured out how to hide the Arduino and wiring temporarily for the prototyping (put it all in a box where the mic wires come out through a hole along with the computer cable)

making the 3d model based off of the microphone and button measurements
Test print to see if the size would work for the sleeve
sleeve printing for 2.5 hrs
The arduino setup connected to a max patch for looping

Nicola demoing how the patch works with the two button setup

Oct 31st

We made a 3D model for microphone in Maya according to the measurement. Here is the screen shot.

capture

 

Oct 30th

We first measured the microphone and the button in order to make 3D model based on real world scale

img_0471 img_0507

006 Timeline and team roles

HIGH LEVEL TASKS

*just made this up so feel free to change anything… just getting some terms to talk about in the roles

  1. Mic interface
    1. Create soft designs
      1. Get feedback from user / classmates
    2. Prototype ideas
    3. Test ideas with user
    4. Iterate on design
  2. Lap interface
    1. Create soft designs
      1. Get feedback from user / classmates
    2. Prototype ideas
    3. Test ideas with user
    4. Iterate on design
  3. Communicating with data management team

TEAM ROLES

*no idea what the best way is to split it up. I figure we can do whatever we need to when the time comes – this is just for class hw sake. Also, because i wasn’t in class, not sure about how long these should be – plz let me know if it shld be longer / or in another format

Ada Zhao: Design, test and make the Mic Interface. Part 1 above.

Ishaan Chaudhary: Lead Lap Interface Designer. Will help team by maintaining ergonomic focus in design stage. Will be focused on items 1.1, 2.1, 2.1.1, 2.2, 2.3, 2.4 above.

Kelly Chang: Focus on implementation

Nicola Carpenggiani:  Prototype of mic looper and lap, testing mic interface.

Satbir Multani: Focused on user testing/feedback to inform the build out of the mic/ lap pad, work on fabrication of the mic/lap pad, communicate with the data management team

SCHEDULE

10/20 Design and paper prototyping, Low-fidelity prototype
10/27 Midterm Presentation

(Focus more on microphone. Knowing how to implement microphone. Lap prototype can be low-fidelity, but the concept of microphone must be concrete and know how to build it and test it.)

11/3 Field Testing 1 (A working prototype of microphone that we can test)
11/10 Iterate on prototype we built. Start to think about lap prototype.
11/17 Building high-fidelity prototype for mic and low-fidelity for lap
11/24 Field Testing 2

(improved prototype for mic and working prototype for lap)

12/1 High-fidelity prototype for both mic and lap
12/8 Final Presentations

005 Group Paper Prototype

Link to midterm presentation

PDF: midterm-presentation

Our idea in general:

capture

  • Handheld mic looper station
  • Modular lap station
    • Trigger + fader station

Prototypes of the Mic:

Prototype of the lap based interface – 1

1 2

Prototype of the lap based interface – 2:

3

004 Research Paper

Definition

Humans have a multitude senses. Sight, hearing, taste, smell and touch are the five traditionally recognized senses. People usually use these five senses to get input from the outside world. However, how would visually impaired or low vision people get input with the rest four senses, especially hearing and touch? What can we do, as designers, to make them utilize the other senses to the maximum extent to have an equal playing field with their peers.

In this paper, I will research how changes are made for visually impaired or low vision people to interact with computers in order to conclude a general design principle for them. I will focus on how the information is conveyed through other senses. I think this research topic can help our group design more meaningful interactions.

Historical View

The traditional solution towards how to make computer accessible to blind people is interpreting the regular graphical user interface through sound or tangible interface. The corresponding technology is screen reader and refreshable braille display.

Jim Thatcher created the first screen reader at IBM, called IBM Screen Reader[1], which transfers the text into speech. The most commonly used screen reader nowadays is JAWS (Job Access With Speech), initially launched in 1995. It reads the content line by line, level by level.

After 1980s, refreshable braille display devices enable visually impaired people to touch the content in the computer by means of round-tipped pins raised through holes in a flat surface [2]. The braille display usually has built-in buttons as shown in the image below.

Technology Intervention

Recently, with the rapid development of technology, there are far more than two ways to interact with computers.

Virtual Audio Reality(VAR)

VAR enables visually impaired people to navigate the computer as they do explore rooms they do not know in their everyday life [3] since human beings are able to use directional hearing for orientations. It uses binaural simulation techniques and room acoustics [3] to recreate sound in 3D environment. In order to interact back to the computer, the user uses a joystick to move the cursor. A good user experience would require a pair of headphones in this case. Compared with using screen reader and keyboard to go over every button on the screen one by one, VAR will save much more time if the user has previous experience using a software so that he can remember the location of a button and get to it based on the sound.

Haptics Interfaces

Another special approach is haptics interfaces. An example would be “the PHANToM”, a high performance, 3D haptic interface from SensAble Technologies(video link) [4]. With a mechanical arm and a pen on top connected to computer, users are able to “feel” the graphic user interface with pull or push feeling given by the pen. Force feedback joysticks use the same principle. In the past, when people touch something, the information he gets is often vertical to the surface. Now, with force feedback, more information is added along X and Y axis.

Statement

Here are some design principles that I conclude through the study of interaction methods listed above. I think they can be helpful for designing the physical interface.

Hearing

Judging from the screen reader, if we are going to use sound to deliver information, it is important to keep things in order. Especially when the information get complicated, it would be helpful to keep it in layers just like the top bar menu of a software in computer. Keeping things organized will give convey more understandable through hearing.

The biggest problem for visually impaired people, in my opinion, is having no sense of which state they are currently at since they can’t see any visual indication. If the user has access to headphones, 3D audio can bring up efficiency a lot. Sound can convey more information than the actual content if we consider the location associated with it.

Touch

If we take a look at the buttons on the braille display, we can find the buttons are relatively large compared with, let’s say, buttons on a camera. The size also matters when designing tangible interface for blind people. Alongside with the size, shape, orientation and texture are also important to tell the difference between multiple objects. In terms of tangible interaction tools, a force feedback can help indicate the state as well.

References

[1] History of Accessible Technology

https://cs.stanford.edu/people/eroberts/courses/soco/projects/2005-06/accessibility/firstwave.html

[2] Refreshable Braille Display Wikipedia Page

https://en.wikipedia.org/wiki/Refreshable_braille_display

[3] 3D Audio Interface for the Blind

http://www.icad.org/websiteV2.0/Conferences/ICAD2003/paper/68%20Frauenberger.pdf

[4] Using Haptics in Computer Interface for Blind People

http://www.arkiv.certec.lth.se/doc/usinghapticsin/usinghapticsin.pdf

003 User Journey

The whole project would contains three parts: a server, a hardware for students and a mobile app for other users.

img_0067 img_0068fullsizerender-2

002 User Profiles

Potential Users:

Lavelle School for the Blind Students

Beat Rockers Instructors

Kids’ Parents

Speech Therapy

Potential Beat Rocker Customers

Individual User Profile

User Profile: Lavelle School for the Blind Students

Name: Alex

Age: 8

Occupation: Student

Needs: Something that I can refer to as a tutorial when I wants to practice but the instructors are not with me. Something that can give me feedback on his practice

Challenges: The feedback needs to be meaningful for me. The interface cannot be just a touch screen.

Motivations: I want to make better B-Box in the next class. I want to know how to practice at home.

User Profile: Beat Rockers Instructor

Name: Anna

Age: 25

Occupation: B-Boxer, Bear Rockers Instructor

Needs: Needs a tool to track students’ progress. It is better if it can help me identify students who need more help. It can also be a communication tool between me and speech therapist.

Challenges: Needs to be easy to use, easy to start.

Motivations: I want to ensure that my classes are efficient. I want to know that these kids are indeed making progress in speeching

User Profile: Potential Beat Rocker Customers

Name: Ben

Age: 52

Occupation: Headmaster of a Blind School

Needs: Needs a tool to see whether B-Box and what Beat Rockers did are really effective for children with speech problems.

Challenges: Needs to understand because I am a bit old

Motivations: I want to bring Beat Rockers into my school but I need to ensure that it actually works first and then persuade others.