I recently participated in the Smart Craft Studio camp and spent 3 weeks in Hida Japan for a series of intensive wood joinery and internet of things(IoT) workshops. During one of the IoT workshop, Shingeru Kobayashi sensei, the creator of Arduino Fio, walked us through how to set our Raspberry up with Amazon Alexa. It was really exciting when all of us heard Alexa’s greeting for the first time.
“I am interested in collaborating with Alexa on something”, I said it to myself. The first thing that came to mind was King Curtis’s Memphis Soul Stew. I was wondering if we (me & Alexa) can build something up together visually like how King Curtis built up the intro in Memphis Soul Stew.
Alexa is on IFTTT. The first pipeline I thought of is Alexa to Google Drive to Unity.
It works, but not ideal, here is why. I quickly loaded an demo scene and hided all the Game Object by setting their mesh renderer to false. By saying specific sentence to Alexa, a new row will be added to a spreadsheet on Google Drive. Then Unity will show Game Objects based on the data in the CVS file. The CVS file doesn’t get updated quick enough. With the current setup, going from the verbal commend to IFTTT to Google Drive to Unity takes about 30 secs. This is not ideal… The wait time is too long.
The next pipeline I am going to try is 8266 with IFTTT’s Maker Channel + beebotte. It will be faster, theoretically speaking….. And It did, after testing. However, it doesn’t feel like we are collaborating… We are more like Tony Stark and Jarvis where I am giving some kind of formatted language/verbal instructions, and Jarvis execute them…
“Comfortably lying on a couch in any position you want, stripping a Cardboard VR on your face, bearing your arms behind your head, and let your tongue take over all the snapping, just like a hungry frog boss.”
Here comes my next experiment. The tongue controller is already working, the player has to reach his/her tongue out and touch the controller, and it will register a hit in the game. The feedback, SFX and Visual, has to be satisfying to make it a worthwhile experience.
Makey Makey Go is able to simulate a touch easily with just 1 contact (capacitive touch), in this case, it is the player’s tongue. This setup will require a USB adaptor from Makey Makey Go to the android phone.
I am experimenting with designing my own Google Cardboard, and possibly using for BFADT promotion. I need a simple, 1-sheet, and custom designed solution for the Google Cardboard, so I can brand it with Parsons BFADT and hand them out at design week. 04/30/2016 This mostly not going to happen, but I will make this anyway into promotional material for our program and the new minor.
Are the optics used in Google Cardboard the same one used in the paper binocular telescope? YES. (objective lens-convex lens) I ordered one and it works perfectly! 3X Magnification, 25 mm Lenses give about 3 inches focal length.
First Attempt: My first attempt creating a prototype with a sheet of cardstock paper and our newly invested Cricut Air. The template I used is inspired by VRSHEET. I found out about it after reading the magazine バーチャルリアリティで二次元へ行ける本, and love the idea of a 1-sheet VR. The initial experiment was a modification of their sample template with improved support that will lock the phone in place with a rubber band.
* It is not completely cut through with the given setting due to the thickness of the cardstock paper. I am going to try double-cut in my next attempt. Otherwise, with some cleaning up, the quality looks great. Most importantly after putting in the lens, it works beautifully.
Precedents & Reference: xReality (週刊朝日Mook) Monoscopic VR Viewer by ハコスコ:
The new Ghostbusters is coming out, I saw the first trailer and decided to watch the original movies for nostalgic reasons. It brought back so many good memories, went on eBay immediately to browse vintage Ghostbusters toys. Well, can’t really afford any, so I decided to make some toys myself, in VR. I wanted to make a Kekkaishi play experience a long time ago, thought it was great for Kinect sensor. Maybe this is an opportunity to bring the two together, two ghosts with 1 stone, but instead of the gesture input, I am going to see if I can do it all with just gaze. Well, at this point, I am more interested in recreating the visual effect of the beam and ghost A.I.
Experiment with the beam.
Added particle effects, point light halo, city, and stuff.
Went through a few visual iterations and decided to go everything low-poly, got a few low-poly assets from the asset store including a nicely built low-poly city block. It helps me create coherence in visuals without too many 3D works. Play with the beam more, added a thinner blue beam in the orangy beam. I also wrote a small code block to generate the sound effects for the beam so the electricity/zap sound is continuous till player lost trace of the ghost. I like what I have so far, it is a wrap for what I wanted to achieve in this experiment.
The next step is to create an entire level with a few of these ghost with different color, movement, and abilities. The boss – The Stay Puft Marshmallow Man. Already had the fight sequence planned out, it is going to be an on-rails experience with a time-based shooting gallery mechanic.
The design of the main lantern of this year’s Taiwan Lantern Festival, Fusho Monkey, has gone viral on Taiwanese social media. The comments are mostly negative due to the weird aesthetic that are drastically different from the previous main lanterns. It was courageous for the designer behind the new lantern to risk for innovation, but it seems like his artist side took over the design process and left the audience stunned with a poorly executed piece that supposed to represent the highest craftsmanship and appreciation for aesthetic in Taiwan.
Fusho Monkey has become the subject of fan arts. When I saw it for the first time, I thought the design failed as a main lantern for the prominent Taiwan Lantern Festival, but the character is interesting. It instantly reminded me of Pokemon, and I finally found some time to put together this small Google Carboard experience that allows players battle Fusho Monkey as Pikachu and Satoshi (Ash). This is not a complete game!! You can zap Fusho Monkey to your heart’s content in this little play.
Made with Unity 3D and Google Cardboard. Ash 3D model is originally made by MMDSATOSHI for MMD and was ported into Unity 3D and applied mecanim animations on it. I built Fusho Monkey myself with simple blendshapes motions. Background music is from Pokemon X/Y. The forest is from the Unity asset store. This is a personal project as a prove of concept, not meant for commercial use. You will need a Google Cardboard to fully enjoy the VR experience.
I was recently invited to teach a workshop on Immersive Storytelling and Virtual Reality at Shih Chien University. Unlike other media technology advancement in the past, designing and producing for VR shares similar tools as those ones used by current digital media designers and producers. The only technical hurdle might be picking up Unity 3D or Unreal Engine during the transition.
With that in mind, I put together a set of tutorial scenarios that are based on various mechanics we found in VR explorations – 360 video playback, virtual tour, and gaze-centric interaction. These scenarios can be utilized for making fun and immersive Google Cardboard VR experiences without writing a line of code. These scenarios also set artists and designers free from the technical challenges and enable them to focus more on creating unique and transcendental aesthetics for that imagined world of theirs.
The week-long workshop was very productive and quality of works were amazing. Workshop participants, mostly sophomores and juniors media design students, were encouraged to bring their media production expertise to this real-time and interactive new media frontier. In this particular mixed of talents, there are glitched video artists, experimental documentarians, 2D illustrators, poets, sound designers, and Cinema 4D magicians. They successfully brought their screen-based works into omni-immersive narrative explorations and rides in this workshop.
I am very proud of all the workshop participants and grateful for the opportunity to test out these tutorial scenarios. I am excited to bring the results back to Parsons and contribute them to our 2016 Fall minor of the same name. Before that I will run the workshop a few more times at Parsons, especially the VR extravaganza on π day with our Design & Technology students and a group of middle school minecraft experts!
實踐大學媒體傳達設計學系 創新媒體國際設計工作營！六天工作營（三天課程＋三天取材製作）產出4組VR虛擬實境作品（每組均包含組員個人作品）授課教授：Parsons School of Design 設計與科技系 Design and Technology 系主任 Kyle Li 客座講師 Shiny Lee
moments from the workshop:
↑↑↑↑Food Hunter: Roller coaster + grocery shopping + gourmet cooking.
↑↑↑↑Drift In: A realistic but glitched world you can’t escape.
↑↑↑↑Bag: Life inside bags.
↑↑↑↑Rainbow: Walking into an illustrator’s rendering of a poetic world.
The word perfume derives from the Latin perfumare, meaning “to smoke through.” Perfumery, or the art of making perfumes, began in ancient Mesopotamia and Egypt, and was further refined by the Romans and Persians.
The world’s first-recorded chemist is considered to be a woman named Tapputi, a perfume maker who was mentioned in a cuneiform tablet from the 2nd millennium BC in Mesopotamia. She distilled flowers, oil, and calamus with other aromatics, then filtered and put them back in the still several times.
The art of perfumery prospered in Renaissance Italy, and in the 16th century, Italian refinements were taken to France by Catherine de’ Medici’s personal perfumer, Rene the Florentine (Renato il fiorentino). His laboratory was connected with her apartments by a secret passageway, so that no formulae could be stolen en route. Thanks to Rene, France quickly became one of the European centers of perfume and cosmetics manufacture. Cultivation of flowers for their perfume essence, which had begun in the 14th century, grew into a major industry in the south of France.
Copied from the movie Perfume:
Now, pay careful attention to what I tell you. Just like musical chord, a perfume chord contains four essences or notes, carefully selected for their harmonic affinity. Each perfume contains three chords: The head, the heart and the base, necessitating twelve notes in all. The head note contains the first impression, lasting a few minutes before going away to the heart chord, the theme of the perfume, lasting several hours. Finally, the base chord (la note de fond), the trail of the perfume, lasting several days.
Mind you, the ancient Egyptians believed that one can only create a true original perfume by adding an extra note, one final essence that will ring out and dominate the others.
Twelve essences could be identified, but the 13th, the vital one, could never be determined.
I taught a 3-day workshop on Projection Mapping to a group of media design students at Shih Chien University’s Kaohsiung Campus. The goal of this workshop is to broaden their design horizon by learning something new and relevant to their background. According to my initial conversation with the program coordinator at FSDC, I learned that all the students have backgrounds in Illustration, Photography, Graphic Design, Motion Graphics, and have very little experience in programming. The first idea that came to my mind was Projection Mapping.
Other ideas were Littlebits or Makey Makey. I have used these two technologies extensively in my own teaching at Parsons. However, building interfaces with these two technologies lead to interaction and eventually software design. There is very little room, due to the technical hurdle, for this group of students to experiment after the workshop. I ditched the first idea and did brainstorming on their backgrounds. I drilled deeper into the common attributes and characteristics of those mediums. I found that “the presentation” of these mediums can be the focus of the workshop, and after some evaluations on available tools and software, I decided to run the workshop on Projection Mapping – a technology-less and code-less presentation method that can bring their works to life.
Workshop: 1. Design & Technology Lecture: This is an introduction to the design processes and theories of design & technology. It shows how design and technology go hand-in-hand and sparks innovations for each other throughout the history of arts and crafts in America. This lecture helps students understand their design education system and encourages them to explore new technologies, platforms, mediums for the growth of their own design/craft. The lecture also covers examples of various trends in New York including Bio Art, Internet of Things, Serious Game Design, Virtual Reality, Augmented Reality, Holograph, Fashion Tech, and Projection Mapping.
2. Projection Mapping: Also known as video mapping and spatial augmented reality, is a projection technology used to turn objects, often irregularly shaped, into a display surface for video projection. These objects may be complex industrial landscapes, such as buildings, small indoor objects or theatrical stages.
The way I designed this workshop is to have a lecture and design exercises in the morning and technical workshop and hands-on experiments in the afternoon. Contents wise I always over-prepared, that way I can skip certain contents or exercise according to the students in the workshop. I have prepared 2 design exercises for this workshop.
Design Exercises: 1.Irrational Forms: Participants are split into small groups of 4 or 5. Each group uses materials found in the environment to construct a tower as high as possible. Once all the tower is built, the facilitator will reveal the true purpose of the tower. Groups then deconstruct the tower into pieces and used them for the assigned purpose.
2.Art Machine: Participants are split into groups of eight and each group sit around a different table. Each participant takes a piece of paper and folds it into 4 rectangular cells on each side. Starting with the top cell, each participant writes down a descriptive sentence that includes at least a person, a place, and an activity. They have a minute to do so. When it’s done, pass the paper to their right. The next participant will create a drawing that best describe the written message. Alternating between writing and drawing and participants only allow to see the writing or drawing of the last participant.
The workshop went smoothly, I have so much fun working with all the students in the workshop! The survey on the first day shows a devise interest in digital and analogue media. Projection Mapping worked well as a strong visual output for all of them. I had each student brought an object to be projected on at the end of the second day. It could be anything as long as it has a non-flat surface. Most of the students used something from their backpack as the mapping object which felt unprepared and less creative. This might because the assignment/object was loosely defined and I did mention pencil boxes and notebooks as examples… However, there were fun objects beyond my imagination and the original video contents that some students brought were very well made.
The morning sessions were short, so I cut the design exercises out and focus on the lecture. I would like to run the design exercises next time tho, did feel like something was missing in brainstorming. Projection Mapping was a great topic for this group of students, everyone has a library of assets and videos they created for other courses and it is very easy to import them to VPT and project them onto an irregular surface which instantly upgrades the presentation of their work.
Hopefully, their Summer-long project will find that balance and make sense of the design and the technology. The home-brew cube surface demo was a success. It is very effective and portable. I made an interactive version of the original demo with Leapmotion, students lined up to try it which is awesome!
Students definitely loved the Oculus Rift Rollercoaster demo during the lunch break on the 2nd day. Broadening their horizon, checked! This won’t happen without Ruei, Mars, and the program director’s assistance. Students were amazing at what they do and the mapping software was a cake for them. Hopefully, it opens up a new way of expressing their thoughts and works. Looking forward to collaborating more!
I was invited to be a part of the design panel in LEVEL UP 2015 TAIWAN GAME DESIGN FESTIVAL (http://levelup-twgs.com/) at TECO New York. The curatorial team made a great effort to gather all the talented hard-working Taiwanese Indie game developers in New York. I really wanted to participate but was caught up with other projects and commitments. In order to contribute to this great event, I decided to give my presentation a playful kick by hacking together an 8-bit presentation on a NES cartridge!
I dug out my tool bag and all the sample codes I used to teach in class. With a little bit of CHR mashup, I was able to put the overall look-and-feel plus the basic platform mechanic together quickly. I soon ran into a problem on placing all the text in the presentation. Letters and symbols are treated as graphic tiles in NES, so if I want to write stuff on the screen I have to arrange and place them on the screen letter by letter. It is a PIA to do so. I decided to write a tool to do this for me – a typing tool for editing nametable.
The only challenge in building this tool is to decode CHR file with limited/raw binary support in Processing in comparison to C++. This could be fixed by writing my own shifting functions which turned out to be a breeze. I am started to see the potential of this tool and want to develop it further with extra functions like color picking, stamp, and creating nametables from an image file or from camera.
Every row (8 pixels across) in the 8×8 tile is 2 bytes of information. In order to get the wanted bits, I left shifted both bytes, masked the first byte for the least significant bit, right shifted the second byte by 1, added the two byte together, and lastly masked the sum for 2 least significant bits. This will give me either 0, 1, 2, or 3, which matches the 4 palette colors.
