Leap Motion 3D Jam Future DJ\VJ Virtual Fan Experiences
A downloadable game for Windows, macOS, Linux, and Android
PROJECT Design
Leap Motion 3D Jam Future DJ\VJ Virtual Fan Experiences
OBJECTIVES
The goal of this project is to gain embedded systems interfacing of live DJ\VJ hardware components with Leap Motion to offer Live Broadcasting & Virtual Experiences for Fans.
- The project's audio\video & broadcasting designs are leveraging free or most cost effective method available to illustrate the fundamentals in isolating a professional mixed audio integration, visual displays, motion controls & live steaming. To illustrate what possibilities fans might experience in the future. Most of these tools are used by current:
- Bands
- Music & Visual Artist
- Commercial
- Movie
- Audio Dub overs
- Live concerts
- Production events
- Studio recordings
- Award Shows, etc.
We were responsible for all of the following:
- Coding research
- Application usage and research
- Vendor engagements for product usage
- Testing and providing reviews of these applications
- Leveling these applications based on:
- Cost
- Skill
- Application ease of use
- Overall, the cool\fun factor
- Providing a solid musical composition experience
- Applications that would be interesting or, engage current and new fan base
- Audio
- Creation
- Remixing
- Stems loop selection editing
- Stems mastering & mixing
- Vendor inquires to validate trade mark usage
- In a live audio video visual broadcast
- End user licensing agreements for platform development
Projects design and research, bettering of VR streaming for:
- Audio that is from Drum machine,
- Application based music creations
- Open Source Instruments,
- Effects instruments,
- Audio sampling techniques,
- Visual art Video rendering and
- 3D CAD modeling
- Visual manipulation for VJ set
- VJ audio sequencing
- Live broadcast streaming
- Audio
- Visual
- Leap Motion VR environment
- Special Effect,
- Shadows with depth
MEASUREMENT DATA
Key elements of Artist and Fan experience
- The fan experience testing, we found the following key elements in obtaining and retaining fans:
- Connection to the Artists or, the Artist's work
- Popularity of the Artist
- Physical attractiveness to the Artist
- Artist sexual orientation
- Musical style
- Musical energy & tone
- Uniqueness in the delivery or, method of creation in a live setting
- Availability of live events
- How often fans were able attend a live show
- How large the events were
- How rare it was to attend a live event
- Who the Artist was booked with
- Age of Fan base
- Age of the Artist engaging the Fan base
- Youth\Teen 1 to 20 years
- Adult 20 to 40
- Adult 40 to 60
- Adult 60 plus
- This affected sales event need for delivery & overall popularity
- Age was also, a factor where long time musical Fans were won or, lost
- Since a Fans musical style tended to be influenced rapidly & was focused on the style that had more energy & tones at a young age
- Teenage years musical styles & fan base was based more on the type of personal style that the audience wanted or, was associated with.
- Teens to those around 25 years of age were when fans were either won or, lost were determined by the longevity of the musical style the fan base was interested in for what seemed to be ongoing and for the remainder of their life
- 25 plus years seemed to have their musical style set & the change of musical style was usually less often. Even when the same songs are repeated for years with minimal or, no new Artist\songs
- Connection to the Artists or, the Artist's work
- Mixing within different generation at events (family or friends)
- Kids
- Grand kids
- Siblings, etc.
- Seemed to allow for the tolerance of styles not commonly experienced
- The ability to convert these users to FANs, was sometimes common in the intermediate family as the styles were commonly heard or, were experienced more often
- Personal preference of the musical style that a user adapted seemed to always be the Fans most common style from Teen to 25 years of age
- Unless there was a cover remake or, the Artist somehow associated their sound that reminded them or, sparked memories of past events
- Remixing of a Fan's musical style can be both inspiring to an audience which can attract new fans or, cause a fan that is set in their ways to not be willing to make changes and will repeatedly listen to the same style without change.
- Remixing or, modifying a song seemed to upset and at times even cause a fan to show disinterest therefore, causing a fan to not want to continue listening and have no interest in the creativity of the remixing of a song.
- This is common with Electronic and Hip-Hop styles of music commonly noticed in sites such as YouTube. Comments are also made by a disturbed fan in which they will go out of their way to inform someone or, enter comments that are about the original Artist and their work and should not be tampered with.
- Disturbed fans that push back and make negative comments of Remixed work are more apt to do so when a song is performed by a different artist versus the sampling of the original artist and remixed with completely with different sounds or, in a different style
- Cover Band Artists were far more successful, with these types of users, when generating a new fan base.
- It was found that these users were sometimes tolerant of Mashups using original artist's samples across their same style of music
- This did seem somewhat generational with users of age 40 plus
- As for most other generations, they have experienced the remix mashup and sampling of musical styles throughout most of their years.
- Remixing or, modifying a song seemed to upset and at times even cause a fan to show disinterest therefore, causing a fan to not want to continue listening and have no interest in the creativity of the remixing of a song.
During this study:
Almost all generations of musical fans were attracted physically to people that preformed the music and would rather experience the Artist's live through videos or, concerts if the content was of good quality and as the song was remembered. Thus illustrating the key element of this project was to broadcast the event to better Fan experience & offer Artists the ability to visually express their ideas and inspiration behind the music creation for their fans.
Inspiration for researching the future Fan experience
- One of our inspirations to build a better experience for the fan's experience was from:
- Muse Application was created by BT & Dr. Richard Boulanger searching for New Sound
- a Virtual gesture based application & using MIT's CSound,
- Fan not only seeing & hearing the artist,
- Being engaged in how the music is created & delivered is commonly missing from even today's live DJ shows.
- Artists are often hidden behind a wall or, the instrument & method of creation is not fully visible like a concert with a Guitarist, Drummer or, even a Pianist.
- Which is why, turntable scratching is still so compelling to DJ Fans.
- 2 records and 2 turntables offering a sound that should be annoying
- Provide a harmonic and curious sound
- Almost everyone has attempted to create a sound at one point.
- Could not reproduce with a standard turntable & LP.
- 2 records and 2 turntables offering a sound that should be annoying
- Which is why, turntable scratching is still so compelling to DJ Fans.
- With You Tube's popularity
- You can now illustrate for your fans with the right equipment and skills to apply a quality scratch placement during the record cutting of tracks.
- Get feedback from fans & deliver content to your audience that allows for a user's engagement through comments.
- You can now illustrate for your fans with the right equipment and skills to apply a quality scratch placement during the record cutting of tracks.
- Example; The BT Muse Application - Unless fans understand why they are waving their hands in the air and will wonder what they are doing.
- The non-standard DJ form could heighten the experience when Fans can visually see what BT movements are doing to generate the sound. Live.
- Thus unlocking engagement and the curiosity that lies within each and every one of us.
- The non-standard DJ form could heighten the experience when Fans can visually see what BT movements are doing to generate the sound. Live.
Curiosity about Muse @ how it might have been built?
For sure the CSound was used, below is a Leap Motion controller module I found from a Berkley upload…Was thinking it maybe just be part of the Muse Source core or inspired it!
- CSound documents – Developed @ MIT
- Microsoft OneDrive View - http://1drv.ms/1Ok5iBF
- CSound_606_LeapInto01_ZIP_Installers – Developed @ Berkley
- Microsoft OneDrive View - http://1drv.ms/1Ok5bGm
AR\VR Application R&D Usage & functionality
- This, section has been written to assist with the understanding of why we are offering advice and guidance for users looking to have the option to choose which application should be explored versus, the understanding of how and why AR &VR are parallel capable devices:
- In so many areas AR & VR offer capabilities that are lacking from one or, the other's program design.
- VR – Offers users an alternative reality
- Virtual reality needs to offer more than a single option such as hearing and not only the sensory of visual sensation
- AR – Brings computing & entertainment in physical reality
- Offering a shared physical reality & virtual computing space.
- The application for our 3D Jam (DJ\VR Future Fan Experience) project suggests that;
- It has to be up to be which reality each user wants to fulfill or experience.
- During this project, we discovered VR & AR technologies were calling out for futurist programmers to offer new capabilities, technologies and ideals that progressively moved programming and computing technologies to move forward. Warning. that we should turn away from recoding habits & idealisms that hinder the progression of technology for nothing more than self-profit and personal gain.
- We compared applications versus deploying only the application we felt was best for the project and our futurist and forth coming fans.
- We wanted to obtain a collection of audio, visual and broadcasting tools that offer futurist programmer's the following options:
- Usage capabilities
- Capabilities that worked
- Capabilities that offer and had the ability to offer idealistic forward thinking functionality
- Apps leveraged were:
- Free
- Open sourced
- Purchasing based on both low & higher costs
- Computing space that is both minimal & HPC (High Performance Computing) that comes with very costly overhead
- Hoping to locate applications:
- That could assist in minimizing computing footprints
- That had the best chance for zero exceptions & error handling designs for future quantum computing tests.
- Offering Users and Developers a virtual experience focused on enhancements and new feature design requests versus wasting, time and energy on bug\fix-action requests
- That could assist in minimizing computing footprints
AR –vs- VR: The age of shared realities, thoughts on (Physical Reality & Virtual Computing)
- Which is better?
- This is an answer that I could not produce a conclusive AR\VR answer for, as they both have elements that appeal to different Virtual computing needs.
- Each user is different while some will desire virtual elements in shared physical space like the AR offer. Others might more enjoy the ability to be visually entertained through VR style elements which enable them to enjoy a virtual 3D design of the artist's creative idealisms that inspire their creation of music works.
- This is an answer that I could not produce a conclusive AR\VR answer for, as they both have elements that appeal to different Virtual computing needs.
- Is VR even a desired technology for future development? Since modem computing developments have illustrated the capabilities to enhance augmented reality capabilities through projection illustration & mapping methods?
- It is suggested that VR can achieve the capabilities to virtualize more than just our visual senses other than a VR environment that is appealing for a number of applications:
- We imagine back to the days when we were young kids at the Freedom Fest of 1990 in Dallas, Texas in which one of our favorite musician's Stevie Ray Vaughan performed live with his brother Jimmie Vaughan . I was amazed after experiencing that performance. Then to hear that he was killed in a helicopter crash going to Chicago. What a sad day to know that Freedom Fest would be the last time anyone would ever experience a Stevie Ray Vaughan concert again.
- Stevie Ray was one of my favorite blues artist's growing up.
- The ability to experience a live concert he was headlining would be such a cool experience.
- Amongst so many other artists from beyond my generation, I have only experienced within Audio such as; YouTube and other music video sites.
- With VR technologies, it is definitely possible with the 3D rendering capabilities to generate the visual experience of these live concert events again!
- However, VR would provide a much more realistic experience when we finally achieve core sensory virtualization. Allowing for human perception virtualization. We believe that this would require at least 4 of the 5 senses to truly feel the VR emotional experience. These four senses are the most effective:
- Sight
- Sound
- Smell
- Touch
- The ability to experience a live concert he was headlining would be such a cool experience.
- It is suggested that VR can achieve the capabilities to virtualize more than just our visual senses other than a VR environment that is appealing for a number of applications:
- VR application thoughts on what it could offer us:
- VR could possibly be leveraged for Cryo-Stasis within a long term space flight, I developed a Music composition in the early 2000's called Cryo-Stasis with thoughts in mind such as; Demolition Man, The Matrix and Vanilla Sky.A time when Cryo-Stasis environments were leveraged successfully.
- Each of the movies illustrated the user within the environment who must believe in the reality of an environment they were in before the acceptance of the reality.Or, that the individual needed to have capabilities of the active thought process during a Cryo-Stasis state.
- My CD was developed with this concept in mind.
- That Cryo-Stasis will only work if, the mind is active. The thought process was that the mind could generate enough electrical energies through stimulation and awaken from a Cyro-Stasis state allowing, for successful regeneration.
- My musical projects were designed in an abstract form of music that had an energy that did not increase beyond the point that would cause an awakening or, go so slow that it would not be affective.
- Also, taking into account that if a song sparked memories of an event in their life, this could cause issues both, negative or, possibly offer a positive result that even within a Cryo-Statis state, the mind and memories would still be active
- So, the elements I leveraged used audio virtualization with repetitive euphoric electrical and classical sounds.
- With the belief that this would produce minimal impacting memories from musical styles the Cryo-Statis user had adopted.
- Allow for the mind to stay active during the Cryo-Stasis generated time line.
- A possibility of success seems viable to me since sound has been known to elevate objects so therefore, why not keep the mind dreaming?
- The thought process and belief system of "I think therefore, for I am.
- Developing VR systems that stimulate other neuro senses could only assist in the delivery that is believable and enjoyable therefore, becoming a useful experience.
- My CD was developed with this concept in mind.
- Each of the movies illustrated the user within the environment who must believe in the reality of an environment they were in before the acceptance of the reality.Or, that the individual needed to have capabilities of the active thought process during a Cryo-Stasis state.
- I also think that people will desire leverage VR capabilities for an escape from their everyday realities. I am sure we all have had this thought process at some point or another in our own lives.
- This quote is used by a number of emotionally distressed people often leading to\or, are already having;
- Psychological issues
- Addictions
- Suicidal thoughts
- Or, just thoughts within a depressive state
- It's a wonder if, these states can and are the leading causes ofdivorce, broken homes or, dysfunctional ways of living
- If that is the case then, a virtualized break through could be the cure for aliments or, some of those situations. Some people can overcome issues without experiencing alternative realities
- This quote is used by a number of emotionally distressed people often leading to\or, are already having;
- Another thought for VR that I myself and almost anyone I have encountered say that they would give anything to spend time with a lost family member or a loved one again.
- To have the ability to have any kind of time at one point or, another that felt like a realistic experience with a lost family member or, a loved one could be uplifting and would be something anyone could say they would enjoy again.
- VR for medical usage such as, paralyzed individuals could mentally gain the abilities they once lost in a physical world.
- The Avatar movie sense of methodology
- Using VR to experience historical and educational events
- Travel to Eras that are appealing and of interest which AR could not offer.
- The movie, The 13th Floor comes to mind
- Travel to Eras that are appealing and of interest which AR could not offer.
- VR could possibly be leveraged for Cryo-Stasis within a long term space flight, I developed a Music composition in the early 2000's called Cryo-Stasis with thoughts in mind such as; Demolition Man, The Matrix and Vanilla Sky.A time when Cryo-Stasis environments were leveraged successfully.
- AR applications for my fan based Leap Motion 3D music project:
- In my experience, the ability to use physical instruments is a must have so, AR would be a desired element for my performances.
- Currently, there are no MIDI\OSC controllers that offer the ability to;
- Scratch vinyl records
- Set the step sequencer note settings on my legacy drum modules
- The inability to interact with physical and virtual object causes a less desirable experience for my current performance would be the uses of VR integrated hardware head gear
- Leap Motion hand gestures for effects controls is a very nice addition that I can see progressing greatly in the future with additional applications and the integrations of more MIDI instruments controlled applications and musical instrument companies with their vendors adapting to the support of a VR musical development platform
- AR development could allow for extended functionality controls to my physical hardware as well as the ability to manage the software based application in a more dynamic manner.
- Through configuring personalized controlled environments around my workstation to ensure quick response times and application management flows with in my DJ set
- When looking at the capabilities of AR within a live set & projection mapping support of the computer based applications
- This offering with the hand gesture capabilities in Leap Motion I could even leverage online applications like http://tonematrix.audiotool.com/
- Which is a compact version of my physical DJ production audio creation & live workstation setup.
- Just Web based & functionality not intended for Live performance usages
- Offering capabilities to bring applications like Tone Matrix into my physical space enhances my ability to deliver a better performance for the audience.
- While allowing me an extra workstation suit that can be preconfigured with needed settings to create a seamless flow from instrument to instrument.
- Offering me more time to focus on effects
- A Capella's
- Cueing samples
- Vocal changes
- Energy buildups
- Heightening the fan experience
- While allowing me an extra workstation suit that can be preconfigured with needed settings to create a seamless flow from instrument to instrument.
- Future supportability for application based turntables could deliver capabilities for longevity of vinyl records since scratch\cutting records are easily damaged beyond playable capabilities after 40 to 50 sessions that integrate scratch style approaches.
- This would also be cool since individual audience members can enjoy visually seeing the standard DJ's scratch styles. By being engaged in the music set with less worry of breaking equipment or, causing live performances with audio train wrecks.
- Thus offering the capabilities to get the crowd involved similar to an illusionist's show. The audience's engagement in a live set is something I only get to do in the studio or, at home. Because of the concerns with audio mix integrity and issues of breaking hardware.
- Which is a compact version of my physical DJ production audio creation & live workstation setup.
- Currently, there are no MIDI\OSC controllers that offer the ability to;
INTERFACES
Project Audio hardware selections
There are plenty of Music based applications both on Leap Motion & Unity's app store's web offerings.This is not the true excitement of the overall project task we were attempting to achieve nor, was it the real reason of interest in our project design. The Fan or, user experience seems to be of a greater interest & was unexplored territory
- We felt that using both parts of the artist and fan experience would bring exposer to a core problem with the Musical & Fan experience.
We, found that the music application offering within these app stores, all fail to meet what is truly the exceptional fan experience. The VR Music applications that are available today offer a uniqueness which in turn make for amazing fun or, integration functionality\capabilities.
- However, each are limited to the constraints of current sensor technologies & with in a VR world are limited by the inability to experience or leverage augment controls in real world instruments within real time.
Audio Mixing\Mastering:
- Documents about Audio applications tested & downloads for application our Microsoft One Drive
- View - permissions
- http://1drv.ms/1lc9Wp9
- Audacity
- Audacity might well be the best-known & most widely used open source music application. Can do everything you'd get from a commercial audio editor. Edit samples and songs, process audio files, burn CDs and export a wide range of audio file formats including WAV, AIF or MP3.
- CLEF
- LEF is a Max-based modular environment for composition and performance of live electronics, structural design concepts and allows users to rapidly create digital signal processing topologies and control data taking advantage of the provided infrastructure and feature sets.
- CSound
- As new generations of sound synthesis's come of age it is important to develop tools to communicate the formative studies of sound and science. The sound designer or software developer may take advantage of this opportunity by creating interfaces to translate classic examples of psychoacoustics for the burgeoning synthesis and others to learn from
- LmmS
- There are 19 included instruments (embedded ZynAddSubFX) and effects, plus VST support. Alternative to programs like FL Studio and Orion. It gives you everything you could need to produced polished electronic music on just about any computer. Import MIDI files, Hydrogen and FL Studio projects.
- Integra Live
- Cis an application that makes it easy to use interactive audio processing to create new music. The software has applications in education as well as professional artistic practice and opens up new possibilities for those seeking an entry point into interactive 'live electronics'
- ISSE
- audio denoising, music transcription, music remixing, and audio-based forensics, it is desirable to decompose a single-channel recording into its respective sources.
- LEGO Music Box
- LEGO Music Box is a collection of software to generate sound using Mindstorms EV3 and PC music software.
- Marsyas
- Music Analysis, Retrieval and Synthesis for Audio Signals is a framework for developing systems for audio processing. It provides a general architecture for connecting audio, sound files, signal processing blocks and machine learning.
- Sonic Candle
- Given a music file, generates a video file with a visualization of frequency spectrum bars generated for the music. Outputs high-quality, rendered video. Intended for publishing your own music on YouTube and stuff
- Wavvy.Audio
- Listen music in your player, just setup «Audio In» in Wavvy. •Audio signal processing (compressors, amplifier). •Serial connect with Arduino
- Cecilia
- Cecelia is a CSound-based graphical environment for music and signal processing. Additive, subtractive, granular synthesis and processing and more are presented in an easy-to-use
- Pure Data
- A sweeping visual programming language for multimedia
- Mixxx
- Mixxx provides a professional quality DJ mixing environment that can read MP3, Ogg Vorbis, WAV, AIFF and FLAC formats, among others
- Rosegarden
- Around since 1993, if you can believe it, and it shows in the highly evolved, mature workflow. Rosegarden is a fully realized MIDI and audio workstation with all the trimmings.
- Supercollider
- Designed as a programming language for real-time synthesis and algorithmic composition, Supercollider is dense and deep. Books, seminars and workshops exist for those inclined to take the plunge.
- ZynAddSubFX
- Microtonal and multitimbral, and with a slew of built-in effects, there's not a lot you can't do with ZynAddSubFX. The sound is nothing short of spectacular. combines Subtractive analogue with a powerful additive engine and a wicked pad synthesizer for serious sound design fun.
- TAL-Noisemaker
- A revamp of the earlier TAL-Elek7ro, it's a three-oscillator powerhouse with a self-oscillating filter section and some tasty built-in effects. The sound is thoroughly old-school, but there are some modern niceties hidden behind TAL-Noisemaker's collapsible windows, including custom envelope shaping.
- Giada
- Giada combines a sequencer, piano roll and loop machine with the ability to host VST plugins. Live sampling is its strong suit, with built-in sample editing and slicing. Latency compensation is included. Designed for DJs and live electronic musicians
- Ardour
- Ardour is an open-source alternative to the likes of Pro Tools, Cubase and Logic. A fully-fleshed out, mature DAW with all of the trimmings, Ardour began life as a Linux program before being ported to OS X and now, in version 4, Windows.
AR\VR - Visual Image rendering & Live Art Performance Applications
- Documents about Visual arts & downloads for application tested on our Microsoft One Drive
- View - permissions
- http://1drv.ms/1REZX5S
- Blackmagic
- DaVinci_Resolve_12 real-time video controller playback
- Fusion GPU Accelerated Creative Workflow. Fusion is an extremely fast visual effects software tool that's responsive enough to be used in high pressure
- Roland's MD-P1-S: Motion Dive Tokyo Performance Package
- Software plus controller playback and performance of real-time video, images, and sounds
- LiviD Cell & CellDNA
- Software playback and performance of real-time video, images, and sounds
- LiviD Controllers extra cost add-on - block, Code, and Ohm64 controllers, Or controllers like the APC40, APC20, & Novation Launch Pad.
- Resolume Arena 5 Media Server
- advanced options for projection mapping and blending projectors
- Control from a lighting desk and sync to the DJ via timecode
- GLMixer
- Open Source decide on the fly how much they should be visible, arrange the images in the screen, and which effects are applying to the
- LFDisplay
- Light Field Rendering real-time light field viewer/recorder called LFDisplay a Post processing software that leverage High performance quantitative digital CCD, EMCCD and Scientific CMOS cameras for industrial imaging applications & QImaging camera formats for science microscopy along with 3D rendering\viewing\mapping
- SGI VizServer server with NICE DCV
- integrate NICE DCV modernizes EnginFrame to ensure that NICE DCV offer same cutting edge performance, it did when it became the first software product on the market to allow sharing of a single physical GPU between multiple Windows and Linux sessions, while maintaining full OpenGL application acceleration and workstation-class performance to provide 2D/3D session management via a web browser providing interactive session for collaborative working
- Aestesis Electronika
- Elektronika Live is the intuitive free VJ software for visual performers. This powerful, modular software will have you designing live visuals for your performances within minutes. Elektronika Live is also a midi video station, designed for the needs of live audio/video performers. Create your own live video compositions using your Windows PC. Just plug together modules from the list of over 30.
- chdh for performance.
- It is based on five different audiovisual instruments made of chaotic and physical modeling algorithms that you can directly control.
- digital cinema package creator
- Program to e.g. avi video files to split into individual images in order to create aDCP, Audio Video & Image tool for media creation
- Perceptron
- Perceptron is a video feedback engine with a variety of extraordinary graphical effects. Perceptron is an endless flow of transforming visuals.
- Virtual Stage Light
- The goal of this software is to provide an easy way to set a complete new light show on stage up (for theater, live music, etc.), based on a unique video projector point of light emission.
- Multi_Projector_Mapper
- The multi-projector-mapper (MPM) is an open-source software framework for 3D projection mapping using multiple projectors. It contains a basic rendering infrastructure, and interactive tools for projector calibration. Allowing for Augmented Reality projection displays
Visual art Video rendering
- Documents about Visual arts & downloads for application tested on our Microsoft One Drive
- View - permissions
- http://1drv.ms/1REZX5S
- Double Fractal
- is a program that creates fast and easily high complex images. No specific knowledge is needed to use this program; it is a program that can be run just for fun.
- CHAOS_PRO
- Calculation and Visualization of Fractals and other Elements of the Chaos Theory
- Chaoskop2
- Calculation and Visualization of Fractals and other Elements of the Chaos Theory
- factory.
- Workshop that allows fractal objects from brick objects (generators, initiators, grammars
- FraktPro
- fractal objects & Mandelbrot
- LightAndGeometry
- Raytracing application
- Aleph One
- is a highly sophisticated program for generating fractals.
- Apophysis
- A version of the fractal flame editor Apophysis which is ready to take advantage of modern versions of Microsoft Windows, provides a big set of new features and has a polished user interface.
- Mandelbulber
- is an easy to use, handy application designed to help you render 3D Mandelbrot fractals called Mandelbulb and some other kind of 3D fractals like Mandelbox, Bulbbox, Juliabulb, Menger Sponge
- Art of Illusion
- is a full featured 3D modelling, rendering, and animation studio. It is written entirely in Java, and can run on almost any operating system.
- Structure Synth
- is all about states. A state describes the current coordinate system and the current coloring mode. The coordinate system determines the position, orientation and size of all object drawn while in the current state.
- Syfluid
- is a funny toy as well as a real simulation. It simulates a fluid or a gas in a very colorful, graphic and sometimes psychedelic way
- SSbump
- A GUI interface to a tool for generating SSBumps (Self Shadowed Bump Maps). Includes a CUDA GPU rendering extension.
- ARToolKit for Unity
- The fastest way to develop AR-enabled games and interactive applications, using the ARToolKit plugin for the Unity game development environment.
Live streaming & broadcasting Fan experience
- Documents about Allowing for broadcasting of Visual & Audio Content on our Microsoft One drive
- View - permissions
- http://1drv.ms/1XWrBzg
- CasparCG Server,
- a professional software used to play out and record professional graphics, audio and video to multiple outputs. It has been in 24/7 broadcast production since 2006.
- Open Broadcaster Software
- is free and open source software for video recording and live streaming. Supported features include
- OBS Multiplatform will eventually support many of the advanced requested features not present in the original OBS, such as multiple stream outputs and scene previewing. It does not have these features in this release.
- is free and open source software for video recording and live streaming. Supported features include
- Microsoft's Skype for Business Online meeting\broadcast,
- Preview available via Office 365 will receive .MOV\WAV\MP3 files then transmit then as a broadcast representation of the .MOV\WAV\MP3 data, thus generating the Live Broadcast Event User experience.
Leap Motion Apps Reviews
Basic level
Music composition:
Syntheremin
- Syntheremin
- https://apps.leapmotion.com/apps/syntheremin/osx
- Review
- The sound quality & the app itself does not remotely, come close to Synth or Theremin. Audio experiences, much less even offer a learning basic to either instrument The way of controlling the sound is not like controlling the sound of a Theremin.
- Cmon Man, your stament about annoying the cat was as clever as the use of advertising this app as a Theremin or MIDI learning tool…to make money.
- Roland's MC - line of drum machines used sound similar to Theremin with its D-Beam feature would have got a better result if you had of sampled that. the MC line of units. They can be grabbed as low as 90$ used at Guitar center.
- variation of the Theremin used in Billboard Top music tracks - Surprised none of this was in the learning part of the app,
- oh what ya there wasn't any experience of learning!
- Jimmy Page of Led Zeppelin used a during performances of "Whole Lotta Love"
- The Beach Boys: "Good Vibrations",
Gravilux
- Gravilux
- Gravilux is an interactive musical star field visualizer. As you wiggle and wave your fingers before the screen, gravity draws simulated stars to your fingertips.
- https://apps.leapmotion.com/apps/gravilux/windows
- Gravilux is an interactive musical star field visualizer. As you wiggle and wave your fingers before the screen, gravity draws simulated stars to your fingertips.
- Review
- I have to be nice because I love me some Bjork so that is super cool however the application is so limited in its capabilities to do basic game or computer functionality it's kind of got me wondering what or how much did they pay to get solid reviews from...or this was there fist exp with VR,
- "Apps like Gravilux awaken an 'Avatar'-like sensitivity to electricity in the body, power in the palms and general connectedness." —NY Times
- "Of the millions of apps out there, none quite capitalize on the sheer fun and beauty of interactivity the way that Scott Snibbe's do." —CoolHunting
- I have to be nice because I love me some Bjork so that is super cool however the application is so limited in its capabilities to do basic game or computer functionality it's kind of got me wondering what or how much did they pay to get solid reviews from...or this was there fist exp with VR,
- Scott Snibbe...
- bro computer basics - Save & Save Often is the most common computer term we tell ourselves next to, take a backup & a backup for a backup & so on.
- I see kids going crazy, when they can't save the hours of work.
- There are other's that reported the inability to save as annoying, but over looked because of the possibilities in total coolness this app could have or might become.
- Looking to develop similar game,
- use one these apps & a nice space explores 3D mod. From Source forge…If, I remember, there is even an Art application that will generate music from color schemes on Source forge. Good Luck, remember (Save please)
- Fractal painter
- http://sourceforge.net/projects/fractalpainter/files/latest/download?source=directory
- Fraqtive
- http://sourceforge.net/projects/fraqtive/files/latest/download?source=directory
- Fractal Zoom Explorer
- http://sourceforge.net/projects/fractalzoomexplorer/files/latest/download?source=directory
Soundscape VR
- Soundscape VR
- https://apps.leapmotion.com/apps/soundscape-vr/windows
- Review
- fail to meet real world real time integration or a solid gesture based capability
- Recommendation
- Would not download or waste your time this should have been developed by Audio tool, working with Leap Motion direct for better hand gesture experience & a real DJ style experience that tone matrix currently offers.
- I believe that if developed by Audio tools, this would provide a more realistic VR music production experience, and been worth playing on a regular base.
- I would have pull from my list but giving the possible legal issues with use of tone matrix & there being no true Open Source or Developer SDK available, any longer.
- Leaving me with the feeling that this app is more of a legal issue and one that brings more copyright concerns to the music creation industry when we have enough of them already
- Legal Note
- I would review the credibility of this application unless (sander snake) has approval from (audio tool or andre-michelle tonematrix creator).
- Since andre-michelle tonematrix merger to Audio Tools after 10 years @lab.andre-michelle.com
- This application appears to be a direct violation of the Audio Tools usage policy not for commercial purposes (sandersnake) should provide Leap Motion with legal proof that (Soundscape VR) in not violation of German law or audio tools legal terms of use.
- Validating approval to commercially distribute the hirable attempt at recreating an amazing online Audio creation tool.
- legal page:
- http://www.audiotool.com/terms
- Tonematrix developed originally
- http://lab.andre-michelle.com/
- legal page:
Mid-level
Music composition:
Veil
- Veil
- an interface for virtuosity using a Thin fabric object in front of you; touch it, strum it, bend it, the fabric responds and generates a sound.
- https://apps.leapmotion.com/apps/veil/windows
- an interface for virtuosity using a Thin fabric object in front of you; touch it, strum it, bend it, the fabric responds and generates a sound.
- Review
- Mid-level score because of price & as its own maker states Veil is neither an ever-good-sounding toy or Noise maker...It's a real instrument & can be a keyboard, it's a guitar, it's a violin, it's a harmonica
- Not even sure how to rate this,
- other than unique...the fabric seems odd for use but so far my remedial understanding of the application is that it must offer offers vibration emulation. Which is key to how string music instruments work.
- Just going to be hard to explain if someone is watching why your banging and strumming a fabric.
- other than unique...the fabric seems odd for use but so far my remedial understanding of the application is that it must offer offers vibration emulation. Which is key to how string music instruments work.
OscilloScoop
- Sculpt Musical Grooves Out of Thin Air
- https://apps.leapmotion.com/apps/oscilloscoop/osx
- Review
- Good "musical" creation with this is possible but not going to win anyone an Emmy award, Application has preprogrammed audio & Lacks customization for loading loops, samples and modifications to them. The ability to change or integrate standalone effect parameters, like reverb, flange, LFO
- I really hope the developer keeps updating the app and making it better, as the app's usage and capabilities does not live up to the name Oscilloscope's are most useful to audio engineers looking to sculpt audio files modify or change them. The Application calling them crowns is kind of a joke...So I hope coming from renown musical talent and application developers.
- The rating was given because some coronations of audio created Good Music that could have been played with other affective musical instruments and effect processors. Also cost not being free, while there as a few good moments of music creation the lack of extended music creation was just that. Lacking and then the inspiration was gone.
- Solid game or toy for youth learning about audio and sound based coronations.
Dropchord
- Dropchord while a game the controller of sounds and interest was given a Mid-level score because of cost & some of the gaming levels were challenging
- Also Since it was possible to Mix alternative sound with the music in the drop cored game, if you ran the audio out to a mixer and had mix capabilities
- https://apps.leapmotion.com/apps/dropchord/windows
- While cool in sparking creativity, this will not be the app you create your next Emmy award with
MIDI/OCS:
Chordion Conductor
- Chordion Conductor
- https://apps.leapmotion.com/apps/chordion-conductor/osx
- Review,
- offered the best user interactivity from beginner to advanced level the widest range of internal instrument base music creation as a MIDI controller
- However, you only available for use with MAC,
- Cmon Man - Project Darwin anyone?
- I care because the root of my demo project for 3D jam is based on the long standing issue with projecting live streaming format.
- Cmon Man - Project Darwin anyone?
- Which, has been the toll that is put on the graphic card and that most users would not have supportable GPU to process the data. Windows Live Broadcast features in Azure help control and lift some of the load so I needed a Windows based application.
- Sorry, you're not number 1.
- You are on the list as the quickest for users to learn & the integration of cord to help teach music reading and cord scaling fundamentals. Nice work, with this music learning function.
- Sorry, you're not number 1.
- I so wish, you were on the Microsoft platform!
Advanced level
Music composition:
Muse
- Muse empowers and inspires anyone to compose and perform beautiful ambient music.
- Designed Ideal, unbound music composition from a typical or standard instruments
- https://apps.leapmotion.com/apps/muse/windows
- Designed Ideal, unbound music composition from a typical or standard instruments
- Created by BT & Dr. Richard Boulanger the Search for New Sound
- Using MIT's with a Virtual gesture based motions & MIT's CSound, one of my inspirations to build a better experience for the Artist Fan experience
- https://www.youtube.com/watch?v=fTWn2vLTMIk
- Using MIT's with a Virtual gesture based motions & MIT's CSound, one of my inspirations to build a better experience for the Artist Fan experience
- Live w/BT in the Studio on Leap Motion Technology
- prior to the new Leap Motion Hand & gesture being used in Leap Motion 3D Jam 2015
- https://www.youtube.com/watch?v=u6BhiRi-sno
- prior to the new Leap Motion Hand & gesture being used in Leap Motion 3D Jam 2015
MIDI/OCS:
Cubase iC Air
- Steinberg Cubase 7 & Cubase Artist 7 controller for leap motion
- https://apps.leapmotion.com/apps/cubase-ic-air/weblink
- Review
- What can you say its Steinberg & Cubase!
- Does not surprise me that the first Music development company To deploy with Leap Motion. They are integrated in almost all of Roland's product suites & every other major music Hardware\Software vendor.
- Steinberg & Cubase integrates well, with legacy & current MIDI hardware. Steinberg & Cubase is amazing an easy. It's also, rare to find a MIDI mapping that is not already in the preset list when connected with MIDID to USB.
- Support of Hardware vendors with Steinberg & Cubase is & has been amazing from my experience since the early 2000's.
- One thing I noticed with Cubase iC Air
- Platform: Web link & Steinberg eLicenser
- wondering if there could be come cool HTML5 or Unity integrations in the future for simplistic music creation for Web based gaming?
- Online podcasting or stream live broadcasting, don't know but cool app from one of the legendes in this industry Steinberg always a Pioneer for new area Dev.
Geco MIDI
- GECO doesn't produce or manipulate sound directly, its purpose is to control other software or hardware that understands MIDI, OSC or CopperLan.
- https://apps.leapmotion.com/apps/geco-midi/windows
- CopperLan
- Creates virtual MIDI ports to connect to standard MIDI applications
- http://www.copperlan.org/index.php/download
- Creates virtual MIDI ports to connect to standard MIDI applications
- Note:
- Geco MIDI does not create music - it a gesture MiDI controller that modulates the MIDI signal sequences between MIDI based Instrument
- The Geco MIDI/OCS/Motion & gesture controller does offer some unique integration capabilities when leveraging CoppereLan to extend Geco motion application manipulation integration to legacy hardware & across multiple MIDI devices with the virtual port capabilities of CoppereLan
- Geco - Leap Motion Video
- show cool capabilities
- but are misleading as to the true usage for the Geco MIDI/OCS widget is not a sound creation
- DJ SelArom Creates Live Music with Geco MIDI
- https://www.youtube.com/watch?v=OcnrPbKCVSk
- Creating Live Music with Leap Motion
- https://www.youtube.com/watch?v=N8uFiJZpkgY
- show cool capabilities
- Review
- the Geco application does offer an insight into where the future DJ\VJ Integration is going. However, this capability off manipulation to sound has been around for many years with Roland's Drum machines that were integrated with D-Beam.
- Thus our usage of the Geco application was nothing more than another effect processor layer, for individual MIDI unit in Parallel to the Roland's SP-555 D-Beam that applied as a master effect.
- The Roland's D-Beam currently one up the Geco application widget as it offers the ability to create sounds...with hand gestures while the sounds it creates are cheesy and a bit annoying unless you know the Theremin controller pitch's.
- So giving this review Geco a solid thumb up for cross-platform capabilities and I see good thing with Geco.
- What is the D Beam controller?
- http://roland.com/support/article/index.cfm?q=faq&p=V%2DSYNTH&id=1831585
Tekh Tonic
- Tekh Tonic is an OSC/MIDI control interface reimagined for today's high-tekh motion capture input devices
- https://apps.leapmotion.com/apps/tekh-tonic/windows
- Review
- Tekh Tonic One ups the Geco MIDI with the ability to manipulate sound & the Grain Plane synthesizer available within Tekh Tonic's Morph controller.
- Grain Plane turns any sound you load into it into a rich, gesture-controlled pad. Similar to the Roland's D-Beam functions on some legacy samplers & Drum Machines, which the One up as well. Offering ability to load individual, sounds of your choosing.
- However, the MIDI Controller capabilities are only supported on OCS/MIDI supported units & applications.
- Thus the competition Geco MIDI/OCS one up's and would be my product of choosing at this time because of the cross platform and overall extendibility with the CoppereLan product suit
- Even with them being the same price, and I have plenty of other music instruments and would rather use more legacy application\Instruments with the Leap Motion MIDI/OCS then another virtual synth.
- However, If the competition is any sign of the future to come Tekh Tonic's Morph controller is a solid product for anyone needing an added Instrument with MIDI capabilities and a Ableton Live user, Grain Plane synthesizer could meet your needs.
Our Finalized Setup for 3D Jam
- Documents about our Project Blue Print Diagram on our Microsoft One drive
- View - Permissions
- http://1drv.ms/1lcIzes
4 Computer were used as this assisted in, minimizing the CPU & GPU so we did not saturate the display on GPU or CPU with more data than the devices could handle
1st Computer – Windows 10 64 bit Pro - build 105866.0
- There Leap Motion hardware controllers
- Leap Motion or Unity app capabilities of controlling Windows computer mouse
- Mixed in Key Flow 8 Application
2nd computer - Windows 10 64bit Enterprise - Build 105866.0
- There Leap Motion hardware controllers
- Leap Motion or Unity app capabilities of controlling Windows computer mouse
- Native Instruments F1 Controller w/ Traktor Pro 2 software
- Yamaha MG10xU Mixer, added for Effects application & its design to record audio for upload to sound cloud, which we leveraged for stems-music stem recording
3rd computer - Windows 10 32 bit - build 105866.0
- Roland's Motion Drive Tokyo software connected installed USB to hardware controller
- Motion Dive Tokyo Hardware controller USB to Roland SP-555
- V-Link SP-555 instrument connected direct from Motion Dive Tokyo with a XLR cable.
4th computer - Windows 7 Ultimate
- Gibson Cakewalk Sonar application
- This computer will recorder the live mixes in real time that are being sent from the Master Mixer to computer via RCA to 1\8th inch jack.
- This allow for the compliance for Native Instruments Stems-Music creator tool. Which requires 4 individual audio files and a Master Mix File.
- The leap motion controller hardware on this box is optional, since the design was to capture the audio stream
Methodology behind why we used this configuration
This, brings us back to the BT (Muse) application Inspiration for Future Fan experience earlier since offering the ability to Stream\Broadcast events through virtual software.
An AR Head Mounted Device would have been Ideal for the DJ\VJ operators.
- This way at a live event, User can physical see functionality of real application that the DJ\VJ have set for augmented display from the music workstations.
- Allowing users to validate the artist is using real-time software for live performance.
It's common for the new age controller artists to attempt to leverage a single laptop design for playing music, however with.
- Music
- Video
- Augmented projection mapping
- As well as Broadcast streaming
All of which are becoming core requirements for a DJ\VJ Live Performances setup. Most of these applications can be a very robust, requiring High pro computing hardware for the extensive load being placed on some single systems.
- CPU
- But the biggest issues are the GPU become saturated & over loaded till it can't handle the processes.
- then lock the Monitor display until GPU load decreases.
- This however fails to happen since the Visual affect will keep moving in a real-time loop until the computer is rebooted
- Unmarking CPUs on multi core systems did have a positive effect on the added load a single computer was able to absorb.
- Broadcasting the event through Microsoft's Office 365 services, also assisted in lowering the load experienced as the Broadcasted connection requests were not leveraging the computer being used for the live performance
When integrating Lighting and visual art's often the visual arts network is overlooked a the larger the event, the more projectors added, & users connecting to the broadcast a single laptop could not handle this load without significant over heating GPU issues like stuck frames or a complete display blackout.
Since we leveraged so many platforms like:
- Gibson Sonar
- Korg,
- Mix In Key
- Native Instruments
- Pioneer
- Roland's
- VDJ display
We have to leverage our Networking and IT background to design a MIDI network segmented by isolating the platforms based on platform & services.
Then configuring each of the isolated platforms with a standalone Computer, Mixer, & other desired device (effect processer, MIDI Interfaces) allowing for products & platform Isolation.
We found the MIDI clocking of same product was more accurate & the non MIDI interfaced drum\synth modules were more successful to Live Artist timing with is set in the platform based isolated\segmented network design.
- When in a production studio work environment lowering this cost overhead, for studio recording time…by not missing cue causing retakes.
- Example, we found that the Korg MIDI devices were almost 300 milliseconds off from Roland's devices & a MIDI merge devices would be required to keep the accurate
Once platform Isolation is completed, we then run each segmented platform based network to a Master Mixer, allowing us to lower cost, one large mixer for all MIDI & audio interfaces. This is a huge expense ranging from 1,000's to 10,000 plus dollars.
- we then loop the Master Mix through an Analogy EQ, back into its self-using (tape records out\in functions)
Thus users experience the live event while we record analog data that has the digitized effect and This clarity is not available with some of the older mono MIDI\Audio systems, so pushing through digital devices apply needed effects & running through the Analog EQ, is also signal elements are converted to Analog for recording.
Offer's expedited ability to press vinyl & gives a Master Mix Steam-Music file required for our Native Instruments Stems-Music creator tools. Isolated Digital Stems files, are created on each of the platform isolated network computers we used. So the artists can leverage the stems audio files if a digital audio file is requested, however most of the artists still prefer Analog samples for live mix processing, since affects & converting to digital is easily done in their own live mixes.
- Some people think Dj, only use Native Instrument Traktor or Serato scratch with CD\Loops & Sync Timing, through MIDI clocking
- or if Artists have a vinyl, show most of the Fans have little clue the vinyl played can be pre pressed from production track mappings during a studio setting.
- To be performed as trick mixing sets, which is a 2 to 4 turntable event, using vinyl records to remix tracks back and forth.
The audio track mapping was an Idea, we got after being bored with turntable mixing that only allowed us to playing others peoples music with the single enjoyment of Mashup creation using turntable trick mixing.
Purchasing drum machines, to preform remixes and samplers for live loop creation during the events, extended the enjoyment of trick mixing.
- The time has finally come where the drum machines are unable to be replaced & the value has gone from 450$ to worth 2400$ for example on a Roland's 303 or 808 drum machine.
- There's also was that overhead for paying the cost of scratching records,
- Pressed DJ vinyls were typical 50 play because DJ presses we lower cost. Since they were super thin
- if it was not for that wet scratch sound that made the thin DJ pressed vinyl popular, they would always useful as a Frisbee for Fans
- Scratching, other vinyl records can cost a wide range depending on popularity availability, in today's age let's, just say there not 0.1 cents like they use to be from Columbia house.
- It's rare you find a single under 10.00 anymore,
Most individuals will never know that hundreds of musical & visual applications loops samples that are need to be integrate in to the creation of a single live concert experience.
- However, if you are attempting this DJ\VJ challenge the once the network is complete & you have selected from the application & loop samples we provided you free.
- The network design you will use is done,
- Then the samples for the audio & video have to be timed ensuring proper, MIDI clock timing was setup for the final show.
Client viewing will need for Skype Live meeting Broadcast:
- Unity Web browser
- http://unity3d.com/webplayer
- Rendering and Special Effects in Unity setting configured
- Special Effects with Depth, setting configured properly
- Unity setting for Full screen if desired
- Rendering and Special Effects in Unity setting configured
- Option for other devices can enable Broadcast keys
- Unity Setup for Broadcasting
- Unity setting for Full screen if desired
- Unity configurations for WWW
- Microsoft app store Skype, Skype for business. Or Lync meeting
It's time to play
3D Leap Motion Virtual DJ\VJ Broadcast experience for the future
Once the music kicks in, the screen display shows the Virtual 3D Graphical Arts Videos, sync with the Roland's Motion Drive Tokyo. Which will connect via Roland's V-Link allowing the Motion Dive Tokyo video a real time tempo sync with the SP555 instrument.
were designed to allow for the comparison with Mix In Key Flow leveraging already spliced Stems & Native Instruments Traktor Pro using F1 controller and Yamaha MG10xU MU10 mixer for effects to create stems compliant audio & illustrate the DJ\VJ fan experience of the future.
We found that allowing for a Parallel Multi DJ music production using both Native Instruments & Mix In Key features as a Dual DJ was more enjoyable in which offered capabilities such as:
- Live musical collaboration
- By leveraging both musical application in Parallel extended the CueUps provided creativity and sharing of audio samples with each other that the other person testing did not have
- DJ Battle style play
- Where we created remixes and audio compilation then used a pump it or dump it style of judging to determine which version was better
- DJ Mix Off type play
- Where you have to find relevant & compatible audio based on what the other artist played that utilized Vocal samples & beat on beat Live (Mashup\Remixing)
- Along the way using the Dual DJ method
- It was found that a buy product of testing was offering an automated method for creating STEMS-Music Compliant audio files.
- Capabilities to test Leap Motions application stores (Music and Visual) applications
- Offering a Parallel head to head comparison of similar products was very nice
Our Projects Audio Mixing\Mastering:
- Native Instruments Native Instruments
- F1 Controllers using
- Traktor Pro software
- Stems music creator tool
- Yamaha MG10xU
- Used as an affect processor
- As well as send audio from Native Instruments Traktor Pro to Master Mixer
- Its feature for recording audio played, was leverage to obtain Isolated recorded stems from the Native Instruments Traktor Pro2 audio
- Master Mixer to Analog EQ
- Design to generate & convert the Digital Audio to Analog
- This was for the ability to Have an Analog Master file
- The data was then transferred to Cakewalk Sonar
- Which recorded the master Stem compliant file needed for
- Native Instrument Stems-Music creator tool compliance
- Design to generate & convert the Digital Audio to Analog
- Mix In Key Flow 2 & 2.5 referred to as (flow 8)
- Usage with Leap Motions Windows control abilities
- Also using MIDI configured setting with
- Native instruments Maschine
- Native instruments Mikro
- Flow 8 by far accelerated live mixing and CueUp of Tracks & Stems vs Flow 2
- Flow 2 solid application however was limited to 2 track CueUps
- Both Flow applications allow for connection with VST plugins
- Offering usage with come cool legacy Open Source Instruments & Effects
- Gibson Cakewalk Sonar
- This application was set on an Isolated computer to recorded the Master Mix from all channels, this was used to create an analog audio copy of the complete Mix from all workstation.
- This was done, Because the Native instruments stems music application requires a master audio copy.
- Since each workstation records their Isolated mixes and then also set each song or loop used in the mix in to a folder on the device, we were able to functionally load, play & records all Native instruments required stems music files samples needed.
PROJECT ANALYSIS
Educationally
We learning how to transmit data packets from an existing MIDI based Audio\Video controllers using Synchronous Audio\Video MIDI Mapping, Clocking, Sync functions using Leap Motion Hardware & Software applications.
Which will all be Broadcast live on Microsoft's Skype for Business Online meeting, through Office 365 services.
- Each live meeting Broadcast number can maintain up to 10,000 users per. Online dial in bridge number.
Broadcast Pre mixed audio, video & leverage live audio\video applications that allow for presentation displaying for Movie Arrangements. Using Video Controller, Wireless communication, and transmit a .MOV & WAV file, through the Microsoft Skype for Business Online meeting, Office 365 service. For remote fans.
- If preforming for a live audience, while broadcasting leverage multi projector mapping & visual light stage.
Various aspects of our project, specifically with the Leap Motion & Widget App interfaces. Both of these interfaces were difficult to debug, in addition. We know from the beginning that it was important that we get the Microsoft Skype for business Live meeting configured and tested, since the application is in preview release.
- Isolate the Native Instruments Stems-Music creator tool Audio files mix & master with the F1 controller & Traktor Pro 2 software.
- Validate if the Device we obtained could handle the extensive load, from a live audio & video performance.
- I was found that we did not have available Computers to maintain the load of both, audio & Video without saturating the GPU.
- Recommended extending a 52-bit computer to 8Gig RAM best available processing speeds.
- Same for 64-bit computer, Expand with Max RAM and High quality processing capabilities
- Graphics card, needed to be 3D capable & of the highest quality available
- CPU multi cores need to be unparked and set to not throttle processing to a single sided core.
- We found frame rates @ 60 plus would start hanging lower processing capable devices
- Frame rates @ 120, would lock GPU system visual display capabilities with error Display saturation, requiring reboot
- Need to open Feature enhancement request with Microsoft, & provided this testing overview & my telemetry logs…to request Live broadcast that can leverage 3D capable systems.
- HD is supported, but the load this would place on hosted servers might not be reviewed fully,'
- However, the Hosted server took an extreme load of a single computer based system. Kudo, for the Microsoft load throttling with Skype for Business & Azure Live Meeting Broadcast systems
- Rendering and Special Effects in Unity setting configured is highly important
- Special Effects with Depth, setting configured properly
- Reference Visio loaded on our Microsoft One drive
- View -Permissions
- http://1drv.ms/1MPB0Xn
Our thought was validated that the Roland's Motion Drive Tokyo w/ V-Link SP-505 instrument needed to have its Video controller elements mixed in a VR built as soon as possible, because it would be difficult to debug software integrations, hardware compatibilities, while offering the traditional DJ\VJ experience without working controller's & Audio\Video Content.
- Documents from our research on our Microsoft One drive
- View - Permissions
- http://1drv.ms/1MhET1F
- Data and setting about Multi core CPU unparking
- View - Permissions
- http://1drv.ms/1MhERaj
- Documents for setting up Skype for business Microsoft One drive
- View - Permissions
- http://1drv.ms/1MPB0Xn
- Documents for setting up Unity Rendering & Webplayer for business Microsoft One drive
- View - permissions
- http://1drv.ms/1NiG9BB
- Documents for Music_Track_ASCAP_Copyright_Approval_Log Microsoft One drive
- View – permissions
- http://1drv.ms/1RH3Ozr
- Documents for Live Stems audio samples Microsoft One drive
- View – permissions
- http://1drv.ms/1XWrqDU
- Documents for Live Stems video samples Microsoft One drive
- View – permissions
- http://1drv.ms/1WHtCly
References that inspired our collaborative futurist project design:
VR: Virtual Reality - Paul Haeberli, October, 1994
Carlos Castaneda – Reference from Don Juan (shadow doors)
Amazing! Given that VR technologies suffered high Application & Hardware costs which diminished 3D rendering due to shadowing issues until recent years. Now that GPU Technology costs and performances have finally been evolving to support the evolution of VR within the future is quite amazing.
"Don Juan pointed directly to a boulder in front of us. "Look at the shadow of that boulder," he said. "The shadow is the boulder, and yet it isn't. To observe the boulder in order to know what the boulder is, is doing, but to observe its shadow is not-doing."
"Shadows are like doors, the doors of not-doing. You may say that there is movement in them, or you may say that the lines of the world are shown in them, or you may say that feelings come from them."
AR: Augmented Reality - Pablo Valbuena Sep. 2007
Le Corbusier. Modulor I. – Quote on: Architecture flow
Amazing! Given projection mapping displaying 3D objects on 2D Architecture spaces:
"Architecture is judged by eyes that see, by the head that turns, and the legs that walk. Architecture is not a synchronic phenomenon but a successive one, made up of pictures adding themselves one to the other, following each other in time and space, like music."
SPECIAL THANK
Leap Motion
For allowing us to be part of the 3D experience and for reminding us of futurist methodologies. For experiencing the creativity and imagination that Leap Motion illustrated with Generation 1 Technologies.
- Leap Motion provided:
- Artists,
- Corporations
- Educators
- Researchers and Scientists,
- A powerful low cost AR\VR device that has futurist designs
- Inspire uses with the physical hand gesture and motions within a specific application environment
- Integrating controls. Leveraged hand movement that offered specific hand gesture placement that was user configurable and made it stellar within itself.
- Providing a low cost HMD (Head Mounted Device) with a large field-of-view that is more viewer-centric, derivative to the compelling Oculus Riff as leverage to Leap Motions device design
- Partnering the 3D jam with the application firm, Unity and other major gaming platforms to assist Leap Motion users and developers with preprogrammed environments & code templates;
- Offering quick access to a variety of computer operating systems and the specialized libraries that needed to develop, test and develop quickly.
- Assisting with validating functionality of applications for integration with our project designs.
- Unity, Oculus and the Leap Motion SDK's were extremely useful and assisted in progressing the overall ideals of our 3D jam project.
- Allowing projects for the 3D Jam to have an open platform not constrained by over bearing rules that could have caused lower than expected results or, even a desire to abandon the project and 3D jam submissions because, it was not an exact spec or desired idealistic design that did not allow for creative R&D within areas that are less explored within the VR development space.
- Leap Motion has allowed us to work on compelling and needed features for future VR success. VR Audio & Broadcasting of live environmental content usage with our 2015 3D jam project submission in making the time spent was quite enjoyable.
After our review of the Leap Motion's product and its capabilities, offer multiple integration functionalities during our 2015 3D Jam project. Leap Motion illustrates everything that AR\VR technologists that came before us and challenged a programmer to offer capabilities in their futurist application designs.
Microsoft
Escalation engineering team's director in Dallas
for giving me a chance to work with them @ MSFT, while a short time.
- I was allowed to functionally build, Office 365 Enterprise 3 infrastructure integrated with Skype for business PSTN\PBX Online calling & Live meeting with broadcast services since 6-2015.
- Illustrated, through the Live Broadcast, Blueprint architecture diagram.
- Please send my thanks, to anyone that my still be with Microsoft, whom was part of the Media Player 9 encoding project team. We leveraged an application that's support started on NT, last major code investment 2000 and Minimal support with (XP, Vista) & was never ported for supported with 64 bit devices.
- The media player 9 encoding codex assisted with us obtaining functional support on Windows 10, 32 bit.
- When all tests on windows 7 failed because the application was never code signed, and the original driver company Edirol is no longer in business. From what we can tell, there have been nothing but quick patch driver updates in XP & Vista.
- Then when to zero support from Roland's, after fall out with the 3 companies involved in the project. Roland's, Edirol & Motion Dive
- The Media 9 codex allowed for us to leverage the Windows hardware driver support pack and correct the visual .dll event that cause use issue for quite a while.
- The media player 9 encoding codex assisted with us obtaining functional support on Windows 10, 32 bit.
- Skype Broadcast with Unity on a leap motion device, was found if set properly for Broadcast, Streaming Functionality & the client users add the Unity Microsoft Streaming Player application properly for HTML5 support. This was a smooth functioning service.
- The Live meeting service will be an amazing service offering, Kudo's for supporting and allowing us to successful test.
- Only known issue was with Office 2016 for 2 ½ weeks in Sept ember October 2015 – added my fix-action
- http://answers.microsoft.com/en-us/office/forum/office_2016-office_install/office-2016-2013-parallel-install-without-office/fa5d066d-84fb-4a89-aea4-7378c1d0f42c
- I believe that looking into, supporting 3D broadcasting with the following will ignite the Skype services in spaces looking for this services but not wanting the SGI HPC systems cost overhead.
- vDGA
- vSGA
- vGPU
My current preview evolvement and architecture overview Microsoft Skype for Business Online PSTN, PBX & Live Broadcast meeting services.
==========================================
MSFT consulting Services:
Below are the current configurations and services used during my design and deployment of the new Microsoft skype for business with PSTN calling and conferencing in the cloud.
- Using the following configuration to offer a live Internet Broadcast experience for up to 100,000 users & having the Microsoft online preview team available for real time support. Has been amazing, I would like to see if we could be the largest Live Broadcast test of the Microsoft Skype for Business Online preview services…by hitting 100,000 users!
This has been one of the coolest functional test to work on, even though I have leveraged it less than any other test. This has mainly been due to validating Vendor approvals prior to broadcasting.
Also, a 2016 update issue where the .cab files, failed to extract…reminded me on an issue with the 2003 support tools years ago.
Environment configurations:
Microsoft Azure and the Red Hat Cloud suite with testing using Apache Web servers as a FW & Rev proxy in Sky4buizz Unified communications
- Linux - I have been testing with Red Hat Cloud https://access.redhat.com/products/red-hat-cloud-infrastructure/ With the Microsoft releasing Azure-consistent experiences and services in your datacenter, Azure Machine learning, multi-platform virtual machine deployments Azure 2016 pack on premise, AWS, Windows Server, Linux, VMware, OpenStack, or OSX.
- http://www.microsoft.com/en-us/server-cloud/products/windows-azure-pack/
- http://www.microsoft.com/en-us/server-cloud/products/azure-in-your-datacenter/ Next gen cloud - Microsoft operations management
- http://blogs.technet.com/b/server-cloud/archive/2015/05/04/microsoft-brings-azure-to-the-datacenter-for-the-next-generation-of-hybrid-cloud.aspx
- http://www.gartner.com/technology/reprints.do?id=1-2G45TQU&ct=150519&st=sb Microsoft 2012 & 2016 and Cisco UCS
- http://blogs.cisco.com/datacenter/cisco-unified-computing-system-and-microsoft-windows-server-2012-get-the-most-complete-solution
- http://www.cisco.com/c/en/us/solutions/data-center-virtualization/microsoft-applications-on-cisco-ucs/index.htmlNote:
- http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/unified-computing/whitepaper_C11-732222.htmlCisco Dcloud labs: Connected Testing lab Cisco Call manager server direct to DCloud along with multiple Cisco UC hardware services which allowed for Virtual lab testing with live hardware, integrations for configuration validation by connecting OnPrem hardware to Cisco virtual labs.
- https://dcloud-rtp-web-1.cisco.com/dCloud/Modeling Labs -Test you network design and future integrations
Hachster.io
For allowing us to be part of the 3D experience, for seeing the needs for remote broadcasting & Voting me the honor to be a Head Start Finalist winner for Leap Motion 3D jam.
- I hope that the projects design to promote future huckster.io team members is admired, as well as not >sniping< open source code to develop another one off application design only as a recording of another CAD simulation.
- Legally as a cofounder for VR-Tactical Evolution Inc. & Nu Vision Sound Labs, this is not allowed.
- My admiration for the futurist programmer idealisms, compelled me to report on the multitude of available application on the market that had source codes readily available.
- I reviewed and provide amazing applications for Hackster.io Futurist programmers, these applications extremely useful and further development for quality live broadcast streaming with vDGA, vSGA, & vGPU are needed in the VR\AR space moving forward.
- I hope that other see that the historical data and reference material illustrates a future of a collaborative AR\VR experience. While assisting with the ability to expand their VR\AR graphical rendering. Looking forward to the capabilities of Musical artisits abilities to leverage these technologies in a live concert setting within the next 5 years.
- I look forward to future huckster.io projects, that spark my interest & can't wait to see how other's progress the Broadcasting, Audio & Video experiences with the data & application information provided.
- Expands the Virtual & Augmented Musical experience.
- I look forward to future huckster.io projects, that spark my interest & can't wait to see how other's progress the Broadcasting, Audio & Video experiences with the data & application information provided.
Mix In Key
Originally, researched as a lower cost review application for future DJ\VJ experiences in the VR\AR space. I quickly realized the ease of use setup & performance was truly an award winning application that could offer artists Emmy awarded quality music content.
After purchasing a legacy version of the Flow application, Mix In Key support rep Chad assisted with getting me. I am honored for him seeing my request to leverage their products in the Leap Motion 3 D jam Challenge. Then getting me in touch with, Yakov Vorobyev - President, Mixed In Key.
Yakov Vorobyev, honored me with an overview of his experience as a DJ and Artist turned application programmer technologist.
Whom, challenged me with the usage of the Flow 8 features in my Leap Motion Challenge. Not only challenging me the Mix In Key family could have easily charged me for their new product, but offered the new Flow 8 release to me without an extended cost.
- I was also honored since my project was a Microsoft services based test & there was no release on the Microsoft foundation yet available, they expedited me a release of the Windows version of Flow 8 in time to test, create and validate the quality of the application is superior to any other in its price range.
- The Flow software, even out's many of the competitor application that have an expense of 2 to 3 time more than Mix In Key's applications.
- I can't wait to see the day when Mix In Key natively supports the use of Leap Motion controller as a MIDI hardware instrument.
Of all the Music application I have purchased and leveraged, the Mix In Key product line across the board meets the futurist programming Idealistic moto I admire.
The support and feedback, is exceptional
Native Instruments
Take a moment to thank the Native Instrument organization for allowing me the ability to use their product in the leap Motion 3D challenge. Also for the legal team Maike Weber, for approving the capabilities of live broadcasting using the Native Instruments Products & services without jeopardizing or causing issues with trade mark, branding compliances.
As this is a critical requirement of Microsoft service based usage, the have always been very open in the ability to test and leverage their applications in testing & research. So I wanted to ensure that there would be zero compliance or later reports that harmed the Microsoft or native instruments brands.
Native Instruments support as always been helpful & quick to respond. This request of usage direct to the Native Instruments legal team, lived up to all the other experiences I have had with their organization over the many years as a product user & proud supporter in their Hardware & software solution.
Hands down Native Instruments is an organization that user's, promoter & vendors can be proud to affiliate with.
I am truly grateful for your timely responses, so I could meet the short time constraints of this project request.
Red Bull
Man, what an honor it was working with all the organizations involved with your company.
- Red Bull Music Academy,
- Red Bull Music,
- Red Bull Media House
- Finally, the Red Bull BC One team
Truly, each department assisted in validating each department individuals to assist with validating if I could use, Red Bull media materials within my leap motion broadcast.
I can see why the Red Bull organization promotes all sorts of extreme sports, customer based performances, events, shows & sponsors so many individuals that lead the way in cutting edge performances, extreme sports that captivate the Red Bull customer base.
I was honored that all the individuals within the Red Bull organization took the time to assist me, I hope that Red Bull sees the future for sponsorship with R&D style events like Hackster.io and how future programmers are revolutionizing products\services many of the Red bull sponsored evens leverage.
- Currently, I plan to Bring my Leap Motion 3D Jam & the future DJ\VJ Fan experience to Thre3style 2016.
- Pending my prearranged VR\AR cloud project with Microsoft & Authentix http://www.authentix.com/, offers availability.
- I have a good feeling Red Bull Thre3style 2016 will experience the new revolution of augmented & virtual reality based DJ performances?
- Offering VR live Broadcasting & a projection mapping Augmented style live event
Our NVSL & VR-Tactical Evolution Inc. Team:
Bela Chavez
- At 12 Bela, was the reason for our submission her interest in the future gaming experience. Her desire to promote NVSL Music with new age ideals & her love for unique new technologies lead use to the Leap Motion 3D jam. After review we wanted to help her with project ideals, development & testing that illustrates & provides developers with applications to create a new DJ\VJ fan experience in the near future.
Connie S Bryson
- Graphics project review & documentation support
Ian Neveu
- Thanks for assisting with the Dual DJ experience mix off & battle testing
Rachel Neveu
- Graphics project development testing supporter
Shawn K Bryson
- Primary Research, design and project development test
Download
