Core player finished
I started this week off with a consult with my coach on monday. We discussed the project, the goals and the planning for this week.
Raycasts #
Firstly I got started on getting raycasts working. I checked out the three.js boxes demo and the raycaster docs to get a grasp on how to use it. I implemented the raycaster in the same fashion as the demo where it shoots a ray from the controller and gets all objects that it collides with. This is used to put a highlight effect on the object that you're currently pointing at.
Now that I had raycasts in place and could detect which object you are selecting, I can now look up the object in the sound group easily via sounds[obj.name] and simply change the volume of the stem playing.
To know when to mute/unmute the stems I needed to detect controller trigger inputs. This is very easy but has a fatal flaw. The function that detects if the trigger is pressed runs on each frame. This means that if you're pointing at an object and the trigger is pressed, the stem will be muted and unmuted constantly.
This was the basic (flawed) code:
if (controller.userData.isSelecting) {
const volume = sounds[INTERSECTED.name].getVolume();
switch (volume) {
case 0:
sounds[INTERSECTED.name].setVolume(1);
break;
case 1:
sounds[INTERSECTED.name].setVolume(0);
break;
default:
sounds[INTERSECTED.name].setVolume(0);
break;
}
}
You can see that this is just a simple switch case that updates the volume based on the current volume. This could've been a ternary operator, but I chose to use a switch case here because it allows me to add cases for other volume levels too later on, like .5 or .25, etc...
To fix the while loop created by just checking if the trigger is pressed, I added a boolean isHolding to tell me if this is the first frame that the button is pressed, or if the trigger is being held down.
if (controller.userData.isSelecting && !isHolding) {
isHolding = true;
const volume = sounds[INTERSECTED.name].getVolume();
switch (volume) {
case 0:
sounds[INTERSECTED.name].setVolume(1);
break;
case 1:
sounds[INTERSECTED.name].setVolume(0);
break;
default:
sounds[INTERSECTED.name].setVolume(0);
break;
}
} else if (!controller.userData.isSelecting && isHolding) {
isHolding = false;
}
Now that I got this working, I was able to easily mute and unmute the stems.
Here is a demo showing this working with sound:
Optimization #
The initial demo code I wrote was a messy proof of concept, so I cleaned that up with a more permanent data structure.
I managed to eliminate a little over 200 lines of redundant code which is about a 40% improvement.
Instead of copy/pasting the same function over and over for each stem, I organized it into a single config object.
const objConfig = [
{ name: "atmos", color: 0xddd3c3, sound: "atmos" },
{ name: "bass", color: 0x812ade, sound: "bass" },
{ name: "drums", color: 0x3444e3, sound: "drums" },
{ name: "guitars", color: 0xc714d6, sound: "guitars" },
{ name: "lq", color: 0xe4406e, sound: "lq" },
{ name: "pads", color: 0xffa211, sound: "pads" },
{ name: "plucks", color: 0xffde4a, sound: "plucks" },
{ name: "tags", color: 0x02d37a, sound: "tags" },
{ name: "vocals", color: 0x51c5c5, sound: "vocals" },
];
I loop over this object and create all the materials, meshes, sounds and analyzers. This gets pretty much everything done, but I can stilla access and modify the objects later on like this meshes['guitars'].position.set(- 100, 30, 100). This sets the position of the guitar mesh in an easy to read way.
I also group each sound, stems and meshes into their own group.
Loading videos #
Loading videos was pretty easy, until I ran into CORS issues. I wanted to load a video from my cdn and somehow ran into issues because the access-control-allow-origin header wasn't set.
Sooo... to Cloudflare.com it was. Under Transform Rules I was able to modify the response headers for my cdn domain.
Initially I wanted to set the header to whichever domain requested it. I thought this could be easily done by taking the origin header and just serving that back, but Transform Rules apparently doesn't allow you to access an array. This seems reserved for Firewall rules.
Looking around the web for quite some time I figured out that the dynamic rewrites are based on cloudflare's firewall rules. This helped me find this page which has a list of all available dynamic objects, one of which is http.referer. I used this one to set the header to the domain that requested the video.
This worked for a bit until for some reason it didn't. I think I know why, but have not been able to confirm it. CORS takes your origin, say http://127.0.0.1:5000 and checks if the access-control-allow-origin header is set to the same thing. Apparently when using http.referer, it sets it to http://127.0.0.1:5000/. Notice the trailing slash. I think because these dont match, the CORS check thus isn't valid and doesnt load the video.
I have temporarlily fixed this by just setting a static rewrite to allow everything until I can test this further.
