Category: Serra Week 1-10

Returning from the break, I dove into a new project: Artefact, guided by Serra. The challenge was to create an engaging user experience. As an aspiring environment artist, I aimed for stunning visuals and compelling compositions. Serra suggested a playful concept: a Star Wars blooper reel. In this story, stormtroopers, Darth Vader, and other characters are actors who unwind at a pub after a long day of shooting.

To achieve realistic animation, I used Vicon motion capture and created the terrain in Houdini for the establishing shot, honing my skills in procedural modelling. This technique allowed for highly detailed and customizable landscapes. The scene was assembled in Unreal Engine 5 (UE5), known for its speed and powerful capabilities, and the final compositing was done in Nuke. The process of procedural modelling in Houdini was incredibly intuitive once I grasped the underlying math and logic, and I was thrilled with the geometric foundation I created.

For the character animations, I booked a Vicon motion capture session. My friend, a Star Wars enthusiast, acted out the shots, including stormtroopers standing guard and interacting in a pub. Despite some initial challenges with the hip movements in the mocap data, Serra and I identified the issue, and new exports were provided. Cleaning up the animations, particularly the intricate finger movements, was time-consuming but rewarding. The characters dancing and interacting brought life and realism to my environment.

Creating the pub scene was a critical part of the project. Using Quixel Bridge assets, I built the structure and set a warm, inviting atmosphere with detailed props and moody lighting. After some layout adjustments and rendering challenges, I resolved issues like noise on metallic surfaces and selected appropriate royalty-free music to match the vibe. Through this process, I refined my technical skills and learned the importance of attention to detail and problem-solving. Now, with the pub scene complete, I look forward to sharing the final product and continuing to explore the nuances of digital art and animation.

This is the Showreel:

It was time to create the pub scene and bring everything together for my project. To construct the pub, I used assets from Quixel Bridge to build the structure of the building, ensuring to include windows for optimal lighting. Once the basic structure was ready, I laid out the scene with various models I had created, along with some additional assets like the Guinness pint.

Initially, I experimented with different layouts. My first attempt featured two stormtroopers sitting and talking at a table, with two more dancing in the background. However, upon rendering the scene, I realized something was off—the dancers in the background were drawing too much attention away from the main interaction. To remedy this, I revised the layout, positioning the dancers in the foreground to better balance the scene.

I paid close attention to the details that would make the pub feel authentic, such as including a pool table and setting a yellowish, moody lighting typical of many pubs. This lighting choice helped create a warm and inviting atmosphere. However, I encountered some challenges during rendering, particularly with noise on metallic surfaces. After some adjustments, including increasing the roughness, I managed to resolve the issue and achieve cleaner renders.

Another challenge was selecting the right music for the shots. During the motion capture session, my friend danced to “Canned Heat,” but due to copyright restrictions, I couldn’t use that track. I spent some time searching for royalty-free music that matched the vibe of the environments and the characters. Eventually, I found a suitable track that complemented the scene well.

After resolving these issues, I brought everything together with a simple composition in Nuke, followed by editing the final sequence in Premiere Pro. This process not only enhanced my technical skills but also taught me the importance of attention to detail and problem-solving in creating a cohesive and engaging scene. Now, with the pub scene complete, I feel confident about the overall direction and look forward to sharing the final product.

To seamlessly transition between the Star Destroyer scene and the pub scene, I needed to include some filler shots. I went for simple shots of a bridge and some buildings on a street to create a cohesive flow. For these filler shots, I utilized pre-made assets, which made the process much quicker and more efficient.

The goal was to maintain a consistent visual style and atmosphere while providing a smooth transition between the distinct environments of the Star Destroyer and the pub. By incorporating these additional shots, I aimed to enhance the narrative flow and maintain the viewer’s engagement throughout the sequence.

This week was a major challenge; creating the Star Destroyer scene for my project. The scene portrays a trooper on guard duty as Darth Vader passes by, with the director calling “cut,” revealing it as part of a filming set. After modeling the environment and adding characters and animations to the scene, I realized that one of the key elements in bringing this scene to life was the lighting.

To ensure authenticity and capture the essence of the Star Destroyer environment, I rewatched “Rogue One” to reference the lighting used in similar settings. Then I began the lighting process in UE5, trying to make each light source mimic the atmospheric and dramatic lighting of the Star Destroyer. Using ACES color space, I rendered the scene elements as EXR files, for the high dynamic range and colour accuracy.

Once the lighting was finalized, I imported the rendered EXR files into Nuke for a simple composition and colour grading. This step allowed me to fine-tune the overall look and feel of the scene, enhancing its cinematic quality. Overall, this process underscored the importance of attention to detail and the significant impact that lighting can have on the mood and atmosphere of a scene.

This week has been quite a rollercoaster for me as I started importing mocap data to UE5. It’s been an exciting journey, but also one filled with challenges. My main focus was on cleaning up the animations and preparing them for import into my scenes. One of the key aspects I worked on was the rig retargeting system in UE5, which allowed me to seamlessly integrate the mocap data into my projects. This system made it easier for me to map the animations to different characters and ensure they moved realistically within the game environment.

However, I encountered some problems along the way. Despite my best efforts, the hip movements in the mocap data looked off, and I couldn’t pinpoint the issue. After asking for help from Serra, we discovered that the export from the mocap team was incorrect; every animation had 2 skeletons. This setback meant I had to request new exports, leading to a delay that added to the pressure of meeting submission deadlines. To keep things moving, I improvised by using Mixamo for one of the shots, allowing me to continue with my plans while waiting for the corrected data.

Thankfully, after a few days, the team sent me the revised data, and some of it worked seamlessly. I began importing and cleaning up the animations, focusing particularly on refining the finger movements, which proved to be time-consuming but worthwhile. The end result is truly impressive—dancing, sitting, and talking animations that look incredibly realistic. I can’t wait to bring them to life in my environments and see how they enhance the overall experience. Overall, this experience has taught me a great deal about problem-solving and perseverance in the face of unexpected challenges. It’s been a valuable learning curve, and I’m excited to see how these mocap animations elevate my projects in UE5.

This week, Serra introduced us to the world of VJing with Resolume VJ, and it was an incredibly fun and insightful experience. VJing involves mixing and manipulating visuals in real-time to accompany music performances or events. Resolume VJ is a powerful software tool commonly used by VJs to create captivating visual experiences.

In our class session, we tried various features and functionalities of Resolume VJ, learning how to manipulate videos, apply effects, and synchronize visuals with music. Emily and I even experimented with setting up controllers to VJ to “Animals” by Martin Garrix, which added an extra layer of excitement to the process.

However, VJing also comes with its challenges. Matching the visual elements to the vibe of the music and ensuring that the visuals complement the overall mood of the performance can be quite stressful. It requires quick thinking and creativity to adapt to the rhythm and energy of the music while maintaining a cohesive visual narrative.

Despite the challenges, the experience was incredibly rewarding, and I gained a deeper appreciation for the artistry and skill involved in VJing. I look forward to further exploring this creative medium and honing my skills as a VJ in the future. Who knows, maybe one day I’ll be behind the decks, creating visual experiences for audiences to enjoy.

As mentioned before, one of my main focuses for this artefact is my establishing shot and the environment. Initially, I planned to render the environment in Houdini using the Arnold render engine. However, due to time constraints, I decided to move everything to UE5 and create a digital matte painting.

Creating a digital matte painting in UE5 involves leveraging the engine’s real-time rendering capabilities to build highly detailed and dynamic environments. UE5 offers a range of tools and features, such as Nanite for handling complex geometries and Lumen for real-time global illumination, which can significantly enhance the visual fidelity of the scene. This approach allows for quick adjustments, making it easier to experiment with different compositions and lighting setups. Additionally, UE5’s integration with other software and its support for ACES colour space ensure that the final output maintains a high level of colour accuracy and realism, essential for creating immersive and believable environments.

One of the most important aspects of matte paintings is colour accuracy. For this project, I rendered everything in ACES colour space, which is known for its wide colour gamut and accurate colour representation. This step was crucial in ensuring that the colours in my scene were true to life and consistent across different devices and screens.

After rendering, I moved the renders to Nuke for compositing. Compositing in Nuke adds realism and depth to the shot by combining various elements seamlessly and enhancing the overall visual impact. The next step will be finalizing the shot that takes place on the Star Destroyer. This will involve ensuring that all the elements come together cohesively and that the environment looks polished and professional.

As I mentioned before, I decided to use Vicon motion capture for the character animation of my artefact because it’s both fast and realistic. To get started, I booked a session with Kay and enlisted the help of my friend, who is also a huge Star Wars fan, to act out my shots. For all the animations, I needed detailed finger movements, so we had to attach sensors to the nails using double-sided tape. This process was quite tedious, but essential for capturing the nuanced hand gestures.

After calibrating the cameras and getting Stanley suited up, it was time to record. We captured all the necessary animations: stormtroopers standing guard, Darth Vader passing by, and two stormtroopers sitting and holding a drink. At the end of the session, Stanley even danced to a song, and we recorded the entire performance. If time permits, I’m planning to include this fun dance in the bloopers of my artefact.

Vicon motion capture, widely used in the entertainment and gaming industries, is known for its precision and ability to capture complex movements. The system uses multiple cameras to track reflective markers placed on the actor’s body, translating their movements into 3D data. This data can then be applied to digital characters, ensuring realistic motion and behaviour. Vicon’s system is highly valued for its accuracy in capturing fine details, making it ideal for animations that require lifelike motion.

The session lasted about four hours, and everyone involved had a great time. I’m now looking forward to integrating these animations into UE5, as Serra taught us, using retargeting and cleaning up the animations to fit perfectly into my project. The experience was incredibly rewarding, and I can’t wait to see how these animations will enhance my artefact.

This week, Serra and Suny introduced us to TouchDesigner, an exciting software that merges real-time 3D graphics with interactive control. What struck me most about TouchDesigner is its ability to seamlessly integrate with various inputs and outputs, allowing for a high degree of interactivity and customization. Whether you’re incorporating live camera feeds, sensor data, or MIDI controllers, TouchDesigner can handle it all, providing a dynamic canvas for creative expression!

In our sessions, Serra and Suny highlighted the immense potential of TouchDesigner for creating stunning visual effects and interactive experiences.

We also had three sessions with Pierre, who guided us through the practical aspects of using TouchDesigner. Pierre’s workshops were incredibly informative, covering a range of topics from project mapping to creating audio-reactive visuals. Through these sessions, I gained a deeper understanding of how to utilize TouchDesigner’s tools and features to bring my creative ideas to life. One of the most valuable lessons was learning how to manipulate geometries and transform them into dynamic visual elements that react to music.

As a practice project, I created a visual piece by twisting two geometries and then transforming them into thousands of points. These points were then mirrored and expanded to respond to a track that I love dancing to. This exercise not only helped me apply the technical skills I learned but also allowed me to explore the artistic possibilities within TouchDesigner. The end result was a captivating visual that moved and evolved in sync with the music, showcasing the power of TouchDesigner to create immersive and interactive experiences.

As a start for my Artefact project, I focused on creating a procedural terrain using Houdini. Procedural modelling in Houdini involves using algorithms and mathematical functions to generate complex shapes and environments dynamically. This approach is particularly powerful for creating terrains because it allows for highly detailed, customizable landscapes that can be easily adjusted by changing parameters, without the need to manually sculpt each element. By using Houdini’s node-based workflow, you can create intricate terrains that can be replicated or altered with minimal effort, making it an essential tool for environment artists.

My experience with Houdini was fantastic. Once you understand the math and logic behind procedural modelling, it becomes incredibly powerful and intuitive to use. I had an amazing time working with this software and managed to create the geometric foundation needed for my establishing shot. The next step in my project will be to create a matte painting using these assets in Unreal Engine 5 (UE5). I’m excited to see how these elements come together and to continue refining my environment creation skills.

After coming back from the break and submitting the outcomes of our collaborative unit, it was time to start a new project: Artefact, under Serra’s advisement. Serra wanted us to create something that involves user experience. Initially, I had no idea what I was going to do, so we discussed the skills I wanted to improve, and she helped me come up with a solid idea.

As someone who aspires to work as an environment artist, having stunning visuals and compelling compositions is one of my main goals. For the story of my artefact, Serra suggested a playful twist: a blooper reel of Star Wars. The concept is that stormtroopers, Darth Vader, and all the characters are actors on a set. After a long day of shooting, the stormtroopers decide to unwind at a pub and enjoy a drink or two!

To achieve realistic animation, I’ll be using Vicon motion capture. For the establishing shot, I’ll create the terrain in Houdini, which will be great practice for improving my skills in that software. The scene will then be assembled in Unreal Engine 5 (UE5) due to its speed and powerful capabilities. Finally, I’ll composite everything in Nuke to polish the visuals.

In the coming weeks, I’ll be posting about my process and the various things we learn in Serra’s class.