Matthew Carlson

Computer Scientist
Game Developer
Immersive Technologies Advocate
World Traveler

My Resume


Throughout my life, I had the good fortune to be able to travel from Ecuador, Brazil, Panama, Nicaragua, and now Colombia. My family traveled from country to country following my dad, who despite having lost his sight, continued to tirelessly work in organizations like Plan International, and currently in the Peace Corps providing humanitarian and developmental aid to impoverished communities worldwide. Having seen my father lose his eyesight at the age of four, and watching him not give up, has given me the drive to develop my passion to help others. I just had no excuse, because if he could do it, then I can do it too. My passion to help others manifested itself in my love for computer science and game design, as I continue to figure out ways to push the boundaries of hardware and software to help aid peoples day-to-day lives, be it through learning, or mental health.

My goal is to form part of the development teams that work on finding new ways to bring information, and commodity to those who suffer from handicaps, working on research aimed at using immersive technologies to aid in learning and helping mental health patients. I hope to bring to light the potential behind these immersive technologies by using the more commercially available virtual, augmented, and mixed reality headsets, alongside custom hardware, to bring these solutions to the general public.

As a recent 2020 graduate from Rensselaer Polytechnic Institute, with a dual bachelor of science in Computer Science, and Games and Simulation Arts and Sciences, I have had the chance to try the many flavors and dimensions of my fields ranging from data science, machine learning, and data analysis to artificial intelligence, shaders, game design, and development. I aim to keep working on research, and games that mix both of my fields, to bring out the best of both worlds while creating memorable and cohesive experiences to all players alike.

I am a Computer Scientist, Game Developer,
World Traveler, & Immersive Technologies Advocate

Work 1

I hope to help people understand each other better through the use of technology, providing a platform for empathy.


C++ / C / C# / Python

Unity / UE4


English / Spanish / Portuguese


Hypnosis Virtual Reality
Eco-Resilience Games
Cognitive and Immersive Systems Lab
Magicians and Minions
Fear Factor
Quest Hall
The Mystery of the Hansom Cab

Current Research

Hypnosis Virtual Reality

A personal research study that I started as Junior during my undergrad at Rensselaer Polytechnic Institute. This interdisciplinary research is my attempt at finding the critical point between computer science, game design, psychology, and cognitive science. My goal is to use the advances in immersive technologies, with biometric data, to create more immersive user experiences aimed at helping with user learning, and mental health patients.

IRB Research Paper

Past Research Project


Eco-Resilience Games

As a team member of Eco-Resilience Games I had the opportunity to be the team manager, and lead programmer for the various projects worked on under Professor Kathleen Ruiz: The Aquatic Messenger, The World of Plankton, and Algae Bloom Dynamics. We had the mission to educate others about the importance of freshwater ecology by using virtual and Augmented reality, to create worlds where a user can immerse themselves into the world of ecological awareness. The player will then be able to interact with different zooplankton, phytoplankton and microfossils of different sizes, and in different periods. Our freshwater ecology is beautiful and our need to conserve it is urgent. We hope to create more awareness by creating these programs for museum settings as well as the general population worldwide.

We have been using Unity to develop both projects, and as of recently, we have been exhibiting our project in several museums and to investors.

Undergraduate Team: Matthew Carlson, Eric Lujan, Brendan Courson, Sol Toder, Harrison Essig, Hongyang Lin, Autumn Walters, Hantian Jiang

Cognitive and Immersive Systems Lab

Working on The Mandarin Project the project aims to use immersive technologies, specifically a 360 panoramic projection of China (Studio 2 in EMPAC) to immerse users into a setting of China. By using this kind of immersion, the hopes are that it would work similarly to learning a language via assimilating in real-world China.
My specific role in 2018 was to integrate four HoloLenses onto the existing system. This allowed me to get hands-on experience in reading and documenting existing code while trying to create a new novel experience with mixed reality technology.
In 2019 I worked on a subset of The Mandarin Project working on a research-oriented tool, called SpeakEasy, to aid students who are currently enrolled at RPIs Mandarin courses. Being part of the undergraduate development team, we have been tasked with building a platform that will use various tools, and games to gather user data, in order to have an AI help users learn Mandarin more efficiently.

In Development


Terrium is a longterm online vehicular laser tag game under the direction of Eric Lujan. Terrium is played on land, sea, and air, as well as having a world creation mode where players can create maps that they can play in. The game revolves around the concept of non-violent combat, where instead of players having guns and blowing each other up, we decided to take a different approach. With the use of lasers, relaxing original music, and beautiful sound effects, we hope to create a serene, engaging game that players can enjoy. My roles thus far have been helping with debugging, adding components such as the tutorial, develop the business model, and pushing the game from alpha to beta.

We are using Unity for this project and recently have been showcasing our project in competitions as well as conventions such as GDC. Below is a video I made for our submission to the New York State Game Dev Challenge

Team: Matthew Carlson, Eric Lujan, Nicolas Aragone, Lamp

Magicians and Minions

A project I started a year ago intending to create a turn-by-turn strategy board game, revolving around the use of unique minions and spells to dominate one's opponent. The game was created on a physical board, and character sheets that the players use to keep track of the values. All of the information is then displayed on the UI for the players to process more easily. I worked on this as part of my Game Development 2 project, but have since been developing it on my own to create a more seamless experience.

We are using Unity, 8-bit sprites for characters, unique UI experience, and original music.

Alpha Release: October 2020

Team: Matthew Carlson, Alex Giles, and Ben Eid


Initially a project started in Experimental Game Design, we developed a cooperative interactive game, with custom hardware controls. To play the game, it is required that two players have to figure out what combinations of controls they need to be able to manipulate a half organic, half robotic mouse, in a 3D world environment. This game aimed to show the importance of cooperation and communication, to be able to advance and explore the secrets of the in-game world.

We used Unity as our game engine, as well as Maya for modeling the character and objects in the 3D environment. We were also chosen to exhibit our game at the Rensselaer Polytechnic Institute booth at the 2019 Game Development Convention.

Team: Matthew Carlson, Cole Carpenter, Connor Griffin, Jose Sanez, Maria Salmon, Eric Lujan

Work 1

Symbionic Gameplay

Voder 2017

The original Voder was created by Homer Dudley in Bell Telephone Laboratory, during the years of 1937-1938, and then showcased in the New York World's Fair in 1939. After that the Voder was stowed away forever. We believe this old, but not yet lost technology, could be of some use, especially as a different approach for a more efficient and lightweight (in terms of usage) text-to-speech system. We are working on digitizing the Voder in our Voder 2017 speech sythensizer, using spectrograms, and sound frequencies for consonants and vowels. We are using the results to create scripts to ease the usage of the digital Voder we have created.

We are using C++ as well as The Synthesis Toolkit to emulate the Voder. We have been actively reaching out to AT&T and Bell Labs for information about frequencies and graphs for pronunciation, and relevant archived information.
Currently we are conducting our own research, learning of existing Machine Learning approaches to natural language processing approaches, to be able to solve our data problems.

Team: Matthew Carlson, Sinclair Gurny, Matthew Dineen, Nitesh Dagli

Work 1

Finished Game


Fear Factor

As a 3-week assignment our group developed a game based on the theme (paraphrased), "working together for a common goal". With this idea we decided to build on a mechanic where the player can switch between the characters in the game. Using that mechanic, we then developed puzzles based on handicaps of the characters fears (i.e. afraid of the dark, can't be alone). This requires the player to use all the characters to solve the puzzles and get through the area. We are addressing individual fears, as well as recognizing, empthaizing with and working together around those fears for a common goal. The characters in our game are children, as we were playing off the theme of child-like fear, but also creative problem solving.

Our project was made in Unreal Engine 4 and Maya.

Team: Matthew Carlson, Max Sichel, Mengyi Li, Tim Kim, Claire

Work 1

Fear Factor Gameplay

Quest Hall

As a 2-week assignment our group developed a game based on the adjectives "elude" and "creep". We decided to make a very basic dungeon crawler with the place being one of the haunted building on our campus, and the character as a RPI student.

Using Haxeflixel, we were able to create a mechanically sound character, as well as having the character and enemies (police and ghosts) be made as 2D sprites.

Team: Matthew Carlson, Matthew Yoder, CJ Legato, Sara Bruton, and Erik Gunderson

Work 1

The Mystery of the Hansom Cab

As a 2-week assignment our group developing a game based on the book "The Mystery of the Hansom Cab", as well as only having a tap and drag (click and drag) mechanic.

Using Phaser and Javascript, we were able to create a simple, but fun mystery game having players search the homes of suspects, and finding items in the environments. All the art is hand drawn to create a noir effect, as well as having original music.

Team: Matthew Carlson, Axel Stewart, Claire Zhu, Christopher Chen, Daniel Hendricks

Work 1

Git Commits


Current Projects


Research Projects


Cups of coffee

  • "A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects."

    -Robert A. Heinlein


Send Me a Message, Recommend a Book or Game, or Share Your Thoughts

2005 15th Street, NY, 12180

+1 (925)391-8261