Category Game Development

---

It’s been a while since OpenAI launched Sora to the public (available to ChatGPT Plus users, by the way), and I finally got a chance to experiment with it. Spoiler alert: it’s incredible!

You can check it out here: Sora Library.

I spent some time playing around with it, and here’s one of the results I came up with:

This is a scene from a puzzle game we’ve been working on. The game has a unique twist — it combines classic puzzle-solving mechanics with a meta-layer where players can fix up different parts of a room and earn stars as rewards. Think of it as a mix of puzzles and home improvement, similar to some of the trending games out there right now.

So, here’s how I experimented with Sora: I kept it simple — just straightforward prompts, nothing too fancy or overly detailed.

Prompt: animate the girl and the cat. move the camera and keep the rest of the scene static (attached the image as the source)

Result:

Prompt: animate this scene (attached the image as the source)

As you can see, it’s pretty unbelievable how Sora has managed to take the entire image, turn it into a three-dimensional world, and move the character while keeping everything spot-on with the static image. The attention to detail, from the shadows to the textures, is absolutely flawless.

Merging the two videos and adding some sound effects turned it into a solid video that’s perfect for use in marketing materials or as an engaging landing video before the game kicks off.

End Result: 🥁🎶

I’m honestly fascinated by the potential of Sora. The ability to use 2D artwork as the source opens up a whole new world of possibilities!

---

So hear me out… Has this ever happened to you? You’re looking for an icon or an image to use in Unity. You open up a website, search for an image, and instead of saving it, you right-click and Copy image.

So far, so good, right? Now you go back into Unity, and oops, you can’t paste it because, well, it’s not possible. The only thing you CAN do is go back to the website, right-click, and select “Save Image As.”

Now you’re going to see this familiar dialog (attaching a Windows screenshot, but Mac users can relate as well).

There are two options now:

1. You try to find the project directory and save the image there directly.
2. You save it in an easy, accessible folder and after that, drag and drop the file into the Unity project window.

Both these options have some issues.

#1: You’re never able to find the project folder. You can’t have all your projects in your favorites/quick access, and it’s just a lot of trouble to navigate your way into the Sprites or whatever folder you need to (often again and again because for some reason the OS forgets the last path)

#2: You’re going to end up with a desktop looking like this.

Hey, and what if the file is a webp file? You might need to convert that to a .png before you can add it in Unity. Or what if the filename looks like 
icon_452_Clock_final_final1_final.png when it’s saved? There are a lot of unnecessary clicks, keypresses, and frustration.

That’s something I faced for many months until I decided …

There’s got to be a better way!

First, I looked for a ready-made solution. None exist. Okay, no problem — let’s create an editor utility that reads the clipboard. Sounds easy right?!

An editor utility in Unity is a custom tool or script that enhances the Unity Editor’s functionality, streamlining the development workflow. These utilities can automate repetitive tasks, create custom inspectors, or add new features to the editor interface.

All I need to do is use EditorGUIUtility.systemCopyBuffer and get the image data?

Well, unfortunately you CAN’T read the raw clipboard data. Yes, you can get text, but not an image.

---

After numerous back-and-forth and trying multiple libraries, I gave up. There is no way this can be done just within the Unity editor. We need to be able to read the entire clipboard and perform certain operations that are just not exposed in Unity editor scripting.

So drum roll… 🥁 presenting the “actual” use of multiple tech stacks that I mentioned in the first line.

I turned to my favorite framework for creating desktop apps: ElectronJS.

Now you can read the clipboard using the built-in Electron.Clipboard library. It has built-in functions called readText and readImage that do pretty much what they are supposed to.

Okay, so now I can get the image that I’ve copied in an ElectronJS desktop application. Now I need to transfer this image to my Unity project/editor.

The tricky thing here is inter-app communication. Or in my case, standalone Electron app to Unity editor script communication.

Still with me? Let’s recap the problem and my proposed solution clearly now:

Problem: I want to be able to paste any image that I’ve copied into the Unity project window (preferably in the folder that is currently open). I just want to be able to copy images from anywhere (web browser, Photoshop etc. ) and paste them in Unity without any complication.

Proposed Solution: Electron app and Unity will work in tandem. The Electron app will monitor and manage the clipboard — if there is an image copy event triggered on the OS, the Electron app will process it and create an image file in a common folder.

The Unity editor script will monitor the Ctrl + V keypress (only if inside the project window) — if triggered, the function will copy the image (created by the ElectronJS app) from the common folder and paste it in the current active folder inside the Unity project.

---

Challenges:

• How to capture Ctrl + V hotkey inside the project window of the Unity Editor.
• How to capture the clipboard onChange event in ElectronJS.
• Image metadata is not saved in the clipboard, so what should the name of the image be?
• Handling all error scenarios: What if text is copied? What if the image is in webp format? etc.

So, it took the whole of Saturday (20th July 2024), but I managed to do it! Yay! 😺

---

Some workarounds (or “jugars” as we call them in my native language):

• Clipboard change event does not exist. The only way to monitor the clipboard is to have a function running (setInterval) that compares the clipboard data to the last data.
• Ctrl + V inside the project window of Unity can only be done using Reflection. Unity doesn’t expose this function, so we have to use reflection to peek inside the class and get its value.
• We cannot get the name of the image; that’s just not stored in the clipboard. Instead, what we can do is quickly show a dialog and ask the user to enter the name of the image that they are pasting. This is actually better because the developer can use their appropriate naming convention instead of leaving it for later and ending up with a whole bunch of sprites with ridiculous names.
• There is no need to show the Electron app. So I decided to put it into the system tray.

Here is another look at the code in Electron that handles the clipboard.

And here is the C# snippet that handles the Ctrl + V hotkey and pastes the image inside the Unity window.

Results

It works as expected. The Electron app runs in the background like a service — and does it’s job of saving images to a common folder. Unity editor works like a charm as well — capturing the paste hotkey and using file operations to paste the image.

So while wrapping up, I had another AHA moment.

When I completed the image paste functionality and tested it multiple times, including on a live project I was working on, I figured, what if we had the ability to paste different files as well? — and not just images?

For example, if we’re copying some code from ChatGPT — we have to create a C# file, name it, change its location and then paste the code … etc.

Q: What if I could copy the code generated by ChatGPT and simply paste (CTRL + V) it inside Unity, and that would create a .cs class for me?

Q: What if I could copy a JSON file and just paste it to create a .json file in the current project window folder that I have opened?

So, here it is, presenting — Instant Clipboard 1.0.

The features are as follows and the utility (both electron and Unity is available on GitHub: https://github.com/saadnkhawaja/instant-clipboard-unity

• Paste images from any source directly into Unity.
• Paste C# code, and the utility will create a C# class for you automatically.
• Paste .webp files directly as .png in Unity with one hotkey.
• No need to convert .SVG or other complex image files (all images will be converted to .png with alpha channel intact)
• Multiple formats supported: XML, JSON, markup (MD), shaders, etc.

*** Save a lot of time and improve your productivity.

The code is available on GitHub.

How to use:

Step 1: Run the Electron app first, maybe put it in the startup folder (or equivalent on Mac), so that it runs when your PC starts.

Step 2: Make sure the InstantClipboard Electron app is running (check system tray)

Step 3: Copy the InstantClipboard.cs file in whichever Unity project you wish to use this. (inside an Editor folder)

Step 4: Copy and paste away…

Download:
https://github.com/saadnkhawaja/instant-clipboard-unity

Enjoy pasting whatever you want directly inside the Unity editor!

Here’s a video for you to enjoy.

/media/4d5470fba56738f2dafb7504b3d5d5c1

Follow me on Twitter: https://www.x.com/saadskhawaja
My Unity Asset Store Page: https://assetstore.unity.com/publishers/5951

Since it’s AI season, let’s briefly see how we can add AI using Unity in games. There are other ways of using AI to aid game design but let’s only focus on development for now.

AI in game development is all about using algorithms and machine learning techniques to make non-player characters (NPCs) and game environments smarter. Instead of relying on scripted behaviors, AI-driven entities can learn, adapt, and make decisions based on what players do. This leads to gameplay that’s more realistic and engaging.

Unity and AI

Games have always had some form of AI in them since forever. Whether it’s making NPCs, pathfinding etc. If you’ve not used AI in Unity before, I suggest checking the following out:

1. ML-Agents Toolkit: Unity’s Machine Learning Agents (ML-Agents) toolkit is great! It lets us create intelligent agents using reinforcement learning, allowing them to learn from their interactions within the game environment. Learn more about ML-Agents.
2. NavMesh: Unity’s Navigation Mesh (NavMesh) system helps NPCs find their way around complex terrains and avoid obstacles dynamically. It’s like giving your game characters a GPS! Explore NavMesh.
3. Behavior Trees: With behavior trees, we can model complex decision-making processes for NPCs without having to write tons of code. This makes it easier to create sophisticated AI behaviors. Understand Behavior Trees.
4. Third-Party Integrations: Unity’s flexibility means we can easily integrate third-party AI frameworks like TensorFlow to create custom AI solutions. Integrate TensorFlow with Unity.

The Next Steps

After you’ve done a bit of tinkering with the above, this is what’s happening right now.

1. Adaptive Gameplay: AI can adjust the difficulty based on how good you are, making the game more personalized. Imagine a shooter game where enemies get tougher as you improve—keeps things interesting, right?
2. Smart NPCs: AI-powered NPCs act more lifelike. They learn from their surroundings and player interactions, making their actions less predictable and more realistic. This makes the game world feel more alive.
3. Procedural Content Generation: AI can generate game content on the fly. Whether it’s levels, maps, or even storylines, AI tailors everything based on player preferences and behaviors. Read about Procedural Content Generation.
4. Player Analytics: AI analyzes player data to give us insights into behavior, preferences, and pain points. This helps us tweak game mechanics and design for a better player experience. Discover Player Analytics.

The Future?

The future looks bright for AI in Unity-based games, and here’s why:

1. Deep Learning and Neural Networks: As deep learning tech gets better, our NPCs will be able to handle even more complex tasks and make decisions like real humans.
2. AI-Driven Storytelling: Imagine games where the storyline evolves based on your choices, thanks to AI. This could lead to some seriously immersive experiences.
3. Real-Time Adaptation: Future AI systems might adapt the game world in real-time, responding to player actions in ways we can’t even imagine right now. It could change environments, NPC behaviors, and mission objectives on the fly.
4. VR and AR: Combining AI with VR and AR could create some mind-blowing experiences. Imagine intelligent agents in VR/AR environments interacting with players in super realistic ways.

Wrapping Up

AI is set to revolutionize game development. By leveraging Unity’s AI capabilities, we can create games that are more immersive, adaptive, and engaging than ever before. As AI technology continues to advance, the possibilities for Unity-based games are truly limitless. The future of gaming is intelligent, and it’s being built with Unity.

Let’s hope we’re not told to charge per install per NPC in the future though. xD

Thanks for reading, and happy developing!

The “No valid user created or default window found” error in Unity typically arises from issues related to corrupted or missing files in the Unity layout folder. This can be caused by improper shutdowns, system crashes, or recent updates.

“Revert Factory Settings” option does nothing to resolve this error so if you’re getting this error, chances are you won’t be able to open your project at all.

To resolve this error, you can delete the Project\Library\ArtifactDB file. Here’s a detailed step-by-step guide:

1. Close Unity: Ensure that Unity is completely closed before proceeding with any changes.
2. Navigate to the Library Folder: Open your project folder and navigate to the Library directory.
3. Delete the ArtifactDB File: Locate the ArtifactDB file within the Library folder and delete it.
4. Reopen Unity: Launch Unity again and open your project. Unity will rebuild the necessary files, potentially resolving the issue.

In some cases, you might also consider the following steps:

• Delete the Entire Library Folder: If the above method does not work, try deleting the entire Library folder. Unity will recreate this folder when you reopen the project, though this process might take some time.