Using AI to create a game

AI has been core to the development of our game in several ways. Almost all graphics has been generated by an AI, the story has largely been influenced by an AI and, an AI is a key component that you can talk to within the game itself!

Story and media

Both the story and media assets within the game has been created using custom GPTs from OpenAI. https://chat.openai.com/g/g-rjjz1GE5I-pixel-art has proven itself to be able to create imagery with a remarkable consistent look and feel based on our prompts, and https://chat.openai.com/g/g-dmgFloZ5w-storyteller has been a great sparring partner in the development of the story

The village butler

Meet the village butler, a core character in our game. He is omnipresent on your adventure and is always ready to answer any questions our adventurers may have.

The butler lives as a custom prompt in Copilot studio, and gets his personality there. In addition we feed him with context aware information based on where the adventurer is in the game together with every prompt he is sent, in this way adventurers can probe him for possible game clues or sometimes straight up answers to the clues.

Also, our guy knows who Botten Anna is, if that isn’t existential risk, we don’t know what is!

Integrating OpenAI GPT4 in Canvas App (The Existential Risk)

In this project, we demonstrate and integrate an AI system that can perform a simple task: identifying good or bad characters in Mario World for Princess Peach. However, we also explore the ethical and moral implications of this system, and whether it has a conscience or not by customizing the prompt and the AI behavior for custom output to the user and narrowing its conciousness as well.

The system consists of three components: a Power Automate flow that connects to the OpenAI Web API Service, a customized prompt that instructs the AI to analyze images of Mario World characters and swipe them left or right depending on their alignment, and a feedback from OpenAI based on our choices and a summary of the character relationship between the user and the character subject.

In this example we can somewhat restrict the behaviour and the information flow it spits out in the context of ethical response in relation to GDPR rules.

Putting Security-central Koopas out of Work: Automatically detecting miscreants for “apprehension”

One of our team members, Odin, has had the unfortunate experience of working in the security industry for 11 years. And he does not want anyone else to suffer the same fate.


That’s part of the reason why we created the Mario Detection System!

By creating a system for automatically detecting unwanted person (GDPR be damned), many security centrals around Bowsers kingdoms can now be emptied of all fault-prone flesh-based detection mechanisms!

And fewer Koopas have to be bored out of their mind on a 12-hour Sunday night shift in a dark and lonely security central 😉

How to see without eyes?

Azure Custom Vision!

We had a problem. A huge problem. We needed to know where in our floor plan the different components are placed.

By training our own custom vision model based on training data of bathroom floor plans, we managed to obtain an AI model able to detect components such as:

  • Toilet
  • Shower
  • Bath tub
  • Sink
  • Main pipes
Bathroom components detection with our custom vision model
Model scores on the training data

The model was published and the API made available for our backend application in C# to consume.

Super easy!

Copilot. The Rise of Machines

We claim badges:

  • Existential risk – for risk of existence of software developers

App should collect, list, track or managed the incidents that was assigned to engineers. The table must contains technical info for resolving the incident. Also the app should include the chart with summary of incident statuses. 

Specify prompt for app generation and submit:

Wait until the entities are created

Adjust if needed and proceed with app creation:

Just 10 seconds and you have an app that you are not ashamed to show on a demo to a client or to prove to a manager that you have been actively working and here are the results of your work. Isn’t it?

Find the right Mario for you

We claim badges:

  • Plug N’ Play – for Dynamics 365 plugin that updates Mario for each support request based on the context using OpenAI
  • The Existential Risk – for AI decision making and work assignment without human involvment
  • Go With The Flow – for automating of the technician assignment
  • Crawler – for an innovative approach of searching for the right Mario technician

Sometimes Princesses are confused and don’t know where to start from when they have technical issues. So we are here to help them with AI! Princesses just need to add the description of the problem and the Dynamics Plugin will suggest Marios that are currently available and best match the case.

So the Princess creates a new support request, and that’s were plugin is triggered. The plugin sends the problem to OpenAI’s smart ChatGPT. The prompt:
Given the following case description, rate its severity on a scale of 1 to 4, where 1 is minor and 4 is critical:

I seem to be having a bit of a predicament with our kingdom’s email system, and I’m in dire need of assistance. This morning, when I attempted to send out invitations for the upcoming royal ball, I noticed that none of my emails were being delivered. They seem to be stuck in the outbox, and no matter how many times I try to resend them, they simply won’t go through.

The result of the OpenAI request is the severity level, in our case it is 3. So the plugin is looking for Marios who can cope with this severity and assigns the case to this Mario. Also, plugin updates the details of the case:

It is an extraordinary way to search for an engineer, but also a risky for administrators of the support team, as AI is taking away their job! On the other hand – it follows the flow and automate the routine process, which is a major benefit for the business itself….

Mario Onboarding Center

We claim badges:

Hipster: We protect privacy by not saving any user information except the name in our database. Also, provide cartoon avatars instead of real face of the employees.

The Existential Risk: Our AI generates interesting background stories where humans may fail to do so.

Mario badge: Gamification of the onboarding process. Instead of boring onboarding we provide users a game character creation experience when joining the company

Thieving Bastards: We use paid ChatGPT api to do api requests.

Do you want a fast onboarding process for your agents? We have the right solution.

In our company, we use AI to generate images that look like your agents. We have customizable prompts to generate different characters. We can create a quick backstory for an immersive customer experience.

You can choose different appearance options. For Image generation, we will need a prompt and an image of the person. Based on the selected character AI will make your avatar a lookalike.

Take your picture and generate an image. We are using chatgpt API for image generation. It is a subscription model for text generation and pay as you go model for image generation. Using existing api instead of building a stable diffusion solution save from time. We use power automate cloud flows to call API requests and retrieve responses. We let the ai combine user picture and prompt to create a cartoony avatar.We do not save any user information in our dataverse to keep up with privacy requirements

Choose your theme for the background. AI will create a nice-looking story for you. Based on what is said in your story it will set a skill level too. This time we are using text generation apis of chatgpt with flows. Based on selected background option ai will generate a background story and skill level. Skill level considers generated story too, so it is meaningful.

.

Let us show you our potentyally world destroying AI

Responsible AI implementation is critical, especially as we delve deeper into advanced technologies. Microsoft is leading the way with its six foundational principles of responsible AI. In this post, our spotlight is on one of these key principles: reliability and safety.

Consider a scenario where a language model, which I developed, exercises complete control over another language model’s instructions. This setup immediately poses a challenging question of accountability, particularly if the subordinate language model engages in undesirable behavior, such as generating insults. While it might seem straightforward, the responsibility ultimately falls on me, as the developer. This underscores the importance of integrating robust oversight mechanisms and ethical guidelines in AI systems to ensure they operate within the realms of reliability and safety. By doing so, we can not only mitigate risks but also enhance the trustworthiness of AI applications in our increasingly digital world.

The described solution represents a high-level design overview. Leveraging the Bot Framework, it analyzes user sentiment and feeds this data into PromptFlow. However, it’s at this juncture where we encounter the more nuanced, potentially contentious aspect of our solution.

Our current PromptFlow process is straightforward yet effective. It primarily requires two inputs: sentiment and text input. Utilizing the meta-prompter LLM, the system communicates with an OpenAI instance located in Norway East. The core function here is to relay instructions to a secondary language model, ensuring the generated content aligns with the specified sentiment.

While this mechanism might not seem revolutionary, its effectiveness is largely due to the robust moderation capabilities of Azure’s OpenAI instances. These safeguards are crucial in preventing misuse or unintended consequences.

However, it’s important to acknowledge the potential risks. In the hands of someone with advanced prompt engineering skills and access to the right language models, this system could be manipulated into producing outputs that are ‘very non-compliant’ with ethical guidelines.

We recognize that our self-prompting AI solution might not be groundbreaking in practical applications, but it has already earned us the Dooh badge, showcasing our innovative approach within this technological space.


Bowser Quiz

With this blog post we aim for the Go With The Flow, The Existential Risk, Mario badge and Plug N’ Play.

  • The Mario Badge for the gamification of Copilot.
  • The go with the Flow and Existensal Risk, because the use of Prompts with CHAT. One prompt to check if answers are correct or not The other prompt tells if the answer is Correct or Incorrect and will use this to send an action to power automate to populate a dataverse table.
  • The plug N´play for the use of prompt plugins in copilot.

Video of the in progress solution:

Steps:
Opening the admin center at https://aka.ms/ppac

Enabled Copilot for the Environment:

The Copilot component is ON in Canvas App:

Copilot

Found a website with Mario facts: https://kids.kiddle.co/Mario
Also uploaded a PDF with mario facts.

Conversation start

Conversation end or start again

  • Tested Generative AI inside the copilot studio, got answers, but could not check the answers with generative AI, it was good to give the answers, but not good to check if the answers is correct.
  • Made a power automate flow that generates a text with AI builder. – But it had limitations, it stopped with that you had to have an approval of text before i could process it.

So the next step i did was to try the new Prompts in Copilot studio: (It took many iterations of the prompt to make the copilot say if it was correct or incorrect).

Started making the Quiz topic that is the “game”, to test if it would work like i thought:

It works

Next STEPS:

Send correct answer to dataverse table to score a point:

Add the Copilot(chatbot) control to the canvas app:

Captain’s log – data at your hook-tip

The captain log business value is that he has an app with all the tools he needs at the tip of his hook.

A new feature:

Behold the “Reveal your true face feature”

Revealing the true face

Some pirates find their interest in hiding away, and then it is handy to have an app to show the true face and spot if this is a know fellow shipmate or a rivalling pirate that deserves to be keel hauled.

The service uses a mobile or tablet camera to take a picture of the person in-question and then posts it to mutiny services for identification and face recognition in order to find who this person is.

Claiming the ACDC Craftsman since it is created using actual code with some best practices in react development: https://github.com/TheBastards-ACDC2023/Bastards-NativeApp