Final Delivery

The Final Solution: KoopaKonstruct Manager

The BPMN diagram below is a simple representation of our KoopaKonstruct Manager.

In summary we are using Hololenses to inspect a chosen area where we want to e.g. hang up a coat rack, build a table, and similar. In order to make the Hololenses smart, we have integrated OpenAI. We are using Function calling to provide instructions about the requested project, information about necessary materials and quantity.This is where we are making our KoopaKonstruct really smart.

As real #Crawlers we use OpenAI’s Function calling to dynamically create the search criterias based on natural language. Google Speach to text is used to capture the natural language query. We are really integrating into everything that flips bits #Embedding numbnuts! With help of the search criterias, we are able to provide critical information further into the supply chain, meaning all the way down to the FinOps where we are improving critical business processes (this will be explained later in this blog post).

Below is a picture from the Hololenses where we use a button to speak with AI:

GPTVision helps us interpret work drawings and breaks it down into the steps needed, so that we can get a clear instruction of what materials we need, how many and how to build the desired object. Using Power Automate, we are able to share and send the item details and necessary quantity into FinOps.

As an addition to the OpenAI magic, we have also created a canvas app that is integrated in the Hololense #PlugN’Play. From this app you can perform business processes, such as triggering sales orders, check inventory status, and 4 different Power BI reports to monitor sales orders and follow up late deliveries #Dash it out.

And here is where we really improve business processes; when the information is sent to FinOps we use X++ code to check inventory status. If materials are in stock, a sales order is created and sales order confirmation is sent. If there are no (or not enough) materials in stock, we also create a purchase order in addition to the sales order. A link between the purchase order and sales order is created, enabling the ability to understand where the demand is coming from. Additionally, to make the KoopaKonstruct Manager adaptable to e.g. construction companies, we have created the possibility to link the purchase and sales orders with projects. This provides a lot of possibilities for detailed project management for the finance geeks.

Below is our argumentation to why we believe we should score high on all 4 categories!

Most Extreme Business Value

Our team embarked on a quest to redefine the meaning of business value, and we present to you KoopaKonstruct Manager – a solution that not only orders products through Hololenses but catapults business efficiency into a realm beyond imagination.

Automating the Complete Supply Chain with Hololenses

We harnessed the power of Hololenses to revolutionize the way products are ordered. One user, equipped with the mighty Hololens, registers the necessary quantity with a mere wave of the hand, triggering a Power Automate flow like a seasoned Mario collecting power-ups.

The adventure doesn’t stop there! Our Power Automate flow seamlessly transmits the order information to FinOps using the Execute action. The magic unfolds as FinOps automatically generates a sales order and a purchase order based on inventory stock. But here’s where the plot thickens – these orders are not solitary entities; they are linked together, creating a dynamic duo that unravels the mystery behind each purchase. There is no need to spend as much as a second wondering how or why the orders were created; the link indicate that there is a demand behind each purchase order.

See yourself sitting in an ordering office at a company where the customers are able to generate sales orders by only entering a demand. And, in addition to the automatic creation of sales order, imagine that you did not have to study the master plan to generate the purchase order as it is automatically created! And as a side salsa, you did not even have to trigger the sales order confirmation as it is sent automatically. So, while you are sitting there wondering about what to do with your life (and how long it takes before the CEO finds out that you are useless as the power of Power Automate and X++ took your job), you can relax and focus on other more important things in life, such as playing a Super Mario game.

Power Stars: Automated Project Linkage

But wait, there’s more! We also increased the business value for construction and engineering enterprises. With the ability to automatically link both the purchase and sales orders to projects, we’ve bestowed upon industries that thrive on projects a golden opportunity to enhance their efficiency and project management. It’s not just about ordering; it’s about constructing a bridge to a new era of project-linked prosperity.

The Warp Whistle to Most Extreme Business Value

We make a bold claim for the “Most Extreme Business Value” category in this year’s ACDC Hackathon. Here’s why our adventure deserves the highest score:

  1. Efficiency: Our solution isn’t just about ordering; it’s about efficiency power-ups that automate the entire supply chain process, from order initiation to linkage in FinOps.
  2. Dynamic and Time-saving: The automatic creation of linked sales and purchase orders provides unparalleled visibility into demand and supply, laying the foundation for strategic business decisions.
  3. Project Linkage: For construction and engineering businesses, our automated project linkage is a treasure trove of golden coins, offering enhanced project management capabilities and business value beyond the ordinary.
  4. Adaptable: Our solution is not only limited to the B2B-market, but can also be used for the B2C-market. Making the app available to customers, they can create orders automatically and receive high quality service as the complete supply chain has become even more efficient.

Excellent User Experience

We have created a Super Mario inspired canvas app that is adapted into the Hololenses. In other words, our app transforms ordinary into extraordinary!

Features:

Killer AI

Whether you’re a seasoned AI veteran or just dipping your toes into the mushroom-filled waters of innovation, our solution is a testament to the power that AI holds in shaping the future.

KoopaKonstruct Manager is not just a game-changer; it’s a game-transformer. Through the mastery of machine learning, we’ve empowered our solution to evolve, adapt, and learn from the challenges it encounters. From predicting order quantities to optimizing supply chain processes in the real world (and in FinOps), our AI-driven approach is a power-up for efficiency and foresight.

In the quest for the Killer AI “crown”, KoopaKonstruct offers more than just automated processes. It also includes magic in form of predictive insights, providing a glimpse into the future of demand, supply, and project landscapes. Strategic decision-making becomes second nature as our AI-driven system transforms data into actionable intelligence.

A mind of its own

In the pursuit of the existential risk badge, our AI solution goes beyond mere execution—it thinks, learns, and adapts with a mind of its own. Through autonomous decision-making capabilities, it navigates the complexities of supply chain processes and provides responses not even a 20 year old is able to find without searching the web.

KoopaKonstruct isn’t just smarter than a 5th grader; it engages in intelligent conversations that transcend the limitations of a classroom. It doesn’t just provide answers; it grapples with FinOps and supply chain complexities, understands nuances, and offers insights that reflect a level of intelligence resembling a dynamic dialogue.

Why KoopaKonstruct Manager deserves the Killer AI “crown”

  1. Innovation Beyond the Blocks: KoopaKonstruct isn’t confined to the traditional boundaries. It breaks through the blocks of conventional solutions, paving the way for a new era of AI-driven innovation.
  2. Adaptability in Every Pixel: Our solution dynamically adapts to the ever-changing business environment, ensuring resilience and adaptability in every pixel of our solution.
  3. Mind Mastery: Our AI isn’t just a tool; it’s a mastery that elevates KoopaKonstruct into a symphony of efficiency, intelligence, and strategic foresight.

Pandoras Box

Equipping a Hololens with AI that can answer questions, order materials, and assist in selecting the right materials and tools for any project based on online instructions transforms DIY into an accessible experience for the less handy. This technology takes the guesswork out of building, offering tailored advice on what to build, the best materials and tools for the job, and even suggesting projects that match your interests and available space. It’s like having a personal tutor and assistant rolled into one, guiding you through the entire process from conception to completion. This seamless integration of technology with hands-on work not only makes crafting projects more approachable for beginners but also enriches the experience with educational value, ensuring that anyone can undertake and succeed in their DIY endeavors with confidence.

PromptFlow: A Multifaceted AI Solution in Microsoft Teams 

Introduction 

PromptFlow, our innovative AI chatbot integrated into Microsoft Teams, not only streamlines race strategy and collaboration but also stands as a testament to advanced cloud technology integration. Targeting three distinguished badges – “Plug N’ Play,” “Crawler,” and “Stairway To Heaven,” PromptFlow embodies the pinnacle of AI-driven solutions in Microsoft’s ecosystem. 

Plug N’ Play: Enhancing Microsoft Teams with AI 

PromptFlow elevates Microsoft Teams by offering automated, data-driven insights for racing strategy development. By utilizing over 5,000 race statistics, it enables users to query lap times, kart performance, and player stats in natural language, demonstrating a perfect blend of AI and user experience. 

Crawler: Transforming Race Strategy with AI Search 

The “Crawler” badge highlights PromptFlow’s innovative use of AI search to navigate extensive race data, revolutionizing the way racing teams strategize. This feature addresses the critical need for quick and accurate decision-making, offering a significant competitive edge in the racing industry. 

Stairway To Heaven: A Symphony of Microsoft Cloud APIs 

Achieving the “Stairway To Heaven” badge, PromptFlow masterfully combines multiple Microsoft cloud APIs: 

  • AI Search: Powers the core functionality, enabling efficient data retrieval. 
  • Sentiment Analysis: Enhances user interaction by adapting responses based on detected sentiment. 
  • PromptFlow Technology: Our proprietary tech, ensuring smooth and natural conversational experiences. 
  • Azure OpenAI Instance: The backbone providing computational power and advanced LLM capabilities. 

This integration not only meets the badge’s criteria but also sets a new standard for AI solutions in cloud environments. 

Technical Integration and Future Prospects 

Each API plays a crucial role in making PromptFlow a robust, intelligent system. This multifaceted integration exemplifies our commitment to leveraging cloud technology for creating advanced AI solutions, paving the way for future innovations. 

The solution/demo:

Conclusion 

PromptFlow is more than just a chatbot; it’s a groundbreaking tool that revolutionizes race strategy development and team collaboration. It showcases our expertise in blending AI with cloud technologies, earning us the “Plug N’ Play,” “Crawler,” and “Stairway To Heaven” badges. As we continue to innovate, we look forward to exploring new horizons in AI and cloud computing. 

Note: This combined draft succinctly captures the essence of PromptFlow’s achievements in earning the three badges. It emphasizes the chatbot’s functionalities, technical prowess, and future potential in the realm of AI and cloud computing. 

Mining into Peach’s Kingdom

With this post we claim the badges:

  • Dataminer – for using dataverse as a datasource in Power BI, with data from the games.
  • Plug N’ Play – for embedding the Power BI report into teams.

The data from the Power BI report is coming from a table in dataverse called “Leaderboards”. You have the score of each player for a certain game, and what number of try there was for that game. We then use this data in the Power BI report to show all of the players score in a dashboard. There is also the possibility of seeing one specific players data. Which is of course presented in the same graphical theme as the other apps.

And of course it doesn’t stop there. We then also made sure to embed the Power BI report into Teams. Into our Peaches-team where we can then keep track of the players progress. The report addresses a crucial business need by allowing us to efficiently track and analyze the progress of players in the various games. We can then make data-driven decisions, like seeing which game the kids find more fun and entertaining. This can then be used to further develop games for the kids, and enhance their learning experience. Which will allow us to constantly evolve and improve.

Piping our way through the Azure Cloud

To make our amazing service Tubi work, a lot of cloud is needed. We aim to make the plumber’s job easier by recommending the best layout for where the pipes should go, and for that, we need AI. We have trained a model in Custom Vision to recognize all components in a bathroom that need water. So, when the plumber uploads a floor plan to our Static Web App, the image is sent to our Azure Function App backend in C# Asp.net through our own API. But both the image and the equipment list must be stored somewhere. Therefore, we have also connected to Azure Blob Storage. Then last but not at all least. The people working in the back office have instant interactive reports available to help them with filing and billing through Power BI and alerting the using an automated flow (Badges: Feature Bombing)

Sometimes it works, and that’s plenty

Databases are good, but sometimes it’s easier to just dump everything in one place until you need it again. Yes, it might not be very scalable or very normalized. SQL became too heavy, and we already needed a Blob storage to store the images, so we also dump the order data in the same blob storage as JSON files. It’s old fashioned way of serverstorage, and a bit dirty, but it works! (Badges: Nasty hacker, Retro badge)

Power the backoffice

As the final list of components are decided, they still have to be approved from the accounting team in the office. To make sure they have all the information they require, we have developed a Power BI dashboard to crawl through our registered data and make sure the orders are handled properly (Badges: Crawler, Dash it Out, Dataminer). And to make sure the orders are handled easy and fast, the dashboard is embedded into teams and an alert is automated by using a logic app to make sure the workers can receive and cooperate in realtime (Badges: Embedding Numbnuts, Go with the flow, Pug N’ Play, Power user love, Right Now, Stairway to heaven).

Plug N’ Play Badges – Send reward to the customer

Behold, as the final flourish to this grand adventure, the customer’s inbox lights up like a beacon of achievement! With a twinkle in its digital eye, the Microsoft Outlook connector springs into action, delivering a virtual diploma straight to the customer’s awaiting hands. It’s not just an email; it’s a messenger of triumph, carrying the seal of excellence from the Awesome Plumber Firm. With every diploma received, another customer joins the ranks of plumbing champions, their victory celebrated far and wide in the kingdom of email!

Imagine the triumphant moment: the work order completed, the last wrench tightened, and voila! Like magic, a path unfurls in the digital realm, leading straight to the treasure trove of work order documents! It’s like discovering a hidden treasure map in the depths of SharePoint! Every document, every detail neatly arranged along the path, beckoning with the promise of knowledge and accomplishment. With each completed task, another chapter is added to the epic saga of plumbing prowess, all waiting to be explored in the enchanted halls of SharePoint!

Analog Odometer Visualisation through PCF – Make it rain!

While this is a great visualisation, we must announce that the CSS and JS was not created by us, but by the great Developer VictorNP, which proposed a vanilla JS solution which we transformed into React functional components. As stated by the designer, It is mainly designed to be a read-only component used to display information – Therefore we’ve limited to bounded context fields.

This is an great example of how the limited possibilities of Dashboards in Dynamics directly, can easily be mitigated, as well making the visualisation reusable for different contexts through PCFs, making custom, code-first visuals in our Low-code solution. By harnessing the power of client-side processing, we ensure that our solutions are not burdened by heavy monolith front-ends, guaranteeing a seamless and responsive user experience every time. Of course, memorisation is applied to improve the transitions and performance of the visualisation.

The PCF component on a CRM form

Moreover, the PCFs functions as plugins for Power Apps, which further enriches the user experience through customisation. Our use cases will be mostly related to displaying important business and plumbing related KPI’s to keep track on the current state of our business. Lastly we wanted others to be able to reuse this visual and provide improvements to it, so we Open sourced it and made available on Github! Contributions are very welcome!

Star world to the cloud

Choo choo! All aboard the star train to the microsoft cloud! The sole action of finding and identifing bananas is not enough. We need to alert people of the danger that has been detected in traffic before they can cause more harm.

Introducing the Banana Notifier Bot (BnB):

The BnB helps our users by sending teams notifications when bananas have been detected.It uses our notification API (which gets data from our banana-scanning app), Bing Maps API and Graph API with location data and timestamp.

Our Notification API also utilizes the SharePoint REST Api to fetch next of kin information from a SharePoint list. Lastly, we store all our banana findings in Dataverse using the Dataverse Web API. Utilizing a total of four (!) Microsoft APIs.

Find the right Mario for you

We claim badges:

  • Plug N’ Play – for Dynamics 365 plugin that updates Mario for each support request based on the context using OpenAI
  • The Existential Risk – for AI decision making and work assignment without human involvment
  • Go With The Flow – for automating of the technician assignment
  • Crawler – for an innovative approach of searching for the right Mario technician

Sometimes Princesses are confused and don’t know where to start from when they have technical issues. So we are here to help them with AI! Princesses just need to add the description of the problem and the Dynamics Plugin will suggest Marios that are currently available and best match the case.

So the Princess creates a new support request, and that’s were plugin is triggered. The plugin sends the problem to OpenAI’s smart ChatGPT. The prompt:
Given the following case description, rate its severity on a scale of 1 to 4, where 1 is minor and 4 is critical:

I seem to be having a bit of a predicament with our kingdom’s email system, and I’m in dire need of assistance. This morning, when I attempted to send out invitations for the upcoming royal ball, I noticed that none of my emails were being delivered. They seem to be stuck in the outbox, and no matter how many times I try to resend them, they simply won’t go through.

The result of the OpenAI request is the severity level, in our case it is 3. So the plugin is looking for Marios who can cope with this severity and assigns the case to this Mario. Also, plugin updates the details of the case:

It is an extraordinary way to search for an engineer, but also a risky for administrators of the support team, as AI is taking away their job! On the other hand – it follows the flow and automate the routine process, which is a major benefit for the business itself….

Bowser Quiz

With this blog post we aim for the Go With The Flow, The Existential Risk, Mario badge and Plug N’ Play.

  • The Mario Badge for the gamification of Copilot.
  • The go with the Flow and Existensal Risk, because the use of Prompts with CHAT. One prompt to check if answers are correct or not The other prompt tells if the answer is Correct or Incorrect and will use this to send an action to power automate to populate a dataverse table.
  • The plug N´play for the use of prompt plugins in copilot.

Video of the in progress solution:

Steps:
Opening the admin center at https://aka.ms/ppac

Enabled Copilot for the Environment:

The Copilot component is ON in Canvas App:

Copilot

Found a website with Mario facts: https://kids.kiddle.co/Mario
Also uploaded a PDF with mario facts.

Conversation start

Conversation end or start again

  • Tested Generative AI inside the copilot studio, got answers, but could not check the answers with generative AI, it was good to give the answers, but not good to check if the answers is correct.
  • Made a power automate flow that generates a text with AI builder. – But it had limitations, it stopped with that you had to have an approval of text before i could process it.

So the next step i did was to try the new Prompts in Copilot studio: (It took many iterations of the prompt to make the copilot say if it was correct or incorrect).

Started making the Quiz topic that is the “game”, to test if it would work like i thought:

It works

Next STEPS:

Send correct answer to dataverse table to score a point:

Add the Copilot(chatbot) control to the canvas app: